ProRealCode - Trading & Coding with ProRealTime™
Moderation stepping in (yes, even on a sunday…) : I lock the topic temporarily, everybody calm down and enjoy the rest of this sunday please.
I will unlock it tomorrow morning, and any further post from this point onward is expected to be only about Gauvel’s points, and/or performances queries, feedbacks, and/or fair product challenges.
The slightest hint of half a word towards adding a reply to the reply of the reply of an argument which started somewhere else God knows how long ago (and I’m not really asking, please don’t answer that), and I’ll delete such post in full directly regardless of what else might have been of value in the post. At this point, don’t count on me to start sorting within a post which phrase is ok to remain, and which phrase should be moderated out.
I sure hope I am making myself clear.
Have a good sunday everybody.
Hi Gauvel,
Thanks for sharing. Your answers made it a bit clearer in some ways, but in other ways it raised some concerns.
We appreciate you taking the time to put this together, but in order for us and for any readers of this thread to understand and get any values out of your images, and what it means in practice, I think we have to dig a bit deeper.
I will get to your final questions a bit later if you want. I suspect that they might not continue to be relevant for you, once you’ve understood the shortfalls and limitations of this correlation analyis.
So I’ve bundled your comments together into a bullet list, you say that:
1) The images indicates correlation between loosing positions for each algos.
2) The analysis are on all of the algos
3) Prime and PTAF are algos recently added by PRA
4) The analysis are based on backtested data
5) The analysis are only weighing in losing trades and not weighing winning trades
6) The analysis are made with a backtest with the “minimum/default” position size.
7) The analysis are made with a backtest with the “default” spreads.
8) The analysis looks at daily correlation
9) The analysis are made with a backtest with the data back from 2013-2015.
MY COMMENT
I think your analysis can provide some value, but it’s not really saying anything about the total results and experience of a live user of our algos, as you are only looking at losing trades and that you are only looking at a daily interval.
As an example an algo can make 1 losing trade of (-500€) and 1 winning trade of (+€800) in the same day with a total daily results of +€300. That algo, in your method of analysis, would be considered having a high correlation with an algo that only made one losing trade that day. Which quite frankly, is not high correlation in my mind.
It’s not unusual for algos, during high volatility days, to make both big losers and even bigger winners in the same day. Our algos trade more often in days like that, and if only looking at losing trades, that ofcourse will have a huge impact on the data and possibly on the conclusions you draw from that data. Also, it’s not clear, atleast to me, how you are weighing the lossing amount of the algo.
Also, I see that an algo COULD have a high daily correlation with another algo (looking at the total daily results including both winning and losing trades), but a low intraday, weekly, monthly correlation. I don’t think that the daily correlation, even looking at both losing and winning trades, is the most relevant time interval to look at.
With a proper ‘stay in the game’ position sizing it’s probably more relevant to look at weekly or monthly intervals, as I don’t see that the results of a single day can have any significant affect on the capital of the account.
Regarding point 2), let be clear for anyone reading, that this analysis is not actually on all algos. Some algos are left out.
Regarding point 3): Actually no. The Prime algos, which is the biggest part of our algo portfolio were released many years ago. The 2 PATF algos however were just recently released.
Regarding point 4, 6, 7, 9): So the devil are in the details. As probably most people are aware of, the backtesting engine are missing those details and is undesirably blunt, especially if you are running backtests over long period of times. Just look at the fact that the overnight fee of having a trade on US Tech 100 for example is around €3.5 today but was less than €0.5 only 18 months ago. The backtest engine can’t take that 700% increase in overnight fee into account, and the same goes for spreads, slippages etc. A backtest can with the proper spreads and fees, on a very limited time period, give you a picture of how the algo performs, but as the spreads are not unusually 5-6-7x higher in the later parts of the backtest than the start (and even intraday), individual trades of such a backtest can’t be entirely trusted. So backtests are a powerful tool to get the bigger picture, but it’s not a adequate tool to extract individual trades from and to make a detailed correlation analysis on. Quite frankly, the analysis and it’s subsequent conclusions, can by definition never be better than the data that you put into it.
I will Gauvel, with that said try to provide you a list of real trades from a real account and you can re-do the analysis to provide full value of such a correlation analysis, perhaps incorporating the rest of the feedback that I have supplied here.
I appreciate a strong due dilligence, but I would be distressed Gauvel, if you only after running our algos for a couple of days, decide to stop running them based on this fragmenteded(Sorry!) backward-looking analysis, that’s based on backtested data and not real experience.
PS. If anyone wants to see the exact correlation of our algos with real trades on a real account, I think one of the best metrics for that is looking at the Maximum Drawdown, and that can be found on our website ProRealAlgos.com/results.
Asking our Verified customers on our discord is another way to do it.
Coming from a neutral stance, but perhaps to add some perspective.
I do look at PRA results daily because I’ve only been live (with all my own bots) for 14months. I find it useful to manage my expectations in algo trading compared to manual because without the experience it is hard to have formed a solid perspective. Particularly on a portfolio level.
I note the NDQ bots in particular having a very tough time. 3 months ago when I do correlate my own backtest results to put into MC testing and then adjust for risk units (ie position size , max DD) I have the most amazing performance (who didn’t on NDQ longs?). Super reliable and some of which I had and some of which I lost as I size up. As it stands my NDQ longs where I have high conviction have been absolutely reamed in the past few months. Is this a surprise? Not really. Starting high conviction sizing into September/Oct when both the bull and bear case are extreme and valid is my own undoing. Correlation on 1 min data is extremely dangerous although I do have now 12 months with live data attached. With the knowledge of live performance still holding or gaining during 8% -11% pull backs. To me I learn that high conviction means half the size I thought I could handle and most definitely I need (as all traders) to learn to manage my expectations through market cycles.
On the flipside, my B-grade bots which makes up the bulk of my activity have very little data (again 1 min with 12-14months total), I have to oversize them a little but produce beautiful equity curves…over time. I could not have possibly know how reliable they would be at the start and had to make trader judgement calls. Everytime I think they have gone bad, they get good again! Such is algo trading.
So I guess, regardless of correlation methodology and even time frame then we all commit to the risks we take upon ourselves through some level of subjectivity. As soon as that’s paid for we have someone to blame. That is not good as a trader.
These days I spend more time measuring my own processes than building and testing news bots. Systems for my systems if you will. Everything that happens is my fault. However after over a year of auto only trading and making some slow money I have returned to some manual trading to have some control over DD’s and the like, particularly income. It is very easy to wake up a victim to a string of algo losses, but hey one can can always take matters into our own hands. Turns out testing and learning for 18months has transformed my manual trading.
I’m grateful for any algo trader to display and discuss their performance and processes and particularly grateful for this thread and forum. So, what ever experience traders here are having just get as much of it as possible and keep moving methodically through the problems.
Very keen to read more correlation conversation to continue, I think we could all grow from this topic.
Just typing outloud 🙂 …
What Gauvel is after (and he will correct me when I am wrong) is whether there is some kind of master plan behind the ProRealAlgos algos. An example would be Long algos only and their correlation undoubtedly would be that they perform worse on bearish days. Or at least they would not make more profit.
Here we readily have the difference between looking at the performance in general and observing losses only. The latter would – in my view – show a stronger correlation, when there.
Gauvel suggests overfitting as a reason for the correlation showing losses and I don’t think he is claiming that ProRealAlgos algos are the lousiest in the world (but it is the undertone). Watch this opening sentence :
It’s seem nobody at ProRealAlgos try to build a solid ‘portfolio’.
which to me tells nothing more than it says : ProRealAlgos has many algos available, but all together they lose (is the clear suggestion) and nothing is hedging each other, etc.
That is really all.
That it brings Gauvel losses of $7500 (I took it that this is for real), is something which should not be, once a sales argument would be that “if this goes wrong, that goes right”. I don’t know whether this is the proposition with ProRealAlgos. Maybe that is not important and all Gauvel may tell is that he is over-disappointed with the results.
Maybe a small lesson for Gauvel (from personal me) : of course all is overfitted. If not, then Carl and the other names will tell how to do that; sure ready to learn here !
And so Gauvel, your best mistake was to backtest back to 2013 for this reason alone already – the overfitting which occurred at developing the algos (add the reasons Carl suggested). But the real point sadly is :
Live will fail just the same for the same overfitted reason.
So all what is happening IMHO is that you are possibly inexperienced with obtaining third party algos, and thought that you could buy some automatic trading system from someone for a few $ and get rich. You know, those programs everybody fails on unless you can find one person in this community who has it all under control and makes a million+ out of it, each day (because why not, if it works anyway – right ?).
Hi PeterSt,
Thanks for your comments. I see that you like to hang around in this thread.
I think it goes without saying that the algos we develop and offer aim to be as diversified as they possibly can be. That is probably what you call ‘a master plan’. One example of that is that out of the 19 algos we are offering are actually built on 12(!) different strategies. We are proud to always be benchmarked as the leading developer, and just the sheer number of different strategies really sets us apart from all other developers.
I don’t agree with the statement that “Long algos only and their correlation undoubtedly would be that they perform worse on bearish days. Or at least they would not make more profit.”.
We got multiple long only algos that are actually mean reversion algos, meaning that they will enter beautiful positions in bearish days, when the market is oversold. These long only algos actually prefer bearish days. If you would bundle up all long only algos into a group, that group of long only algos would perform worse on bearish days – but what’s the point of bundling up long only algos in such large binary category/group? It’s much more interesting to look at mean reversion strategies as a group, or trend following strategies, or bearish hedge strategies and most interesting it’s probably to look at the full portfolio of all algos (not leaving out some).
The quote you refer to is based on the fragmented correlation analysis we are currently discussing and not on any live results (as Gauvel has only been using the algos for a couple of days). I’m confident that Gaufel re-evaluate this statement based on the feedback that we’ve provided above, and also based on the real results he will experience over the coming months.
I can assure you PeterSt that Gauvel has not experienced any $7,500 losses. He only started the algos just a few days ago. He is referring to something else, but I can’t really understand where that number comes from, but it’s not results from his own account – the same way that this correlation analysis is not based on results from his own account.
Again, I can’t speak for Gauvel, but if he’s dissappointed in the results after running the algos for a few days then, I’m sorry, but I’m much more interested in hearing what Gauvel thinks after running the algos for 1 month, 3 months and 12 months. One day in trading means nothing, one week barely means anything, one month means something and twelve months gives you the proof. I don’t think his alleged disappointment comes from the 4 days results of a 365 days license but rather comes from this fragmented correlation analysis, which we are now discussing and that we’ve now highlighted some shortfall in.
I certainly don’t agree that all third-party algos programs fail and I would suspect that ProRealCode doesn’t agree with you either.
Hi coincatcha,
Thanks for sharing, it’s always refreshing to hear from other traders in the trenches. I’m happy to hear you’re following our results on a daily basis,
Yeah those high-conviction trades is relatable. What’s even more relatable is the what you’re saying about ‘bad’ algos going good. We learned that pretty early on in our algo development careers that we kept stopping algos that would just weeks after being stopped would make new all time highs. One thing led to another and we’re now following a strict book of rules when deciding whether or not an algo should be stopped or if it deserves another month of trust. Another measure to avoid that is to only make these decisions on a monthly basis.
I agree, sometimes the market can throw curveballs, especially when emotions are running high. Managing expectations and sizing positions accordingly is a valuable lesson.
I’m also eager to see more discussions on correlation here. Sharing experiences and knowledge can help us all become better traders. Thanks for contributing to the community, and let’s keep the conversation going!
Firstly, I am in any case doubting the overall return of the last 3 years. I guess you wouldn’t have survived if it weren’t the case.
I’m simply pointing out the strong correlation. If I buy 19 algorithms, I hope to find 19 authentic ones, not inbred ones from only 5 close relatives. (Joking here)
On your part, the biggest problem is that the algo portfolio is down 7k (standard contract size) and on standard position sizing the portfolio is marketed to require à 10k balance. If I loaded the algos at the mid-July peak. I would have 3k left to spare, unable to even run half of the 19 algos.
When too much overfitting occurs, a thesis among others*, working on overall correlation, W+L, is kinda shady. Taking losses for granted, doubting gains, that should be the way. For decimal R:R like your portfolio, the emphasis should study the losing side before merging the L to the W.
*not fully mine yet concerning the portfolio, it would be tough to go deeper because I only have a fraction of the code for each algos
Building a portfolio, ones of the first questions should be :
Additional questions :
1) Concerns about the analysis:
You express your belief that the current analysis lacks depth as it only considers losing trades and daily intervals. You mention that on high volatility days, algorithms can have both significant losses and even bigger wins in a single day, affecting the correlation analysis. You question how losing amounts are weighted in the analysis and suggest that daily correlation might not be the most relevant interval to examine, advocating for weekly or monthly intervals.
Answer : Daily correlation should be predominant when answering ‘which algos are similar ?’ ( In the loss point of view ). It’s still too high in the weekly based approach.
2) Clarification on the scope of the analysis:
You clarify that the analysis doesn’t cover all the algorithms and that some are excluded from the assessment.
Answer : Yes. It’s not voluntary. Those track records ( in $ / Yen ) need to be translated in Euros and I didn’t have the time to that. I’m not trying to hide negative correlation ( By guess I would suggest that the Yen one is weakly correlated which is good )
3) The release of algorithms:
You specify that the Prime algos have been in use for many years, while the two PATF algos were recently introduced.
Answer : I didn’t know that.
4, 6, 7, 9) Issues with backtesting and details:
You point out that the devil is in the details, emphasizing that backtesting engines lack precision, especially over extended periods. You mention issues with overnight fees, spreads, and slippages that can’t be accurately accounted for in backtests. Backtests offer a general overview but are insufficient for detailed correlation analysis of individual trades.
Answer : That’s true.
5) Offering real trades for analysis:
You propose providing a list of real trades from a real account to allow for a more thorough correlation analysis, considering the feedback provided.
Answer : Yes, I would be glad to get the live trade Track Record since inception.
6) Request to avoid hasty decisions:
You express concern that making decisions about running algorithms after only a few days of analysis based on historical backtested data might not be the most prudent approach.
Answer : I have lost far too much in two weeks. Since the PRA portfolio was already in bad DD I wanted to make further investigations.
coincatcha : ‘So I guess, regardless of correlation methodology and even time frame then we all commit to the risks we take upon ourselves through some level of subjectivity. As soon as that’s paid for we have someone to blame. That is not good as a trader.’
As I said ‘I should have done it before giving away my money.’ l, I’m only blaming myself.
We lose some, we gain some, that’s how it works. In the end it’s just some money.
So all what is happening IMHO is that you are possibly inexperienced with obtaining third party algos
First time renting algos, yes
and thought that you could buy some automatic trading system from someone for a few $ and get rich.
No, that wasn’t my plan, but money is still a good factor to assess performance and work. I wanted to diversify my current portfolio and the next one with a third party one.
Gaveul, just to clarify, you are saying that you are NOT doubting the overall returns of the last 3 years, right? Just to clarify as that sentence was a bit ambigious.
Yeah, I really get that you want as many authentic strategies as possible – and that’s why we are proud to say that we’re the one developer offering the highest number of different strategies. You get all of them when buying the All algos license from us. As mentioned they are 12 completely different strategies and codes, and more are being added as we speak.
I see again that you are referring to a -€7k drawdown, but the drawdown from our ATH(aug 1st) to the latest datapoint (oct 1st) is -€2,67k (€36619-€33953). That should be put in perspective of the +€7,4k gains from January until Aug 1st.
So far in October we are at a loss, but we’ll see the end results for the full month in a week but overall we are at good profits this year.
As you have only run the algos for a couple of days yourself I assume that you are basing that -€7k number on something else, perhaps the livetrades site on our webpage, which until just a few days ago were showing trades from algos that were stopped months ago and didn’t include any new algos released. This I have explained to you. So any results you see on the livetrades site are correct starting this week but not before that.
Regarding your question why some algos does not start before 2015, I don’t understand what you mean. There are no limitations built in to the algo on how far back you can make a backtest. Any limitations of the time period of a backtest is solely related to the historic data you have access to. You might not have data before 2015.
I don’t agree that a daily correlation analysis should be the predominant way to make correlation analysis. You say that it’s still too high in the weekly based approach. If you’ve done such an analysis it would be interesting to see but again… I think you agree, that we are wasting time discussing details regarding a correlation analysis, based on trades produced by a blunt backtest instead of trying to re-make this correlation analysis with real true trades. The correlation analysis and the conclusions you can draw from that, can by definition never be better than the quality of the input data.
Regarding release dates of the algos, you can see all release dates at prorealalgos.com/results
Let’s keep the conversation going! Have a good one
Before embarking on any mathematical correlation studies (of which I have not found the best way), I do this.
Here is a manual study I do when I’m looking to pair a set of algorithms.
There is a few things to note here being:
1) There is very limited data here to work with.
2) The Nasdaq was on a tear up until July, adding a very strong recency bias to the data. The short version I have used for over one year so it’s a trusted system and the consecutive losses are a known/expected part of the system.
3) Clearly the long has collapsed and both systems are currently positively correlated since I have started them together live.
I have sized down the Doji system by half shortly after the start of its second (continuation DD). If I didn’t know the markets were currently on a knife edge I would absolutely panic at this and would, as with any trade recognise a change in character of the pair. Like Gauvel, of course I must look around and test every NDQ long I can get my hands on to see if it’s just mine and get some answers. It is not. The NDQ has been un-tradeable long for a few months and has cost me a packet. No new car for Christmas for me this year and fortunately this is only one account.
Carl, thanks I have heard you say this about waiting until the end of the month to review and take action and have rules for myself to do this. Should the seasonal bias not play out (maybe known tonight or over the next week), regardless I will do what needs to be done without panic.
I don’t ever see many traders post or really publish fails and losses so I thought I would for some perspective as this is just a moment in time where something is very much not working out!
The bottom line is that there is ZERO evidence of someone making money with the algos from PRA over time. Everything else is noise.
During the 3 years or so I have been ware of this company, I have not heard about anyone making money over time. Instead, the road is littered with people who got brunt and threw in the towel.
Nobody is able to verify their results. Nobody is making money over time. That is my conclusion and opinion.
I am open to be proven wrong.
Hi Allan. Thank you for granting us and this thread your FIRST forum reply of all time 🙂 Welcome! I’m happy that you are ‘open to be proven wrong’.
The last few replies in this thread have been a discussion regarding the validity and quality of Gauvels correlation analysis, not about the validity of our historic results. It’s a big jump, in both theme of topic aswell as in logic to proclaim that “ZERO evidence of someone making money with the algos from PRA over time”.
With that said let’s keep the topic you now bring up as separate from the correlation analysis, and let me address this topic on doubting any results of our clients. You are claiming that 1) there is ZERO evidence of someone making money with the algos from PRA over time. That you haven’t heard about anyone making money over time 2) That the road is littered with people who got brunt and threw in the towel. 3) Nobody is able to verify the results.
Let me adress these 3 points one by one
So which kind of reviews fulfills these two conditions? Well, I see that there are two different third-party pages where you can read reviews that fulfills these two conditions. One is the ProRealCode Marketplace and the other on is Trustpilot. We have a link to our trustpilot on the start page on our website – and that’s what I mean that you are actively turning a blind eye to any evidence that opposes your claim.
Have a good one! Best regards / Carl
Thanks @coincatcha for sharing. Yeah it’s undoubtedly a rough time for many algos!
How much historic data do you got access to? On 1 minute it’s as you say quite limited and it’s easy to build something overfitted.
Yes, OT given the recent discussion but relevant to the main topic? My first post on this forum was many years ago btw.
A very detailed reply, thank you.
1. The Trustpilot reviews;
-How many of these people having left reviews are still actively using your services?
– Most reviews are from 2021 or before. I recognise a few of them and I know for a fact that they have stopped using the algos altogether.
-From 2023 I recognise 2 people, I am not sure they are thrilled by the results anymore, judging by your own discord.
– Spring 2023, I count to 7 reviews from Pakistan. May I ask if IG is operating in Pakistan, or which platform do they use?
-One guy gave a 5 star review because you refunded the package….
– The marketplace reviews came with a gift back in the days, I can not remember what and the telegram channel has been closed down (so evidence lacking) See screenshot for incentive on trustpilot reviews.
All together: The reviews do not prove that anybody is making money long term.
2. Looking at the cycle of people arriving excited and hopeful on the discord, to then vane and bail out. There are many examples of this and you know it. The explanation you give as to why is credible. Still doesn’t prove that anyone is making money.
There is very limited if any interaction with your posts in social media. Anyone making good money from this would surely show some more interest?
3. Great, people can download and do their own BTs. I have not seen anyone trying to compare results to yours since the guy doing the weekly BT board on discord left the building. Even then, it does not include costs as rentals, overnights, slippage, tax etc. So again, does not prove that people are making money.
So to sum up: I am still of the opinion that there is ZERO evidence that people are making money over time. As in science, nobody can prove a negative, so I might be completely wrong, positive evidence pending.
I do not blame people mismanaging their accounts on you. I want you and everyone else to be successful.
Perhaps one of the 99% you talk about could step forward and give some hard evidence of actually making money?
Thank you.
Yes I noticed. That’s why we feel honored that we’re considered such an intriguing/engaging topic, that people that never post on the forum, that doesn’t seem to take much interest in other ProRealCode related topics, take the time and energy to post at this particular thread. Don’t know why though, but it makes us feel honored.
The topic about reviews, how results can are verified, how to come in contact with our customers, how to be the most transparent developer out there have been discussed in length over the years, even in this thread but I will adress your points here aswell. I feel however that the discussion is derailing a bit, as you are now also questioning the autheticity of third-party reviews. I’m sorry but reviews on a third-party site from verified customers, is as good as it’s ever going to get in terms of testimonials.
If you are still of the opinion “that there is ZERO evidence that people are making money over time.” I don’t think there is much for me to do about that. You say that you are ‘open to be proven wrong’ but my overall assessment is that you probably aren’t. I would just be wasting time, but I think that the time I’ve taken to write this message can be benificial for readers of this thread later.
Have a good night Mr Allan.
Best regards / Carl G Eriksson
ProRealAlgos? Real or fake
This topic contains 226 replies,
has 29 voices, and was last updated by ProRealAlgos
2 years, 3 months ago.
| Forum: | General Trading: Market Analysis & Manual Trading |
| Language: | English |
| Started: | 07/28/2020 |
| Status: | Active |
| Attachments: | 55 files |
The information collected on this form is stored in a computer file by ProRealCode to create and access your ProRealCode profile. This data is kept in a secure database for the duration of the member's membership. They will be kept as long as you use our services and will be automatically deleted after 3 years of inactivity. Your personal data is used to create your private profile on ProRealCode. This data is maintained by SAS ProRealCode, 407 rue Freycinet, 59151 Arleux, France. If you subscribe to our newsletters, your email address is provided to our service provider "MailChimp" located in the United States, with whom we have signed a confidentiality agreement. This company is also compliant with the EU/Swiss Privacy Shield, and the GDPR. For any request for correction or deletion concerning your data, you can directly contact the ProRealCode team by email at privacy@prorealcode.com If you would like to lodge a complaint regarding the use of your personal data, you can contact your data protection supervisory authority.