Backtest results change when I stop using variables
- This topic has 56 replies, 3 voices, and was last updated 4 years ago by
.
-
Thanks for the correction GraHal – it did concern me that the long and short exits were not at all balanced as that is another instant curve fitting alarm bell.
Attached is the 20/5 robustness test with your correction and the ITF file of the strategy with the robustness test added showing how TRADEON should be added to the entry conditions when a strategy can reverse direction.
My concerns about the equity curve still stand unfortunately.
Attached is one of the worst equity curves from the robustness test. You have to ask your self when looking at any equity curve whether that is an equity curve that you would have traded for very long and also analyse where you think all the profit was actually made.
If you had decided to go on holiday and switch auto trading off while you put your feet up or take short periods of time off from trading then that is the equity curve that your strategy could give you. If an awful lot of the robustness tester results give similar equity curves then there is a very strong chance that you could actually end up with one on your trading account that looks like that!
1 user thanked author for this post.
GraHal, Vonasi
Thank you for your input. I haven’t yet tried out the various suggestions although I did spot the short trade “imbalance” and so did not go Live. Thank you for your revisions. I only go Live with 20p trades at first anyway to see how things work in the real world. I note your equity curve comments but the variables set in this system are trying to address the sideways market. I’ve only ever tested it from 1 Jan 2018. I have another set of variables to use in trending markets. Like you suggest, I would never trade a curve like that – too risky for little old me. Interestingly the system produces poor result on the S&P500, which you’d think would track the DJI. Tests on the FTSE 100 were better than on the DJI. This suggests to me that the variables are over-fitted to the DJI, whereas I’d rather have less potential profit and a more robust system that could be plugged in to any of the major indices. Also, Vonasi, I take on board your comments about “so many variables” and I think a simplified system may become less over-fitted…And, yes, wise advice about becoming rich!
It’s been a great experience learning from you two, and that’s just so far.
OK, late start today – so I’m away to get on with it
1 user thanked author for this post.
I think I may have taken the code that Vonasi wrote for the robustness tester and it had your old code in it…Not sure, but I’m glad we’re all in agreement now! Thanks for coming back to me.
1 user thanked author for this post.
On Sunday when I was testing your code, I kept hitting / getting stopped by the tick by tick limit.
There is a limit of 2500 bars where – when using tick by tick – which if > 2500 times that the TP and SL are hit in the same bar then we get an error message asking us if we want to turn tick by tick off or close the Backtest.
Does above account for Issues you are seeing / having @Mike?
M FV produces 9007 and M HC collapses to 4063
1 user thanked author for this post.
I saw that message from time to time a few weeks back when I was just starting to use PRT. I haven’t seen it in these tests. I saw you and Vonasi seemed to be testing back to 2000 or earlier. I am only testing 21 months, which is way under the 2500 limit. Short answer is “no” 🙁
A thought: your code and its variables suffered less in tbt than mine – possibly because tbt had less impact on your wider stops than on my narrower ones..?
1 user thanked author for this post.
I checked on my CFD Account and got same values … see attached and bear in mind that this is for the DJI 2$ CFD
1 user thanked author for this post.
Hers the M-FV and M-HC … same value again
1 user thanked author for this post.
Find exclusive trading pro-tools on 
