Have you ever seen that the Worst Parameters are the first results on Backtest ?
- This topic has 27 replies, 7 voices, and was last updated 5 years ago by
.
-
I think the most important problem is that it’s not very easy to explain
Just remember it’s just a method to test the robustess and to find the best factors/indicator to obtain a correlation between IS/OOS results
I will re-try next week to explain if I have some time
Happy week-end
1 user thanked author for this post.
Well, after thousands of simulation on OOS, on differents codes, I have some first results…
I study 15-20 Pearson correlations between IS and OOS results (Gain, drawdown, runup and so on..)
First : It’s absolutely not easy ! 🙂 and seems not to be reproductible in all algos
In general it confirms what I said, there is a quite good correlation between the rankings in IS and OOS results, may be at 70- 80 %
But it confirms what I sais in this post the best variatiions are not the first in the ranking in the IS Backtest= the classical backtest !
The better evolution +15, +20 or +30 % are between rank 80-100 in IS Backtest results
Yes you can have a very bad curve and a wonderful OOS evolution !
The 1-20 rank in IS Backtest results give less than 10 % and often -5 %, -10 %
It confirms what we already know, the more curvefit we do (A lot of variable, a very long code and so on..), the worst results we will have in OOS !
If it’s possible to “predict” the first results in the ranking on OOS, at this time I find no correlation to predict the variation (Some ideas but need to be confirmed) and differentiate a good and bad evolution.
In other way, Max drawdown, % winning trades, runup and so on..can not predict a good evolution in OOS results = You can have a high % winning trade, small Max drawdown, high runup and so on and have an evolution of -10 % …
To be continued
An example (OK it’s only on 33 transactions on IS backtest, but it’s the first I found)
Good % winning trade = 78,79 %, good average win 574 dollars, very small max drawndown, the first in the ranking in IS Backtest Results
BUT
Number 47 in the OOS results/rank (after 2 weeks), and a loss 18965 dollars ->17172 dollars = – 10 %
It confirms what we already know, the more curvefit we do (A lot of variable, a very long code and so on..), the worst results we will have in OOS !
I agree partially, just sharing that I have system with plenty variables and long code and so far performing well in live for 100% of the IS period with 65% win rate, and each month so far is gaining without changing the parameters.
I think bottom line is depending how you use and optimize the variables, I have much more variables because of MTF, market structure, trailing, breakeven, price action parameters, distance to the nearest SL, overbought/oversold, volatility, etc…And I don’t always pick the best result but more on stable result and each variable undergone the 5 iterations 60/40 (or 70/30) WFA and I optimize long and short separately (with WFA too). I also conduct the Vonasi’s robustness tester and monte carlo analysis, if result not so good, I repeat whole thing again. All these consume time and may tend to let people impatiently assign the most optimized value. Btw, one can have few variables, but if 1 variable is used on multiple items/indicators, for me, it is like using multiple variables too.
One thing I didn’t optimize though is the SL, I assign it with 0.3% of the close price and fixed 1 contract only. Due to low SL, so generally it open trades on low to medium volatility only, i.e. it will open trade and wait for increasing volatility to take profit or hit SL if wrong direction. It generally has 2-3 trades per day, so fast for recovery too, it means I won’t be worrying if losing 1-2 trades, and I have to wait long to recover. And more affordable for me to diversify with other assets too with lower SL.
In any case, I appreciate your sharing, just adding my experience so it can act as additional source of thought for others.
Find exclusive trading pro-tools on 
