Anomaly between walk forward and normal backtest outcomes

• 04/07/2020 at 5:21 PM #125111
  AutoStrategist

  I ran 2 backtests on the same data optimising the variable xx which was used for a stoploss value, one with walk forward analysis and the other without.  The walk forward statistics show the selected variable (xx) as 140 across all test ranges (I could only capture the one as shown in the attachment but all periods show the same value for xx at 140).  Looking at the optimisation outcome without walk forward shows the best outcomes with an xx value of 165.  The 2 overall outcomes are very different based on which value is selected as shown in the attachments, the walk forward one being considerably worse, surely at least one of the test periods in walk forward analysis should have selected a value of 165 ?

  

  AutoStrategist

  Participant

  Topics: 6

  Replies: 125

  Been thanked: 71 times
  04/08/2020 at 8:36 AM #125151
  Nicolas

  Normal optimization (without WalkForward and OOS periods) consider all the time period as IS. So there are more data in classical optimization than in WF optimization.

  The WF testing is just a basic optimization but only on a part of the data (InSample), then the best variables settings found for this period is tested (without optimization) on the rest of the data (OutOfSample).

  Backtesting on different length of data results on different variables values.

  1 user thanked author for this post.

  AutoStrategist

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4741 times
  04/08/2020 at 11:10 AM #125195
  AutoStrategist

  Thanks Nicholas that was me not having a full understanding of how WF works as I don’t use it very often.  Am glad I asked the question though as it appears to highlight the value of using both types of optimisation.

  

  AutoStrategist

  Participant

  Topics: 6

  Replies: 125

  Been thanked: 71 times
• Author
  Posts