Anomaly between walk forward and normal backtest outcomes

Viewing 3 posts - 1 through 3 (of 3 total)
  • Author
    Posts
  • #125111 quote
    AutoStrategist
    Participant
    Veteran

    I ran 2 backtests on the same data optimising the variable xx which was used for a stoploss value, one with walk forward analysis and the other without.  The walk forward statistics show the selected variable (xx) as 140 across all test ranges (I could only capture the one as shown in the attachment but all periods show the same value for xx at 140).  Looking at the optimisation outcome without walk forward shows the best outcomes with an xx value of 165.  The 2 overall outcomes are very different based on which value is selected as shown in the attachments, the walk forward one being considerably worse, surely at least one of the test periods in walk forward analysis should have selected a value of 165 ?

    DAX-opt-with-walk-forward.jpg DAX-opt-with-walk-forward.jpg DAX-optimise-results.jpg DAX-optimise-results.jpg DAX-walk-forward-outcome.jpg DAX-walk-forward-outcome.jpg DAX-opt-no-walk-forward.jpg DAX-opt-no-walk-forward.jpg
    #125151 quote
    Nicolas
    Keymaster
    Master

    Normal optimization (without WalkForward and OOS periods) consider all the time period as IS. So there are more data in classical optimization than in WF optimization.

    The WF testing is just a basic optimization but only on a part of the data (InSample), then the best variables settings found for this period is tested (without optimization) on the rest of the data (OutOfSample).

    Backtesting on different length of data results on different variables values.

    AutoStrategist thanked this post
    #125195 quote
    AutoStrategist
    Participant
    Veteran

    Thanks Nicholas that was me not having a full understanding of how WF works as I don’t use it very often.  Am glad I asked the question though as it appears to highlight the value of using both types of optimisation.

Viewing 3 posts - 1 through 3 (of 3 total)
  • You must be logged in to reply to this topic.

Anomaly between walk forward and normal backtest outcomes


ProOrder: Automated Strategies & Backtesting

New Reply
Author
Summary

This topic contains 2 replies,
has 2 voices, and was last updated by AutoStrategist
5 years, 10 months ago.

Topic Details
Forum: ProOrder: Automated Strategies & Backtesting
Language: English
Started: 04/07/2020
Status: Active
Attachments: 4 files
Logo Logo
Loading...