Walk forward repetition quantity.

• 02/08/2018 at 10:59 PM #62079
  GraHal

  I’m not saying you are wrong as I do the same myself, but I do question myself … there is no true OOS period?

  If variables have been optimised over 100k bars and then a dummy variable is used to enable WF over the same 100k bars … where is the OOS period??

  All we see is which period(s) out of the 100k bars are making you big profit, small profit or loss.  We can see this without WF from the big ups, small ups and downs of the equity curve backtest result of optimisation?

  I also am happy to be proven wrong on this?

   

  

  GraHal

  Participant

  Topics: 203

  Replies: 8389

  Been thanked: 2035 times
  02/08/2018 at 11:16 PM #62080
  GraHal

  Vonasi said … if the variables have a big enough range to choose from then you could make every IS and OOS period look pretty damned good … this gives an indication how often we need to optimise?

  If – to make the IS and OOS period ‘good’ – I need 20 repetitions over a 5 year period then I would need to re-optimise my Strat every 3 months?

  If – to make the IS and OOS period ‘good’ – I need (only) 5 repetitions over a 5 year period then I would need to re-optimise my Strat every 12 months?

  Now to find out what ‘good’ is!? !? 🙂

  

  GraHal

  Participant

  Topics: 203

  Replies: 8389

  Been thanked: 2035 times
  02/09/2018 at 8:56 AM #62093
  Vonasi

  GraHal – and that is exactly why it feels like the ultimate curve fitting. For example you might have a moving average variable and in a WF of 5 repetitions it decides that each IS/OOS period would have been best run with 2, 75, 200, 14, 105 as the optimized period. Clearly each WF period’s requirements for the MA setting are very different to the previous ones so why on earth should 105 be the best setting to use from now on? I guess this example is an example of a result that is ‘bad’.

  SO if we then move on to how many repetitions to get a ‘good’ (still needs to be defined) result we can then curve fit a bit more. 5 doesn’t work so try 20…. ah 20 is better so best if I change the variable quantity every ten minutes to something that worked in the last ten minutes…. I’m struggling to like that idea.

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1810 times
  02/09/2018 at 9:41 AM #62096
  GraHal

  Hahahaha we share a similar sense of humour!

  Re the  2, 75, 200, 14, 105 (extreme example as we all know) but I have tended to use the modal value (no mode in these values) or an average value of all 5 values. I bet you do the same?

  I agree … I wouldn’t have confidence going forward with the 105 value.

  If WF results were as inconsistent as above then I’d abort / re-code or optimise every 10 minutes! 🙂 🙂 🙂

  Another thing I don’t like  … with non-WF optimise I rarely use the most profitable result. But I have no option with WF optimise as I only get to see 1 result per optimise period.

  

  GraHal

  Participant

  Topics: 203

  Replies: 8389

  Been thanked: 2035 times
  02/09/2018 at 1:02 PM #62115
  verdi55

  Walk forward analysis is useful when it shows that optimized parameter values do not change a lot over the different samples and the best values remain quite stable (i.e., fall into a narrow range). When single optimized parameter values are wildly hopping around like “2, 75, 200, 14, 105 “, it simply means that the overall strategy is optimally curve-fitted, I think.

  

  verdi55

  Participant

  Topics: 2

  Replies: 311

  Been thanked: 84 times
  02/09/2018 at 2:53 PM #62134
  verdi55

  And a strategy is the better the more walk-forward repetitions you can have without great variation in the best parameters. Let’s assume for example, some moving average shows an optimized value of 30 periods over the whole 200.000 bars, and a range from 20-41 that gives positive results. In a walk-forward analysis, this parameter shows best values between 27-33 periods for 5 WF repetitions and best values between 25-35 for 20 repetitions. When also most out-of-sample periods give positive results, it would probably mean that the strategy is quite robust with regard to the period number of this moving average.

  On the other hand, when 30 periods as the optimized period value over 200.000 bars already show insular character, that is : most strategy results are bad, but only period values between 28 and 32 give positive results, a walk-forward analysis will most likely yield jumping period values such as 7, 39, 21, 27, 52. This will mean that the period parameter of this moving average  is not useful even when back-tested, curve-fitted and optimized and you should not use it in the strategy. Or do something to make it behave in a more stable manner.

   

  1 user thanked author for this post.

  GraHal

  

  verdi55

  Participant

  Topics: 2

  Replies: 311

  Been thanked: 84 times
  02/09/2018 at 5:21 PM #62151
  Leo

  verdi55 wrote:

  And a strategy is the better the more walk-forward repetitions you can have without great variation in the best parameters.

  Sounds logical

  

  Leo

  Participant

  Topics: 17

  Replies: 357

  Been thanked: 144 times
  02/09/2018 at 6:28 PM #62159
  verdi55

  Leo wrote:

  verdi55 wrote:

  And a strategy is the better the more walk-forward repetitions you can have without great variation in the best parameters.

  Sounds logical

  but unfortunately does not yet guarantee a strategy will make money in the future…too bad…

  

  verdi55

  Participant

  Topics: 2

  Replies: 311

  Been thanked: 84 times
• Author
  Posts