Technique help to reduce over-fitting

• 09/02/2020 at 7:04 PM #143234
  zilliq

  Always Keep It Simple (In code and strategy). 3 Variables only, 100 000 units.

  Strategy without overfitting (or as little as possible) based on Stochastics I develop today, but no matter for the strategy on backtest. Need to test on OOS now

  Have a nice evening

  

  2 users thanked author for this post.

  MAKSIDE, Paul

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  09/02/2020 at 7:27 PM #143238
  Nicolas

  No loosers?

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20830

  Been thanked: 4744 times
  09/02/2020 at 7:45 PM #143239
  Dow Jones

  Your new strategy looks nice, @zilliq, thanks for the tips too.

  Btw, recently I picked up a topic thanks to @vonasi, I think it is helpful on reducing the optimization time as well (which I believe everyone is so eager).

  Control of yearly/monthly/weekly profit/loss => https://www.prorealcode.com/topic/annual-percentage-returns/#post-93664

  The article is originally for yearly, but you can transform it to monthly/weekly (below I share example for monthly). The better is even set a range of highest profit and lowest loss allowed.

  Since we added a quit once the bad month happened, thus the back test is faster, it will not continue to test for the rest of the months.

  You can even add a condition check to not have consecutive month/weeks (even days) of loss, e.g. do a summation[3](monthlyreturn) <= 0 for no 3 months consecutive loss.

  capital = 5000 once lastcapital = capital runningperc = ((strategyprofit - laststrategyprofit) / lastcapital) * 100 if month <> month[1] then monthlyreturn = runningperc[1] laststrategyprofit = strategyprofit lastcapital = capital + strategyprofit total = total + monthlyreturn count = count + 1 //monthlyaverage = total/count endif if monthlyreturn < -0.5 then QUIT endif

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

  capital = 5000 once lastcapital = capital runningperc = ((strategyprofit - laststrategyprofit) / lastcapital) * 100   if month <> month[1] then monthlyreturn = runningperc[1] laststrategyprofit = strategyprofit lastcapital = capital + strategyprofit total = total + monthlyreturn count = count + 1 //monthlyaverage = total/count endif   if monthlyreturn < -0.5 then QUIT endif

  Since the optimization timing is faster (thanks to skip the bad parameters), you can have a bigger range of optimization.

  

  Dow Jones

  Participant

  Topics: 10

  Replies: 135

  Been thanked: 75 times
  09/02/2020 at 7:47 PM #143240
  Dow Jones

  GraHal wrote:

  I guess you are teasing us and you are not going to share your excellent looking Strategy??

  @GraHal, you can subscribe when he put in market place, haha 🙂 But I’m sure you also have such strategy, right? Perhaps same for many people here as well…

  

  Dow Jones

  Participant

  Topics: 10

  Replies: 135

  Been thanked: 75 times
  09/02/2020 at 8:18 PM #143242
  zilliq

  Nicolas wrote:

  No loosers?

  Of course there is losers;- 87 % winners
  100 % winners on 100 000 units = Overfitting for me

   

  @dow jones LOL not even.

  I consider to sell an algo is a scam because it need always some work and retesting

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  09/02/2020 at 8:28 PM #143243
  Paul

  Zilliq, pure a simple risk/reward strategy with a certain ratio or other exit methods included?

  

  Paul

  Participant

  Topics: 40

  Replies: 812

  Been thanked: 660 times
  09/02/2020 at 9:55 PM #143253
  zilliq

  Classical exit based on ATR @Paul

  1 user thanked author for this post.

  Paul

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  09/03/2020 at 8:31 AM #143260
  Nicolas

  zilliq wrote:

  Of course there is losers;- 87 % winners

  Good! Any chance to share the same screenshot with position size window below the equity curve?

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20830

  Been thanked: 4744 times
  09/03/2020 at 9:14 AM #143265
  zilliq

  Of course Nicolas, I just need to do another backtest. I will tomorrow

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  09/03/2020 at 9:19 AM #143268
  zilliq

  First OOS day for this Algos (Mean nothing as you now, but good start).

  Always a 1000 USD/day strategy (I don’t need more 🙂 )

   

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  09/04/2020 at 11:05 AM #143365
  zilliq

  To continue to minimize the overfitting, today I begin an OOS demo test with a strategy on SAR to have only 2 variables

  58,99 % winners on the backtest but on 556 trades ! 100 000 units EUR/USD 1 mn Sharpe Ratio 1.1.

  We will see if it was a good idea 🙂

  Have a nice day

  Zilliq

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  09/04/2020 at 11:09 AM #143367
  Nicolas

  @zilliq you should open a fresh new topic for your OOS forward test about this specific strategy, we are a bit off-topic now here, if you don’t mind.

  2 users thanked author for this post.

  Paul, Francesco

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20830

  Been thanked: 4744 times
  09/04/2020 at 12:20 PM #143374
  Dow Jones

  Dow Jones wrote:

  Control of yearly/monthly/weekly profit/loss

  An update to this one, below is an example to control on the consecutive monthly and weekly loss.

  You can probably extend it to maximum win for monthly/weekly too (I didn’t do it as it is already difficult to achieve no monthly and weekly loss over long period).

  I kind of think that a stable strategy should have stable income, so no huge spike or huge drawdown on certain period that resulted good total gain only.

  Adding this means the back test will stop for the combination if you will anyway don’t like it, e.g. if you won’t accept strategy that loss 2 consecutive months, then no point to continue the back test for whole period.

  So making the back test a bit faster. Hope you guys like it and suggestion welcome.

  //1. Check monthly not lower than expected profit and no consecutive month loss capital = 5000 monthlymingain = -10 maxconsecutivemonthlyloss = 1 maxconsecutiveweeklyloss = 2 once lastmonthlycapital = capital once lastweeklycapital = capital once consecutiveweeklyloss = 0 once consecutivemonthlyloss = 0 runningmonthlyperc = ((strategyprofit - lastmonthstrategyprofit) / lastmonthlycapital) * 100 runningweeklyperc = ((strategyprofit - lastweekstrategyprofit) / lastweeklycapital) * 100 //Ensure none of the month less than expected gain if month <> month[1] then monthlyreturn = runningmonthlyperc[1] lastmonthstrategyprofit = strategyprofit lastmonthlycapital = capital + strategyprofit if monthlyreturn < 0 then consecutivemonthlyloss = consecutivemonthlyloss + 1 else consecutivemonthlyloss = 0 endif endif if monthlyreturn < monthlymingain or consecutivemonthlyloss > maxconsecutivemonthlyloss then QUIT endif //2. Check consecutive week of loss is acceptable if opendayofweek < opendayofweek[1] then weeklyreturn = runningweeklyperc[1] lastweekstrategyprofit = strategyprofit lastweeklycapital = capital + strategyprofit if weeklyreturn < 0 then consecutiveweeklyloss = consecutiveweeklyloss + 1 else consecutiveweeklyloss = 0 endif endif if consecutiveweeklyloss > maxconsecutiveweeklyloss then QUIT endif

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

  //1. Check monthly not lower than expected profit and no consecutive month loss capital = 5000 monthlymingain = -10 maxconsecutivemonthlyloss = 1 maxconsecutiveweeklyloss = 2 once lastmonthlycapital = capital once lastweeklycapital = capital once consecutiveweeklyloss = 0 once consecutivemonthlyloss = 0 runningmonthlyperc = ((strategyprofit - lastmonthstrategyprofit) / lastmonthlycapital) * 100 runningweeklyperc = ((strategyprofit - lastweekstrategyprofit) / lastweeklycapital) * 100   //Ensure none of the month less than expected gain if month <> month[1] then monthlyreturn = runningmonthlyperc[1] lastmonthstrategyprofit = strategyprofit lastmonthlycapital = capital + strategyprofit if monthlyreturn < 0 then consecutivemonthlyloss = consecutivemonthlyloss + 1 else consecutivemonthlyloss = 0 endif endif if monthlyreturn < monthlymingain or consecutivemonthlyloss > maxconsecutivemonthlyloss then QUIT endif   //2. Check consecutive week of loss is acceptable if opendayofweek < opendayofweek[1] then weeklyreturn = runningweeklyperc[1] lastweekstrategyprofit = strategyprofit lastweeklycapital = capital + strategyprofit if weeklyreturn < 0 then consecutiveweeklyloss = consecutiveweeklyloss + 1 else consecutiveweeklyloss = 0 endif endif if consecutiveweeklyloss > maxconsecutiveweeklyloss then QUIT endif

   

  1 user thanked author for this post.

  Nicolas

  

  Dow Jones

  Participant

  Topics: 10

  Replies: 135

  Been thanked: 75 times
  09/04/2020 at 1:35 PM #143380
  Nicolas

  Very good idea and thank you for that great snippet DowJones. But, while it should improve backtest time, you will also introduce a form of over-fitting by avoiding bad months, yeah i know nothing’s perfect 😉

  1 user thanked author for this post.

  Dow Jones

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20830

  Been thanked: 4744 times
  09/04/2020 at 1:56 PM #143386
  Vonasi

  Nicolas wrote:

  you will also introduce a form of over-fitting by avoiding bad months, yeah i know nothing’s perfect 😉

  Nicolas is right (as always!) Also remember that a strategy cares nothing about things like when the name/number of the current month changes.

  You could have a bad second half of a month and a bad first half of the next month and that back test is included in the results because both months are average results whereas if they same results had just happened a couple of weeks earlier and occurred all in the same month then that test would be excluded. Surely it is better just to have a rolling period or rolling quantity of trades and if the results are below what you would be happy to trade then quit the back test to save optimising time?

  1 user thanked author for this post.

  Dow Jones

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
• Author
  Posts