Issue with ProBackTest: gives different results on same algorithm and conditions

• 03/17/2021 at 12:57 PM #164468
  Nono1981

  Hello all,

  I have an issue with ProBackTest. I developped a couple of algorithms and tested them last week, each one of them multiple times, on different days, and always had exactly the same result.

  I tested them starting Aug 19, 2019 at 01:00 and until March 10, 2021 at 12:00, always with the same starting amount, always with the tick by tick on, and the same spread.

  Yesterday I backtested the exactly same programs, with exactly the same dates, spread, amount and tick by tick. And the result was extremely different e.g. for one of them my gains were divided by (roughly) 4, the percentage of winning trades dropped from 77% to 41%, percentage of time in the market dropped from 62% to 57%… I rebacktested again this morning, and have the same result as yesterday (very different from the original one).

  Does anyone have an idea of why the same algorithm under the same conditions would give very different results in ProBackTest?

  Thanks you very much if you can help me!

  Best regards,

  Bruno

  

  Nono1981

  Participant

  Topics: 1

  Replies: 7

  Been thanked: 0 times
  03/17/2021 at 1:07 PM #164474
  Nicolas

  What account type / broker / instrument and timeframe please?

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4741 times
  03/17/2021 at 1:17 PM #164479
  Nono1981

  Hi Nicolas,

  Thanks for the quick answer. I’m using an unlimited account (a real one, not demo), at IG, the program I mentioned is on Brent (Brut Brent (1€)) and 8h, but the same issue also happens on other programs on Brent and other timeframes (I have several programs on Brent and they differ for each timeframe).

  

  Nono1981

  Participant

  Topics: 1

  Replies: 7

  Been thanked: 0 times
  03/18/2021 at 10:19 AM #164568
  Nicolas

  So this is just for Brent? I am wondering if this is not related to a change in trading hours for this particular instrument, I will find out.

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4741 times
  03/18/2021 at 10:22 AM #164570
  Nicolas

  Were the before and after results obtained following an optimization? Or are they individual backtests (without optimization)? Thank you.

  1 user thanked author for this post.

  Nono1981

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4741 times
  03/18/2021 at 12:41 PM #164582
  Nono1981

  Hi Nicolas,

  They are individual backtests, and the program is exactly the same.

  Yes, it’s only Brent. I also thought it could be related to the change in trading hours since last week. The change in the results is massive, therefore the only explanation I see (for now) related to timing would be that all orders woud be passed in the backtest with an 1-hour delay after the conditons are met. Is there a way to check that?

  

  Nono1981

  Participant

  Topics: 1

  Replies: 7

  Been thanked: 0 times
  03/18/2021 at 1:11 PM #164586
  Nicolas

  Just try to change the custom trading hours to verify your assumption, right click on the price chart and choose “custom trading hours”.

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4741 times
  03/18/2021 at 1:19 PM #164589
  Nono1981

  I just put all my settings to New York time, restarted PRT, ran the backtest and it gives me the same results as yesterday (different from last week).

  

  Nono1981

  Participant

  Topics: 1

  Replies: 7

  Been thanked: 0 times
  03/18/2021 at 2:24 PM #164592
  Nicolas

  ok, so I’m encouraging you to send a technical report through the platform and with a reference to this topic please.

  1 user thanked author for this post.

  Nono1981

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4741 times
  03/18/2021 at 2:35 PM #164593
  Nono1981

  Thank you Nicolas. Actually I’m not sure I want to share the code with them for now, but if  I do and get an answer, I will keep you updated. Thanks for the effort!

  Best regards,

  Bruno

  

  Nono1981

  Participant

  Topics: 1

  Replies: 7

  Been thanked: 0 times
  03/18/2021 at 2:47 PM #164594
  Nicolas

  I asked questions and put a link to this thread in my own query about it. BTW, without the code itself, it seems very hard to replicate what you talk about, and as far as I know you are the first complaining about that issue.

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4741 times
  03/18/2021 at 7:54 PM #164610
  Nono1981

  Hi Nicolas,

  Yes, I understand that it would be much easier to help me with the code. I was also hoping someone had had the same issue and knew where it came from.

  Thanks again for all the support and info.

  Best regards,

  Bruno

  

  Nono1981

  Participant

  Topics: 1

  Replies: 7

  Been thanked: 0 times
  03/29/2021 at 12:21 PM #165630
  Nono1981

  Hi Nicolas,

  I wanted to keep you updated on the topic above. I can now confirm that the issue is related to the change in time difference between the US and France. Somehow the PRT servers must not have duly updated that.

  I have been trying the same algorithms on the different timeframes on Brent for the last two weeks (and having the same different results), and waiting for today, that the time difference is back to what it was when I first tested. And “miraculously” this morning I got back the exact same results I had before the US time change.

  I understand that you have direct contact with PRT, if you could, I believe it would be worth informing them of the above.

  Thanks anyway for all the support!

  Best regards,

  Bruno

  

  Nono1981

  Participant

  Topics: 1

  Replies: 7

  Been thanked: 0 times
  03/29/2021 at 12:46 PM #165631
  nonetheless

  Here’s a piece of code that might help, adjusts for the temporary time differential in the switch to American Summertime, spring and autumn 2010-2024 based on Wall St open hours. There are probably other ways of doing it but this works for me.

  To be honest though, I dont see why your backtest results would change. Regardless of what the time difference is today or last week, historically you would still have ~3 weeks each year when the hours were out of sync, so results should be more or less the same, no?

  //adjustment for American Daylight Savings time DLS =(Date >= 20100314 and date <=20100328) or (Date >= 20101031 and date <=20101107) or (Date >= 20110313 and date <=20110327) or (Date >= 20111030 and date <=20111106) or (Date >= 20120311 and date <=20120325) or (Date >= 20121028 and date <=20121104) or (Date >= 20130310 and date <=20130331) or (Date >= 20131027 and date <=20131103) or (Date >= 20140309 and date <=20140330) or (Date >= 20141026 and date <=20141102) or (Date >= 20150308 and date <=20150329) or (Date >= 20151025 and date <=20151101) or (Date >= 20160313 and date <=20160327) or (Date >= 20161030 and date <=20161106) or (Date >= 20170312 and date <=20170326) or (Date >= 20171030 and date <=20171105) or (Date >= 20180311 and date <=20180325) or (Date >= 20181028 and date <=20181104) or (Date >= 20190310 and date <=20190331) or (Date >= 20191027 and date <=20191103) or (Date >= 20200308 and date <=20200329) or (Date >= 20201025 and date <=20201101) or (Date >= 20210314 and date <=20210328) or (Date >= 20211031 and date <=20211107) or (Date >= 20220313 and date <=20220327) or (Date >= 20221030 and date <=20221106) or (Date >= 20230312 and date <=20230326) or (Date >= 20231029 and date <=20231105) or (Date >= 20240310 and date <=20240331) or (Date >= 20241027 and date <=20241103) If DLS then Tradetime = time >=143000 and time <210000 elsif not DLS then Tradetime = time >=153000 and time <220000 endif

  1 2 3 4 5 6 7

  //adjustment for American Daylight Savings time DLS =(Date >= 20100314 and date <=20100328) or (Date >= 20101031 and date <=20101107) or (Date >= 20110313 and date <=20110327) or (Date >= 20111030 and date <=20111106) or (Date >= 20120311 and date <=20120325) or (Date >= 20121028 and date <=20121104) or (Date >= 20130310 and date <=20130331) or (Date >= 20131027 and date <=20131103) or (Date >= 20140309 and date <=20140330) or (Date >= 20141026 and date <=20141102) or (Date >= 20150308 and date <=20150329) or (Date >= 20151025 and date <=20151101) or (Date >= 20160313 and date <=20160327) or (Date >= 20161030 and date <=20161106) or (Date >= 20170312 and date <=20170326) or (Date >= 20171030 and date <=20171105) or (Date >= 20180311 and date <=20180325) or (Date >= 20181028 and date <=20181104) or (Date >= 20190310 and date <=20190331) or (Date >= 20191027 and date <=20191103) or (Date >= 20200308 and date <=20200329) or (Date >= 20201025 and date <=20201101) or (Date >= 20210314 and date <=20210328) or (Date >= 20211031 and date <=20211107) or (Date >= 20220313 and date <=20220327) or (Date >= 20221030 and date <=20221106) or (Date >= 20230312 and date <=20230326) or (Date >= 20231029 and date <=20231105) or (Date >= 20240310 and date <=20240331) or (Date >= 20241027 and date <=20241103) If DLS then Tradetime = time >=143000 and time <210000 elsif not DLS then Tradetime = time >=153000 and time <220000 endif

   

  

  nonetheless

  Participant

  Topics: 83

  Replies: 1237

  Been thanked: 978 times
  03/29/2021 at 10:31 PM #165684
  Nono1981

  Hi nonetheless,

  Thanks. To be honest I don’t see why neither … but it’s a fact: for two weeks, when the time difference was not the normal one, the exact same code gave different resuls, than came back to the original one. Anyhow, this should now work for the next 6 months or so… until next time change.

  Thanks for the adjustment code.

  Best regards

  

  Nono1981

  Participant

  Topics: 1

  Replies: 7

  Been thanked: 0 times
• Author
  Posts