Different Backtest Results using different PC’s

• 08/13/2023 at 11:25 AM #218988
  Mr.Crni

  Hey Forum

  I code togehter with an other guy. We use PRT via IG.

  The following problem appeard:

  For coding, we use our private Laptops (let’s call them “Laptop 1” and “Laptop 2”) and we also use a shared PC (called “PC”). We installed PRT on all the three machines but at different times (laptop 1: january 2022, laptop 2: summer 2022, pc: summer 2023).

  a couple days ago, he sent me a optimized code from laptop 2, ready for goin live on the PRT Demo. I imported the itf. file on the PC and made an controll Backtest without changing anything in the settings. The results were far from beeing the same. Then i imported the same file on laptop 1, made a backtest an got the same good Backtest results like on laptop 2.

  We noticed the same problem in reverse with optimized codes we made on the PC and then transfered to our laptopts.

  has anyone made the same expierience? can the timing of the installation play a rule? are there different subversions?

  We also recognized on PRT-Demo at least once a difference between executed trades and backtest trades (trades were executed in Demo-Live but were not displayed during a later backtest). I thought I had already read about this in the forum, but this is a different topic.

   

   

  

  Mr.Crni

  Participant

  Topics: 3

  Replies: 6

  Been thanked: 1 time
  08/13/2023 at 12:16 PM #218989
  GraHal

  Depending on the code, the date / time starting the code can give different results, sometimes very different.

  You would need to eliminate above as a possibility by starting the code at same date / time on both laptops & PC.

  

  GraHal

  Participant

  Topics: 204

  Replies: 8401

  Been thanked: 2041 times
  08/13/2023 at 6:10 PM #218997
  Mr.Crni

  exchange and backtest was all done within one day, so this should not be the problem.

  

  Mr.Crni

  Participant

  Topics: 3

  Replies: 6

  Been thanked: 1 time
  08/13/2023 at 7:32 PM #219001
  GraHal

  Mmm without trying as I suggest then you cannot know for sure? It is not just about the same day, the 3 Systems on separate Platforms may have to be started at the same time to prove or disprove my suggestion.

  You may well be right, what is your next move?

  Maybe others will make suggestions for you to check out.

  Without seeing your code we can only make wild guesses.

  Is PRT version same on laptop 1, laptop2 and the PC?  Are these all running in tick by tick mode?

  

  GraHal

  Participant

  Topics: 204

  Replies: 8401

  Been thanked: 2041 times
  08/14/2023 at 10:33 AM #219025
  Nicolas

  Same instrument? Same start and end date of the backtest? Same fees applied?

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4742 times
  08/14/2023 at 1:24 PM #219038
  Mr.Crni

  Hello Grahal and Nicholas

  An Example:

  Picture “[…] Laptop1” shows the “good” results, letting run the file on the Laptop 1. Picture “[…] PC” shows the “bad” results, letting run the file on the PC.

  Same IG account, same PRT-Demo-account. Just different machines and different download-/ installation-times of the PRT-Software .

  The Picture “Comparing […]” shows the difference between letting run the file as a Backtest and the “live- version” running in the Demo account since two months with the same file. different results. The results from backtest on the Laptop is equal with the live-Version running on the Demo account.

  Can you guys also import the file and test it? Would be interestin to see your results (relevant file is attached).

   

   

   

  

  Mr.Crni

  Participant

  Topics: 3

  Replies: 6

  Been thanked: 1 time
  08/14/2023 at 1:59 PM #219044
  GraHal

  Attached …

  1.  100K bars on US Tech 100 on M5 TF.
  2. 200K bars on US Tech 100 on M5 TF.

   

  

  GraHal

  Participant

  Topics: 204

  Replies: 8401

  Been thanked: 2041 times
  08/14/2023 at 2:18 PM #219054
  GraHal

  In your screenshots, why do you not show what Instrument on NASDAQ you are testing on?

  

  GraHal

  Participant

  Topics: 204

  Replies: 8401

  Been thanked: 2041 times
  08/14/2023 at 2:43 PM #219064
  Mr.Crni

  US Tech 100 Cash (1€), spread 1.0, tick-by-tick mode is on

  

  Mr.Crni

  Participant

  Topics: 3

  Replies: 6

  Been thanked: 1 time
  08/14/2023 at 3:24 PM #219072
  GraHal

  I propose we all 3 do a backtest starting on 1 Feb 21 at 08:00 and ending on 11 Aug 23 at 21:00 and then we all post a screenshot of the result?  US Tech 100 Cash (1€), spread = 1.

  Attached is mine.

  1 user thanked author for this post.

  Mr.Crni

  

  GraHal

  Participant

  Topics: 204

  Replies: 8401

  Been thanked: 2041 times
  08/14/2023 at 3:27 PM #219075
  PeterSt

  To be honest … I see only backtest comparisons which – as told – may start out with the slightest difference (for first trade) and all can be totally different.
  I also observe the relative few trades this system applies, which makes my previous sentence even more important or true.

  Best would be to show a list of trades like you see below (from the Detailed Report). For that you need to do one thing only : let the 3 systems start with the same trades somewhere in time. In my example this would be August 2 2023, 14:22:00 (first line in the overview, at the bottom). If you are correct and that there are differences, it may be hard to achieve this to begin with. But otherwise once you have found a “synched” trade, all should follow the exact same sequence and revenues and all. Of course take care that each system buys the same contract quantity.
  If things get out of sync (after an initial in-sync) then there’s a problem somewhere. We could then proceed with the list of differences and attempt to explain it.

  2 users thanked author for this post.

  Mr.Crni, GraHal

  

  PeterSt

  Participant

  Topics: 70

  Replies: 2287

  Been thanked: 695 times
  08/14/2023 at 3:49 PM #219077
  GraHal

  If we get 3 inputs from Mr.Crni (and / or friends? 🙂 ) then I will simulate your test Peter … good idea!!

  Finally we are collaborating on something! 😉

  

  GraHal

  Participant

  Topics: 204

  Replies: 8401

  Been thanked: 2041 times
  08/16/2023 at 9:54 AM #219156
  Mr.Crni

  Thanks to you guys. will try to respond asap (heavy workload at the moment).

  1 user thanked author for this post.

  GraHal

  

  Mr.Crni

  Participant

  Topics: 3

  Replies: 6

  Been thanked: 1 time
• Author
  Posts