Different Backtest Results using different PC’s

Viewing 13 posts - 1 through 13 (of 13 total)

Author

Posts
08/13/2023 at 11:25 AM #218988 quote

Mr.Crni
Participant

New

Hey Forum

I code togehter with an other guy. We use PRT via IG.

The following problem appeard:

For coding, we use our private Laptops (let’s call them “Laptop 1” and “Laptop 2”) and we also use a shared PC (called “PC”). We installed PRT on all the three machines but at different times (laptop 1: january 2022, laptop 2: summer 2022, pc: summer 2023).

a couple days ago, he sent me a optimized code from laptop 2, ready for goin live on the PRT Demo. I imported the itf. file on the PC and made an controll Backtest without changing anything in the settings. The results were far from beeing the same. Then i imported the same file on laptop 1, made a backtest an got the same good Backtest results like on laptop 2.

We noticed the same problem in reverse with optimized codes we made on the PC and then transfered to our laptopts.

has anyone made the same expierience? can the timing of the installation play a rule? are there different subversions?

We also recognized on PRT-Demo at least once a difference between executed trades and backtest trades (trades were executed in Demo-Live but were not displayed during a later backtest). I thought I had already read about this in the forum, but this is a different topic.

08/13/2023 at 12:16 PM #218989 quote

GraHal
Participant

Master

Depending on the code, the date / time starting the code can give different results, sometimes very different.

You would need to eliminate above as a possibility by starting the code at same date / time on both laptops & PC.

08/13/2023 at 6:10 PM #218997 quote

Mr.Crni
Participant

New

exchange and backtest was all done within one day, so this should not be the problem.

08/13/2023 at 7:32 PM #219001 quote

GraHal
Participant

Master

Mmm without trying as I suggest then you cannot know for sure? It is not just about the same day, the 3 Systems on separate Platforms may have to be started at the same time to prove or disprove my suggestion.

You may well be right, what is your next move?

Maybe others will make suggestions for you to check out.

Without seeing your code we can only make wild guesses.

Is PRT version same on laptop 1, laptop2 and the PC? Are these all running in tick by tick mode?

08/14/2023 at 10:33 AM #219025 quote

Nicolas
Keymaster

Master

Same instrument? Same start and end date of the backtest? Same fees applied?

08/14/2023 at 1:24 PM #219038 quote

Mr.Crni
Participant

New

Hello Grahal and Nicholas

An Example:

Picture “[…] Laptop1” shows the “good” results, letting run the file on the Laptop 1. Picture “[…] PC” shows the “bad” results, letting run the file on the PC.

Same IG account, same PRT-Demo-account. Just different machines and different download-/ installation-times of the PRT-Software .

The Picture “Comparing […]” shows the difference between letting run the file as a Backtest and the “live- version” running in the Demo account since two months with the same file. different results. The results from backtest on the Laptop is equal with the live-Version running on the Demo account.

Can you guys also import the file and test it? Would be interestin to see your results (relevant file is attached).

NASDAQ_Impulsmover_5min.itf Comparing_Results_Impulsmover_Nasdaq_PC.png Results_200k_Impulsmover_Nasdaq_Laptop1.png Results_200k_Impulsmover_Nasdaq_PC.png

08/14/2023 at 1:59 PM #219044 quote
GraHal
Participant

Master
Attached …
1. 100K bars on US Tech 100 on M5 TF.
2. 200K bars on US Tech 100 on M5 TF.
Screenshot-2023-08-14-135622.png Screenshot-2023-08-14-140224.png
08/14/2023 at 2:18 PM #219054 quote

GraHal
Participant

Master

In your screenshots, why do you not show what Instrument on NASDAQ you are testing on?

08/14/2023 at 2:43 PM #219064 quote

Mr.Crni
Participant

New

US Tech 100 Cash (1€), spread 1.0, tick-by-tick mode is on

08/14/2023 at 3:24 PM #219072 quote

GraHal
Participant

Master

I propose we all 3 do a backtest starting on 1 Feb 21 at 08:00 and ending on 11 Aug 23 at 21:00 and then we all post a screenshot of the result? US Tech 100 Cash (1€), spread = 1.

Attached is mine.

Mr.Crni thanked this post

Screenshot-2023-08-14-152327.png Screenshot-2023-08-14-152454.png

08/14/2023 at 3:27 PM #219075 quote

PeterSt
Participant

Master

To be honest … I see only backtest comparisons which – as told – may start out with the slightest difference (for first trade) and all can be totally different.
I also observe the relative few trades this system applies, which makes my previous sentence even more important or true.

Best would be to show a list of trades like you see below (from the Detailed Report). For that you need to do one thing only : let the 3 systems start with the same trades somewhere in time. In my example this would be August 2 2023, 14:22:00 (first line in the overview, at the bottom). If you are correct and that there are differences, it may be hard to achieve this to begin with. But otherwise once you have found a “synched” trade, all should follow the exact same sequence and revenues and all. Of course take care that each system buys the same contract quantity.
If things get out of sync (after an initial in-sync) then there’s a problem somewhere. We could then proceed with the list of differences and attempt to explain it.

GraHal and Mr.Crni thanked this post

image_2023-08-14_161110293.png

08/14/2023 at 3:49 PM #219077 quote

GraHal
Participant

Master

If we get 3 inputs from Mr.Crni (and / or friends? 🙂 ) then I will simulate your test Peter … good idea!!

Finally we are collaborating on something! 😉

08/16/2023 at 9:54 AM #219156 quote

Mr.Crni
Participant

New

Thanks to you guys. will try to respond asap (heavy workload at the moment).

GraHal thanked this post
Author

Posts