Demo vs Live Acc Backtest Results Different

Forums › ProRealTime English forum › ProRealTime platform support › Demo vs Live Acc Backtest Results Different

Viewing 15 posts - 1 through 15 (of 16 total)
1 2
  • 07/18/2018 at 10:56 AM #76218
    juanj

    Good Day

    I have just come across something extremely worrisome that could potentially invalidate much of what I have done on PRT over the last 2 years.

    And worse pull the entire legitimacy of ProRealTime’s backtest engine into question.

    I personally test all my strategies on my Live account albeit I do leave them alone approximately 1 to 2 months after development to see if the OOS backtest data confirms the in-sample backtest data.

    Yet today for the first time in probably a year I ran the exact same strategies I have been testing and running in my Live account on the Demo account.

    And to my horror, the backtest results differ wildly between the Live account and Demo account!

    See example attached. I have already confirmed TimeZone settings to be the same on both Demo and Live accounts!

     

    1 user thanked author for this post.
    avatar GraHal
    07/18/2018 at 11:05 AM #76221
    Vonasi

    By comparing trades are you able to work out where the difference is? If you use stop or limit orders in the strategy then that may be the issue as I highlighted a while back that my backtests with limit orders were giving incorrect results but if I changed the order to a SET order the results were more accurate. I am still waiting for PRT to fix this major issue but we may have to wait till v11.0 I fear!

    I’m guessing that you have the same spread and tick by tick data on or off for both strategies?

    07/18/2018 at 11:55 AM #76231
    juanj

    Just to clarify both are backtest results. Only difference is one of grin the Production Acc and the other en the Demo Acc.

    Spread, TF, and tick by tick settings are identical.

    So far luckily live results match the production backtest. But it worries me that there is such big difference between the two backtest engines?

    07/18/2018 at 12:44 PM #76234
    Vonasi

    It would be interesting to know exactly where the difference is – are the entries all the same but the exits wrong? Do you use sell limit or stop orders?

    07/18/2018 at 2:11 PM #76240
    Maria PRT

    Dear Juanj,

    In order to allow us to analyze your platform, please send us a technical report by following these instructions:

    • As soon as you encounter the issue, go to the “Help” menu on the ProRealTime toolbar.
    • Click on “Technical support”
    • Select the appropriate option
    • Give as many details as possible, explain the issue you are having in the text box and mention the name of the code you are referring to.
    • Click “Send report”

     

    This will send us a report containing technical information concerning your platform, which will allow our technical team to run a diagnostic analysis of the issue you have come across.

    Many thanks in advance,

    Maria

    07/18/2018 at 2:41 PM #76241
    juanj

    @Vonasi, comparing the orders list side by side in Excell is like comparing two different strategies.

    I often times noted a 5 to 10min difference in entry times (i.e. 1 to 2 bars)

    So playing around with trading times, but mostly I am at a complete loss to understand it.

    The code is around 800 lines with many references to time so very difficult to troubleshoot.

     

    07/18/2018 at 2:46 PM #76242
    Vonasi

    If the codes are definitely identical and being run on the same time settings with the exact same start date and time and same tick by tick and spread then it can only be a data issue or the orders not being correctly carried out in the tick by tick backtesting.

    07/18/2018 at 3:03 PM #76244
    juanj

    @vonasi so I found the issue:

    ProRealTime has really screwed this up big time. By graphing the time elements ‘hour’, ‘minute’, ‘dayofweek’ for the two different accounts look what I found:

    Real Time: 09h45
    Demo Acc Time: 09h50 (See attached)
    Production Acc Time: 10h00 (See attached)

    How the hell am I supposed to work with this kind of data???

    Oh and PRT Moderators please stop asking me for a Technical Report, I have reported similar issues in the past yet here we are.

    07/18/2018 at 3:18 PM #76248
    GraHal
    Vonasi wrote:

    If the codes are definitely identical

    I just my twopenny worth … I often use the site on link below to check for differences in code versions … sooo easy to use (copy and paste a version in right and left panes and press check diff) and interesting to see the differences! 🙂

    https://www.diffchecker.com/

     

    07/18/2018 at 3:32 PM #76251
    Vonasi

    I always like to use openhour and opendayofweek etc as I have found strange things in the hour, dayofweek etc data before. It would be interesting if you can compare that in live and demo for us.

    07/18/2018 at 3:45 PM #76252
    Vonasi

    Maria from PRT has politely asked you to send in a technical report as that is her job. Prior to PRT having a presence in this forum the only way to communicate problems to them would have been via a technical report. Now at least we can discuss a problem on here but then without the technical report the problem is unlikely to be looked into. PRT do not just investigate every issue posted in this forum – they need a report to start the process. It is the work of a moment to send a report with a link to the topic here so I don’t understand your or anyone else’s reluctance to do so. Yes once you send it you may be left wondering if anything is happening but I have been surprised by the number of faults that I have reported that have with time suddenly started working correctly.

    07/18/2018 at 3:46 PM #76253
    juanj

    Thing is I need to reference different times in the day. Just confounded as to why minutes are so different!

    07/18/2018 at 4:17 PM #76258
    robertogozzi

    I attach 2 pics, one is from my Demo IG account and the other one is from my Real IG account, both on Dax €5, 15-minute TF.

    They show the same results with the same code.

    For the candlestick labelled 09:45 (today) they show 10:00, because the strategy is executed at the closure when IT IS indeed 10:00, should you need to know the opening time you may want to use OPENHOUR instead.

    What’s really odd is trhe difference in minutes, are you sure they they are same TFs?

    07/18/2018 at 5:26 PM #76262
    juanj

    They are exactly the same time frame.

    I have reported the time difference issue to PRT sometime last year already, got told it had to do with timezones. This is not a sufficient answer in my opinion. Back then I didn’t even test in demo account like now. And last time I submitted a technical report I was told PRT didn’t receive it, although IG apparently did.

    The fact that I have different minutes for the same code as exported from Production and then imported in Demo points to a obvious bug either related to my time zone (GMT +2) or instrument (South Africa 40 Cash).

    My screenshots don’t lie.

     

    07/19/2018 at 5:08 PM #76341
    juanj

    My suspicion here is that PRT has different servers, and this time issue is only applicable to a specific server.

    I have a friend of mine confirming that he also gets the same time difference issue when graphing hour and minute between production and demo accounts.


    @Vonasi    , you say “I have found strange things in the hour, dayofweek etc data before” have you reported this to PRT?

  • Author
    Posts
Viewing 15 posts - 1 through 15 (of 16 total)
1 2

Find exclusive trading pro-tools on ProRealCode Marketplace

Similar topics:
different backtest results using different pc’s
ft100 vs fdax – futures vs cash – ig vs ib vs saxo
backtest vs live trade
different backtest results depending of atr code placement
optimization results different from backtest
backtest giving strange results
backtest shows wrong results
differnce between live & demo
live algo trading vs papertrading with ib account
prt backtest in demo is very slow

Create your free account now and post your request to benefit from the help of the community
Register or Login