massive errors in performance report

Forums › ProRealTime English forum › ProOrder support › massive errors in performance report

Viewing 15 posts - 1 through 15 (of 17 total)
1 2
  • 04/08/2021 at 10:33 AM #166491
    nonetheless

    Does anyone know why there should be such a discrepancy between the strategy performance as reported by PRT and what actually happens in real life?

    The attached image shows what PRT thinks happened on the left, and my IG history on the right (live trading).

    Any trades on the DOW can only have been with this strategy. Where some of the dates and times are the same, the profit/loss amounts are way off, with PRT reporting several trades that simply never happened.

    It wouldn’t surprise me to find that the two were not identical, but in this case they bear no resemblance to each other – to the extent that the data is meaningless. Has anyone else noticed this?

    (apologies if this has been discussed before)

    04/08/2021 at 10:48 AM #166494
    Nicolas
    Attached is a non-exhaustive list of the elements that can impact a live trading strategy and create differences with a demo account and/or backtests:
    • Spread
    • Slippage
    • Orders rejections due to one of the above reason, but also because of the allowed distance from current price to put pending orders (known as “minimal distance”)
    • Different trading hours (ProOrder code launched in a different time zone / custom hours, by the user)
    • Coding problem: division by zero error, null or negative periods for indicators, ..
    • Lack of responsiveness of IG demo servers (if IG is the broker), although this has improved considerably since last year.
    • Make backtests without tick-by-tick option
    • “set stop trailing” instruction that give IG the total control of your stoploss, can be moved differently between accounts due to points above
    • Limited risk accounts and their rules
    • Guaranteed stoploss rules and fees
    • Starting a strategy at a different time (1 hour or even 1 minute later): depending on the code of the strategy, the results of some calculations could be different.
    • Margin required on the trading account (no demo or backtest tests are made on this subject)
    • Overnight and overweekend fees
    04/08/2021 at 11:19 AM #166511
    nonetheless

    Thanks Nicolas, I am well aware of how those factors can effect the accuracy of backtests but just to be clear, this is not a backtest, it is the supposed record of live trades for that period, so most of those factors do not apply. Also, you specifically mention DEMO account, but this is with my live account; does that make a difference?

    How can it record trades that never happened? Does this mean that (if I had been monitoring it) it would have shown as an active trade on the PRT platform, with nothing happening on the IG side?

     

    04/08/2021 at 11:38 AM #166514
    GraHal
    nonetheless wrote:

    How can it record trades that never happened?

    A quick glance, I can see a trade on 31 March on PRT data that is not shown on the IG data.

    Also for all / most trades, profit / loss difference between PRT and IG data show values of magnitude not accountable by Nicolas list.

    Your findings are worrying and frustrating Nonetheless.

    I know you know, but you likely will get quicker resolution by reporting to PRT using the form below.

    Refer to this Thread then it be a 2 minute job to Report using the PRT WebForm.

    https://www.prorealtime.com/en/contact

     

    04/08/2021 at 12:13 PM #166521
    nonetheless
    GraHal wrote:

    for all / most trades, profit / loss difference between PRT and IG data show values of magnitude not accountable by Nicolas list.

    Yeah, that’s what I thought too. Obviously at the end of the proverbial day, with live trading what you get is whatever lands in your account and that’s that. I just hadn’t realized that the PRT reporting was quite so meaningless, in terms of % win, profit/loss etc. Possibly the biggest practical problem would be any coding that refers to ‘StrategyProfit’ (MM for example) when PRT is apparently incapable of measuring that with any accuracy.

    Thanks for the link, I’ll send it to PRT and see what they say.

    1 user thanked author for this post.
    avatar GraHal
    04/09/2021 at 12:36 PM #166617
    VinzentVega

    Ich have some problems with the backtest report. A lot of details are missing.

    04/09/2021 at 12:44 PM #166619
    nonetheless

    do you mean backtest, or live trading performance report?

    04/09/2021 at 12:53 PM #166620
    VinzentVega

    Backtest. But after a reboot it works fine again.

    04/09/2021 at 1:32 PM #166629
    nonetheless

    Backtests are potentially inaccurate because of all the factors that Nicolas mentioned above. This I find understandable because they’re reporting imaginary trades that ‘might’ have happened in the past.

    I had (wrongly) assumed that the live trading reports would be free of that because the real data would be available – either a trade happened or it didn’t, where it entered / exited etc etc.

    Unfortunately this is not the case.

     

    04/09/2021 at 9:51 PM #166661
    VinzentVega

    The backtesting reports (V11.1)  are sometimes pretty strange. MAE are somtimes massiv higher than the SL. MFE is also wrong.

    04/09/2021 at 10:05 PM #166663
    GraHal

    I’ve had some weird stuff happening earlier today!

    Been getting great BT results almost as of tick by tick was disabled, but it was enabled.

    Then a few hours ago it was if PRT turned tick by tick back on and the good results have now largely gone!!

    Early night I think and see what shows tomorrow!

    04/09/2021 at 10:56 PM #166665
    XORANDNOT
    VinzentVega wrote:

    The backtesting reports (V11.1) are sometimes pretty strange. MAE are somtimes massiv higher than the SL. MFE is also wrong.

    See https://www.prorealcode.com/topic/mae-and-mfe-values-in-the-detailed-report-are-wrong/.

    The PRT team have reproduced the problem and are working to correct the bug.

    1 user thanked author for this post.
    avatar VinzentVega
    04/10/2021 at 1:23 PM #166686
    XORANDNOT
    VinzentVega wrote:

    The backtesting reports (V11.1) are sometimes pretty strange. MAE are somtimes massiv higher than the SL. MFE is also wrong.

    By the way, I think it is quite normal that MAE can be higher than stop loss, because the detailed report cannot know what your stop loss is. MFE can also be higher than take profit for the same reason. MAE and MFE are simply the highest high and the lowest low of all bars since opening of a position, including the last bar.

    04/10/2021 at 2:07 PM #166687
    VinzentVega

    I don´t think so. In V10 MAE  & MFE worked fine. I checked it many times directly in the chart. This is definitly a bug.

    04/10/2021 at 6:09 PM #166721
    XORANDNOT

    That’s strange. For me, MAE and MFE values are identical in v11 and v10.3 (see images). However, this means that MAE and MFE have always (also in v10.3) been calculated with the wrong close prices instead of low and high. But nobody has noticed.

    sl and tp are both 500 in the images attached. But sometimes, MFE or MEA are higher than 500, especially for positions closed on a week-end, it seems.

    Sometimes, MFE values are all = 0 in v11 when the detailed report is shown for the first time. Only when starting  the report for a second time, real MFE values show up.

  • Author
    Posts
Viewing 15 posts - 1 through 15 (of 17 total)
1 2

Find exclusive trading pro-tools on ProRealCode Marketplace

Similar topics:
proorder autotrading performance report not opening
beginner make simple errors
customizing optimization report (adding variable)
dashboard of detailed report remain empty
detailed report-backtest different to detailed report
ema logic screener errors
how to refresh the detailed report when modifying the tf?
in-store performance updates
performance considerations – custom indicators
rejection errors at top of running screen – waste of space and time!

Create your free account now and post your request to benefit from the help of the community
Register or Login