Why is backtesting so unreliable?

Viewing 15 posts - 1 through 15 (of 21 total)
  • Author
    Posts
  • #110841 quote
    nonetheless
    Participant
    Master

    I’ve developed a few codes that appear to be successful in backtesting, but whenever I run them in demo nothing comes close to that kind of profit.

    I adapted the following from the Pure Renko Strategy posted by Doctrading.

    This version is designed for the DJI 5min chart on UK time and shows incredible results – 75% wins over 17 months with only 3 losing days. Well, you know what they say about things that sound too good to be true. Run it live and it loses money every day. Same story with other seemingly successful codes I’ve developed. I’d be very curious to know why there should be such a huge discrepancy.

    Thanks in advance for any enlightenment.

    // Definition of code parameters
    DEFPARAM CumulateOrders = False // Cumulating positions deactivated
    // The system will cancel all pending orders and close all positions at 0:00. No new ones will be allowed until after the "FLATBEFORE" time.
    DEFPARAM FLATBEFORE = 143000
    // Cancel all pending orders and close all positions at the "FLATAFTER" time
    DEFPARAM FLATAFTER = 210000
    boxSize = 13
    once renkoMax = ROUND(close / boxSize) * boxSize
    once renkoMin = renkoMax - boxSize
    IF high > renkoMax + boxSize THEN
    WHILE high > renkoMax + boxSize
    renkoMax = renkoMax + boxSize
    renkoMin = renkoMin + boxSize
    WEND
    ELSIF low < renkoMin - boxSize THEN
    WHILE low < renkoMin - boxSize
    renkoMax = renkoMax - boxSize
    renkoMin = renkoMin - boxSize
    WEND
    ENDIF
    buy 1 CONTRACT at renkoMax + boxSize stop
    sellshort 1 CONTRACT at renkoMin - boxSize stop
    // Stops and targets
    SET STOP $TRAILING 12

     

    #110847 quote
    Vonasi
    Moderator
    Master

    nonetheless – Welcome to the forums. When posting code you are expected to use the ‘Insert PRT Code’ button to make the code more readable to others. I have tidied up your post. Please use the button in future posts. 🙂

    nonetheless thanked this post
    #110863 quote
    Nicolas
    Keymaster
    Master

    Same story over and over again 🙂

    These are the things you should be aware of:

    1. try to not over optimize your settings, it will fails if you don’t validate an OOS period (overfitting)
    2. spread in backtests, did you set it?
    3. broker slippage: no real way to simulate that in backtests
    4. renko bricks construction: only dependent to price moves, so results will be completely different if you start a backtest with a different time period, because price will start moving at first bar of history
    5. set stop trailing instruction: doesn’t respect the IG trailing stop step, don’t use it
    6. always make backtests with tick by tick box checked
    nonetheless thanked this post
    #110876 quote

    You should never do tests with 5 minute time frames, 17 months are very few.
    Normally you only have profits on paper but then you lose money.

    #110883 quote
    nonetheless
    Participant
    Master

    Thanks for the advice – I think I’ll stick to manual trading, at least I know how that works.

    Best of luck to all!

    #110887 quote
    GraHal
    Participant
    Master

    Just confirm that you had the tick by tick Mode  box selected in your backtest @nonetheless  … as without using the tick by tick mode then bt results look like  the holy grail!!

    Just tying to save you getting too disillusioned with Auto-Trading 🙂

    #110889 quote
    GraHal
    Participant
    Master

    Is attached something similar to what you got @nonetheless … because attached is without tick by tick as that is all I could get due to too many instances of … well, there’s a lot to it and you be better to read up elsewhere on this website.

    Suffice it to say that Real Live  is guaranteed to be nothing like attached due to TP and SL being met in the same bar  … in short your Trailing Stop is way to low / close to Trade Price.

    NOne.png NOne.png NOne-2.png NOne-2.png
    #110893 quote

    The problem is not the tick by tick, but the reduced time of the test ….. And even as it is done, with the computer it is easy to get back exceptional results.

    nonetheless thanked this post
    #110897 quote
    GraHal
    Participant
    Master

    Below looks like it might be more realistic … see how it is Short during a down period and vice versa?

    I’m putting below on Forward Test on Demo for a while … never had a Renko System before! 🙂

    Nonetheless Thank You! 🙂 🙂

    EDIT / PS

    I’m sure below could be improved more, e.g. by adding one of the many coded Trailing Stops available on here.

     

    //https://www.prorealcode.com/topic/why-is-backtesting-so-unreliable/#post-110889
    // Definition of code parameters
    DEFPARAM CumulateOrders = False // Cumulating positions deactivated
    // The system will cancel all pending orders and close all positions at 0:00. No new ones will be allowed until after the "FLATBEFORE" time.
    //DEFPARAM FLATBEFORE = 143000
    // Cancel all pending orders and close all positions at the "FLATAFTER" time
    //DEFPARAM FLATAFTER = 210000
    boxSize = 110 //50
    once renkoMax = ROUND(close / boxSize) * boxSize
    once renkoMin = renkoMax - boxSize
    IF high > renkoMax + boxSize THEN
    WHILE high > renkoMax + boxSize
    renkoMax = renkoMax + boxSize
    renkoMin = renkoMin + boxSize
    WEND
    ELSIF low < renkoMin - boxSize THEN
    WHILE low < renkoMin - boxSize
    renkoMax = renkoMax - boxSize
    renkoMin = renkoMin - boxSize
    WEND
    ENDIF
    buy 1 CONTRACT at renkoMax + boxSize stop
    sellshort 1 CONTRACT at renkoMin - boxSize stop
    // Stops and targets
    SET STOP PLOSS 75
    SET TARGET PPROFIT 150
    
    nonetheless thanked this post
    NOne-3.png NOne-3.png NOne-4.png NOne-4.png
    #120355 quote
    GraHal
    Participant
    Master

    The above code (or a fork of) was doing very well for me on Demo Forward Test but for the last several days it gets Rejected / Stopped every time the code is read.

    I get the Rejected with the error message … System Error.

    Can anybody see anything in the code that ProOrder or IG may not like??

    Is While, Wend approved terms etc ??  Thee must be some reason surely??

    I know IG have had big Issues today, but lots of my other Systems taking trades just fine today so why is this System getting Rejected for System Error??

    #120356 quote
    Paul
    Participant
    Master

    perhaps it’s not the code, I got trouble too in recent days on several systems, mainly system error

    GraHal thanked this post
    #120357 quote
    GraHal
    Participant
    Master

    Yeah thanks Paul … I just checked and lots of others are now being rejected for System Error.

    #120359 quote
    GraHal
    Participant
    Master

    I’ll correct myself … the above code is the only System that is getting Rejected on my SB Account.

    Lots of Systems are repeatedly getting Rejected on my CFD Account for System Error.

    #120361 quote
    Paul
    Participant
    Master

    not only system error on my cfd demo account.  It’s messy a.t.m.!

    Screenshot-2020-02-25-at-00.16.22.jpg Screenshot-2020-02-25-at-00.16.22.jpg Screenshot-2020-02-25-at-00.16.29.jpg Screenshot-2020-02-25-at-00.16.29.jpg
    #120365 quote
    nonetheless
    Participant
    Master

    I’ve had stuff getting rejected all day, esp on the DOW and DAX, NAS systems working fine.

    Rang IG about it but got a recorded message acknowledging platform problems and to stay on the line only if I wanted to close a position. Presumably they’re inundated with complaints.

    BTW @Grahal glad to hear that Renko thing was actually working for you! Maybe I should take another look at it…

Viewing 15 posts - 1 through 15 (of 21 total)
  • You must be logged in to reply to this topic.

Why is backtesting so unreliable?


ProOrder: Automated Strategies & Backtesting

New Reply
Author
Summary

This topic contains 20 replies,
has 3 voices, and was last updated by Nicolas
3 years, 9 months ago.

Topic Details
Forum: ProOrder: Automated Strategies & Backtesting
Language: English
Started: 10/22/2019
Status: Active
Attachments: 6 files
Logo Logo
Loading...