Backtesting with Spreads

Forums › ProRealTime English forum › ProRealTime platform support › Backtesting with Spreads

Viewing 15 posts - 1 through 15 (of 24 total)
1 2
  • 02/08/2017 at 12:06 PM #24309
    Andy11

    Hi

    Would It be possible to check if there is any detailed information on the way the ProOrder backtesting applies spreads during the testing, please? I’ve been experimenting with a few systems that trade the Wall Street index on 5m chart.

    When I run the test with the ‘spread’ option set switched off, some of my systems work quite well but when I switch the ‘spread’ option ON to 2pts those same systems blow up the account very quickly. I expected a difference in performance obviously but such a huge one is quite surprising.

    Even more confusing, when I test with spread OFF and Target Profit at say 10pts it works well. When I also test it with spread OFF and TP at 8pts or 12pts it also works ok but when I turn spread ON and set TP to 10pts it blows up the account. Shouldn’t a 2pt spread with 10pt TP look similar to no spread with TP at 12 or 8?

    Thanks for any information you can share. I would like to understand the method backtesting applies when spread is ON so I can understand the results better. It will help me decide whether these systems are only any good for Futures with no spread or IG with 2pt spread. It’s also just very confusing to me why it would be so different.

    Thanks

    Andy

    (btw this is version 10.3)

    02/08/2017 at 2:05 PM #24330
    Eric

    you pay spread when you enter the trade and when you get stopped out

    you can also look at the average gain per trade if you trade 1 contract (1 euro per point) then you need >1 avg gain per trade

    02/08/2017 at 2:09 PM #24331
    Eric

    you can also look at the average gain per trade if you trade 1 contract (1 euro per point) then you need >1 avg gain per trade

    (if you test dax with 1 point spread and spread off)

    02/08/2017 at 2:19 PM #24332
    GraHal

    @Andy My understanding is that if spread is set at 2 points then every deal that is done incurs 2 points bid/ask spread. However, your results  …  2pt spread with 10pt TP look similar to no spread with TP at 8? … indicate that backtest does not work as I thought?  I’ll have a go to verify.


    @Eric      you pay spread when you enter the trade and when you get stopped out … so you’ree saying we pay 2 lots of spread? So with spread = 2 we pay 2 pts on entry and an additional 2 pts spread on exit? Or an additional 2 points only on a stop exit?

    Cheers
    GraHal

     

    02/08/2017 at 2:38 PM #24339
    GraHal

    @Andy I did a brief test and I agree, results aren’t ‘mathematically logical’. Maybe differences depend on your / my  code, the exit of a Long could be via your TP @ 10 Pts or via a SellShort (PRT exits Longs if SellShort conditions are met). Probably needs spreads taken account of in relation to individual entry & exits?

    Interesting topic and I’ll keep thinking / testing, but maybe our Master may come along and make a definitive statement!? 🙂

    02/08/2017 at 2:43 PM #24341
    Eric

    @GraHal

    no i mean if the spread is 2 you “pay” 1 when enter and 1 when exit with stop or at market order, but maybe i am wrong?

    i use mostly average gain per trade  with spread zero to see how the systems perform

     

     

    02/08/2017 at 2:54 PM #24345
    Eric

    about limit orders

    does it really work with buy at limit in backtest 10.3 tick by tick? or is it only sell at limit?

    i mean open a trade with a limit order (if close is 11600 buy at limit 11580 and set target pprofit 40

    i had som problem with backtests showing good result but bad on forward test in demo

    i will have a closer look when i have the time

     

    02/08/2017 at 3:34 PM #24362
    Eric

    forget about my last post

    i found the problem

    i had exitshort at limit it should be exitshort at stop

     

    02/08/2017 at 3:35 PM #24363
    GraHal

    @Eric I agree … we base profit / loss of Auto-Strategies (ATS) on backtest results and then when Demo Forward Test reults seem so different it requires lots of time and patience looking at individual orders, tick charts etc to validate that backtest works as we think / expect it to!? And what we want to do is make money! 🙂

    Maybe though the ATS you had a problem with was ‘optimised for the backtest period’ and ‘not so optimum’ for the Forward Test period?  This is why we should all do Walk Forward in sample / out of sample testing, Monte Carlo analysis etc etc??

    I’m coming to the conclusion that if I’d spent the time Manual Trading that I’ve spent on Auto Trading (with all the above extra tasks) then I’d be worth a mint!? 🙂  But I’m not giving up now else it will all have been a waste of 2 years of my life!?:)

    Cheers
    GraHal

    02/08/2017 at 4:38 PM #24380
    Andy11

    Thanks for your thoughts!


    @Eric    , yes, I understand (I think!) how spreads should work (or at least how they do in manual trading).

    My question is about the backtesting method. @GraHal, I’m glad you found it too! At least I’m not mad, unless we’re both making the same unknown error.

    I would appreciate any more thoughts from anyone who knows how the backtesting interprets spreads. For example, is there a bias issue like there used to be before tick by tick testing was brought in?

    Thanks!!

     

    02/09/2017 at 7:08 PM #24541
    Derek

    Hello Andy11,

    without looking at your exact results it is not easy to provide a definitive answer. You probably did check your trades already. Just post them here for a better insight and a Spread vs. No Spread comparison.

    Maybe the following considerations can help:

    10 points can (!) be almost nothing. If you are backtesting an overoptmized system the slightest change will throw it off. Simply put: Substracting the number of trades times the spread to calculate the results with spread will most likely fail under these conditions. This problem will be reduced on larger timeframes.

    Which brings me to another point: higher values are likely to be more stable. This holds true for timeframes and indicator parameters.

    02/09/2017 at 7:12 PM #24542
    Derek

    Can’t edit post on mobile clarification:

    5 min timeframe and 10 point TP are very restrictive conditions for trading with PRT.

    02/10/2017 at 3:37 PM #24692
    Andy11

    Thanks Derek

    I must confess I didn’t look into the detail of each trade to try and understand it. I should have!

    However, like you say I have for now decided to leave short timeframe short TP strategies alone and focus on more robust ones. I just wanted to see if anyone knew of any issues with the backtesting systems for spreads like their used to be for tick by tick

    Andy

    02/10/2017 at 4:35 PM #24716
    random45689

    As far as I understand it historical spreads are not incorporated into backtesting so any timeframe will suffer this same issue and give incorrect outcomes as it will only be the spread you manually set in the backtest window that is used over the entire backtest period whereas in reality spreads vary and spike (sometimes a LOT!)

    I also found out today the minimum order size of an instrument changes at IG so in a similar way an autostrategy backtested for example with a fixed order size of 1 will run in demo/backtesting no problem but fail and stop (and should also fail and stop in the backtest) when run live if the minimum order size ever increased above 1 – same goes for minimum stop distance being a similar issue – I guess being aware/knowing about it all helps at least! (..?)

    02/10/2017 at 11:23 PM #24768
    Derek

    @Andy: i have experienced this myself before and yes, that is frustratring. It is very important to question the good backtest results because there might be a problem with the system that is not recognized by the backtesting engine or the backtesting engine itself might be faulty. Unfortunately PRT is generating quite a few bad backtest results. You should Incorporate in- and out-of-sample testing at least(!) and add the spread and slippage not too late in your system development.

    @MaxT: You can try to adjust for your first observation by raising the spread for all spreads for all trades (which will also cover slippage) and adding different spreads for different times of day into your code manually. You could also try to estimate the spread based on short term volatility.

    Considering your second observation: Can you show me a few or at least one example. That is very interesting.

  • Author
    Posts
Viewing 15 posts - 1 through 15 (of 24 total)
1 2

Find exclusive trading pro-tools on ProRealCode Marketplace

Similar topics:
backtesting does not work which a great amount of time units
backtesting kaput ? (ib)
backtesting trading hours
backtesting will not begin
help test new backtesting software
is there any point devising strategies and backtesting?
issue with inconsistent backtesting results
no backtesting results

Create your free account now and post your request to benefit from the help of the community
Register or Login