I’m getting different results between backtesting and real trading. I have been reading some of the other posts and realise it is likely because PRT only looks at the bar once in order to satisfy conditions and cannot look at real time intrabar data. Could someone please confirm that this is correct ?
For eg. I used this code (with minimal capital as I was aware I may get losses ) to test out results of a strategy on DAX 4h:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
DEFPARAMCumulateOrders=False
DEFPARAMflatafter=200000
// TAILLE DES POSITIONS
// INDICATEURS
MM=Average[200](close)
iMACD=MACD[12,26,9](close)
// ACHAT
ca1=close>MM
ca2=iMACD>0
IFca1ANDca2ANDtime>=080000THEN
BUY2shareATMARKET
ENDIF
// VENTE
cv1=close<MM
cv2=iMACD<0
IFcv1ANDcv2ANDtime>=080000THEN
SELLSHORT2shareATMARKET
ENDIF
// Stop et objectifs
SET STOP%LOSS0.035
SET TARGET%PROFIT0.35
The problem is everytime a position gets opened, it almost immediately gets stopped out – because the stop is very tight. I deliberately put the stop as tight as possible as I’m new to this so wanted to learn without incurring huge losses. So I’m aware why this happens but in backtesting the stop rarely gets triggered because at some point during the 4hr bar the price has actually gone into profit and hit its limit so PRT backtest thinks that the trade is always closed out in profit and returns a profit in test results whereas in real trading the trade has hit its stop (as its so tight) and closed out at a loss as the broker sets the stop in realtime. I used the above code in real trading this week and every trade was a losing trade but if I apply the backtest today using the last 3 days real data – some of those losing trades are actually winning ones. So the backtest is obviously misleading and I want to find out the limitations of PRT in this regard.
Is my interpretation correct ? Is there any way of backtesting that use real time intrabar data so testing returns the correct results ?
What you are pointing here is something a lot of people have already asked on forums 🙂 Yes the backtests are different from real live trading because conditions are only tested one time per bar as you discovered already. If the target and loss occur on the same candle, the backtester will take the takeprofit in consideration before the stoploss, so a lot of trades that would have been in losses in real live trading are winners in backtests.
To avoid this, you can trade your strategy in minimal timeframe because conditions will be checked more often of course.
There are currently new development in progress at PRT to had new capabilities to the backtester to check conditions in any inferior timeframe at each bar of the timeframe you are testing your strategy on. We just have to be patient for this new version to come 🙂
Great, thanks for the quick response Nicolas. I did suspect that this was the case, but always good to get it confirmed from someone like yourself who is very knowledgeable in this regard. The point you made about the backtester always applying the take profit before the stoploss now makes perfect sense with the results I’m seeing so I understand it better as was worried that the testing results were purely random chance as to if it’s a profit or loss within the bar. For now I will take your advice and look at strategies in shorter timeframes for more accurate results.
Also, in a next update, multitimeframe look will be available. So I believe, it will be possible to launch a strategy in a 1 minutes timeframe and look about indicators values from the 10 minutes one.. hope so 🙂
I just received an answer about trailing stops from IG Markets. I just post it here. Sorry it’s in German, but Google Translate helps:
Die Kollegen von IT Finance (der Herausgeber von ProRealTime) sind nun auf uns zurückgekommen. Im ProBacktest wird stets das Hoch bzw. das Tief zur Berechnung verwendet, da der Backtest nicht mehr Daten zur Verfügung hat und deswegen keine chronologischen Daten hat. Bei hoher Fluktuation innnerhalb einer Kerze würde dies also u.U. zu ungenauen Daten führen.
Bei einer Long-Position würde also bspw. das Hoch verwendet werden, um zu bestimmen, wie weit der Stop im Backtest nachgezogen wird.
Bitte beachten Sie, dass IT Finance aktuell an einer Weiterentwicklung des Backtests arbeitet, welche dies verbessern sollte. Laut den Kollegen sollte diese schon bald eingeführt werden, als Andeutung wurde eine Einführung innerhalb der nächsten vier Wochen angegeben.
Thanks, thats great to hear that the issue is being addressed. The sooner we can get the tester as close to actual trading the more reliable our coding will be, some of the results look really interesting in testing initially until you look more closely at the results and realise that they may not be accurate. I was testing out a simple breakout strategy to see how inaccurate the results in PRT can be compared to real trading. As below, strategy buys/sells DAX when price breaks out of the first 5mins of trading. You can do this 2 ways – code up strategy to look at Five 1min bars at the open and run the code on 1min timeframe OR code up One 5min bar lookup and run on the 5min timeframe. The point being that ideally if the backtester was looking at tick by tick data then both codes should produce exactly the same results ( I hope ?) :
Run on 1min timeframe :
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Defparamcumulateorders=false
Defparamflatafter=162900
n=1
IFTime=080500THEN
rangehigh=highest[5](high)
rangelow=lowest[5](low)
amplitude=rangehigh-rangelow
long=0
short=0
ENDIF
ifTime>080500ANDTime<=162900THEN
IFlong=0THEN
buynshareatrangehighstop
ENDIF
IFshort=0THEN
sellshortnshareatrangelowstop
ENDIF
ENDIF
IflongonmarketTHEN
long=1
ENDIF
IFshortonmarketTHEN
short=1
ENDIF
set stopplossamplitude
set targetpprofitamplitude
This produces a gain of £16.80 (0.17%) over 2 mths. If you change part of the code as below and run on 5min timeframe :
1
2
3
4
5
6
7
IFTime=080500THEN
rangehigh=highest[1](high)
rangelow=lowest[1](low)
amplitude=rangehigh-rangelow
long=0
short=0
ENDIF
It produces a gain of £232.10 (2.32%) – thats a difference of more than 10 fold in the numbers – and thats only over a 2mth test period. The reason is that sometimes the 5min bars are so big that the buys and sells take place in the same bar and thats when the results start becoming inaccurate.
So as pointed out by Nicolas, best to try and run your strategies on a shorter timeframe wherever possible if using a short term strategy. If you are using longer term positioning then it shouldn’t matter as the chances of a say a 4hr bar incorporating both your trade entry and limit/stop in the same bar will be much smaller. If any of my conclusions above are inaccurate, please feel free to correct. Thanks.
As was already mentioned in backtesting loss may be interpreted as profit due lack of information regarding price movements inside of the bar. Does it also mean that winning trade can be recorded as loss in backtesting for the same reason? For example, if you go long and candle close price is lower than entry then your trade will be recorded as loss. However, it does not mean that the price didn’t move up before it went down.
I believe that cannot happen as PRT always assumes that if there is a profit available then that will be the first default trade result so a genuine winning trade will also always be shown as one. It’s just the losing trades that could be misrepresented as winning ones. Therefore, if there is a limit and stop in the same bar then regardless of whether the stop level was hit first in real trading, it will always assume that the limit (ie profit) level has been reached first.
Is there any update when this new version is available? I just ran into the same issue and must say that ProRealBacktesting is completely worthless if it always apply a best case scenario within a candle.
10.3 is out of Beta now which includes inside bar testing. IG / IT-Finance have plans to implement 10.3 at the beginning of next year so hopefully only a couple of months to wait.
hi All, I’m new and still in demo mode (not real cash yet). I’m getting big differences between stop losses on back test Vs live. I’m using tight stops in backtest which (as described in this thread) are not achievable in live mode. I’m running 1min data with a simple MACD strategy. Works amazing in backtest, and does really badly in live mode.
Is there a solution to this? Do I need to develop a strategy that doesn’t rely too heavily on stop losses? Thanks!
Backtests have no ideas of what is the distance from open price allowed by your broker to put stoploss. You will find this information on the website of your broker, and it is different for each instrument.
.
