AlunParticipant
Average
Why do trading results differ for backtested results compared to actual results for the same code, criteria and time period.
I ran my auto trade system in demo mode for 3 weeks. It generated x5 positive gain long trades – which was great!!
I then decided to retest the same auto trade system and code by backtesting it over the same time period exactly as the previous demo time period.
The results were interesting when comparing the two:-
1. The demo had 5 positive gain long trades while the backrest had 7 (5 positive and 2 negative gain)
2. Some trades opened at exactly the same time but closed at different times – generally the backtested trades were worse.
3 The backtest has additional entry trades that were poor.
I am trying to understand why these differences would be the case when the code and period are the same, and would therefore welcome any comments.
Many thanks.
One major difference is that spread varies on Live data vs the fixed value for spread you entered in the box?
You must be doing something right 🙂 if your backtest was worse … as it is normally the reverse due to being over optimised!
I’m sure you have … but make sure you have tick by tick box selected in backtest?
AlunParticipant
Average
Thank you for your comments GraHal.
I will check the points you mention.
In my humble opinion, it’s because backtest engine sucks.
I have several examples when a strategy based on a simple indicator signal (it is 1 or it is 0) it produces different entries from indicator or it misses entries.
So, to not abandon this platform which has its advantages overall, I’m developing only 1 minute or 5 minutes strategy and putting them in live demo, without much consideration on backtest results.
I am interested in your future findings, thoughts and conclusions!
I’m always open to a radical new approach … it may work!? 🙂
Keep us posted!
AlunParticipant
Average
Will keep you posted. I’m now running a version of the system for long trades only as a result of the actual performance mentioned to see if the 100% live demo performance continues.
Will do another comparison in a few weeks.
I do have another question – there must clearly be a technical difference between what is happening on backtesting and what happens in the live demo mode. The data used and/or the trade entry criteria detect different information on backtest to live demo (stating the obvious). Why? Spread seems a good possible difference for variation between the two. There must be other reasons – backtesting is simulating live trading so what is it doing that differs to live data trading.
Understanding this may help create better back tested optimisation improvements to system variables.
there must clearly be a technical difference between what is happening on backtesting and what happens in the live demo mode
Are you talking about backtesting over EXACTLY (start and finish tines) the same time period as you have a;ready run on Live data?
Then you compare the two results??
AlunParticipant
Average
Yes the EXACT same time frame was used to the minute for start and finish, so I would expect the same result to come back for both – live demo plus backtested. But this was not quite the case – see original post for a summary of the differences.
As mentioned, I will check it again as soon as I get another opportunity.
EXACT same time frame
Time Period as Time Frame is like 1 min, 5 min, 1 hour for the charted period.
Forgive me … I’m not being pedantic or trying to be clever … but if we are to bottom this out we need to be clear on the terms! 🙂
Ha I just checked … 1 min, 5 min, 1 hour etc … is described as Timescale on the Platform, but most folks on here call it the Timeframe / TF.
AlunParticipant
Average
Apologies for any confusion, I meant the “Starting date” (and time) and the “Ending date” (and time).
Backtest can’t simulate at 100% the live trading behavior due to effect of spread and slippage mainly. Some instructions like “set trailing stop” behaves differently between backtests and live/demo trading.
Conditions tested are sometimes dependent to history loaded, so a backtest started 15 days before today could not give the same exact results as the exact same code started today in live trading.
