Actual results compared to backtested results

• 06/08/2019 at 10:37 AM #100182
  Alun

  Why do trading results differ for backtested results compared to actual results for the same code, criteria and time period.

  I ran my auto trade system in demo mode for 3 weeks. It generated x5 positive gain long trades – which was great!!

  I then decided to retest the same auto trade system and code by backtesting it over the same time period exactly as the previous demo time period.

  The results were interesting when comparing the two:-

  1. The demo had 5 positive gain long trades while the backrest had 7 (5 positive and 2 negative gain)

  2. Some trades opened at exactly the same time but closed at different times – generally the backtested trades were worse.

  3 The backtest has additional entry trades that were poor.

  I am trying to understand why these differences would be the case when the code and period are the same, and would therefore welcome any comments.

  Many thanks.

   

  

  Alun

  Participant

  Topics: 10

  Replies: 20

  Been thanked: 1 time
  06/08/2019 at 11:09 AM #100185
  GraHal

  One major difference is that spread varies on Live data vs the fixed value for spread you entered in the box?

  You must be doing something right 🙂 if your backtest was worse … as it is normally the reverse due to being over optimised!

  

  GraHal

  Participant

  Topics: 204

  Replies: 8397

  Been thanked: 2039 times
  06/08/2019 at 11:16 AM #100186
  GraHal

  I’m sure you have … but make sure you have tick by tick box selected in backtest?

  

  GraHal

  Participant

  Topics: 204

  Replies: 8397

  Been thanked: 2039 times
  06/08/2019 at 4:01 PM #100205
  Alun

  Thank you for your comments GraHal.

  I will check the points you mention.

  

  Alun

  Participant

  Topics: 10

  Replies: 20

  Been thanked: 1 time
  06/10/2019 at 5:05 PM #100333
  AlgoAlex

  In my humble opinion, it’s because backtest engine sucks.

  I have several examples when a strategy based on a simple indicator signal (it is 1 or it is 0) it produces different entries from indicator or it misses entries.

  So, to not abandon this platform which has its advantages overall, I’m developing only 1 minute or 5 minutes strategy and putting them in live demo, without much consideration on backtest results.

   

  

  AlgoAlex

  Participant

  Topics: 33

  Replies: 64

  Been thanked: 25 times
  06/10/2019 at 6:36 PM #100340
  GraHal

  I am interested in your future findings, thoughts and conclusions!

  I’m always open to a radical new approach … it may work!? 🙂

  Keep us posted!

  

  GraHal

  Participant

  Topics: 204

  Replies: 8397

  Been thanked: 2039 times
  06/10/2019 at 11:26 PM #100369
  Alun

  Will keep you posted. I’m now running a version of the system for long trades only as a result of the actual performance mentioned to see if the 100% live demo performance continues.

  Will do another comparison in a few weeks.

  I do have another question – there must clearly be a technical difference between what is happening on backtesting and what happens in the live demo mode. The data used and/or the trade entry criteria detect different information on backtest to live demo (stating the obvious). Why? Spread seems a good possible difference for variation between the two. There must be other reasons – backtesting is simulating live trading so what is it doing that differs to live data trading.

  Understanding this may help create better back tested optimisation improvements to system variables.

   

  

  Alun

  Participant

  Topics: 10

  Replies: 20

  Been thanked: 1 time
  06/11/2019 at 8:25 AM #100389
  GraHal

  Alun wrote:

  there must clearly be a technical difference between what is happening on backtesting and what happens in the live demo mode

  Are you talking about backtesting over EXACTLY (start and finish tines)  the same time period as you have a;ready run on Live data?

  Then you compare the two results??

  

  GraHal

  Participant

  Topics: 204

  Replies: 8397

  Been thanked: 2039 times
  06/11/2019 at 1:39 PM #100429
  Alun

  Yes the EXACT same time frame was used to the minute for start and finish, so I would expect the same result to come back for both – live demo plus backtested. But this was not quite the case – see original post for a summary of the differences.

  As mentioned, I will check it again as soon as I get another opportunity.

  

  Alun

  Participant

  Topics: 10

  Replies: 20

  Been thanked: 1 time
  06/11/2019 at 2:20 PM #100431
  GraHal

  Alun wrote:

  EXACT same time frame

  Time Period as Time Frame is like 1 min, 5 min, 1 hour for the charted period.

  Forgive me … I’m not being pedantic or trying to be clever … but if we are to bottom this out we need to be clear on the terms! 🙂

  Ha I just checked  …  1 min, 5 min, 1 hour etc … is described as Timescale on the Platform, but most folks on here call it the Timeframe / TF.

  

  GraHal

  Participant

  Topics: 204

  Replies: 8397

  Been thanked: 2039 times
  06/11/2019 at 8:01 PM #100457
  Alun

  Apologies for any confusion, I meant the “Starting date” (and time) and the “Ending date” (and time).

  

  Alun

  Participant

  Topics: 10

  Replies: 20

  Been thanked: 1 time
  06/12/2019 at 8:30 AM #100480
  Nicolas

  Backtest can’t simulate at 100% the live trading behavior due to effect of spread and slippage mainly. Some instructions like “set trailing stop” behaves differently between backtests and live/demo trading.

  Conditions tested are sometimes dependent to history loaded, so a backtest started 15 days before today could not give the same exact results as the exact same code started today in live trading.

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4739 times
• Author
  Posts