PRT execution and backtesting issues experienced

• 07/31/2020 at 11:46 AM #140544
  Philstrading

  Hi, perhaps someone can give this ot the correct person in PRT however it is for general awareness for traders to improve awareness of system anomalies.

  Please also note that I’m not opening this up for comment about how this isn’t experienced, these issues have been consistent over a 3 month period across 6 different accounts and therefore I think they should be shared for awareness.

  I’m just writing up some awareness issues for what I’ve experienced when trading in real time and back-testing that I think programmers and traders should be aware of and isn’t advertised or explained by PRT or brokers. This is a list of anomalies I’ve experienced while running the exact same programmes across 6 accounts.

  1: When running a new program initially and it states it is running, it can take a number of hours for the system to register the program. So if you close off a program in the afternoon, it likely doesnt run in the evening even if running, it won’t kick in orders that it should until the following day. This is reasonably consistent across 6 accounts, sometimes a re-initialisation can help if it isnt kicking in.

  2: PRT will not always execute your signal. I’ve experienced many failures over the course of about 3 months now where out of 6 programs sometimes only 2 may kick in say, and 4 will not, other times you can get 5 kick in and 1 may fail, so the system is not consistent in executing your programs always. This has been on the Medium term timeframes of 4hrs and 7hrs. PRT to date have not explained this, latency aside I’ve only experienced some deflection of the issue when being replied to.

  3: We recently  had an experience where out of 3 programs running that were the same, 2 executed correctly and one executed a candle late, which relates to 4hrs, making a winning signal a losing one, however the signal was not generated on for the candle traded, it was generated on the 4am candle to trade at 8am and executed at 12 noon on one of the accounts. 2 of us run a seperate indicator for the signal and that reflected the trade correctly as well, so the sytem held the trade until the  next candle for some reason, which equated to 4hrs. It is therefore advisable to run an indicator for your trading signal and compare with real time execution. This isn’t about trade execution when in a trade, but when actually commencing the trade from the signal. We did send this for them to look at why this was.

  4: When backtesting the seconds charts the results are not generally reflective of real time data. This is an issue for the seconds charts only, maybe related to the ticks being used, however when forward testing and comparing backtesting the results can differ a fair bit. For the minutes charts and above I find backtesting spot on and accurate, and u can usually backtest and check if you should have had trades and see which trades have been missed by the system as well. I’m informed by PRT that the amount of data available for seconds charts is a system limitation, but may be addressed in future versions.

  Overall I find the system a good one, however I’ve not noticed anywhere a mention of the delay in running times and the issues experienced are reasonably frequent but overall I guess I cant complain too much, it just has some gliches people should be made aware of. For me I would note the following points however:

  1: You should always check your expected executions with that experienced to see how often the system is skipping your trades, which will obviously change your programmes performance, and feedback to PRT as the more people that bring this issue forward the more likely they will look at it properly.

  2: Allow 24hrs approx for the system to register your program in its memory and don’t assume it is actually running for execution when you have just put it up.

  3: When checking the program execution also ensure to check it has executed on the correct candle and not been held by the system and executed late. Feed back such anomalies to PRT so they can get enough complaints to take it seriously.  It is advisable to run an indicator for your trading signal and compare with real time execution.

  4: Check any programs running on the seconds charts by forward testing on the demo platform or run with a minor amount, I’ve been forward testing some proved on the demo briefly by running at 0.04% to ensure effectiveness, backtesting isn’t reflecting  the live or demo performances on the seconds charts but is very accurate on the minutes and hours charts and above.

   

  Hope that helps. I’ve forwarded this to the Tech section also, but perhaps someone can give to someone appropriate, all I get is deflection, but I think its important to have awareness of system limitations when putting your money on the line.

  

  Philstrading

  Participant

  Topics: 24

  Replies: 124

  Been thanked: 42 times
  07/31/2020 at 1:39 PM #140553
  Nicolas

  Hi Phil, I think most of the things you mention are “normal”, market and technical behavior, and this is what makes auto trading so hard for most newbies to understand. in the middle. With more than 10 years of experience and thousands of codes to my credit, I can assure you that the strategy represents only 30% of the success, the rest being a matter of technique (often linked to interactions with the broker). Let’s try to go over your elements together point by point.

  1/ never heard of that before, if the strategy is running (green icon), then the code is read at each bar close of the desired timeframe or the minimal one (if MTF code). If your code needs calculation made over X bars to be able to trigger an order, then it could differs a bit from what you could have when you are backtesting.

  2/ you mean, your broker? If an order doesn’t trigger when it should, have a look at your order rejection list. Sometimes, an order that could trigger in backtest could not in real time due to market situation at that moment (widening spread, slippage, stop distance, ..).

  3/ if it was pending order, then it last one candlestick, and if the strategy was a 4-hour one, then it would last 4-hours until a new code reading at 4-hours bar Close.

  4/ slippage mainly. While backtest doesn’t wait for a broker response to send an order, it is in real-time! A real order book is not paper trading 😉

  What you experienced is what you could expect with any automatic trading platform. But I suggest that you send report to your broker when an order is not set properly, 80% of time it is a broker rejection, 19% a problem with the code (backtests behavior differ from reality), 1% a software issue. This is from my own experience of 5 years of tech support here on this website 😉

  BTW, your post is welcome and I’m sure it might enlighten some members around about important things to know when doing automated trading.

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4738 times
  07/31/2020 at 2:46 PM #140559
  Philstrading

  Cheers Nicolas, helpful as always.. to clarify though

  1: Sometimes it is instantaneous, however the same program put up live with a differing amount say, perhaps changing the trail start or stop loss  but not the trigger criteria, will not kick in whereas the one thats been running a while will. It can be the following day before it kicks in. I’ve put programs onto other accounts that dont kick in till the following day, but kick in where they’ve been running a while, and I’ve added to up to 3 accounts and its consistent across the 3 of them not kicking in till the following day. It happens a fair bit, though admittedly not every time. I notice it as I’m looking across 6 accounts whereas if just my own I probably wouldnt notice. I also probably test and check more than most. This can happen when closing and re-initialising as well. Because I trade and test the programs a lot  I likely come across it more than someone just running a few programs.

  2: They dont show as an order rejection, they dont show or trigger at all, I have to therefore assume its latency or the system missing the execution somehow. I’m not talking of 1 or 2 either, perhaps a dozen times across 6 accounts over 3 months. This could in my view be faulty memory modules, extended latency, not really sure, but there is no evidence of order rejection associated with the issue of programmes not executing wherea they have on other accounts and on the signal indicator indicating they should have. If the system is anything like mine a reboot now and again is required for proper performance.

  3: You are familiar with this one, its your Dax Kama. The other programs are my own, although I amended the stop to 40 and made one other adjustment to the trail if I recall. The signal was generated at 4am yesterday on 30th July. The order was executed at 8am following the 4am candle close and signal. On one of the accounts out of 3 of them it was executed at 12 noon making it a losing trade instead of a winning one at 8am. The signal indicator generated reflects the signal on the 4am candle but the system did not execute it until 12 open. Nothing rejected, and it was triggered at 8am on the other 2 accounts. If executing from the program there should have been no trade triggered at 12 noon yesterday, so it executed the 4am signal that should have been executed on the 8am open at 12 noon open candle where there had been no signal to trade at that time. It could be so many things, perhaps how the sytem may glitch reading the time zones perhaps which are set the same across all the accounts, whatever the reason which I can only speculate about, it reflected in a trade that had no signal to trade at that point in time.

  4: Slippage I understand, but its more about the seconds candles being read properly by the ticks generated i think. They dont just give different amounts, but non comparable results on the seconds charts. Everything running on minutes and hours or above is very accurate on back-testing. I could live with differing amounts due to slippage, in this case the results are just erroneous on the seconds backtesting. PRT menton something about amount of ticks available per candle, perhaps its just too low a timefrom to accurately recall results from but thats just speculation, the results however, a little erroneous, but I still use them to see how a new program seems to look then I forward test it on demo and if I like it start it live on a nominal amounts, compare the demo and the live for synchronisity and go fully live when I have some consistency of good results, so I just have to work around the issue, but its all part of the game of course..

   

  To be honest I dont expect 100% performance, latency executing so many programs, etc etc, I might expect the occasional missed trade, slippage is just commonplace, but when across 6 accounts I’m surprised by how much it is actually happening with trades not executing or running when they say they are running after initialisation. The delay in running I surmised rightly or wrongly that it likely takes the system a short while to store soemwhere, I can be running a program on the same system, put the same one up on the same system or a different one and it wont kick in the same as the one already running until the following day. I have to assume it takes time for the system to run it in its memory, I wouldnt be able to ascertain why, just that in practice that is what it does.

  The code I run is usually free of glitches, because I will have ironed them out on demo and backtesting, my code is usually sound when going live. Overall I run code on 40seconds, 5 minutes, 4hours and 7 hours, and I use mtf to run a 4hour trail on a 7 hour code. I have noticed when running on the demo that if you run 2 codes the same but one with an mtf trail for instance, that when both are not in a trade they dont always kick in together, the MTF appears to read differently somehow, this could be related to the issue around programs not always kicking in, or it could be something to do with the mtf engine, I’m still monitoring that particular issue.

  All these issues are forwarded via the platform tech help. I use IG, however for PRT they always refer me to PRT as they are the IB for me and PRT get the Tech help messages from me via the platform, I have to say that they either have difficulty reading English sometimes.  or they are deflecting the issue, or simpy lack the comprehension for whats being stated, they appear to work off the assumption you dont know what you are on about, sorry I have to disapoint in that respect in some circumstances.

  I’m glad you think the post may be of some use to some, thats why I put it up for awareness. Hope that helps to clarify further. thnaks fo rthe prompt response as always. I don’t think PRT wil lbe of much use with resolutions, but if more are made aware of them, perhaps it will beging to be picked up on some of them. You just have to keep an eye out, not assume a losing trade is a losing trade every time, and make some allowances in working practices to work around some of the issues. Otherwise its doing well for me at the moment, and got dsome nifty programs up and running and they are doing OK at the mo. Onwards and upwards as they say.

  2 users thanked author for this post.

  Kovit, GraHal

  

  Philstrading

  Participant

  Topics: 24

  Replies: 124

  Been thanked: 42 times
  08/03/2020 at 2:08 PM #140731
  Dow Jones

  Nicolas wrote:

  this is what makes auto trading so hard for most newbies to understand

  Indeed, @Nicolas. That’s why we keep learning from prorealcode (thanks for your initiative to start it) and overcome some limitation with code. I think many of us knows a few limitations, (myself also complain time to time) but still continue our passion, hoping one day PRT will be the branded in automated trading (like Apple in smart phone) and those who stay will be the pioneer/veteran. In any case, I like PRT really for the back test and more organized and centralized forum, and the fact it is still maintained and kind of free for IG clients 🙂

  Nicolas wrote:

  If an order doesn’t trigger when it should, have a look at your order rejection list. Sometimes, an order that could trigger in backtest could not in real time due to market situation at that moment (widening spread, slippage, stop distance, ..).

  Today one of my EURUSD system (run for more than 2 weeks with accurate result as back test) just missed a trade at 07:15 (UTC + 1) and there is no any information about the order history in the order list (no rejection etc…). I deliberately increase the spread in the probacktest to 10 points (should be quite widen for EURUSD), but the trade still in the probacktest. Is this a good way to verify if due to widen spread that the trade is missed? From your experience, what could likely be the other root cause? Thanks in advance for your advice.

   

  

  Dow Jones

  Participant

  Topics: 10

  Replies: 135

  Been thanked: 75 times
  08/03/2020 at 2:14 PM #140732
  Philstrading

  Whether spread is a factor may depend on your codes entry criteria, otherwise if based on indicators or the like spread will just change your entry level or exit level by the spread difference.

  1 user thanked author for this post.

  Dow Jones

  

  Philstrading

  Participant

  Topics: 24

  Replies: 124

  Been thanked: 42 times
  08/03/2020 at 2:33 PM #140739
  Dow Jones

  Thanks, @Philstrading. Yes, I wasn’t sure it is purely due to widen spread, thus trying to increase the spread to see if this can help to simulate it if this rule out widen spread issue. At the same time, I was kind of sure not coding issue, because same strategy is also used on other asset and probably EURUSD was moving too fast thus issue more visible here. If due to other reason like you mentioned for example server latency issue, it will be impossible to debug. In any case, I will support your initiative and effort  to describe the issue in this thread (thanks for that) by sending my feedback to PRT as well.

  

  Dow Jones

  Participant

  Topics: 10

  Replies: 135

  Been thanked: 75 times
  08/03/2020 at 2:40 PM #140742
  Nicolas

  @yahootew3000

  Other cause could be the account type (limited risk account has some limitations and rules different from other regular account: ProOrder on Limited Risk Account available for Italy,Spain,Sweden,UK

  The guaranteed stop is also something handled by the broker that ProBacktest cannot simulate: if you move your stoploss in the code at a certain price, the broker will adapt it to its own guaranteed stop level.

  In any case there is an explanation, of course if you want to know why an order has not triggered, it would need an investigation into your code, here publicly or privately with the technical service.

  

  Nicolas

  Keymaster

  Topics: 81

  Replies: 20828

  Been thanked: 4738 times
• Author
  Posts