IMPORTANT : BIG BUG ON SPREAD !

Viewing 15 posts - 1 through 15 (of 66 total)
  • Author
    Posts
  • #184246 quote
    zilliq
    Participant
    Master

    Hi Guys,

    First : Happy new year to all !

    Second: I discover this morning a big bug who can concern you, notably if you do backtest on Forex

    The bug is on the spread

    You can try ..

    First : I do a backtest with a spread of 0 /no spread: I had 74,16 % winning trades, +3870 euros and so on

    Second : I do the same backtest with a spread of 0.9, classical on Forex, and I had the same results…74,16 % winning trades, +3870

    Third: But now, just put the spread to 1 (0.1 more only), and now the results are catastrophic !

    So, I think there is a big big bug on the plateform and the spread. If you put 0.5, 0.8, or 0.9, the plateform replace by 0 / no spread !

    Of course it’s very important because if you have wonderful results on backtest and you put your algo on real you will never have the same results because the results on your backtest were the same as No spread and completely overestimated

    Hope it can help you, and hope PRT correct this big but who is present since many years I think, so strange noone detect it

    Bye

    Zilliq

    PeterSt thanked this post
    sp1.jpg sp1.jpg sp09.jpg sp09.jpg sp0.jpg sp0.jpg
    #184254 quote
    GraHal
    Participant
    Master

    You should have posted the Topic in Platform Support Forum so the PRT Rep sees / reads it.

    Hopefully a MOD will move it if you ask (as next comment).

    #184256 quote
    zilliq
    Participant
    Master

    Thanks Grahal

    But it was to warn the users of forex backtests that’s why I post here

    Bye

    GraHal thanked this post
    #184259 quote
    robertogozzi
    Moderator
    Master

    @zilliq

    Do not double post. Ask your question only once and only in one forum. All double posts will be deleted anyway so posting the same question multiple times will just be wasting your own time and will not get you an answer any quicker. Double posting just creates confusion in the forums.

    Thanks 🙂

    #184263 quote
    zilliq
    Participant
    Master

    ? It’s not a question, just a warning for french and english people in both forum  (because you can be french and not speaking english)

    JohnScher thanked this post
    #184266 quote
    Nicolas
    Keymaster
    Master

    Rules are made to be respected, please do:

    • Do not double post. Ask your question only once and only in one forum. All double posts will be deleted anyway so posting the same question multiple times will just be wasting your own time and will not get you an answer any quicker. Double posting just creates confusion in the forums.

    I’ll get back with more infos once the problem is replicated.

    #184276 quote
    monkeys nuts
    Blocked
    New

    zilliq

    zilliq – It is not uncommon for the PRT platform to throw out odd results once in a while.  Perhaps this happened to you and you can retry and get the right results now?

    #184291 quote
    zilliq
    Participant
    Master

    Thanks Monkeys

    But I did different trials on different algos and had the same bug

    Very easy to observe : Do a backtest with a spread of 0, 0.5 and 1 and you will have the same results with spread of 0 and 0.5 and very bad results with 1

    Bye

    JohnScher thanked this post
    #184297 quote
    PeterSt
    Participant
    Master

    Hi @zilliq,

    In http://this post  (and earlier in that topic), I essentially claim the same. But in there it is way more complicated to explain because that is about aggregated orders.
    In the base the problem is already there as well, but it requires recognizing what the culprit actually is. But anyway, I noticed the same as you tell already years ago – possibly since I work with PRT. This is a bit hard to tell, because from the start I calculated my own spread-loss and then it is fine. Still, the figure entered in the Editor screen also does something, and the nice thing of your topic is that I now definitely know that this should be shut off (set to 0) all together. And this in itself never went into my too thick brain ! In other words, if you always have it at 0.6 (pips) it does something in consistency with your own programming. If you then change it to 0.8, things change, but the checking whether this changes correctly is cumbersome. So it really does something, but what it is … it beats me. It could be a factor of 10 off or so.

    I have just been working on it (again) and I think I know how to solve it for myself. This should be something like setting it to 0 in the Editor’s screen, and next do it yourself throughout. But, in a fashion that you can get rid of that again with the flip of a switch/flag when PRT would be so kind to solve this (we would want it to be dynamic).

    In the attachment you see the self-calculated spread of 38,64 at the top (orange) and the system-implied spread of 12.29 at the bottom (yellow). The 38.64 is correct (against the position) and the 12.29 makes no sense. This is for 0.8 pips in the Editor’s screen. Set the Editor to 0 and the 38,64 applies and I am in business. The system calculates 0 for spread indeed and you’re in control again. Now you can set your own spread to twice as low (0.4 in my example) and the 2nd attachment shows that truly (don’t look at the Profit in there because it currently is a tweaked job and the Spreads needs to be subtracted).

    I am afraid I am not easily done with this yet, because all relates to what the system (PRT) subtracts for eaten spread. This is what my posts in that other topic are about – getting it consistent with a. what PRT reports in the backtests and b. make that in itself consistent with reality. I accomplished #a the other day, but quite explicitly killed reality with that because the calculated spread now was wrong. And to be vague once again (sorry for that) : now I calculate the spread right and my programs know what to accomplish (for say break even), PRT’s backtest fails on me because it won’t subtract spread and my shown results will be higher than reality will show me lateron.


    You won’t believe it, but in PRT-IB I always “live” with the very same problem but with different source  : there we can perfectly work with commission (IB works with BrokerFee) but PRT forgets to show that in the results (Detailed Report and such). So there too the results show too high, but at least there I can do something like 500 trades times 60 of BrokerFee and subtract that from the total result I see. With spread this is a bit more complicated, but it could work out the same.
    I have told this something like 100 times to local support as well as numerous times to 2nd line (Technical) support as well for the IB side of things. Throughout the years nobody is doing anything about this and regarding such (crucial) matters, PRT support virtually does not exist indeed.


    The most essential thing of all is that your program knows where break-even is. So in the last example (2nd attachment) this would be when I have a gain of + 19,32. I am not sure yet what the Detailed Report will show, but most probably 19,32 profit, while in reality it is zero.

    zilliq, thank you. You kind of forced me to work this out to something which will really work for me. All (servo) mechanisms will start to work like reality (Live) works too (apart for spread differences but this is a matter of defining spread on the high side and hope your system can still make something of it (profitable trades)).
    Subtracting e.g. 8 euros on the 100.000 of position times the number of trades, is easy to do and is a linear thing. So for example, see 3rd attachment. Gain is 1915 there and position is 480.000. Spread is 0.8 or 8 on the 100.000. Thus 4.4 * 8 = 35,20 (attachment 1 shows 38,64). This times 20 because of 20 trades = 704. Thus the real result will be 1915-704 = 1211.
    This is fine for me. But  do notice that each time I would show you a result, I always need to tell you “but this is without spread” and nobody will understand. Profit on individual trades I also can’t show without context because they are all too high. See 4th attachment. Each result in there is 35,20 too high now.

    What we also could try is finding the spread number for the Editor which resembles reality (I mean, maybe that 2.5 (etc.) implies a reality of 0.8).

    image_2022-01-02_071212.png image_2022-01-02_071212.png image_2022-01-02_072612.png image_2022-01-02_072612.png image_2022-01-02_075255.png image_2022-01-02_075255.png image_2022-01-02_075330.png image_2022-01-02_075330.png
    #184302 quote
    JohnScher
    Participant
    Veteran

    Many thanks for the information !

    zilliq thanked this post
    #184303 quote
    zilliq
    Participant
    Master

    Hi @PeterSt

    Thanks for your long reply and your time

    Happy you confirm what I see (I work on PRT since so many years, and I haven’t see that..So many wrong backtests since then 🙁

    I can’t imagine people who go on real after good (but wrong) backtests after putting a spread<1…so sad

    I’m not sure I understand all your explanations, sorry, but right with you , it is lineary, so theorically we just need to susbtract spread*number of trades to obtain gains, but it will be approximative. I don’t understand why they haven’t correct this bug if you warn them since so many years ??? I will warn them too

    I have a question : How do you calculate the spread ? Because the spread in backtest and in real trading is quite different, the tradeprice is different, the open is different … You trade forex on PRT/IB, right ?

    Thanks and have a nice sunday

    Zilliq

    Ps: Your link doesn’t work

    #184304 quote
    zilliq
    Participant
    Master

    @PeterSt

    I do other tests, and the problem/bug is worst than I thought

    I do a backtest with a spread of 0 and 0.3 and had a “wondeful” backtest : 92.59 % winning trades +400 euros (Of course because with this bug, spread of 0.3 equal no spread..)

    BUT, after, I put a spread of 1. And now the results are catastrophics 0 % winning trades, -1370 euros !

    The difference is so huge !! Imagine the cnsequence since so many years ?!?

    But why is it worst ? Because if you put a spread of 1.5 for example.. NOTHING change : Same 0 % winning trade and same -1370 euros.

    What does it mean ? It means that PRT consider ONLY the first number of the spread, not the decimal.

    0.3, 0.5, 0.7 equal 0=no spread for PRT, and 1.2, 1.5 equal 1…2.3, 2.5, 2.7 equal 2 and so on

    Do you imagine the number of wrong backtests we did since so many years ??? And the huge loss for some people after false backtests when they go on real

    May be a little “trick” before they correct this big bug.: Instead of spread, use the cost by order.

    For example, on EUR/USD a pip=10USD. So instead of spread of 0.3 (who doesn’t work), use a cost of work of 3 USD

    With no spread you have a winning trade of 92.59 %. Same after 3 USD cost/trade

    And the gain is of course different, but seems correct. Before + 400 USD, and after 400-2*3*81(trades)=-86 USD

    Not a perfect solution but a small trick before they correct this bug

    Have a nice day

    Bye

    2022-01-02_10h17_28.jpg 2022-01-02_10h17_28.jpg 2022-01-02_10h17_50.jpg 2022-01-02_10h17_50.jpg 2022-01-02_10h19_17.jpg 2022-01-02_10h19_17.jpg 2022-01-02_10h22_58.jpg 2022-01-02_10h22_58.jpg
    #184311 quote
    PeterSt
    Participant
    Master

    I have a question : How do you calculate the spread ? Because the spread in backtest and in real trading is quite different

    Yes, this is different, but this does not matter really because if the theoretical spread entered in the Editor would work out as should, the difference would be there as well (compared to reality). I say that this is a matter of staying on the safe side. So if IG tells it is normally in between 0.6 and 0.9 on average, it could be good to stay at 0.9, no matter you may not be able to make profit in backtesting any more (you will find new ways :-)). But this is the ever occurring debate of course.

     

    is quite different, the tradeprice is different, the open is different … You trade forex on PRT/IB, right ?

    Correct. This is why everyone should take the spread-costs at the opening. That will be your Fee to pay and that is thus what you need to gain right from the start in order to become break-even.
    No, I don’t trade Fx manually (on IB or elsewhere) but it is my preferred instrument for IG autotrading. It is only that initially I developed the autotrading on IB, which is possible in backtest and Demo over there.

     

    Ps: Your link doesn’t work

    Here is that link again. Hopefully it now works.

    #184313 quote
    PeterSt
    Participant
    Master

    May be a little “trick” before they correct this big bug.: Instead of spread, use the cost by order.

    This is what I use for IB (because for IB it works like that – (hardly) no spread there). But then the problem is virtually the same regarding the Detailed Report and all  in there – the BrokerFee is not calculated (this depends on the type of report you look at and the main issue is in the Demo (thus mimicking Live)).
    Combined with Spread which is there in reality, I don’t think this will work out – or it becomes more and more vague for interpretation. For example, in Demo you would NOT want to show the BrokerFee, but this is OK because in reality spread will be taken (also in Demo, I’d say). But this does not compare with backtesting, so those two would not sync (Demo with Foward testing).

    Maybe we can find a good modus, and currently I am voting for finding the Editor number which resembles reality of what Backtest should show (for IG – for IB there is no solution that I can see).

    zilliq thanked this post
    #184314 quote
    phoentzs
    Participant
    Master

    Now I’m slowly picking up on the subject. Does that mean the spread in the back test is incorrectly calculated and a back test is therefore wrong ?! I usually use a higher spread than is calculated in real, so I’m on the safe side.
    SP500 always 0.6

    EURUSD always 1

    NASDAQ always 2

Viewing 15 posts - 1 through 15 (of 66 total)
  • You must be logged in to reply to this topic.

IMPORTANT : BIG BUG ON SPREAD !


ProOrder: Automated Strategies & Backtesting

New Reply
Author
author-avatar
zilliq @zilliq Participant
Summary

This topic contains 65 replies,
has 10 voices, and was last updated by zilliq
4 years, 1 month ago.

Topic Details
Forum: ProOrder: Automated Strategies & Backtesting
Language: English
Started: 01/01/2022
Status: Active
Attachments: 57 files
Logo Logo
Loading...