Backtesting kaput ? (IB)

Viewing 4 posts - 1 through 4 (of 4 total)
  • #226899

    I just spent over 2 hours filing a ticket about something being totally wrong today, unless I am blind and don’t see the cause. So if anyone seems to notice similar as this …
    Side note : this is for V11 and V12 and in two accounts of IB. I did not check IG.

    Backtest results are not the same, I think after the platform reset 06:30 (Amsterdam) this morning. But, this depends very much on what the code does. A small quote of what I wrote to support :

    If we look back in ##PRTzz304c, we can see suspects like StrategyProfit, Floor, Min. I use these throughout the program code as well, including Round and Max.
    What I am saying is : it is not necessarily the data because it can be the commands as well. I could add that these commands are almost (or fully ?) exclusively used in the MM code, which is much larger than the small part I showed. I have no reasons to use these commands elsewhere, except for StrategyProfit which I use to determine the exits of trades. Because that all keeps working (as far as I can tell) it should not be that.

    The kind of fun is that I can compare the backtests with Live from the past 3 weeks, and nothing is the same any more. This counts for 3 running Systems I checked.

    When it stopped working I did not restart PRT somewhere – the backtest results still on screen from earlier this morning, just don’t work the same any more for certain situations.
    If someone else also has issues suddenly, it would be good to know about it ?

    #226928

    This appears way more complex than I could initially see. It is possibly (probably ?) related to the last line of code you see in this snippet pulled from the program of concern :

    Obviously I am making my observations from backtesting. In this case I find that where > should work (it is what I want because I want to do things when making profit in the last trade), that doesn’t do a thing and I notice that <> does do something with effect. As if losses are now incorporated too. Special environmental to this “problem” : the backtests do not show losses and so making it <> as a temporary solution is for that moment fine with me. It only means that when there is this one loss out of 100 trades (or whatever I exactly expect), the contract amount will be increased after a loss as well. This is insignificant.

    In practice different things occur than what the eye can see; in practice, with backtesting in IB (!), there are TWO occasions when one trade unwinds/exits and when the code is called in two subsequent events (bars)  this happens – the first change of StrategyProfit is when the trade indeed exits and the second change of StrategyProfit is when the backtesting mechanism adds the broker fees. Careful, this is how it works for IB. For IG this does not happen because IG works with spread for income (in this situation which is about Futures, Indexes for IG).
    This implies a kind of cancellation of the reported profit to the code, because in the first call (at the Exit) there will be profit (can also be a loss), while the second call implies a loss (broker fee is a deduction).

    This is how backtesting works.
    In Live no such thing happens … In Live there is just less gain reported, as the broker fee is right away deducted from the gain of the trade.

    We now arrive at the more complex situation that the contracts-adding code is called twice – which out of all I already noticed and what made me halve the added contracts, so net it would be fine (Peter, this is a wrong thing to do, man). But code like this is everywhere in the program, so many more things go wrong.
    Try to envision that I had to adjust the MM code which originally was triggered by losses only, and which I could not get to work in a situation with “profits only” (MM did not do a thing now). So now I changed the MM by means of letting it act on profits, but because profits also imply losses each time, each trade reports a double virtual profit (envision what that code snippet does on the condition you see) and the MM is down the drain.

    The net effect I can not reason yet, but the outcome of it is that Live performs way way better than backtest, while backtest actually was “fit” to what I see happening. Thus, what I see happening should increase the number of contracts gradually and if that goes too fast for what the portfolio can bear, I slow it down in the code. And that slowing down does not work in Live because of the StrategyProfit acting differently (no “no-profit” because of broker fee).

    And so all ‘n all backtesting is fine. Live is also fine, but very different. Net for the situation, Live is not so fine at all because after each profit 1 contract is added. Totally crazy stuff you see below, and which makes me sit there and wait till I deem it too much and kill the system (which I did twice so far from of Jan 4).
    … And all I thought was an “oh well, *that* is working nicely !”, meanwhile knowing that I must be doing something wrong because I am not really into blowing an account by this strange act;
    In the other topic (about Money Management) I see myself writing about how I approach that, actually not being able to justify this craziness, where you can see things go out of whack. But hey, I derived that from Live and not from Backtest. Backtest works as how I made it (for net result) and there nothing crazily grows. The 2nd attachment shows how it turned out in Live and where I could not deem that wrong. This coincidentally relates to the profit per trade which turned out way better for the latest market situation than I envisioned (better : saw happening in backtest) when designing this, and so I thought that was the reason and let it go for 3 weeks. … Until I wanted to showcase this per backtest in that MM topic and simply could not mimic the Live situation. And then I created the topic you are reading right now …


    Almost the worst is that despite you see something totally illegal in the 2nd attachment, it would work on the longer term just the same. Thus, it would not be so that a loss would make the system go under water. The loss(es) would be huge, but the profits have been more huge to begin with. However, the growth just makes no sense and eventually it would blow up the system itself because of a lack of funds in the account. This is what we talk about in that MM thread … and that it is harmless in itself. But hey …
    Money Managment ideas
    More specifically this post this post and beyond.

    #227218

    On my latest backtest, I am not reproducing last week’s results.

    See above for similar / same Issue

    #227226

    Hi GraHal, Hello @plbourse,

    I hope it is not too confusing, but what I wrote about in this topic can not be related to plbourse’s problem. PRT ne donne pas les mëmes résultats que la semaine dernière

    My last post in this topic (“Backtesting kaput”) , tries to describe that Backtest is not the same as Live for the IB version of PRT. Initially I thought things had changed. But nothing changed with backtesting … the behavior of StrategyProfit is just not the same in both environments Backtest vs Live. Very inconvenient if one does not know, but it is another problem.

     

    2 users thanked author for this post.
Viewing 4 posts - 1 through 4 (of 4 total)

Create your free account now and post your request to benefit from the help of the community
Register or Login