Z-Score to improve strategies

Viewing 15 posts - 1 through 15 (of 19 total)
  • Author
    Posts
  • #174383 quote
    robertogozzi
    Moderator
    Master

    I am posting my version of the code to calculate Z-Score and manage it, in order to embed it into your strategies. It also comes with a simple test strategy.

    What is Z-score?

    Z-score measures the distance between the mean of some set of the statistical results and the given observation.

    A trader might suspect a dependency if he or she experiences a run of several consecutive profitable trades, or a run of several consecutive unprofitable trades. “Obviously, there was some kind of dependency or serial correlation among your trades [in this case], where winners were followed by winners and losers were followed by more losers,” Thomas Stridsman wrote in the April 1998 issue of Futures. “If this happens again, you’ll want to exploit the good times and perhaps avoid trading altogether in bad times.”

    In Forex systems, traders are interested in Z-score not of a trade’s return (profit/loss size) but rather in a Z-score of the outcome — was it a profitable one or a losing one?

    Z-score, calculated using winning and losing streaks, measures the dependence between the outcome of the previous position and the outcome of the next position.

    If we consider profitable positions as positive results and losing positions as negative statistical results, counting the total number of all wins, losses, overall trades, as well as the number of win and loss streaks, we can calculate a Z-score for a given

    trading strategy.

    Usually, Z-score fluctuates between -3 to +3, but sometimes, can go above and below these “limits”. A Z-score value of 0 means that we are dealing with completely random results.

    Each Z-score value has also a probability of dependence (P) associated with it, which informs us of how probable dependence between the trades is. P values below -2 and above +2 have high (>95%) probability of dependence between trades:

    • a positive Z-score means that:
      – a profitable position is likely to be followed by a losing one
      – a losing position should probably be followed by a winning one.
      Basically, it means that the probability of long winning and losing
    • a negative Z-score means that:
      – profitable positions are likely to be followed by more profitable positions
      – losing positions are to be followed by more losing positions.
      Basically, it means that winning or losing streaks are probable.

    It should also be noted that Z-score calculation makes sense only for sufficiently

    large samples. Math literature suggests a sample size of no less than 51 (some talk about 30) to get a reliable Z-score value.

    ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
    //Z-Score  (code)
    //
    //The results can be INCORRECT when, on the SAME bar, the following 3 conditions are met:
    //
    //       1) a trade is closed
    //       2) a new trade is opened (usually due to a Stop & Reverse)
    //       3) the new trade hits TP
    //
    //Furthermore, optimization should be avoided when accumulating positions, unless it's
    //possible to consider all of them as one.
    //
    //(streaks)-https://www.forextraders.com/forex-education/forex-money-management/using-the-z-score-to-determine-trade-size/
    //(streaks)-https://www.earnforex.com/guides/z-score-optimization-in-forex-trading/
    //
    //(runs)   -https://www.mypivots.com/dictionary/definition/233/z-score
    //(runs)   -https://www.referenceforbusiness.com/encyclopedia/Val-Z/Z-Score.html
    //
    //          Counting STREAKS    (+ are Winning trades      - are Losing trades)
    //
    //trades:      +-++--++-----+---+++-  (21 trades)
    //tally:          1 2 3    4   5  6   (6 streaks)     a STREAK is a TWIN pair followed
    //                                                                by a different sign
    //
    //
    //          Counting RUNS    (+ are Winning trades      - are Losing trades)
    //
    //trades:      +-++--++-----+---+++-  (21 trades)
    //tally:       12 3 4 5    67  8  9   (9 runs)        a RUN    is any sign followed
    //                                                                by a different sign
    //
    // --- start of Z-Score code
    ONCE Periods   = 0
    ONCE Streaks   = 0
    ONCE CurTrade  = 0
    ONCE TotalWin  = 0
    ONCE TotalLose = 0
    Periods        = Periods + 1
    MyProfit       = StrategyProfit
    // tally Total winning + losing streaks (streak = consecutive winning or losing trades)
    IF MyProfit <> MyProfit[1] AND (BarIndex > 0) THEN
       IF MyProfit > MyProfit[1] THEN
          CurTrade = 1
       ELSIF MyProfit < MyProfit[1] THEN
          CurTrade = -1
       ENDIF
       //---------------------------------------------------------------------------------
       //            code using STREAKS   (it seems to be returning only negative values)
       //             (a sign change preceded by a twin, i.e. a couple of the same sign)
       //
       //Twins       = (CurTrade = CurTrade[1])       //TWO identical trades to make a STREAK
       //Streaks   = Streaks   + ((CurTrade <> CurTrade[1]) AND Twins[1])//now Different trade +
       //                                                                  prior candle's TWINS
       //
       //---------------------------------------------------------------------------------
       //            code using RUNS   (like PRT seem to be doing)
       //             (a run is each sign change, no matter twins)
       Streaks     = Streaks   + (CurTrade <> CurTrade[1])               //now Different trade
       //---------------------------------------------------------------------------------
       //
       // tally Winning & Losing trades
       TotalWin    = TotalWin  + (CurTrade = 1)
       TotalLose   = TotalLose + (CurTrade = -1)
       N           = TotalWin  + TotalLose
    ENDIF
    IF Streaks > 0 THEN
       P              = 2 * TotalWin * TotalLose
       R              = Streaks
       Zscore         =  ((N*(R-0.5))-P) / sqrt((P*(P-N)) / (N-1))
       Zscore         = round(Zscore * 100) / 100
    ELSE
       Zscore         = 0
    ENDIF
    Zpos              = Zscore > 0    //or whatever else > 0
    Zneg              = Zscore < 0    //or whatever else < 0
    // --- end of Z-Score code
    //
    //*********************************************************************************
    //                     Z-Score management
    ONCE SkipOneTrade = 0
    ONCE MinTrades    = 30    //30  or  51 trades minimum to consider Z-Score Reliable
    //                                     as suggested on some websites
    IF Zpos THEN
       //Positive Z-Score
       IF (CurTrade = 1) AND (Curtrade[1] = -1) THEN //Winning  trade following a Losing one
          IF N >= MinTrades THEN         //use Z-Score only when MinTrades+ have been tallied
             SkipOneTrade = 1
          ENDIF
       ENDIF
    ELSIF Zneg THEN
       //Negative Z-Score
       IF (CurTrade = -1) AND (CurTrade[1]= 1) THEN  //Losing  trade following a Winning one
          IF N >= MinTrades THEN         //use Z-Score only when MinTrades+ have been tallied
             SkipOneTrade = 1
          ENDIF
       ENDIF
    ENDIF
    //*********************************************************************************
    //
    //                           S T R A T E G Y
    //
    ONCE MA    = 100    //100
    ONCE T     = 1      //1=ema
    IF close CROSSES OVER average[MA,T] AND Not OnMarket THEN
       IF SkipOneTrade THEN  //Do not open any trade when SkipOneTrade is TRUE
          SkipOneTrade = 0   //clear it for the next trade
       ELSE
          BUY AT Market
       ENDIF
    ELSIF close CROSSES UNDER average[MA,T] AND Not OnMarket THEN
       IF SkipOneTrade THEN  //Do not open any trade when SkipOneTrade is TRUE
          SkipOneTrade = 0   //clear it for the next trade
       ELSE
          SELLSHORT AT Market
       ENDIF
    ENDIF
    set stop   ploss   250    //250
    set target pprofit 2500   //2500
    //
    //*********************************************************************************
    // Nicolas' trailing stop function
    trailingstart = 25     //25  trailing will start @trailinstart points profit
    trailingstep  = 5      //5   trailing step to move the "stoploss"
    //reset the stoploss value
    IF NOT ONMARKET THEN
       newSL=0
    ENDIF
    //manage long positions
    IF LONGONMARKET THEN
       //first move (breakeven)
       IF newSL=0 AND close-tradeprice(1)>=trailingstart*pipsize THEN
          newSL = tradeprice(1)+trailingstep*pipsize
       ENDIF
       //next moves
       IF newSL>0 AND close-newSL>=trailingstep*pipsize THEN
          newSL = newSL+trailingstep*pipsize
       ENDIF
    ENDIF
    //manage short positions
    IF SHORTONMARKET THEN
       //first move (breakeven)
       IF newSL=0 AND tradeprice(1)-close>=trailingstart*pipsize THEN
          newSL = tradeprice(1)-trailingstep*pipsize
       ENDIF
       //next moves
       IF newSL>0 AND newSL-close>=trailingstep*pipsize THEN
          newSL = newSL-trailingstep*pipsize
       ENDIF
    ENDIF
    //stop order to exit the positions
    IF newSL>0 THEN
       SELL AT newSL STOP
       EXITSHORT AT newSL STOP
    ENDIF
    //*********************************************************************************
    //
    graph Zscore coloured(255,0,0,255)
    graph TotalWin
    graph TotalLose

    One of the attached pics shows correspondance between these calculations and PRT’s.

    I tested it on DAX, h1, 200K units.

    In the attached PIC about the formula, PRT stands for “the above formula in PRT Language”, not “PRT Formula”.

    GraHal, PeterSt, Nicolas and 3 others thanked this post
    Z-Score.jpg Z-Score.jpg Z-Score-code.itf Z-Score-Formula.jpg Z-Score-Formula.jpg
    #174413 quote
    robertogozzi
    Moderator
    Master

    While the computation is working fine, I had to slightly change the Z-Score management and the strategy, as follows (you can copy & past it or you can download the correct version of the ITF file):

    //*********************************************************************************
    //                     Z-Score management
    ONCE SkipOneTrade = 0
    ONCE SkipFlag     = 0
    ONCE MinTrades    = 51                           //30  or  51
    DirectionSwitch   = (ShortOnMarket AND LongOnMarket[1]) OR (ShortOnMarket[1] AND LongOnMarket)
    IF Zpos THEN
       IF (CurTrade = 1) AND SkipFlag = 0 AND ((Not Onmarket) OR DirectionSwitch) THEN
          IF N >= MinTrades THEN
             SkipOneTrade = 1
             SkipFlag     = 1
          ENDIF
       ENDIF
    ELSIF Zneg THEN
       IF (CurTrade = -1) AND SkipFlag = 0 AND ((Not Onmarket) OR DirectionSwitch) THEN
          IF N >= MinTrades THEN
             SkipOneTrade = 1
             SkipFlag     = 1
          ENDIF
       ENDIF
    ENDIF
    //*********************************************************************************
    ONCE MA    = 100    //100
    ONCE T     = 1      //1=ema
    IF close CROSSES OVER average[MA,T] AND Not OnMarket THEN
       IF SkipOneTrade THEN
          SkipOneTrade = 0
       ELSE
          BUY AT Market
          SkipFlag     = 0
       ENDIF
    ELSIF close CROSSES UNDER average[MA,T] AND Not OnMarket THEN
       IF SkipOneTrade THEN
          SkipOneTrade = 0
       ELSE
          SELLSHORT AT Market
          SkipFlag     = 0
       ENDIF
    ENDIF
    set stop   ploss   500    //500
    set target pprofit 5000   //5000
    //
    //*********************************************************************************
    //trailing stop function
    trailingstart = 50    //50  trailing will start @trailinstart points profit
    trailingstep  = 5     //5   trailing step to move the "stoploss"
    //reset the stoploss value
    IF NOT ONMARKET THEN
       newSL=0
    ENDIF
    //manage long positions
    IF LONGONMARKET THEN
       //first move (breakeven)
       IF newSL=0 AND close-tradeprice(1)>=trailingstart*pipsize THEN
          newSL = tradeprice(1)+trailingstep*pipsize
       ENDIF
       //next moves
       IF newSL>0 AND close-newSL>=trailingstep*pipsize THEN
          newSL = newSL+trailingstep*pipsize
       ENDIF
    ENDIF
    //manage short positions
    IF SHORTONMARKET THEN
       //first move (breakeven)
       IF newSL=0 AND tradeprice(1)-close>=trailingstart*pipsize THEN
          newSL = tradeprice(1)-trailingstep*pipsize
       ENDIF
       //next moves
       IF newSL>0 AND newSL-close>=trailingstep*pipsize THEN
          newSL = newSL-trailingstep*pipsize
       ENDIF
    ENDIF
    //stop order to exit the positions
    IF newSL>0 THEN
       SELL AT newSL STOP
       EXITSHORT AT newSL STOP
    ENDIF
    //*********************************************************************************
    //
    graph Zscore coloured(255,0,0,255)
    graph TotalWin
    graph TotalLose
    Z-Score-code-1.itf
    #174418 quote
    GraHal
    Participant
    Master

    Link to above added as Log 295 here …

    Snippet Link Library

    robertogozzi thanked this post
    #174462 quote
    PeterSt
    Participant
    Master

    Roberto, I copy-pasted your Z-score code (partly from your 2nd post), but something seems to go wrong / off ? The strategy is my own.

    -1.13 (PRT) vs +11.2 (yours). Notice the ever increase throughout the chart (of ~15 trading days).

    The difference I see is that I am dealing with Brokerage Fees (which are significant) which influence the amount of gain.

    Thanks !

     

    PS: The one but last attachment is redundant.

    image_2021-07-31_193443.png image_2021-07-31_193443.png image_2021-07-31_193620.png image_2021-07-31_193620.png image_2021-07-31_194704.png image_2021-07-31_194704.png image_2021-07-31_195217.png image_2021-07-31_195217.png
    #174469 quote
    robertogozzi
    Moderator
    Master

    Fees, spread etc… do not affect the results.

    Only accumulation of positions and the three conditions stated in the comments (before the code starts) can affect Z-Score.

    #174474 quote
    PeterSt
    Participant
    Master

    a new trade is opened (usually due to a Stop & Reverse)

    That happens (happened) 3 times, and I figured it unimportant. After switching off that feature, no changes except for the results somewhat. The Z-Score PRT reports also changed a little (1st attachment).

     

    a trade is closed

    That does not happen as far as I am aware of.

     

    the new trade hits TP

    Impossible because I control this myself (no Set Target $Profit, etc.).


    I assed a part of the chart which better shows what happens (2nd attachment) . The red graph is obviously your Z-Score.

    I will try to look into it later. But if you have more hints, please don’t hesitate.

    image_2021-08-01_051824.png image_2021-08-01_051824.png image_2021-08-01_052808.png image_2021-08-01_052808.png
    #174477 quote
    PeterSt
    Participant
    Master

    It should be something with the streaks. You probably can tell (I so far can not);

    The previous post showed a winning streak of 4, which increases the Z-Score multiple times. In the attachment below there’s a losing streak of 2, and this decreases the Z-Score.
    I obviously have more winning streaks than losing streaks (also see 2nd attachment in my first post from yesterday), zo net the Z-Score increases. … Without looking into the code, it seems that a 3rd win (or loss) in a streak again increases the Z-Score, which should not be ? (because it is the same streak). Probably …

    More later.

    image_2021-08-01_053713.png image_2021-08-01_053713.png
    #174482 quote
    PeterSt
    Participant
    Master

    With my own method of using Twins (instead of Runs) I end up with the same “negative values only” as you, Roberto (exact same outcome).

    #174486 quote
    PeterSt
    Participant
    Master

    And if I apply my own method of tallying Runs, I again arrive at your exact number.

    Something else is, that the feeling of it all should tell that I never would have such a low Z-Score as PRT is telling me. Thus Roberto, your remark that PRT would be using the Runs method could be correct, but the way PRT is calculating that should be (way ?) off.

    What remains is why your calculated Z-Score *is* the same as PRT’s …

    #174492 quote
    robertogozzi
    Moderator
    Master

    It’s worth while testing it with several strategies on different TF’s. I have, and I think the formula is the same one.

    PRT, on the other side, can overcome my limitations (as stated in the comments), as they can exactly tell if more than one trade was exited on the same bar, which we can’t, and manage accumulation (though accumulation is discouraged as a general rule according to some websites).

    Anyway, being able to manage Z-Score and also drawdown (https://www.prorealcode.com/topic/when-to-stop-a-strategy-and-money-management-code-snippet/#post-174460) may help improving strategies.

    Neither is the Holy Grail, a poor strategy is likely to remain as such, but a good enough strategy could be improved a bit.

    #174495 quote
    PeterSt
    Participant
    Master

    I would love to have it readily available in my strategies because it would in real time show “what’s next”. Actually as how the real Fx guys apply it. So I guess I won’t rest until I have it going as you intended it. It intrigues !

    Thank you again for making this available. 🙂 It appears not all that difficult, but someone has/had to do it (invest the time).

    #174521 quote
    PeterSt
    Participant
    Master

    I think I found the culprit.

    IF MyProfit <> MyProfit[1] AND (BarIndex > 0) THEN

    It goes wrong at that line (or because of that line used as a trigger);
    The first attachment shows a small drain of money at the entry (this happens at each entry). The difference you see reported as -60.
    The second attachment shows the used BrokerFee (this is IB). It is 60 … (counts for entry and counts for exit again – this latter does not bother regarding the issue).
    And so each entry is seen as a “Streak” and counts as such. Also, it will count as a loss and it confuses everything.
    When I eliminate the BrokerFee, all is fine (fourth attachment) – I’ll grant PRT the small difference (Reverse Orders is inactive though measurement is over 157 trades so maybe rounding errors are in place somewhere.

    Remedy : Let trigger the Z-Score code by a means which does not incorporate the entry. It is not easy (for me) to see how to do that, knowing that we’d want general code for that (like Roberto’s example which nicely has it all at the top of the program code). I myself probably would dare to code something like :

    // Untested :
    
    BrokerFeeAmount = 60   // Set to what's in the Editor/Backtest startup screen.
    
    DontIncreaseStreaks = 0
    If MyProfit < MyProfit[1] then   // We lose some ?
      // Is it the entry which charges the BrokerFee (Commission) ?
      If MyProfit - MyProfit[1] = -BrokerFeeAmount then   // This compares at 5 decimals.
        DontIncreaseStreaks = 1
      endif
    endif
    
    [...]

    I hope this addition helps someone (BackTesting with PRT-IB or other brokers who charge commission).

    robertogozzi thanked this post
    image_2021-08-01_190029.png image_2021-08-01_190029.png image_2021-08-01_190108.png image_2021-08-01_190108.png image_2021-08-01_191217.png image_2021-08-01_191217.png image_2021-08-01_193209.png image_2021-08-01_193209.png
    #212191 quote
    paisantrader
    Participant
    Senior

    P              = 2 * TotalWin * TotalLose

    Do you see a problem changing line 67 to “P              = max(1, 2 * TotalWin * TotalLose)” ?

    There’s a Division by zero error after the first trade since only win or lose has a 1 and the other is 0. Figured this way was more appropriate than “IF Streaks > 0 AND TotalWin > 0 AND TotalLose > 0 THEN”.

    robertogozzi thanked this post
    #212243 quote
    robertogozzi
    Moderator
    Master

    Well spotted paisantrader. Thank you 🙂

    #226771 quote
    Meta Signals Pro
    Participant
    Veteran

    Hi Robertogozzi,

    I am stumbling upon this interesting idea;
    But I cannot see how I can use it concretly in an ON OFF strategy management;

    Can you explain it on this example drawn directly from your code and strat.

    Thanks

    Z-score.jpg Z-score.jpg
Viewing 15 posts - 1 through 15 (of 19 total)
  • You must be logged in to reply to this topic.

Z-Score to improve strategies


ProOrder: Automated Strategies & Backtesting

New Reply
Author
Summary

This topic contains 18 replies,
has 6 voices, and was last updated by Brad
1 year, 11 months ago.

Topic Details
Forum: ProOrder: Automated Strategies & Backtesting
Language: English
Started: 07/30/2021
Status: Active
Attachments: 17 files
Logo Logo
Loading...