Tail ratio as the best IS/OOS correlator ?

• 06/02/2020 at 5:08 PM #134254
  zilliq

  Hi Guys,

  “…tail-ratio (i.e. the ratio between the 95th and 5th percentile of the returns distribution) showed a stronger significant correlation with OOS Sharpe ratio than IS Sharpe ratio did (Pearson R² = 0.025; p < 0.0001)…”

  https://joi.pm-research.com/content/25/3/69

  IE tail ratio seems to be one or the best indicator to correlate IS and OOS of our Algo

  PRT doesn’t give the tail ratio for now. Does someone of you have a PRT Code to calculate it “simply” in a backtest ?

  Thanks a lot

   

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  06/02/2020 at 5:21 PM #134257
  Nicolas

  So if I understand correctly, you mean by dividing the 95th percentile by the 5th one of the returns distribution you get a significant ratio to predict a good OOS? But if the strategy is over-optimized in IS, what is the purpose?

  

  Nicolas

  Keymaster

  Topics: 83

  Replies: 20898

  Been thanked: 4866 times
  06/02/2020 at 5:41 PM #134261
  zilliq

  It’s not indicated in the article @nicolas, but it’s up to us not to overfit to have a better “prediction”

  Do you have an idea to calculate “easily” these 95 and 5 percentile on a backtest (I don’t find an easy method) ?

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  06/02/2020 at 6:37 PM #134277
  zilliq

  I love the conclusion (Based on 888 algorithms) :

  “…While the results described above are relevant by themselves, overall, predictability of OOS performance was low (R² < 0.25) suggesting that it is simply not possible to forecast profitability of a trading strategy based on its backtest data…”

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  06/03/2020 at 8:31 AM #134332
  Nicolas

  The conclusion could also have been written with: despite all the analytical data that we can study, nobody will ever be able to predict the future, not even math!

  Calculate easily: no, but could be possible now with $array.

   

  

  Nicolas

  Keymaster

  Topics: 83

  Replies: 20898

  Been thanked: 4866 times
  06/03/2020 at 10:58 AM #134363
  zilliq

  Thanks @Nicolas

  But I’m afraid that even with Array it will take a lot of time

  Do you have a code/snippet to test ?

  Thanks

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  06/03/2020 at 11:19 AM #134368
  zilliq

  Even with array, I think we will need a For/Next loop who will be very consuming in time

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  06/03/2020 at 1:04 PM #134391
  Nicolas

  Yes, that involved nested loops for sure.

  Simple description found on the web, in order to get it clear in mind:

  1. Collect all the data samples.
  2. Sort the data set by value from highest to lowest and discard the highest 5% of the sorted samples.
  3. The next highest sample is the 95th percentile value for the data set.

   

  

  Nicolas

  Keymaster

  Topics: 83

  Replies: 20898

  Been thanked: 4866 times
  06/03/2020 at 1:32 PM #134401
  zilliq

  Yes @Nicoals it’s that.

  It’s not all data samples, only the gain of each trade (it will be faster)

  

  zilliq

  Participant

  Topics: 58

  Replies: 683

  Been thanked: 152 times
  06/03/2020 at 4:23 PM #134427
  Nicolas

  This is just a generic term, so data would be anything you like. In this case data would be the positionperf.

  

  Nicolas

  Keymaster

  Topics: 83

  Replies: 20898

  Been thanked: 4866 times
• Author
  Posts