Strategy Robustness Tester

Vonasi · Sun, 01 Sep 2019 12:34:14 +0000

Following on from a discussion elsewhere regarding walk forward testing I had the idea to create a bit of code that you can add to any strategy and see how it would have performed if you had traded on only odd days or only on even days, or odd or even months or odd and even years or any combination of the three. By comparing the output of all 23 tests we can get an idea of how consistently a strategy makes profit. The gain figure in the optimisation results should be ignored as the number of trades for each test will be very different. The average gain per trade, win rate and equity curve for each test can tell us a lot about the strategies robustness though. You just add the code to the start of your strategy code and then insert the extra condition ‘and tradeon’ in the conditions whenever you send buy or sellshort orders to the market. Then just optimise with a variable ‘choice’ with values 0 to 22 step 1. Then just click on each result in the optimisation report to see the equity curve for each test. Here is the code to add to the start of your strategy in case it is of use to anyone: //Day Month and Year Robustness Tester //By Vonasi //20190901 //0 = all days //1 = odd days only //2 = even days only //3 = odd months only //4 = even months only //5 = even years only //6 = odd years only //7 = odd days in odd months only //8 = even days in even months only //9 = even days in odd months only //10 = odd days in even months only //11 = even months in even years only //12 = odd months in odd years only //13 = even months in odd years only //14 = odd months in even years only //15 = odd days in odd months in odd years only //16 = even days in even months in even years only //17 = even days in odd months in odd years only //18 = odd days in even months in even years only //19 = odd days in even months in odd years only //20 = even days in odd months in even years only //21 = even days in even months in odd years only //22 = odd days in odd months in even years only oddday = opendayofweek mod 2 = 1 evenday = opendayofweek mod 2 = 0 oddmonth = openmonth mod 2 = 1 evenmonth = openmonth mod 2 = 0 oddyear = openyear mod 2 = 1 evenyear = openyear mod 2 = 0 tradeon = 0 if choice = 0 then tradeon = 1 endif //odd days only if choice = 1 and oddday then tradeon = 1 endif //even days only if choice = 2 and evenday then tradeon = 1 endif //odd months only if choice = 3 and oddmonth then tradeon = 1 endif //even months only if choice = 4 and evenmonth then tradeon = 1 endif //even years only if choice = 5 and evenyear then tradeon = 1 endif //odd years only if choice = 6 and oddyear then tradeon = 1 endif //odd days in odd months only if choice = 7 and oddday and oddmonth then tradeon = 1 endif //even days in even months only if choice = 8 and evenday and evenmonth then tradeon = 1 endif //even days in odd months only if choice = 9 and evenday and oddmonth then tradeon = 1 endif //odd days in even months only if choice = 10 and oddday and evenmonth then tradeon = 1 endif //even months in even years only if choice = 11 and evenmonth and evenyear then tradeon = 1 endif //odd months in odd years only if choice = 12 and oddmonth and oddyear then tradeon = 1 endif //even months in odd years only if choice = 13 and evenmonth and oddyear then tradeon = 1 endif //odd months in even years only if choice = 14 and oddmonth and evenyear then tradeon = 1 endif //odd days in odd months in odd years only if choice = 15 and oddday and oddmonth and oddyear then tradeon = 1 endif //even days in even months in even years only if choice = 16 and evenday and evenmonth and evenyear then tradeon = 1 endif //even days in odd months in odd years only if choice = 17 and evenday and oddmonth and oddyear then tradeon = 1 endif //odd days in even months in even years only if choice = 18 and oddday and evenmonth and evenyear then tradeon = 1 endif //odd days in even months in odd years only if choice = 19 and oddday and evenmonth and oddyear then tradeon = 1 endif //even days in odd months in even years only if choice = 20 and evenday and oddmonth and evenyear then tradeon = 1 endif //even days in even months in odd years only if choice = 21 and evenday and evenmonth and oddyear then tradeon = 1 endif //odd days in odd months in even years only if choice = 22 and oddday and oddmonth and evenyear then tradeon = 1 endif Then alter your strategy code to use the TRADEON variable so that trades are only opened if the day, month and year meet the test criteria. if (your long entry conditions) and tradeon then buy 1 contract at market endif if (your short entry conditions) and tradeon then sellshort 1 contract at market endif

This topic has 161 replies, 20 voices, and was last updated 3 years ago by Roger.

Tagged: casino, robustness, walk forward

Viewing 12 posts - 151 through 162 (of 162 total)

← 1 2 3 4 5 6 7 8 9 10 11

01/20/2021 at 11:26 AM #158647

nonetheless

XORANDNOT wrote:

to apply position scrambling already during parameter optimization

this is an excellent idea, although i wouldn’t know how to code it either. Best would be if Random Position were one of the optimization options built into the engine (along with gain, drawdown etc)

Another thing I’m experimenting with, is to move this idea of alternative entries out of the testing zone and into live trading. So I make alternative versions of the same algo that are just different enough to keep them out of synch, so that each sees different trades. For example, one with a % TS, another with ATR TS. Even if they take the same trade, they will exit at different places so usually one or the other will be off market and ‘available’. Obviously the basic structure has to be sufficiently sound that the alternatives still work comparably well.

nonetheless
Participant

Topics: 83
Replies: 1237
Been thanked: 978 times

01/20/2021 at 11:44 AM #158653

GraHal

nonetheless wrote:

one with a % TS, another with ATR TS.

Yes I often do above … 4 versions running on Demo Forward Test having same entry, but different exit criteria.

Most times, the version 1 comes out best … probably due to less specific criteria producing less curve fitting.

1 user thanked author for this post.

nonetheless

GraHal
Participant

Topics: 203
Replies: 8375
Been thanked: 2024 times

01/20/2021 at 12:00 PM #158656

Vonasi

nonetheless – You have basically described my SEB testing technique. For example if I create a strategy on the daily chart and it passes the SEB test then I put it on forward live test with 6 strategies that can each only open a trade on one day of the week. So I have a Monday strategy, a Tuesday strategy and so on. Unless a trade is open for more than a week I will get every possible trade available.

On an hourly strategy the ideal is to have a strategy for every hour but 24 strategies on demo annoyingly takes up pretty much all of our allowance for running strategies in demo live.

Another option is to have two strategies, one that opens on only even barindex bars and the other only on odd numbered bars. At least this way we have two strategies slightly out of sink. We do still however miss a lot of possible trades but not as many as if we had just one strategy on forward test.

1 user thanked author for this post.

nonetheless

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times

01/20/2021 at 1:24 PM #158671

Monochrome

The even odd barindex is interesting. Could be useful for short timeframes. Can you code that for us please vonasi? Would greatly appreciate it

Monochrome
Participant

Topics: 16
Replies: 109
Been thanked: 44 times

01/20/2021 at 2:19 PM #158680

Vonasi

Monochrome wrote:

Can you code that for us please vonasi?

I already did way back in the early days of this topic! I called it Robustness Tester Lite.

https://www.prorealcode.com/topic/day-month-year-strategy-robustness-tester/#post-106748

1 user thanked author for this post.

Monochrome

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times

01/20/2021 at 3:14 PM #158688

XORANDNOT

nonetheless wrote:

XORANDNOT wrote:

to apply position scrambling already during parameter optimization

this is an excellent idea, although i wouldn’t know how to code it either.

It could be possible by using arrays. Each different random entry pattern gets an index number in the array, and then you add up all positions from different entry patterns by cumulating them (defparam cumulateorders = true). Then you manage them with partial closings. This means : you add up all different possibilities of the robustness tester in one single, large system by using arrays and position cumulation.

All

set stop ploss sl

1

set stop ploss sl

and

set target pprofit tp

1

set target pprofit tp

would have to be replaced by

sell at positionprice - sl stop sell at positionprice + tp limit

1
2
3

sell at positionprice - sl stop

sell at positionprice + tp limit

and there are a few other complications when going over the zero threshold in total position size.

But as long as v11 yields chaotic results in the parameter optimization of cumulating systems, this will not be doable, I think.

1 user thanked author for this post.

nonetheless

XORANDNOT
Participant

Topics: 12
Replies: 178
Been thanked: 38 times

01/21/2021 at 9:11 AM #158771

nonetheless

Vonasi wrote:

So I have a Monday strategy, a Tuesday strategy and so on

Vonasi, you are a genius. I didn’t pay attention to the SEB idea as I use short TFs and it wouldn’t be practical. But 5 copies of the same algo running each day of the week is a v interesting idea, see attached.

The first image shows the global performance.

nonetheless
Participant

Topics: 83
Replies: 1237
Been thanked: 978 times

01/21/2021 at 9:16 AM #158777

nonetheless

This offers a wealth of data to pick over – number of trades, comparative gain per trade etc.

Interesting that Thursday is notably the worst (over those 6 years), and friday the best. Which means that in many cases, poor trades taken on a thursday would be blocking the possibility of better trades to be taken on a friday.

nonetheless
Participant

Topics: 83
Replies: 1237
Been thanked: 978 times

01/21/2021 at 10:19 AM #158795

Roger

If I understand the idea behind the VRT correctly (not 100% sure about this point), wouldn’t the following tip be enough to check the robustness of an algo ? :

Replacing this :

If c1 then buy 1 contract at market endif If c2 then sellshort 1 contract at market endif

1
2
3
4
5
6
7

If c1 then
buy 1 contract at market
endif

If c2 then
sellshort 1 contract at market
endif

By this :

If c1 then exitshort at market buy 1 contract at market endif If c2 then sell at market sellshort 1 contract at market endif

1
2
3
4
5
6
7
8
9

If c1 then
exitshort at market
buy 1 contract at market
endif

If c2 then
sell at market
sellshort 1 contract at market
endif

Roger
Participant

Topics: 9
Replies: 93
Been thanked: 30 times

01/21/2021 at 10:29 AM #158799

Vonasi

I don’t understand your post Roger or what at all it has to do with robustness testing.

Both of your codes do exactly the same thing. An order in the opposite direction closes all currently opened orders anyway.

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times

01/21/2021 at 10:34 AM #158801

Vonasi

nonetheless wrote:

This offers a wealth of data to pick over – number of trades, comparative gain per trade etc.

It would be great if we could combine all this wealth of extra data. If we could run six strategies in the optimiser (one for each day) and then combine their results to give a combined equity curve that would be fantastic. I did suggest it to Nicolas but he didn’t think that PRT would be that enthusiastic so I’m left with the slow process of dragging results into Excel and sorting them to create my own equity curves.

1 user thanked author for this post.

nonetheless

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times

01/21/2021 at 10:53 AM #158804

Roger

Vonasi wrote:

Both of your codes do exactly the same thing.

Wow I did not know that. An order in the opposite direction closes the trade, ok, but then I did not know it took a new trade. Good to know !

Roger
Participant

Topics: 9
Replies: 93
Been thanked: 30 times
Author

Posts