Strategy Robustness Tester

Vonasi · Sun, 01 Sep 2019 12:34:14 +0000

Following on from a discussion elsewhere regarding walk forward testing I had the idea to create a bit of code that you can add to any strategy and see how it would have performed if you had traded on only odd days or only on even days, or odd or even months or odd and even years or any combination of the three. By comparing the output of all 23 tests we can get an idea of how consistently a strategy makes profit. The gain figure in the optimisation results should be ignored as the number of trades for each test will be very different. The average gain per trade, win rate and equity curve for each test can tell us a lot about the strategies robustness though. You just add the code to the start of your strategy code and then insert the extra condition ‘and tradeon’ in the conditions whenever you send buy or sellshort orders to the market. Then just optimise with a variable ‘choice’ with values 0 to 22 step 1. Then just click on each result in the optimisation report to see the equity curve for each test. Here is the code to add to the start of your strategy in case it is of use to anyone: //Day Month and Year Robustness Tester //By Vonasi //20190901 //0 = all days //1 = odd days only //2 = even days only //3 = odd months only //4 = even months only //5 = even years only //6 = odd years only //7 = odd days in odd months only //8 = even days in even months only //9 = even days in odd months only //10 = odd days in even months only //11 = even months in even years only //12 = odd months in odd years only //13 = even months in odd years only //14 = odd months in even years only //15 = odd days in odd months in odd years only //16 = even days in even months in even years only //17 = even days in odd months in odd years only //18 = odd days in even months in even years only //19 = odd days in even months in odd years only //20 = even days in odd months in even years only //21 = even days in even months in odd years only //22 = odd days in odd months in even years only oddday = opendayofweek mod 2 = 1 evenday = opendayofweek mod 2 = 0 oddmonth = openmonth mod 2 = 1 evenmonth = openmonth mod 2 = 0 oddyear = openyear mod 2 = 1 evenyear = openyear mod 2 = 0 tradeon = 0 if choice = 0 then tradeon = 1 endif //odd days only if choice = 1 and oddday then tradeon = 1 endif //even days only if choice = 2 and evenday then tradeon = 1 endif //odd months only if choice = 3 and oddmonth then tradeon = 1 endif //even months only if choice = 4 and evenmonth then tradeon = 1 endif //even years only if choice = 5 and evenyear then tradeon = 1 endif //odd years only if choice = 6 and oddyear then tradeon = 1 endif //odd days in odd months only if choice = 7 and oddday and oddmonth then tradeon = 1 endif //even days in even months only if choice = 8 and evenday and evenmonth then tradeon = 1 endif //even days in odd months only if choice = 9 and evenday and oddmonth then tradeon = 1 endif //odd days in even months only if choice = 10 and oddday and evenmonth then tradeon = 1 endif //even months in even years only if choice = 11 and evenmonth and evenyear then tradeon = 1 endif //odd months in odd years only if choice = 12 and oddmonth and oddyear then tradeon = 1 endif //even months in odd years only if choice = 13 and evenmonth and oddyear then tradeon = 1 endif //odd months in even years only if choice = 14 and oddmonth and evenyear then tradeon = 1 endif //odd days in odd months in odd years only if choice = 15 and oddday and oddmonth and oddyear then tradeon = 1 endif //even days in even months in even years only if choice = 16 and evenday and evenmonth and evenyear then tradeon = 1 endif //even days in odd months in odd years only if choice = 17 and evenday and oddmonth and oddyear then tradeon = 1 endif //odd days in even months in even years only if choice = 18 and oddday and evenmonth and evenyear then tradeon = 1 endif //odd days in even months in odd years only if choice = 19 and oddday and evenmonth and oddyear then tradeon = 1 endif //even days in odd months in even years only if choice = 20 and evenday and oddmonth and evenyear then tradeon = 1 endif //even days in even months in odd years only if choice = 21 and evenday and evenmonth and oddyear then tradeon = 1 endif //odd days in odd months in even years only if choice = 22 and oddday and oddmonth and evenyear then tradeon = 1 endif Then alter your strategy code to use the TRADEON variable so that trades are only opened if the day, month and year meet the test criteria. if (your long entry conditions) and tradeon then buy 1 contract at market endif if (your short entry conditions) and tradeon then sellshort 1 contract at market endif

This topic has 161 replies, 20 voices, and was last updated 3 years ago by Roger.

Tagged: casino, robustness, walk forward

Viewing 15 posts - 76 through 90 (of 162 total)

← 1 2 3 4 5 6 7 8 9 10 11 →

10/29/2019 at 4:43 PM #111560

Vonasi

Is the error that the variables QTY and RANDOM are not declared? If so then as well as changing the TRADEON = 1 also remove the // from in front of the QTY and RANDOM setting lines.

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times

12/06/2019 at 6:06 AM #114299

Tony87

hello, Robustness Tester is Great Btw….

I have a excel file iv been working on for test comparison’s, shared below

can someone fine tune the graphs (graph angle of inclination would be a effective robustness indicator), ?

I have been trying to create a automatic recording system from Results sheet to Record, It seems it is only possible with macro’s ?

https://www.extendoffice.com/documents/excel/5216-excel-record-changing-values.html

any help advice appreciated

Tony87
Participant

Topics: 2
Replies: 4
Been thanked: 0 times

12/06/2019 at 9:08 AM #114307

jebus89

This post has so much info right now, i would love to try this out but im stuck and cannot get this to work!

I saw you posting a small guide with points on “how to…” on like page 1 or 2, can you post and updated one @Vonasi and include the whole updated strategy robustness code?

jebus89
Participant

Topics: 118
Replies: 586
Been thanked: 238 times

12/06/2019 at 3:16 PM #114350

Vonasi

I think the guide that you are referring to is this one here:

https://www.prorealcode.com/topic/day-month-year-strategy-robustness-tester/#post-107192

I’m a little busy with another major (boat related) project at the moment so time to write guides and instructions is a little short – perhaps if you have any specific problems or issues I might be able to answer individual items more easily.

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times

12/06/2019 at 3:59 PM #114353

Vonasi

Tony87 – Sorry but I have only basic skills with Excel but I like your idea of automatically recording the results of tests. Yes I think macros is how you would do it but I would have to learn a whole new skill set to help with that one!

Calculating the angle of inclination should not be to difficult as I believe the calculation is fairly simple – perhaps someone better experienced with Excel could help with both of these?

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times

12/07/2019 at 11:47 AM #114382

GraHal

Tony87 wrote:

It seems it is only possible with macro’s ?

The VBA code (to auto-record a cell change) is provided on the link in your post so all you need do is add that VBA code (in the correct way) in a macro code window and you would have the macro you want.

Instructions for adding the provided VBA code (to make the macro) are there also on the link you provided.

I’m sure you want to do it then you will be chuffed / very pleased with yourself and will have a new skill for life!? 🙂

GraHal
Participant

Topics: 204
Replies: 8403
Been thanked: 2043 times

12/08/2019 at 12:27 AM #114398

Paul

@Vonasi I use tradeon as your example on fifi’s v6 for entry, but not for re-entry, is that correct?

parameters are qty 2-20 & random 1-5, no other optimisation parameters

test; fixed settings and no exit criteria, not even a stoploss

Paul
Participant

Topics: 40
Replies: 812
Been thanked: 660 times

12/08/2019 at 12:28 AM #114400

Paul

full strategy1

Paul
Participant

Topics: 40
Replies: 812
Been thanked: 660 times

12/08/2019 at 12:39 AM #114402

Paul

and other full strategy2

full strategies have same sl 2% , pt , ts etc. and both are based on the same concept from the test.

one setup parameter difference between them.

The concept looks stable, but still have to explore what all means exactly.

Paul
Participant

Topics: 40
Replies: 812
Been thanked: 660 times

12/08/2019 at 9:54 AM #114412

Vonasi

I posted Fifi’s code with the robustness tester added here:

https://www.prorealcode.com/topic/day-month-year-strategy-robustness-tester/page/5/#post-111465

When adding TradeOn condition we must ensure that it is only placed where new trades can be started when not on the market. If a strategy adds to a position or reverses a position then we must allow it to do this. Some codes can be more difficult to add the robustness tester to than others. I discussed an example here:

https://www.prorealcode.com/topic/day-month-year-strategy-robustness-tester/page/4/#post-108261

We should also always test with level stakes to get meaningful results.

As for Fifi’s code I seem to recall it tested well but it is so curve fitted with so many conditions that it will always test well! It is a bit like writing a strategy that already knows which red candle was always followed by a green candle and then writing a code that just ignores candles on dates in the past when that did not happen. The back test will always be fantastic and it does not matter which trades you randomly test they will always be good ones because it is data mining curve fit!

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times

12/08/2019 at 10:08 AM #114413

Vonasi

We must always remember what the point of the robustness tester is – it is a way of randomising the data we are testing on. If we only allow trades to start every fifth bar then we have the potential to create trades that we would never have seen if we started trading on bar 1 and it opened a trade and held it open till bar 6. The trade that could open on bar 5 would be missed in our original backtest and it could be a case that bar 5 is a losing trade, bar 10 is a losing trade, bar 15 is a losing trade etc and we did not see them and think we have a great strategy just because trades starting on bar 1 and bar 6 and bar 11 were all winning trades. The robustness tester lets us see that a strategy that we thought won 100% of the time and makes a small profit actually only wins 50% of the time and makes a loss when you add the two tests together. It also lets us see whether our very low draw down is just because we got lucky and opened trades at just the right time in our backtest whereas if we’d opened the trades one bar later our account could have been wiped out by a massive draw down.

1 user thanked author for this post.

Paul

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times

12/08/2019 at 1:13 PM #114423

Paul

I’ve run it again and it takes about 5 to 6 minutes for the complete strategy 200k bars. It uses market orders and there are no restrictions on a new signal in the same/opposite direction. It just closes & (re)-enters. So it’s not curve-fitted I believe as I had before (5.0p). Also they way your robust tester is implemented is fairly surely correct with the example of fifi’s.

There’s a div/0 error in the 2-10 1-10 sheet, because the last filled row is 91 and it points to row 93.

What to make of the score? Compared to your best it’s higher, but what does that mean when the win% chance & average gain here is lower?

Paul
Participant

Topics: 40
Replies: 812
Been thanked: 660 times

12/08/2019 at 5:05 PM #114438

Vonasi

Paul – You haven’t organised your results from low to high in the optimised results window before dragging into Excel. You have to do this to get the correct charts and correct score calculations.

Also I’m not sure that you have the correct Excel locations being used in the calculations as one of your chart titles is showing a % instead of the correct title for that chart – a sure sign that something is wrong!

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times

12/08/2019 at 6:27 PM #114443

Paul

Thnx for spotting the problem.

The excel file is split at row 48, for the part of the 2nd sorted optimisation window to drag to.

I have 100 rows, I could split it myself or just create more rows which will probably work. But do you have a xls file which is suitable for 100 optimisations?

Also the online version of GraHal goes to 46 rows.

Paul
Participant

Topics: 40
Replies: 812
Been thanked: 660 times

12/08/2019 at 7:42 PM #114445

Vonasi

Yes as I said in a previous post we need to set up different Excel pages for different RANDOM and QTY settings as they produce different quantities of results. I’m no Excel expert but perhaps someone better with Excel knows a way to automate it all a little better but until then just having separate pages in the workbook for different amounts of test results is my workaround.

You need to change the chart data settings and calculations to create each new page in the workbook. The calculations just need to know the first and last (highest and lowest) values to do the calculation as described in this post:

https://www.prorealcode.com/topic/day-month-year-strategy-robustness-tester/page/2/#post-107211

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1810 times
Author

Posts