Hi there
I’m a relatively new member and was hoping someone could point me in the right direction regarding my queries.
I have been programming some very basic automated systems and then optimizing the variables the old way (without using the WF tool). This has resulted in some respectable looking equity curves which has made me curious about eventually running the systems Live. However, I am aware of curve fitting I and don’t want to fall trap to the many other pitfalls along the way!
Therefore, I am keen to understand the process of designing, optimizing and running Live automated trading systems. Anyway, that’s the brief background and now here are the issues I am having:
I have attempted basic optimization and then ran a ‘Walk Forward Analysis’ on the system with the optimized variables using a dummy variable. To me the results look OK. By OK I mean that I can achieve an acceptable WFE ratio in most of the 5 IS/OOS periods. For example, the most recent WF I ran had the following WFE ratio’s: 1) 129.05% 2)50.47% 3)93.28% 4)18.35% 5)47.18%.
I know that the WFE ratio should be greater than 50% in all periods and I have achieved 4/5 in some of the systems. However, I am not sure if 4/5 is a good enough indicator of the robustness before going Live. Should each period be above 50%, or is 4/5 acceptable? I can post the system if it helps, along with the results of WFA etc.
Also, I am unsure whether I need to use the WFA analysis tool to optimize the variables for each IS/OOS period. So, if I use a dummy variable and test 5 IS/OOS periods (each with a WFE of 50%+ indicating robustness), how should i use the tool to optimize for each period? My understanding currently is that it is used to give us an indication of when to re-optimize, based on the current market conditions. If I am correct, does that mean I should I re-optimize for the periods where I receive a WFE ration of < 50% and then use those optimized variables in those market conditions?? If someone could explain this process in more detail I would be immensely grateful.
Finally, could someone please point me in the direction of any relevant forum/blog posts.
Many Thanks
Jim
PS: great community for learning!
I’m always hesitant to answer posts if I am not going to answer all the questions. Reason … my comment may mean that other helpful members may move on and answer other’s questions.
But anyway, here goes …
You don’t mention Live testing using your Demo Platform … this is key to assessing suitability to go Real Live with real money.
I find I want to get Algos on Real Demo Forward Test as soon as possible … providing I can see a logical equity curve, meaning taking entries and exits at logical points etc.
So, in short … have you Demo Tested on real live data your Algos?
(I’ve asked above to others before and a few times the answer has been … Demo what is that? Reason they say that is they hadn’t enabled their Demo Platform.)
Thanks for the quick reply!
Yes, I have them running on Demo but they are all only a few weeks old. I forgot to mention about using Demo and you are definitely right! It makes sense to run paper trading before risking capital.
how long should you test them in Demo mode??
thanks again
point me in the direction of any relevant forum/blog posts
You may have seen this already, if not it’s a useful addition to WF analysis. The version at #106992 seems to be the one most people use.
Strategy Robustness Tester
that does look useful!!! cheers
how long should you test them in Demo mode??
Longer the better, but I try and form an opinion by testing in realtime Demo for a minimum of 1 month / 50 trades.
I am more confident if my backtest equity curve shows my Algo copes with sharp spikes and / or extended ranging periods or even trends / counter trends.
Hi @borderlineJim,
Avoid over-fitting risks is the key to success in automated trading. The WF is just a simple model that compares the annualized performance between the IS and the OOS. The idea of this model is that a too-large difference in the annualized performance between the IS and OOS would mean an over-fitting problem.
I do not entirely agree with this interpretation because I consider that the validity of the WFE dépend on the type of trading strategy. For example, if your strategy opens entries only while a few volatile and bull market, the WFE may not be relevant.
Imagine that the IS market is completely bull then the OOS is completely bear. In this case, that would be normal that the annualized performance of the OOS was flat and the WFE was low.
The analysis of over-optimization risks is very complex, there are a lot of discussions on this forum and posts on the internet about this problem.
But you are on the right track by asking these kinds of questions 😊
