Optimising – Curse or a Blessing??
- This topic has 76 replies, 11 voices, and was last updated 5 years ago by
.
-
Referring to what was written in the pathfinder thread, I also made the mistake in the beginning that I set new strategies live after a fancy looking backtest result without having run it on demo for some time. This most often proved as a stupid idea and lost money.
Nowadays I usually run a strategy for several months on demo before I approve it for live trading. I’m still breaking this rule when I’m porting a successful strategy to another instrument. If I’m already running the same strategy with good results live on other instruments I often dare to put it live directly. This (so far) went well. 🙂
1 user thanked author for this post.
That’s why I’m glad we now have a WF tool in the platform, still not perfect but at least usable to educate people about the danger of optimization, and because I’m already in this state of mind since a long time, I tried to teach people here about how to build something relevant enough to be considered as “robust”, since the beginning of the website. There is only way to go: build something you might feel reasonable for your risk aversion in an IN SAMPLE period and validate it in an Out Of Sample period, with WF, backtest, MonteCarlo or even demo trading.
As Vonasi said in the pathfinder topic, even if most of strategies of the library seems curvefitted now, it is not necessary a waste of disk space on our website’s server. They are educational, that’s the way everyone’s learn and how I learned myself. One day you’ll have an edge and the next one your account will go red, that’s trading, not a promised land.. unfortunately. But hey! We still having fun! 😆
I may add a few observations from my own optimizations, and comparison of their results in backtest and reality :
- If an optimized backtest in a few cases yields super gains, but only for a few parameter combinations that emerge from a large sea of mediocre or bad results – do not trust it. These are probably over-optimized insular results.
- If you do not understand (that is, if you cannot describe in simple words that your grandma could understand) why certain parameter combinations yield good results, do not trust them. They may be just accidentally working, because the curve fit is so good.
- If you can tell why a certain result worked well in backtest, and why you may have traded this also in manual trading (e.g., if there is a simple reason, such as : price is already overbought, so I don’t buy now, I wait until price has come back), it may be good : when there is in addition a broad range of optimized parameter values where this is true.
- Optimized parameter values that do not change much over time are usually statistically ok, but when they leap around wildly between different in-sample periods or between backtest and demo trading, they are probably not usable.
3 users thanked author for this post.
One more :
If you get good results for a broad range of parameter values, do not use the ones at the ends of the range, even if they may yield the best gain. This may be like standing at a cliff : The edge is the highest point, but one step further and you fall down.
1 user thanked author for this post.
You can’t know the change of the market phases in advance, only in hindsight you can tell that there was a change – and would have needed other parameters.
Maybe you can distinguish market phases on a very long-term basis (years), and differentiate between bull and bear markets that need different strategies and parameters.
But when you need to re-optimize parameters regularly after a few months, it means that the system is not “robust”, as Nicolas would call it.
In a way, you need to look for stable mediocrity that works well over longer times, instead of contantly re-optimizing systems to get super-stellar results for the last period, and then nothing with these parameters in the next period.
1 user thanked author for this post.
Every trader has his or her different ways of going about trading and your style @GraHal may work well for you but the main issue I would have with it is that nothing can be quantified. How do you not know that the system you just manually closed a bet on for a small loss or small win wouldn’t have had a huge win if you’d just left it running. Then BANG! Sod’s law the next trade hits a huge loss in the blink of an eye before you had a chance to pull the plug on the strategy. You think it is a rubbish strategy when in fact it could be a winner.
Sometimes it is better I think to just have a general look based on recent history and get a feel for where the markets are at. If we have just had five years in a row up then there is a strong chance that this year could be a down year. We have had four quarters in a row going up – how often has that happened and what is most likely to happen next – maybe this next quarter will be a downward correction and so on through the timeframes. This I think is a better quantifiable reason to not trade or pull the plug on a strategy rather than panic closing just because a trade is not in a profit within minutes. So often I have gone to bed depressed that a trade is in a loss only to wake up and find that it is in a big profit. Nine times out of ten if I close a trade early I have cut the profit short. Most of the time it is better to give a trade room and let it do what it has to do.
This maybe of interest and kind of related to what I posted above – I am just working on an indicator/analysis tool to quantify the effect of runs of the same colour candle and calculate the probability of what is most likely to happen next based on history. With it we should be able to calculate a probability of what might happen next which could benefit our trading decisions. For example – did you know that since the start of PRT daily data the DAX has never had more than nine down days in a row and the probability of it going up after four down days in a row is over 62% and if it has gone up two days in a row then there is only a 51% chance that it will go up the next day and the average size of the down days will be bigger than the average size of the up days.
Hopefully once I am happy with it I will post it to the library.
Well, as long as you make money with your system, it works for you. That’s the final judgement.
Stable mediocrity means for me : Be happy with average gains, as long as they drop in quite regularly. Do not dream of becoming a millionaire within months by writing and (re-)optimizing brilliant backtests, it has never worked (for me).
This is just my personal advice to anyone reading here, newbie or not. From time to time, there may be born geniuses, but I am none.
1 user thanked author for this post.
Find exclusive trading pro-tools on 
