On the subject of curve-fitting
- This topic has 37 replies, 4 voices, and was last updated 5 years ago by
.
-
Personally I think that regular optimising just keeps you in tune with what just happened but there is no way to know that tomorrow the markets will behave exactly the same as they did last month. You could just find yourself always one month behind the ideal curve fit and always wrong! Others who do regularly re-optimise might disagree with these thoughts.
More data is always a good thing but then no two market crashes have ever been the same so what fits historical crashes probably wont work the same in tomorrows market crash. We also have to consider that due to the invention of the computer trading and market character is very different after the mid 90’s than before that time. Before the invention of the computer trading was done by only a few and they used a pencil and paper to draw their own candlesticks and a to calculate their own indicators. Most investors were long term buy and hold value investors but after the computer became a usable trading tool all sorts of trading styles and analysis could be used and high frequency trading became a possibility. The markets changed and are changing every day as technology continues. There has also been other interventions such as market controls to stop shorting in crashes which is another reason why future crashes will not be the same as past crashes. Then we have to throw in news and politics and world changing events that effect market behaviour – there was no Brexit in 1941 but there was another different market changing event! Do we optimise our strategy to cope with a world war or a stupid referendum?
If a strategy works from the mid 90’s onward and avoids massive draw down in any crashes then as long as it was not curve fitted to avoid those crashes it should cope in the future. For example having a fuse stop loss of say 10% means that the most you could ever lose in a crash is 10%. However a strategy that catches a falling knife and keeps buying into a crash and only sells if in profit could actually return a brilliant equity curve if it just got lucky and bought on the right candles. Buying just one candle later or just starting the strategy on a different day and the same idea could find you buying at the top of a crash and continuing to buy all the way to the bottom (or until your bank runs out!)
1 user thanked author for this post.
Might it even make more sense to always choose the best fit for the curve you’re in, rather than a compromise that fits any?
I have done this as a trial on several Systems and they do seem to whoosh away into good profits quicker than Systems optimised over 100k+ bars.
BUT above is conjencture because I cannot point to / provide a full set of results for the same System on Forward Test (FT) after optimising over 10K bars and another version (same System) over 100K bars.
I have done FT on 2 Versions (10K and 100K opti) and it has left me with the impression stated in my first paragraph.
We would need to ensure we re-optimise at the 10K bars point until 100K bars have passed and then compare results of the 2 Versions (10K and 100K opti).
1 user thanked author for this post.
I’m a bit confused by what you have discussed. WF breaks the data down into segments and then optimises part of that segment to find the best working variable values. It then uses those values on an out of sample part of the data so that we can see how it would have worked walking forward. Overall results are those OOS tests added together compared to the IS.
To simulate this you have to choose some recent history (a day, a month, a year, a decade? We are already curve fitting!) and optimise it to see what values worked best and then trade those values until you next feel like optimising. So to simulate this in a back test you would need to for example optimise based on December 1995’s data and then see how those values performed in January 1996 and record those results. Then re-optimise on just January 1996 data and use those values in February and record those results. Then re-optimise on February’s data and use that in March and so on and so on. An alternative is to optimise on all of history up till end of December 1995 and then use those values in January 1996 and then at the end of January re-optimise using all data up to the end of January 1996 and use those values in February.
I once coded a strategies that did both of these things and in back testing they worked great but in forward testing they failed terribly as it seems that the most important thing is the robustness of the strategy itself rather than having a strategy that you throw a broad range of variable values at time after time and keep your fingers crossed that they work as well this month as they did last month. I called my strategies ‘self optimising’ and Leo wrote something similar with his Neural Network concept. I gave up on the concept after spending a lot of time and effort on it. It was not a total waste of time as it confirmed to me that re-optimising is not the holy grail – it is just a way of proving whether last months curve fit still works this month!
1 user thanked author for this post.
I can’t get my head around why it has to be random / manual?
My thinking is coming more from a robustness testing point of view. Without picking multiple different starting points you could just get lucky with the five you chose.
1 user thanked author for this post.
Find exclusive trading pro-tools on 
