Thoughts on how to avoid curvefitting
- This topic has 35 replies, 9 voices, and was last updated 3 years ago by
.
-
So from reading, studying, testing and creating algorithms i have a couple of tips to bring to the forum.
As every post i make regardig “how to ..” my opinions on things, are based on my own experiences and thoughts from what i have read in books and heard from listening to podcasts.
So how can we create stuff that is not curvefit to the point of just breaking when put into live?
- Never optimize on all of your data
- Put it in demo to see that it is in fact doing what it is supposed to do. Patience is key.
- You are trying to find patterns within the chart/data, make sure that the pattern you’re trying to trade, is in fact a pattern that happens again and again and again. If your trying to “buy the dip” then you should basically buy every dip you can find in the chart. if you’re only buying the dips that go in your favor, then im gonna guess that you have “picked out” the very best trades from optimizing aka you have curvefit your algo too hard and suddenly the code is not buying the pattern you want in the future, but rather backtest is just picking out the best dips that have happened in history, future dips will look different tho, so make sure your algo is not too spesific and fit to the data.
So what does this mean? Well it means that your code shouldn’t be looking “too good”, do not aim for perfection, aim for robustness. If you manage to buy “every dip” and your still profitable but with less win%, then i would trust the “buy every dip” code rather then a 90% win rate dip-buying code.
Don’t make codes trying to be exact and perfect, make systems that are easy. Easy to explain, easy to show and easy to code.
One of my favorite quotes from trading are (and as always im paraphrasing) “Make sure your code doesn’t fit like a glove, it should fit like a mitten. More wiggle room for the price, more chance for failure, but also more robust when it comes to new data”
My best algos to todays date are in fact my “loosest” ones. The ones with less code, the ones with a “loosely” defined exit (wiggle room) and the ones that do indeed get faked out here and there.
My worst algos to todays date are the ones that have the most conditions, less wiggle room and more room for being stopped out JUST before the price bounces or what have you..if you got 10 conditions with 10 different variables for trying to buy a dip, then it sounds like the code is trying to find exact same patterns as backtest, rather then just trying to buy a dip.
Dosnt mean it wont work, but i would feel better having only 5 conditions even if it means reducing my win rate or even profit.
Remember that “buying dips” (mean reverting) or buying a breakout (momentum) can be measured using 100 different indicators, but that dosnt mean you should include 50 of them, 1 or 2 might do the trick.
When i started out there was one thing i just could not get out of my head, and it has really bothered me to this day: Does indicators work? Should you focus only on “higher highs and lower lows”? What indicators work? What does not work? how can i know?
For me at least, these questions have finally been answered: I have listened to hundreds of podcasts episodes that includes great pro and amateur traders that talk about their own strategies. I have read many books on the subject as well. I have heard about people using only the bollingerbands (combined with price action) and i have heard about others using only the RSI (combined with price action), i have read about people using both the RSI and Bollingerbands combined. I have read about people that swear that no indicators work, only price action can tell you where price might go. Ive listened to experts claiming that if you do not have multiple time frames in your code you cannot be successful, ive read about experts saying that if you do not have a dynamic code, or loads of decision trees, then you cannot be successful. Some say stop loss is key, others say stop loss sucks. Some use trailing stops, some dont. Some use profit targets, others dont.
The final conclusion for me is that everything can work, if you know how to work it.
Indicators can work, people use them and are making profits, i use them and im making profits.
Pure price action can work, people use them and are making profits, i use them and im making profits
Combination of both price and indicators can work. Decision trees can work. Multiple timeframes combined with indicators and combined with price can work. Stop losses are great and stop losses can be your worst enemy.
Honestly i think that everything can work, if used correctly. These are all tools to create a strategy that is profitable. What makes a tool great for you, is knowing the “how, why and when” to use it. Not every tool fits every person and trading style. It comes down to you and your personality.
Even tho it sounds cliche, mean reversion isnt for everyone, same with momentum strategies. Some do great on mean reverting strategies, others dont. They just dont seem to “get it”. Keep working at it tho and you will see the light eventually.
Thank You jebus89 for taking the time to share your thoughts, experiences and summary of the hundreds of podcasts and books you have read!
It is all valuable information and guidance … keep it coming please … whenever / whatever occurs to you.
I use a mixed bag of all the strategies you mention above, I will use anything as long as it can be seen to work … as you say, the acid test is a decent Forward run on Demo with plenty of winning trades.
Even after a successful Forward Test I would still monitor every Live trade until more and more confidence builds up.
1 user thanked author for this post.
But saying “daily close > 200SMA” for S&P minis is probably not such a bad rule, as trading below the 200 day (or whatever variable u choose) seems to add more loosers as price chopps even more when its below than above.
…or better still setting a band either side of the MA200 so that you have a ‘no trading zone’ can work well and take out a lot of the whipsaw losing trades. You only go long if above the upper band and sell any long positions if price falls below the lower band.
On a separate note I have also found recently that using weighted averages rather than simple or exponential has proven more robust in most strategies. I guess that a weighted average responds faster to recent moves and recent data is the most relevant to our strategies.
Find exclusive trading pro-tools on 
