Thoughts on how to avoid curvefitting

Viewing 15 posts - 16 through 30 (of 37 total)

← 1 2 3 →

Author

Posts
04/05/2019 at 12:07 PM #95529 quote

Vonasi
Moderator

Master

djohnson87– Yes it could be data mining or curve fitting but it could also be a fact that when the markets are closed it is not a good time to trade. The only way to properly prove this is to either test on a historic out of sample section of data or if this is not available forward test the strategy live.

04/05/2019 at 4:12 PM #95558 quote

djohnson87
Participant

New

Yeah fair enough, will give the forward test a go as I believe i’ve tested it on the max avail backtest data.

Just a side question, AU200 futures being a derivative product by nature mirrors the underlining index – what causes the moves after market?

04/05/2019 at 5:56 PM #95577 quote

Vonasi
Moderator

Master

Found by a Google search to save a lot of typing! There are also pre open and after close auctions that can move price.

After Hours

It used to be that the only people who could trade after hours were the super-rich and big institutional investors. Nowadays, anyone with internet access can do it. As a result, trading in a stock can keep going even after the stock has “closed.” The same things that move stock prices during regular hours also move them after hours — supply and demand. If big news about a company breaks, that will affect the price in after-hours trading, and the price will rise or fall depending on the news. When the stock opens in the morning, traders will be taking into account the news, as well as the after-hours reaction to it, and that may affect the opening price.

Volume Issues

There are significantly fewer people trading after hours than during the regular day. Lower trading volume tends to make stock prices more volatile, or more likely to jump up or down rather than move smoothly. That’s because a single big buy or sell order can have a major impact on the demand or supply of a particular security.

When the rest of the world starts trading after the opening bell, that kind of extra supply or demand might be easily absorbed. That means the opening price may be radically different from what the stock was trading for after hours. Just as there’s no assurance that one day’s closing price will be the next day’s opening price, there’s also no assurance that the after-hours price will carry over into the regular session.

04/06/2019 at 8:50 PM #95628 quote

jebus89
Participant

Master

Hi to add my own 2 cents to that question: When markets are closed, as vonasi has pointed out: there is less volume and way less players in the market. I have some algos that only trade when underlying market is open. I have some algos that block out certain times of the day (like between 23.00 – 03.00 i have found it in most of my algos on wall st, that trading is no good).

Everything we do is curve fitting, and only true answer to find out is wether or not it you have overoptimized is by going forward… Demo or going live is the only truth hehe.

04/07/2019 at 12:13 PM #95638 quote

Finning
Participant

Veteran

Hi all,

a question I have, after listening to various “better system trader” podcasts is that fewer rather than more conditions for trade entry seems to be more robust. Talking trade setups here and not indicators-though i can see the value in having limited indicators. Do most people find this to be true-fewer setups is better than more? Why can you not have say a half dozen core trade setups (talking trades here and not indicators used) that always work and are not turned off or affected by additional trade setups-the basic amount that a lot of people seem to recommend-and then pile on additional trade setups to have an extinsive a list of trades as you see fit? Why can’t you start out with something that is good, and then more is better?

Also, I can see what people are saying with out of sample testing-break your available data up into quarters, and only use one quarter for system development. Make a simple robust system that works on that first quarter-and then if it has a chance of working in real trading it should work on the other 3 quarters. That’s fine. If I have a simple robust system that works on the first quarter and the other 3 quarters, can i then add to it any way possible by creating trades from the other 3 quarters to make a good robust system “better”? Sort of the same as the question above, but considering the importance of out of sample testing.

Thanks for your thoughts,

Finning

04/07/2019 at 12:52 PM #95639 quote

GraHal
Participant

Master

Here’s maybe some converse thoughts to add in the mix?

Curvefitting is fine as long as the same curve occurs in the future and Price does the same thing as it did during the Optimise Backtest?

If you add more and more Set ups / Price bar patterns then we may find that the same set ups don’t occur together and so we get loads less trades than in the Backtest?

Or the set ups do occur in the future, but Price doesn’t then do what it did in the Backtest and so we make less Profit or even a Loss?

Above could occur (and does!) with one Set Up / Indicator etc but there is less chance for a difference n the future because – with one only – we have fewer variables in the mix / strategy (more means more chance of variation from historical curves).

I have many times knocked up a simple one indicator strategy (v1.0 and then added this that and the other condition (v2, v3 etc) increasing Backtest Profit and Winners to Losers ratio etc with each new version!.

Then months later when I check over the 200 Systems I have on Forward test I see that the v1.0 version is steadily plodding along making a healthy profit, whilst the more complex / more conditions versions have fallen by the wayside and are overall losers. If the > v1.0’s are in Profit it is often accompanied by bigger drawdown that I know I can’t live with trading real money as it saps my energy, motivation and makes me grumpy! 🙂

04/08/2019 at 8:08 AM #95663 quote

jebus89
Participant

Master

Hi Finning.

First of all i wanna say “Yes” to the fact that you need to not make (optimize) ur code on 100% of ur data. You create and optimize on a small part (25-50% is fine in my experience). Then you check if it works on the rest of it. Then u run it live.

If you optimize a new code on 50% and you suddenly think “oh i wanna add this filter to the original code” then you want to got back to your “optimizied data” and start adding filters or whatever.

Ive written about it in a post or 2 that ive made earlier, check my history, belive the title was something like “how i create systems” or smthn.

04/08/2019 at 12:28 PM #95696 quote

Finning
Participant

Veteran

Hi Jebus,

Thanks for that! I’ll find and review your post on systems creation. GraHal, thanks for perspective on complexity with regards to indicators. Have you had many systems with many (like a dozen plus) Set ups / Price bar patterns for long/short entry? Did you find the same sort of issue with number of entry commands in a system as what you did with number of indicators used?

Cheers,

Finning

04/08/2019 at 12:52 PM #95698 quote

jebus89
Participant

Master

Id like to add my 2 cents on that matter as well if i can.
Ive experience the same as Grahal, V1 outperforming V2++ in the long run, and this is because of overfitting

Its easy to think that “oh i only have 2 conditions for my entry, if i add 3 more filters i can tune this baby to the max!”

Problem is that when you start adding, you start curvefitting (more then u already have)

I believe that the best thing to do is have multiple OK looking systems. Winning every trade is hard. Exiting on peaks without giving anything back to the market is gonna be hard.
Trying to just get average, good trade tho, is a lot easier. Do this *10 and suddenly u should/might have a diversified portfolio of strategies. Some win in month A, some loose in month A, on the average, if u make more money than you loose, then ur good2go.

Usually when you keep adding filters, you “remove the average” of the system if that makes sense. Your trying to optimize it so it gets better and better. When what you really should have done is just sticking with the “average good looking” system, because it will take more trades (less filters maybe) and in the future price data, who knows if that “new filter” you just added is going to filter out the next 10 profitable trades.

More trades = more robust backtest in my opinion. Adding filters that remove trades COULD remove your future profits as well.
And of course theres this beautiful balance of bullshit in this game. A filter can remove 100 bad trades in backtest, but at the same time remove 100 profitable future trades. Just try to think about the filters you add: what is it actually doing? Is it removing a day of the week? Why? If the majority of Fridays are bad in backtets, whos to say that fridays wont be ur best day going forward.

Lets say ur trading a breakout strategy: a filter like “Daily close > average 200MA” might be a better filter because when daily price is below, thats when we see the majority of volatile and choppy markets (dosnt mean thats true for future prices tho!)

So i would say, the hardest part is just getting those average systems. It takes so much time to create and fail.
Then when u have a few, the next phase of problems occur: trying to stick with the program.

I have systems that have weeks, months and some even a year or 2 in backtest where the system has just lost money all year long. (loosing small tho) and sticking true with those strategies are hard. but if you have multiple strategies and your equity in total goes up, then it could and should be a lot easier for u.

Use ur head when adding filters, and try not to remove many trades from ur backtest if possible. hope this helps

04/08/2019 at 1:30 PM #95705 quote

djohnson87
Participant

New

Cheers all, lots of helpful advice!

04/08/2019 at 7:55 PM #95757 quote

GraHal
Participant

Master

Finning wrote:

Have you had many systems with many (like a dozen plus) Set ups / Price bar patterns for long/short entry? Did you find the same sort of issue with number of entry commands in a system as what you did with number of indicators used?

I’m not clever enough to code up many / dozens of concurrent conditions using sets ups /Price bar patterns.

I recall trying the Pattern Indicator on various TF’s and markets below on an OR OR basis (not concurrent / simultaneous) but didn’t get enough trades or a good enough equity curve etc. I might give it another go! 🙂
https://www.prorealcode.com/prorealtime-indicators/candlesticks-patterns-indicator/

I have got a Doji strategy that is doing good on long term Forward Test Demo if I recall correctly.

So how are you doing? I’d love to test out one of your multi-set up strategies! 🙂

04/08/2019 at 8:19 PM #95762 quote

jebus89
Participant

Master

Grahal, u say u have like 200 forward testing strategies?
How many are u running live and how long will u test things before putting it live?

04/08/2019 at 9:10 PM #95770 quote

GraHal
Participant

Master

jebus89 wrote:

How many are u running live and how long will u test things before putting it live?

None Live since May 2018 when I disabled my Live Account!

Several reasons … I was trading 1 min TF manual and got fed up glued to the screen and needed the summer to do other stuff. I have found the release from the stress of trading is good! 🙂

I am now waiting for PRT v11 as v 10 has so many frustrations for me for Auto-trading. Great for manual trading though!

I did a dummy / pretend I am Live last month on a separate dedicated account. Chose a few of my best Systems, but by the end of March I was down on Auto, but up on Manual!

So I’m trying again for April, so far nothing to shout about … funds more or less same as on 1 April!

04/09/2019 at 1:42 AM #95777 quote
djohnson87
Participant

New
Thanks for sharing. Couple of questions regarding your experience (link to other threads if this is off topic for these parts please):
1. What were your main frustrations with PRT v 10?
2. How would the auto differ from manual assuming you’re using the same rules?
3. How long do you forward test before going live on a system?
04/09/2019 at 10:02 AM #95795 quote

GraHal
Participant

Master

djohnson87 wrote:

What were your main frustrations with PRT v 10?

Click / read the link below, there are also lots more. Doubt they all be sorted with v 11, but here’s hoping!

GraHal wrote:

I’m fed up clicking on View Performance and trying to remember results to compare when I click View Performance for the next System which is also profitable.

djohnson87 wrote:

How would the auto differ from manual assuming you’re using the same rules?

When I manual trade I often don’t use a fixed system of rules. I look at the price bars and – using all the knowledge gained through countless trades and all I have read over many years – I make a judgement and I open and close trades. I am not conscious that I am even analysing the set ups.

I have tried using Indicators in manual trading, but this has not lead to any significant improvement.

Try the game below for practice, it is real data in real markets.
I think (?) because one sees the Daily price action bobbing up and down for a second or two that it makes it easier to judge what Price is going to do next. Try the Daily Trader (Swing Trader is more difficult). I often get like 80% trades as winners, and even 100%!

https://chartgame.com/

djohnson87 wrote:

How long do you forward test before going live on a system?

My longest Forward Tests have been going for 2.5 years. I have tried to get a sharing of Forward Test Performance going on here (see link below) but nobody seems interested.
Re how long, I think it’s more about how many trades than a calendar time. I get more confident at > 40 or 50 trades.

GraHal wrote:

This Thread enables us to share Systems that have been Forward Tested either Live or Demo.
Author

Posts