In sample/Out of sample of live strategies inside!
- This topic has 23 replies, 4 voices, and was last updated 5 years ago by
.
-
Thank you for this good idea.
The out of sample pic is live results only.)
Is the Out of Sample results Live on Real or Live on Demo.
Also might it not have been more useful a comparison if the In Sample results were only up until you started the Out of Sample Live trading period as then it would give a better comparison of the change in results between an optimised period and a non-optimised period??
1 user thanked author for this post.
All the systems are running LIVE live…
but the screenshots are from backtest because i have started/stopped a few of them, also updated the stop loss / trailing stop here and there and therefore had to restart the code.
Edit: Just to be 100% clear: The “out of sample” screenshots here are from BACKTEST only, but i have been running them live more or less since the “out of sample date” shown in the photo. I can confirm that profits have been good and similar to backtest/out of sample photos 🙂
Edit2: Yes i could have used “in sample” photos without the “out of sample” time lol i didnt think about that until i posted the photos, but all of them have 400+ trades so u can imagine just taking out the last Quarter/few months and the results are pretty much the same numbers. If u rly want i can make a photo of just the in sample period.
Edit3: I feel like i dont have enough “out of sample” data just yet to really compare the 2 🙂 Would like about 100 trades / 1 year worth of data! But so far it looks promising, considering how the markets (ws/dax/nq) looks from february
1 user thanked author for this post.
Btw just to show an example of how i work/create algos.
Following is 2 photos, 1 where i have chosen the first 50% of the data and started optimizing, then the second image is the same algo but backtested over the 100% of all the data after i have optimized on the first 50%..
In other posts i have gone thru my way of creating systems, simply put its:
1. Make ur algo on 100% of the data (dont change 1 single variable at all. If your idea is “Buy when price crosses over supertrend and sell when crosses under supertrend” then this is your entire algo with no optimization.)
2. use only 30-50% of the data for optimizing the algo, optimize everything from stops/targets/trailing stops/variables..
3. go back to 100% of the data to check for overfitting.
As u can see from the Poirot pictures included, the first half is a lot smoother and not as choppy as the second half. This is because the first half is optimized, so my variables have been tuned to fit it “as perfect as possible” while the other half just have the same variables to go after and of course is less smooth then because the “same data” will obviously not occur in the future, but SIMILAR data might come..
Edit: @GraHal The more data the better it is imo 🙂
Edit2: Ill add another strategy thats currently in demo, same thinking here, optimized on 50% of the data, then backtested on 100% of the data on picture 2, notice the difference in smoothness of the EQ curve, but all in all “similar results”. Added photos of the results also (not sure if spread is added in this one tbh.)
1 user thanked author for this post.
Good post! And what creative names you have for your algos.
I work similarly although I do not develop the algo first on all data and then optimize on only a part. I develop completely on say 50-70% of the data and when I’m all done then I test the remaining data OOS. I think using all data to build does introduce look-ahead bias even if you don’t change any variable.
Haha thanks Despair!
I understand what u mean about the “look ahead bias” – But honestly it wouldnt matter if i started with 50% or 100%, if i where to use the exact same strategy with the exact same values, if its profitable for 50% and 100%, it dosnt matter, as long as i dont start optimizing/changing values on 100% of the data.
i Could have used ONLY the 30%/50% data and not “checked the 100%” to see how/if its profitable on 100% of the data, before optimizing. But it wouldnt matter if i checked it or not, because the results wouldve been the same.
Please correct me if im wrong!! 🙂
Honestly, if i had the capital i would just push shit right into live.. I would have made alot more money then i wouldve lost, so far..
Yeah I do push shit right into live (well not shit, but Systems that backtest good!) but I did then monitor every trade closely (NOT, what is it – press play and go away?
).I have gone through a bad patch (on manual trading) and so I have re-evaluated my whole way of working, tidied up my Platform (and my wife’s!
) and I plan to go forward using Auto-Systems … with hopefully no manual trades … through the summer at least! A big part of my major review is I have been too intense on it all for over 2 years and the rest of my life has largely stood still! I feel like I’m doing drug at times
and I can’t stop, but I am weaning myself off it all slowly!I have a penchant for short TFs (hence the 10 sec and 1 min results above) I guess as that is how I trade manual on PRT.
GraHal
Hehe im sure u remember my “Thoughts on profitable trading” or what i called that “post for noobs”.. I couldnt trade manually cus i was/am a horrible trader 😀 I dont have the calmness to do manual trades.. its horrible but my stomach twists and turns on every trade. and somehow i 9/10 times buy at the top and sell at the bottom manually 😀 And ive manually stopped systems from trading on just the WORST possible timings 😀 Honestly ive managed to stop a system at the bottom of a drawdown, watched it “paper-trade” lots of profits, started the system again just in time for another drawdown 😀 Suddenly im looking at 2x drawdowns with no profits in between lol
press play and stay away is now my daily motto lol. If i ever think about stopping systems again i just think about all the profits ive missed out on cus of it.
Edit: From all the books ive read and podcasts ive listened to ive realised 3 major things:
1. There is no 1 holy grail system/timeframe/market. Diversification is key.
2. If you dont have a rigid, robust system that you follow 100% of the time, ur gonna fuck urself more then u shouldve in the long run. Because every time u press “buy/sell” manually, ur gonna have feelings regarding that trade, regardless of how the setup is. Your gonna be very hyped and stoked, or very scared when taking the trade, and thats gonna fuck u up sooner or later (unless ur a trader-god lol) So if u start algo-trading then u remove all of those feelings. The only feeling u should have when doing this (in my opinion) is Confidence – that is gonna help u “press play and stay away” and thats gonna help you make money in the long run and follow ur systems no matter what. Compared to what MIGHT happen if u started to do manual trades.
3. The only way to be confident in your systems is by creating them in a very robust matter, doing everything u can do to reduce the chance of overfitting.
1 user thanked author for this post.
).
Find exclusive trading pro-tools on 
