Robustness in Automated Algorithmic Trading Systems

Forums ProRealTime English forum General trading discussions Robustness in Automated Algorithmic Trading Systems

Viewing 15 posts - 1 through 15 (of 41 total)
  • #7739

    I’ve heard about ensuring robustness in the past and it was recently questioned in the Grid Trading System here:

     

    I’m somewhat skeptical about trying to make A “robust” trading algorithm (of course there are many types of robust so what does robust mean to you?) because I see the trading environment like the transport system. For example there are Trains, Planes, Trucks, Vans, Cars and Motor Bikes, etc… Each vehicle type is usually best suited for a specific purpose. Sure you can use a truck to carry an envelope but that’s inefficient. Now of course with programming we can transform a Truck to become a Motor Bike but then that might me mean we don’t have a Truck to undertake its job if the opportunity arrives unless we have Cumulative Orders set to True. But once we do so aren’t we bringing unwarranted complexity into our algorithm and increasing the potential for errors and other potential trading and testing inefficiencies or other pitfalls?

    I believe in trying to create Portfolio Robustness instead of 1 system with “optimal robustness”.

    What have you been able to achieve when it comes to robustness in your automated algorithmic trading systems or portfolio?

    #7697

    Robustness: this word is always forget in almost all TS that I foun here in the library. System proposed are always working on 1 timeframe and 1 index.

    What does this mean?

    This mean that most of these trading system works only on the past of that index. If you try to put them in live you will lose big money.

    I would like to open a discussion on this with all of you.

    Here my first opinion, one system that works on DAX should work in a similar way on MIB, CAC, SP500 etc…

    one system that work on gold should work also on WTI and silver…

    (the spread should be always the same, spread should not be calculated on this tests because is different on all indexes).

     

    Last thing, a test must be done also with a Walk-Forward and Monte Carlo analysis.

    Now… your’s opinion…

    David

    #7704

    Thank you David for the purpose of this thread. You have all my attention.

    It’s not fair for other people work to say that all strategies from the library are looser. But of course, as a professional TS coder myself, I must admit that some of them may be curve-fitted. Build trading strategies that worked well in past without at least forward test them is almost like gambling. Find the best edge between what have been good in the past and what would be as good in the future is the hard part.. the very hard part and 99% of people who are developing trading strategies as a hobby don’t have a clue about what it involved.

    The strategies of the library are there because people shared them for all other prorealtime users to get benefit from their discoveries. The purpose of the library is to make a central point of prorealtime codes for all users from all over the world. There are many codes proposed that I declined to be not featured in it, FYI.

    About avoiding equity curve fitting, I believe that a “simple” Monte Carlo analysis is already something sufficient to sort the “not so bad” strategies from the worst ones.

    I’m glad that this kind of conversation start on our forums, because I often said to many people that our main work as trading strategies developers is here : avoid curve-fit in our development. 

    #7710

    Hi Nicolas, sorry I didn’t want to say that they are all bad. I apologize.

    I think that I have never create a roboust code so I think that I could be one of them 🙂

    first important thing, let’s put pressure on PRT to have the montecarlo and walkforward 🙂 I think that any very good trader would never use PRT without these tools. I saw in a separate thread that you have done the montecarlo test in excel. If you have time can you explain how? It could be a good workaround while we wait PRT.

    Some suggestion to avoid curve fitted TS:

    If you use the optimizator of the parameters always check that near value give similar performance.

    Use the optimizator only on few parameters

    Try to do not have fixed paramenters, for instance do not choose 1% for the stop loss but try to choose it using some volatility indicator. In this way if you change index the performance should not change too much.

    Any other?

     

     

    #7745

    @David @dwgfx

    I have merged your 2 topics, hope you don’t mind.

    Diversification is the key. In manual trading and the same in automated trading. We often read here and there that money management is the one and only thing needed to perform well in trading and that’s OK. But build a well weighted portfolios of strategies is a must to compound good average result in a long run.

    #7752

    thanks Nicholas!

     

    @dwgfx in my opinion a roubust TS as said before should be a system that give similar performance on similar index. A train should move you from Milan to Rome in the same way and with same performance then from Berlin to Munich. This mean that trains are national Indexes like MIB, DAX, CAC etc…

    A plane could be the commodities that have different volatility and trading time… Gold, Silver, WTI…

    Forex could be the bike for instance…

     

    When I create a Trading system I always ask to myself: “could this index move like another one from tomorrow?” If the answer is yes that means that the system should give good performance on both.

    For instance: TS on Italian MIB give 20% of gains and on DAX lose 5%. This should put me some worries…

     

     

    #7756

    What you are calling ‘similar index’ would involved that these index are highly correlated ones. If they are highly correlated, then they should be traded like the same instrument, so I agree from this point of view. But even highly correlated instruments have not the same price behaviour and correlation coefficient are often calculated on price close only.

    If you look at the CAC40 breakout system on the library, the one I am talking about here : http://www.prorealcode.com/blog/automated-breakout-trading-strategy-french-cac40/

    You’ll find that it does not so well perform on DAX30 which in very tight correlation with the CAC40, without making some adaptations of the strategy.

    Straight equity curve is not what we should focus on, because we always introduce human bias in what we are developing as a coder. At the exact first moment when you said : “I want my new strategy to be in gain in my 1 year backtest”, you already have include a bias that has already curve-fitted your result, so you fail.

    #7791

    I agree with you.

    I know well your “automated breakout” and I think that your chosen parameters should be decided by ATR or similar indicators. In this way it could be more robust to be moved on other Indexes. What do you think?

     

     

    #7794

    @nicolas did @david-1984 and I post at the same time? Now that’s correlated!

    We will raise many good debates in this discussion.

    @david-1984 you state:

    Robustness: this word is always forget in almost all TS that I foun here in the library. System proposed are always working on 1 timeframe and 1 index. What does this mean? This mean that most of these trading system works only on the past of that index. If you try to put them in live you will lose big money.

    Yes, you  will lose money if you try to use a TS that was first designed for M1 and then run it on H1. For me the M1 is like a very fast motor bike and H1 is a slower motor bike or maybe not even a motor bike but they are on the same road. I don’t expect it to perform as well because I haven’t optimised it for H1. I suggest leaving curve fitting out of the discussion for the moment. I mainly believe in Price and Standard Deviation or Error,  as the main building blocks (all other indicators in my opinion are lagging). When you compare the range between M1 and H1 you will get a different range. So how can I expect to get similar “good” results. Now I say “good” because I understand the profit must be different but “good” might be defined as, well… I’m not sure at the moment Win/Loss ratio?

    On to your 2nd point:

    one system that works on DAX should work in a similar way on MIB, CAC, SP500 etc…

    While each of these are indexes they do have different demand and supply, or lets say volume and are at different price levels. How can you expect them to trade the same. Again I refer to the ranges and volatility? Here is an extreme example but I think it’s still valid during the tumultuous Greece bailout negations there was much greater uncertainty and the riskiness and volatility especially to Europe indexes and probably more to DAX than others and definitely more than to the ASX200.

    So I guess this relates to your point about not using 1% for the stop loss but instead try to use some form of volatility indicator. When I look at GBPJPY I see one of the highest volatility pairs. So if I try to code something that will work for GBPJPY  it won’t even trigger for another pair let’s say EURGBP?

    If I go the opposite start with EURGBP and then apply to GBPJPY it will trigger but will most likely make losses. So how can I address this? Should I come up with complex conditions to try and test for which pair it is and then make entries if it matches? But then something strange happens that’s unexpected that as much as I tested for I didn’t expect that to occur and I get a loss that I shouldn’t have. Of course if this is a once off loss it shouldn’t be a big issue but overall I see it as complexity with little added value.

    The illustration of the train from Milan to Rome is great but I don’t think it fits within my theoretical model because in this situation the train is the same, the track guage is the same, the only difference is the train stations and location, so for me Berlin to Munich would by like a EURindex (inveur) if this is one index, but not 2 different indexes?

    #7795

    @dwgfx

    What would make a strategy that fits the same way on all forex pairs is a strategy made of currencies strength. It’s a path I have followed but give up while I need PRT to be multisymbol first.

    @david-1984

    About the breakout strategy, it’s only price action, no need of ATR and that’s the main reason why it worked so good in the last years.. and why i’m so confident in this one. If it’ll fail, well.. we’ll go further to next TS 🙂

    #7802

    @dwgfx… ok I understand your point of view… but now how do you decide if a TS can be used with real money? You create a TS, you test it without fitted parameters, you see that it makes money with a good ratio… what does convince you to go live?

    With only these rules almost all TS would be good for trading with real money but until now I found good for real money only Nicolas CAC Breackout system.

     

    #7833

    @nicolas

    Can you explain further what is multisymbol?

    @david-1984

    I believe the way to commence trading a live account with a TS requires a set of statistical parameters to test for which I haven’t yet completely decide upon but they should be statistically significant and reliable which is an area I need more education in. Nicolas shared the ProOrder Breakout on French cac40 of his account results and he begins to talk about using Excel for working out the averages of positives and negatives of the mean deviation, then kurtosis and there are other statistical confidence measures I’m sure we could use.

    With the basics of PRT detailed report it offers win/loss ratio, number of  trades, consecutive win/loss and you need at least a minimum number of trades (how many? central limit theorem) to test for statistical significance but these is a real limitation with PRT lack of back test data. Within these parameters you should have a basis if the system should be run on a live account but I believe all systems have a limit until they must be readjusted because the market conditions are always changing. You can try and build this in but there’s no set and forget. In life there’s always maintenance and continuous improvement.

    If you create a TS and decide that win ratio should be >= 75%, consecutive loss should <= 3 and you find that that it starts to under perform these criteria you have to readjust your decision to run the TS live and or parameters on the TS.  I haven’t even address profit here. Well Profit is very ambiguous because profit is based on your account size right? and the value of your positions?

    I know I haven’t been able to answer your question on closing a trade with a volatility indicator or something like that but I have a bias against this idea. I much prefer somewhat set % or pips/points because to me they are more consistent and I like consistent over profit maximisaiton. I might build a TS and if profit maximisation is the leading objective this might be the biggest failure. This is probably a common mistake by traders because they compare profit with the risk free rate of a bank account, long time stock investment or a traditional business model, 1 buy this item and ad my mark-up and make a profit but no losses or limited losses. Then they try and make very high profit and the cost of making a more reliable TS. I would say we need to look at robustness of a TS based on what I’ve stated above with consideration of money management (even if I recall somewhere Nicolas believes this is not the #1 priority – I could be wrong), be it the size of positions or the number of positions relative to our account size, and how frequently the TS trades.

    Would you like to have a TS that generates:

    A. 1% profit per month and executes 15 trades per month or

    B. 3% profit per quarter and executes 1 trade per month or 3 times per quarter?

    Then if you can have 10 of these TS running each achieving A. how would that look for you?

    C. I know the real answer is A with 100% per month 🙂

    #7852

    money management is important but in my opinion must be setup in a very good way because if we just increase the number of shares when we reach a particular gain it increase only the risk.

    I would like to have a TS that can perform in a long period like it has performed in the past. The real problem is how to calculate this.

     

    I make you an example on  ProOrder Breakout on French cac40. In the last 3 days it got 4 stop loss and 1 trade closed with losses. What does it means? T

    he system is not working well anymore?

    The system is doing well but during these days the market was not usual?

    It is a normal drawdown?

    What else?

     

    David

     

     

    #7875

    The fact is market not mechanical and the breakout strategy is not as ‘mechanical’ as what we think of, because it doesn’t rely on indicator behaviour calculated over the same last x periods at each new candle. It only deals with a market spread between 2 hours. Intraday trading is a battlefield, and I have not met a lot of TS that came across in a long period so well, believe me. Also, since this strategy deals with let it run profit, it suffers of kind of mean reverting phenomena in intraday that don’t sweet this strategy actually.

    David, would you mind post it here your Excel spreadsheet result of your strategy posted here : http://www.prorealcode.com/prorealtime-trading-strategies/supertrend-against-trend/

    Please add spread in it. Just drag and drop the Probacktest result in Excel and upload the file here, thank you.

    #7878

    Added the spread and done it for CAC, MIB and DAX. For both: optimized parameters and standard parameters.

     

    Let me know if this is enought and if you want also for other indexes.

    David

Viewing 15 posts - 1 through 15 (of 41 total)

Create your free account now and post your request to benefit from the help of the community
Register or Login