What is considered ‘Good’?

Forums ProRealTime English forum ProOrder support What is considered ‘Good’?

Viewing 14 posts - 1 through 14 (of 14 total)
  • #213115

    Hi All,

     

    Very newbie question here – what is considered good performance for a system over a 12 month backtest (for example)?

    Does ‘good’ vary depending on market/index/stock?

    Just trying to get some level of benchmark from the stuff i’m experimenting with.

    Cheers

    #213124

    What about : Have similar profit as you would be able to do manually. Or is that too daft for an answer ?

    The more intelligent answer could be : take care that your backtest results work out in Live to begin with.

    I have more answers. But this is a start ?
    Regards,
    Peter

    #213149

    Thanks for the answer Peter.

    I get what you’re saying, and at the moment I’m only testing out a system in demo to see if it meets expectations of the backtest.

    In instances where I’m experimenting with new systems that I’ve not tried manually, I’m trying to gauge if I’m settling for a level of supposed profit, when most people wouldn’t accept that as a successful system, and would move on. For example – let’s say I’m sitting here thinking that a system is decent if it’s achieving 1.15 P/L ratio, but most people would scrap it and move onto something with a far higher ratio. (arbitrary numbers). I’m just wondering out of interest – I’ve never spoken to anyone about automatic trading systems before.

    Many thanks

    #213151
    JS

    Hi,

    As you probably know, there are several important key figures that must have at least a certain value:

    Profit/Loss ratio, when it is equal to 1, the profit is the same as the loss, so a good ratio must be above 1 where 2 is already a very good ratio…

    %Profit Trades, 50% indicates that as many profit trades as loss trades have been executed, so a good percentage is above 50% and 60% is already good…

    P/L ratio and %Profit trades cannot be seen separately, if your P/L ratio is for example 2 then %Profit trades may be lower than 50% because the size of the profit trades will exceed the size of the loss trades (you are still making a profit)…

    Another important feature is the “Drawdown” or the maximum continuous drop, you always want to keep the DD as low as possible because otherwise it would be possible for your DD to wipe out your entire account… So, the lower the better…

    As mentioned, “good” for everyone is different but with a P/L ratio of 2 or higher and a %Profit Trade of 60% or higher and a DD of 15% or lower I would be satisfied…

    #213154

    A couple of weeks back, I wrote out this : re: Which parameters to tune when developing automated strategies?
    Parameters like JS talks about are in there too, written from a personal perspective.

    The DrawDown “parameter” is not mentioned there (that I can recall), but I know from experience that for a normal system something like 10% is great. So in that post I linked to, I could have told that this 10% is something to strive for. Why ? because I know it can be done.
    (normal system : a system which does not excessively deal with money management – not for protection, nor for exponential growth)

    Important : Whatever you read or encounter from past posts, it is to be interpreted as what people experience while running backtests. 99.99% of people who show backtest results, show this as a reflection of what they hope what will be (in Live, ever) and are all not realistic at all in my view. 0% of people who made that into something running Live, show those results; it would be a moot thing, though it could be shown for showing off (this is all subjective to my thinking).
    Then where is that remaining 0.01% ? well, I guess those are the sellers on the Market Place. They have to show something or else they don’t sell anything. Personally I never saw something which even is remotely close of what I would like to see from an AutoTrading system.


    The people who really apply AutoTrading systems in this community, and I mean, who do that blindly and just let them run, can be counted on one hand. Or else let all those others speak up (please). So for what it is worth for you or other readers, the below could be my answer, derived from systems which really are applied for AutoTrading. The figures count for each instrument because it is something to strive for, again because I know it is doable. It is only that each instrument requires its own very dedicated attention (think many months, once you know what you are doing). So :

    Revenue in one year can be 100%. Thus, invest 10000 – get out 10000 extra in one year.

    If the DrawDown is taken into account, the money at hand (portfolio) becomes unimportant. I mean, would the DD be 10%, then the net loss at any one time can not exceed 1000. But would it occur, it would occur right in the first trades of the system being live, or else it is no loss.


    So “good” for me is 100% revenue in 12 months.
    Good for someone else may be 10% or less. At least that would be a more normal figures. And what to do if the programming skills are not optima forma and you’re just not able to get at that 100% “level” ? then 10% really would be good enough.
    So you see ? there is no one answer to this.

    And never forget : the backtest must reflect reality later; it is the most easy to optimize the system to death and reach way more than 100%. And you wouldn’t even know you over-optimized it.

    1 user thanked author for this post.
    #213223

    This is extremely interesting; thanks to you both for your thoughts, and time to give such thoughtful responses. I know i have a lot to learn in developing automated systems, but i think i’m unlikely to ever be in a position to just let them run, unchecked, each day. I could be wrong….

    But this does give me some idea of benchmarks to work towards.

    It seems that knowing when a backtest and reality are aligned is a bit of a challenge – how long do you leave a system operating on a demo account before unleashing on LIVE? Is a demo account even a fair test (wider spreads, and larger SL needed, etc). Or is it better to run a system on LIVE but with a minimum trade size, to test it truly.

    So many questions….!!

    #213230

    I am a bit reluctant to post this because it seems to have a negative stance. Still it is far from that as it is the greatest hobby I ran into throughout my life.

    Hopefully some decent answers :

    I never test on Demo but for maybe not common reasons :

    1. For me this is a waste of time.
    2. Demo exhibits a lot more System Stops than Live (say 10 times more). N.b.: A System Stop is that the AutoTrading System is killed from the server – you will have to restart it.
    3. Demo does not force me to do it right.
    4. I calculate-in $ for failures (which are my own but which are also mainly broker errors; sometimes these are errors from PRT which relate to broker errors, but these are more rare.

    Ad 2.:
    This is harder to explain without many pages of writing. Almost nobody knows this because relatively few people run systems on Live and compare.
    There is no usefulness in working around problems which exhibit in Demo only. This, while it takes months (you have my guarantee) to get along with Live and work around the issues there. Issues : again stopped systems. Some issues can’t be worked around and will stay forever. You win some on it, you may lose some. N.b.: A System which is killed may leave you with a running system which is in a loss and which you can’t control automatically any more. Now its your turn again as a manual trader, but hey …

    Ad 4:
    Following the latter text, it will cost money because of “errors” (I don’t call those bugs because it is merely bad behaviour of the broker – this forum is full with examples (not for this topic/thread)). But it surely will cost money because of your own bugs or things you did not take into account.
    A not-so-nice part of it all is that we have IG for broker and we have IBKR for broker and both have their own issues while both can not be dealt with by PRT decently. But, we can work around it, once you have experienced the issues. This will be different for everybody. And it is (thus) also different for IG and IB; when I had IG up and running and covered for all what I saw happening “to me”, I could start over with IB. Luckily I have a vast experience with IB and manual trading – I could see the issues coming before IB Autotrading was even live (now 3 months or so back). The time involved is almost immense, because it takes so much throughput time. I mean, apply your solution and retry – and wait for the situation to occur and see whether it helped. Ah, it did not help ? then undoubtedly you will lose some money again. Oh, try it on Demo first ? no no, because there your system will blow up because of “demo reasons”.
    You may well think that I over-exaggerate things, but all I do is do “more” than what is common. Still in the end it is your prospectus.

    Ad 3.:
    Only because you work with real money, you are forced to think in solutions, instead of letting it go and do something else. At least for me it works like that.
    Maybe someone recognizes this : when you perform a dangerous hobby or sport, this can’t be practiced outside of the real game. I know this from driving Ralleys; there is no way to practice this as you will drive slower in order to avoid trees and such. With the real thing you don’t think of trees (and just run into them when it so happens). And so the practicing occurs when performing the real thing. Do it more and you will eventually know more and be better at it. This in itself costs money; it must be calculated-in.

     

    how long do you leave a system operating on a demo account before unleashing on LIVE?

    Still it is a valid question, or at least a decent answer could be useful. Here is my attempt :

    With the so-called knowledge that waiting for Demo to work out is a serious waste of time, plus the other arguments, the skill has thus to be in backtesting. Well, I suppose one could graduate on that when one has that under control. Of course I think I do, or else I should not be writing this. Still, mind you please, it is FWIW.

    I don’t know how many pages the book will be that could explain the pitfalls, what to look at, what to avoid, how to teach yourself not to over-optimize, how to see that you did that after all while you were sure you did not … I guess that book will be thick. At least I can honestly say that I am still learning new things on a say weekly basis, which is quite crazy for someone who is working on it almost full time, for at least the past two years. These two years can be extended by another four, if I take into account the weekend-work I applied the first four years. Anyway, six years in total and I still learn.

    July 2022 was the first time I let go the first system on Live. This was not yet unattended, but at least I dared to do it. Next this required updates each week, sometimes more often because things plainly went wrong. Anyway, the versions of that system are “uncountable”, but let’s say close to a 100 – they all ran Live. Today they run really unattendedly since end of 2022. They do their work as observation post for things which occur at the broker (IG). In the mean time I started working on IB and today this is finished (that runs unattendedly for a month or so). When this week PRT will unveil the new quantities to trade, I will dive on to that immediately (currently this is limited because in Beta).


    This seems to be for discouragement. But the opposite is true : if you want to earn some money by means of AutoTrading, then better think twice instead of thinking that you can let run any system you find in here. Take into account the many years it will take, so that you won’t be discouraged half-way. See it as a hobby and not as a means of income.

    2 users thanked author for this post.
    #213356

    Thanks Peter. This is (again) extremely useful information, and I don’t find it discouraging at all! I want to be realistic in my approach, so if that means running the systems in live, but with heavily reduced trade size, then i feel like this is a worthwhile experiment. And definitely agree that demo doesn’t force to make the same decisions as I would in live – I wish I did, but it’s just not the same.

    One of the reasons that I’m starting to look at the automation of a system is simply because I’m reasonably happy with my manual trading system, but it does require a level of consistency, that I can’t always have when I’m also working and dealing with other tasks (family etc). So i thought it would be a nice idea to try to automate it…. it appears it’s not so simple, but incredibly valuable as an experience. I was under the impression my manual system was fairly black/white in terms of rules, but when applied through automation, it’s not so clear cut. Working this through is making me see the nuances that i’m applying in my own trading.

    That being said, your point about over-optimising is showing itself in what I’m experimenting with at the moment…. for example… I found that in backtesting my system seemed to trigger losses on Tuesdays and, by removing Tuesdays from the algorithm, the performance is much better. This feels distinctly like over optimising, unless there is something significant that happens on Tuesdays that I’m not aware of.

    I’ve actually just discovered a very well performing system, by having a typo in my original code…. it took me by absolute surprise, but it seems to be an incredibly simple reversal system, which triggers roughly once a week, on average, but almost always pays off… as it was a typo, i’m somewhat untrusting of this system, but perhaps it’s worth a go if the back testing is anything go by…. as I say, it’s incredibly simply – maybe only 5 lines of code.

    #213366

    perhaps it’s worth a go if the back testing is anything go by

    I’ve also ended up with a few good Systems by just such a typo or ‘an error’! 🙂

    Costs nothing to Forward Test your ‘Typo System’ on your Demo Account for a month.

    Let us know if it works out?

     

    1 user thanked author for this post.
    #213413

    @PeterSt

    May I ask which values, i.e. profit/loss ratio and drawdown, have the systems that you run unattended? And which TF?

    #213414

    Believe it or not, but I can’t tell you. What you see below is a kind of representative though.

    Because of the number of trades which ate so limited (1500) it only goes back to just over 3 months;
    It really is painful that I don’t have this data myself.

    Today (coincidentally) I have been working the whole day to format statistics that are reliable per instrument and Trading System at the same time. In the 4th (black) screenshot you see an example of *that* (that is Nasdaq) but it will last as only as that particular instrument stays under 1500 trades – or whatever it really is because in that particular  account I was granted 10000 trades for the statistics, but it is messing all up, even limited to a time period. Thus, would I show the both instruments I have running in that “black” account, then all is a mess and nothing makes sense.

    And worse : I actually can’t tell at all what I made on the AutoTrading. It’s a bit of per daily basis remembering how things went.

    I am working on the overall solution for this, but so far I don’t see it …

    TF is 1 minute on all systems these days, but the Statistics on the green screenshots also worked with 6 seconds (I think that was in the period shown Dec. – Jan or so). The 6 seconds are too short for me to let work well.

    Drawdown for that black screenshot is $774. Don’t ask me whether that is correct – I am looking at that for the first time (because you asked). Oh, P&L ratio for that one is 1,28.
    Notice that it is dangerous to value this, thus, FWIW; this is because a. this runs only 3 weeks or so, and b. the System is changed regularly (like this weekend). But I don’t really look at it for it to maybe go wrong.

    #213420

    @PeterSt

    Mmh, I have to keep asking. Without telling your secrets… price action? indicators? With trailing stop? Wins bigger than losses? Because my M1 systems look similar in retrospect. So far, however, I have only blindly relied on my M5 systems. I just have a lot fewer trades.

    1 user thanked author for this post.
    #213425

    Hello phoentzs,

    FWIW for you, in all what I do resides a sauce of strength.  I don’t derive this from any known indicator – they never work out for me.
    The trick of course is, how to do that from within program code. Also, know for sure that this strength works against you half of the time. Thus, in 100% of cases, when a trend changes, this is usually after a strong last exhibit in the direction of the trend of that moment which can be seen in the 1 minute charts. Would I not be in, I will be at the trigger of that exhibit, in the wrong direction.
    What I’m saying is, that each streak ends with a loss (this is almost by guarantee).

    The number of trades is the key to success. Without that, I would be nowhere (nothing to measure – nothing to tune).

    I don’t use PRT trailing, as this is not (yet) possible with PRT-IB. But besides that, it would be too “common”. Please notice that the Trailing – when done right (can I do that ?) – gets you twice the profit easily. Thus, this is totally crucial.

    Still the wins are lower than the losses, but his is not easy to express “as such”.  I mean, while I could say that the SL is maybe twice the eventual TP (including Trailing), it is quite rare that the SL level is reached, because first Strength again indicates to go out or even revert. And now I don’t know the answer, as usual, because that answer requires long term analysis of the same System. Example of this exorbitant environment (of programming) : my Nasdaq system I showed yesterday from the past 3 or so weeks, was without the Trailing being in order in decent fashion. I only made that this last weekend (it was a pending job for a month). Now it is active – so ask me again next month. And see ? I truly can say “next month” because of the number of trades.

    Another example and hint : If I discover a means to excessively extend the number of trades, I right after that focus very explicitly on the Trailing again. This is logic (to me) because the more trades, the more Trailing will bring $. It could easily net outperform losses otherwise.

    Regards,
    Peter

    2 users thanked author for this post.
    #213545

    Hi,


    @Peterst
    Thank you for your valuable feedback.

    I am almost ready to go live with 2 strategies that I will run with micro futures of course as I am not expecting to reproduce the same gains as they did in papertrading instantly.

    What would be your recommendation in terms of trading hours? Overnight fees look crazy on IG.

     

Viewing 14 posts - 1 through 14 (of 14 total)

Create your free account now and post your request to benefit from the help of the community
Register or Login