Optimization results different from backtest
-
Hi, since yesterday when I do optimization and replace the values of the optimization into the code and backtest the results are vastly different. Any explanation to this? I did a optimization on a simple MA crossover with SP500 mini. The optimization shows profit of $455 with the SMA 40 cross over SMA 55. When I replace the opt values in the code the results show loss of -$29. See code and results below.
// Definition of code parameters DEFPARAM CumulateOrders = False // Cumulating positions deactivated AVG1 = Average[55](close) AVG2 = Average[120](close) c1 = AVG1 > AVG2 Possize = 1 // Time Management CTime = Time >= 20000 and Time < 220000 // Longs IF Ctime and c1 Then if Average[40](close) crosses over Average[55](close) Then BUY Possize CONTRACT AT MARKET LE = 0 ENDIF Endif // Stops and targets SET STOP pLOSS 70 SET TARGET pPROFIT 70Thank you Johann.
It is not about the variables obtained fixed in code, it is about the iteration of at least y which internally fails. For now too difficult to explain in easy terms, but it looks like an internal rounding issue (I’d need to prove that). But summarized : when the variable y is set to 40, the result is presented of 50.
To me it feels like the 40 value is internally set to 40.00001 which then is rounded to 50.If you set x to 15 (just do that in the Optimization Parameters window) and let iterate y from 30 to 50 with a step of 1, you can better grasp what I am sensing.
It requires some more digging to provide PRT with the real culprit. I will try to do that …
Exact same for me on v11 …
- Lower curve is the optimisation result.
- 2nd curve up is the opti result for x and y entered as fixed in the opti table.
- 3rd curve up is the opti result for x and Y entered in the code and opti variables deleted from opti variable table.
What am I doing different than what you are doing or vice versa?
Hi PeterSt, ok so I run the test on 50k with and without the c1 condition. The first test I did without the c1 I kept the opt variables x and y (5 – 60 with 5 iterations) and (10 – 100 with 10 iterations). The result I chose was a combo of x = 20 and y = 40. The results after I replaced was different but not as much. Then I tested again but this time I fixed x as 20 and only tested for y (10 – 100 with 10 iterations) and it gave me 40 again. This time around when I replaced y with 40 the results were the same as the optimized result. So it makes sense that there could be a back the moment you have to many variables with iterations. Which is a shame because it makes it a lot harder. I have never experienced this before…
I did bring back the c1 condition and followed the same principle testing it once again with two variables simultaneously with iterations it was not accurate…one had to be fixed for it to be accurate…
Houston …
I don’t know what I did or where it happened, but since the last 20 minutes I would say “Johann, what are you talking about ?!?”. Meaning :
Suddenly it works correctly and whatever I do, I can’t let it go wrong any more. The only thing I did not do yet, is reloading your .itf from disk. … And that I now did too. It does not help.
So Johann, what are you talking about eh ? hahaha.
Unbelievable. There is something wrong beyond imagination. I must give up on this now ?I recall that it started to work after I had obtained a Graph command, which was after I tried to make a trading system for ProOrder of it (see second attachment). Notice that both do things;
The latter has issues to begin with, because look at the x and y you see in there – they are both at 1 and spring from the original loading of the System. They were at 1 just the same when I attempted this ProOrder stuff an hour ago, while long gone I had set them to different values (you can observe that in the first screenshot as well, though this is after reloading the not and not after one hour of messing about). So do notice that if I would hand this System to ProOrder right now, it would use x=1 and y=1 which is always terriblyt wrong of course (because I have filled x=15 and y=40 in the parameters. OK, see third screenshot where you have it combined. N.b.: This can be fixed by exiting the Editor and reloading the System hence Editor again.
Side note : I never gave PRT this in itself gigantic bug because I kind of gave up on reporting bugs.
And so my guts tell me that where I originally could copy your behaviour Johann, possibly something in the realm of ProOrder reset stuff, but quite definitely. But look how difficult it is and how may bugs a man has to conquer before he can do his thang :
I removed the SP500 System from the cloud stuff (“Remove Backtest”) from the list of systems I have in there – Saved and Exited PRT and Restarted it.
Back was the system. Just like that, including the presenting of the backtest. So it also requires to be removed from the Chart. OK, done, Remove it from the list of systems again, Save – Exit – Restart. Now all was finally clean.
But now there was so much hassle in between (including the restart of PRT a couple of times, which I already did outside of removing the system) that all has became moot.The Graph command is special because I am fairly confident that PRT on their servers cache things with that – or around that or whatever. Graph has given me so so many times issues and all we can do is create backdoor solutions to overcome it. And YES that includes strange backtest results, just because things error out quietly internally. But this was the other way around … I added the Graph command.
Anyway since adding that command or otherwise going to the “Automatic Trading” tab (2nd attachment) things suddenly were correct – I did both things at the same moment of backtesting.Crazy.
- You must be logged in to reply to this topic.
Summary
This topic contains 13 replies,
has 3 voices, and was last updated by 
2 years, 2 months ago.
Topic Details
| Forum: | ProOrder: Automated Strategies & Backtesting |
| Language: | English |
| Started: | 11/28/2023 |
| Status: | Active |
| Attachments: | 13 files |
