With above, I just did a quick mash up of an existing code, so not a valid test of Arrays. My understanding of Arrays is (using an xls to illustrate) … imagine a spreadsheet with numbered rows; after each bar the Close value is entered into a cell then next bar / close value is entered under the last entry … repeat etc. When required, a query of the Array (e.g. filled cells as above) can give HH, HL, LL, LH value over x bars. The clever bit is to code into a strategy to determine if price has passed the Highest High and price is now hitting a Lower High (a High lower than Highest High). And so on, so as to determine if the uptrend has ended. Anybody agree / disagree with above? Can any wizard code it? 🙂
(emphasis is mine)
If it only were true that it would work like that. But it doesn’t and I think it can’t.
What you suggest/imply there, @GraHal, is that the array lives over bars. As in
1
2
3
4
5
Once$MyArray// Syntax error
Once$MyArray[0]// Syntax error
Once$MyArray[]// Syntax error
On a side note : and *if* it would work like that, you’d get completely crazy of the fact that arrays can be one dimensional only. Thus, whatever you do, you would need two arrays or more. Think like (not tested for syntax, thus mere pseudo code) :
1
2
3
4
5
6
7
8
9
10
11
12
13
14
IfHighest(Close[20])then// whatever this does.
ElementToSet=ElementToSet+1
$BarReference[ElementToSet]=BarIndex
$HighReference[ElementToSet]=Close// Register highest price of last 20 bars at this moment in time.
endif
Fori=0to$BarReference[LastSet]
HighestBar=$BarReference[i]
HighPriceBar=$HighReference[HighestBar]
HighPrice=Close[HighestBar]
next
// Anyone who can understand this is crazy.
// Anyone who can code this properly *and* understands himself, is ready for the mad house.
The worst part is that this can easily be done without array, and then we suddenly understand.
But meanwhile notice what I attempt there : Writing (for me) useful values in array elements, without consuming an 1M-elements array. This
a. Makes it incomprehensible
b. suggests that the array lives across bars (which it does not).
Conclusion of that :
There is no use to “save” array elements like I suggest doing there, and having them 1:1 with real bars (BarIndex) would
– make it understandable indeed
– make it a useless exercise because we already have that array (it is named Close[]).
Working with arrays in PRT is mind boggling, or it is me who just can’t cope.
Anyways …
I personally use arrays everywhere and my whole life, but not within ProRealCode because there is no need to for me. This could also relate to actually everything already being in an array (or being presented as one), so that stuff like
1
2
3
4
5
6
7
8
// Preserve the last 4 bars' gain.
Gain4=Gain3
Gain3=Gain2
Gain2=Gain1
Gain1=Gain
Gain=ThisBarsGain
should better be done in a nice array, a small loop taking care of this. But this too is useless once you learn that this does all tricks already :
1
2
3
IfGain>Gain[3]then// Gain larger than the gain 3 bars back ?
// All still OK.
endif
So no definitions because PRT does this internally for you, and all is accessible in “array fashion”.
Summarised : I am not debunking the array feature, but its real use-cases for me would occur when
they live across bars;
they are at least two dimensions.
The latter is important so we could attach the value we want to preserve to the bar where it occurs in one go, which combines with not maintaining array elements per (history) bar.
1
2
3
4
5
6
7
8
9
10
11
12
13
IfHighest(Close[20])then// whatever this does.
ElementToSet=ElementToSet+1
$HighReference[ElementToSet,1]=BarIndex// In case we want to know the bar where this happened.
$HighReference[ElementToSet,2]=Close// The price where we detected this (whatever) High.
endif
// Find all the "highest" in subsequent elements of the array.
Fori=0to$BarReference[LastSet]
HighPriceClose=$HighReference[i,2]// A high price occurred.
HighBar=$HighReference[i,1]// Use this at convenience. For example :
PricePriorToHigh=Close(HighBar-1).// This happened right before this high.
next
So never mind this makes sense or not, at least I am now able to be creative and let it do something without being continuously disturbed by thinking of what I’m actually doing or even trying to do.
We should not hallucinate over arrays going to live across bars (hoping that PRT is going to develop that) because I foresee all kind of hurdles regarding referencing bars in the past which are of a too far away history. Thus, if I would be able to store “events” (like my Highs) in an array in subsequent elements and once per month such an event occurs with my 1 second TimeFrame, then personally I would noy know what happens when my Live Strategy is 5 months in, and I still refer to a 1 second bar of 5 months in the past. I know, this is a general problem already, and the topic with the question “I save a barindex in a variable in order to refer to it as “Close[myBarFromDec2019]”, I postponed for later – but sure is due. IMO these things can’t work and my Live Strategy will eventually break (error out) or possibly come up with 0 values which ever back were not 0. Anyway :
An array would be useful for this, if I would only store the resulting values of Close[myBarFromDec2019] and never the index to bars. Thus $MyHistoryArray[element 773] = Close[0]. After 3 years I will surely be able to access element 773 and its stored value. That is, if my strategy still runs, which chances are small either because of error or me being broke in that portfolio. And restarting that Strategy will have lost the value(s).
There is really more to it all and I think that PRT does not have the infrastructure to let this work as I like to. But I’ll survive. 🙂
Peter
Peter … how is attached working? Have I fudged it to get positive results and it’s not using an array at all?
Credit goes to the original code Author (?), I can’t remember to what extent I forked it.
Well, you surely almost fudged me !
😉
In a 1M backtest it emptied my bank account prematurely on Dec 10. Luckily the stockings had been filled already, as this is past data with no guarantee fir future Christmases.
At least the Risk/Reward has been 3,61. But my Reward looks to be being broke now.
I have been working on such a strange thing before; I didn’t understand what was happening, but it won and won and won. I ran into it by a mistake I made. After one day working on it, it appeared to be pure coincidence that it worked in a certain stage. I recall that the equity curve looked as strange as the one you see in the 3rd attachment (this is from this DJi S15, 100K, although I think the 100K units for S15 fail on me – 50K, 100K, 200K all perform exactly the same).
I am sure it is caused by the array used. Haha.
But no, I don’t even understand what it does. Sorry !
It seems that it tries to cover for an earlier drawdown … until that breaks. Double or nothing kind of.
Btw, I recognize from the sudden slowness on 1M bars that it most probably fails on accessing undefined array variables. So yes, instead of providing an error, PRT happily continues and you won’t understand a thing of it (again, Peter ? 😉 )
Something else : careful with judging what actually happens. Compare the two situations below. Each suggests that this is the result of 4 trades, but it isn’t …
Instead it semi-closes the last position because (somehow) the data won’t proceed in real time (I did not put a limit on it). Also see the vertical white bar in the equity curve in the left. Thus, each time I run it, the time you see in the 3rd attachment increases (now 09:58, the previous run of the same 09:50 or so). Thus, this trade is open since Jan 7, and with that the gain is only a little (subtract the 1729).
Something else is, that apparently it is a good idea to go Short on the DJi on Jan 7.
Arrghhh, too late now !
Peter, have you got an analogy of how Arrays work that we can understand please? I can’t get my head around arrays living across the bars? 🙂
Extending my analogy using .xls, if we visualise an .xls in 3D as attached, is that what you are trying to say in your original post as to how data is / can be loaded into an Array?
Data does not have to be loaded into 3 dimensions, it can be 2 dimensions … as my rows and columns xls analogy?
Peter, have you got an analogy of how Arrays work that we can understand please?
First off, this first part is what Nicolas will have explained in the main topic about arrays as well, though probably with different wording or perspective. I will most certainly add some kind of technical perspective as how arrays would not “behave” as usual, which could be a problem for those familiar with them. Thus, not to improve on Nicolas’ explanations :
An array with its subsequent elements, the first element starting at 0 : I sequentially put Prime Numbers in this array, just so there’s some logic in the sequence.
So these 11 first prime numbers have been stored in this array (we could call it $Prime) and instead of calculating them if we want to know the next one to 17, we could go (loop) through the array, wait until we run into our target (which is 17 we’ll find in element 6), go to the first next element and find 19 in that next (7th) element.
Difference between PRT and the mere real world, regarding this
In the real world an array is “Dimensioned” in advance, because it should not reserve unnecessary memory, so you could have a command like
Dimension $Prime[100] // Reserve 100 elements for 100 prime numbers max.
PRT allows to use arrays without dimensioning them. This will mean that accessing them is a simulated act, making the access to them very slow (this is so indeed). The technicalities are beyond this post and usefulness, but let’s say that this is to our convenience that this is so (unless you’d need sheer speed – ahem).
In a similar fashion, in PRT we can write (to) array elements while skipping smaller numbered elements. Thus, if I like my next element in $Prime to be #30 then I’d simply put
$Prime[30] = 97 // Just a guess that this would be 97.
At doing this (why would I want it ??) this poses the problem that at looping through the array I’d run into element [11] which is not defined (I never wrote to it). And thus PRT allows this to error, but without erroring-out – you wouldn’t notice it. You may also not notice that the contents is of no value to you, while expecting the next prime number there.
(ouch, it appears touch for be to stay objective)
More in a next post … (knowing that I also need to pay attention to my backtests 🙂 and some markets).
Very generally, the usefulness of arrays appear when you need more than 1 value of a same kind but for different instances (this is not an official description).
which would give endless lists, without even being able to nicely loop through them.
For PRT we can obviously compare this with a long list of constants (like “hot” seconds in a 5 minute TimeFrame, etc.), those seconds implying an action – could be buy, sell, or stay.
Thus suppose we would comprise an array of 300 elements, each element telling about its relative second whether it’s a buy or a sell, and the Time “constant” tells us that we’re in second #23, we could look in our $BuySell array whether it contains 0 (stay), -1 (Sell) or 1 (Buy).
Of course we could have a separate array for Long and Short.
The purpose of such “constructions” is always the same : have unambiguous code which in a line or 5-10 does all without numberous If – ElsIf ‘s and without 300 OR’s. Thus for this example, if you’d convert the second number in the m5 TF, you can refer to that in the array its element descriptor like
1
2
3
4
5
6
7
8
9
// Currentm5Sec was determined to be 23.
Currentm5Sec=Currentm5Sec-1// Because the Array starts with 0 and we behave like the real world (fill array without gaps).
IfNotOnMarketthen
If$BuySell[Currentm5Sec]=1then
Buy100SharesatMarket
elsif$BuySell[Currentm5Sec]=-1
SellShort100SharesatMarket
endif
endif
Of course defining all the entries for the array may still take 300 lines of code, although the Sell and Buy signal elements could be sufficient, now NOT writing to the elements subsequently (try to think about this yourself). Anyway, the above few lines are now all the lines for each of the 300 seconds to deal with.
So now you’d have a real application for arrays, but look at Nicolas’ examples as well.
So what about that “does not live across bars” thing ?
As we (hopefully) know, variables in PRT can be defined in the program code, but when the program ends, the variables diminish.
Our program code is always called at the close of a bar, be that a 1 second bar, a 1 minute or a 10 day.
At the beginning of our code, variables have no value.
Sadly PRT allows us something to refer to them without that we have defined them for a first time (give them a value), which is such wrong behavior (this is not subjective) that many people will never get to understand how this “calling of our code” works. Anyway try to remember :
Our code is (without exception) called per close of each bar, and each time of that call all is fresh and new.
EXCEPT
Except when we “declare” a variable with Once :
1
OnceMyVariable=0// or 1 or whatever you want.
This means that this would happen :
1
2
3
OnceMyVariable=0// or 1 or whatever you want.
MyVariable=MyVariable+1
GraphMyVariable// Shows 1,2,3,4,5,6,7[,...]
And this contrary to this situation :
1
2
3
MyVariable=0// or 1 or whatever you want.
MyVariable=MyVariable+1
GraphMyVariable// Shows 1,1,1,1,1,1,1,1[,...]
Now what I was saying earlier on, is that an array can not be declared with Once. And for this reason it will always lose its content at the next bar’s call.
In other words :
Would we like to use arrays e.g. for building up Artificial Intelligence data, then this is not possible. Self-learning is possible, but only by means of normal variables, declared with Once.
In other words, supposed we would like to capture/register monthly averages or something (mind you, in a LIVE strategy) then all we can do is
1
2
3
OnceJan21=ThisMonthsAvg // Whatever we calculated.
OnceFeb21=ThisMonthsAvg // Whatever we calculated.
OnceMar21=ThisMonthsAvg // Whatever we calculated.
Registering that on the monthly basis.
But this is moot anyway, because if our strategy quits for some reason, the data will not be available to it at a next start, unless we keep track of that with backtesting in parallel (always capture the last month’s performance of “whatever we calculate”) and feed that data to the program code, to be used in a next startup of it.
Obviously it could be useful again to put that in as static data in an array, which obviously can be done, just like our array with Buy and Sell signals (not that I told about that explicitly, but notice that that too would be “defined” over and over again at each bar call, millions of times (think 1 second TF) while one really should be sufficient. Thus, with normal “Once”d variables it can be done one time only, but with arrays this is not possible (that I know of).
About history for arrays, I think it’s impossible to achieve. In a 200K backtest (not to use 1M) with just, say, a 100-element array, more than 20M (32+ bit) memory slots would be allotted to it, which is impossible to achieve.
Even if users’ram were used, nobody can have TBs of ram!
But users’resources, if available, could only be used for backtesting, not for autotrading, as this would imply keeping users’PCs on 24/7 to allow trading.
.
