robertogozzi wrote:

About history for arrays, I think it’s impossible to achieve. In a 200K backtest (not to use 1M) with just, say, a 100-element array, more than 20M (32+ bit) memory slots would be allotted to it, which is impossible to achieve.

Yes, I agree with your general stance. However …

This assumes the most-normal PRT procedure that we would want something like $MyArr[30[200]] (element 30, 200 bars back). But this approach IMO is just not how arrays should be used. Or maybe better : this is not how you’d want them to be used, if they only would live normally across bars *plus* if we’d only use them “normally”. Thus :

LastUsedElement = LastUsedElement + 1
$MyArr[LastUsedElement] = BestSignalForThisDay   // Occurs once per day.

implies one new element to occupy and with the already allowed array size of 1M, it would last 1M days, right ?
However, PRT-habit(-coding) would be something like :

LastUsedElement = LastUsedElement + 1
$MyArr[LastUsedElement] = Close   // Occurs each bar.

// From here sort out the highest occurrences per day which ...
// ... which already can be done without arrays.

Please keep in mind : I made this array to live across bars (in my mind).

… And *that* thus does not work, because even without the possible [30[200]] desire, it already goes wrong after 1 month and the 1 second TF.

In my first post I said “there is much more to it”. Actually so much so, that this won’t ever happen. Still nothing is lost when the [30[200]] syntax just would not be allowed. And well, currently that is not allowed already (because the array lives the current bar only to begin with).

In the end all relates also to this (I referred to that in my first post as well) :

Once BarCount
BarCount = BarCount + 1

If Close[Barcount] > close then
  // I apparently now have a lower price than when my Strategy started.
endif

Can someone tell me what will happen with that when LIVE ?
OK, people from PRT Development will be able to tell. But not “us” because we did not even attempt it (me neither, afraid of what might happen after … how many bars actually ?).
But possibly dev people thought of this and it might even work. Mind you, the backtest data of 1M bars is only a commercial thing. The data is just there. Especially the data which builds up during the strategy running. … But it conflicts with our thinking and (about) backtesting. Hey, we have 1M bars there, not more. … But in Live we logically have already because our code passed them all (if my s1 code runs for 12 months it has seen 12M bars).

On a side note and thinking “machine learning”, indeed I am building up my own averages (not moving averages but normal averages). Or say “statistics”. Just because I can predict it won’t work otherwise … Say lowest lows an highest highs; possibilities to take into account. Thus for example, what would the S&P500 report for all time highest high ? Probably nothing much with my 1 second 1M bars. But my program would have known it, if it only keeps on registering the all time highs. And maintain that in an array as how I would propose it ? how many elements would it consume for the past year ? maybe 20 ?

GraHal wrote:

Using, for example, the last 60 x 5 sec TF bars; is it possible to set up an array (via the code) that allows the array to be scanned / queried (correct term?) to provide the value of the highest[60]close and / or the lowest[60]close?

You hit the nail on the head by using the term “query”. Or, what I would like in these modern times, a literal query as how it could be used in the SQL language to dig out certain values and elements from the array. But *that* is not going to work, because it is too far out of line for PRT. So what remains is :

Sure GraHal, that is possible, but by means of scanning the array (looping through all of it, you remembering the Highest etc. you run into). But if anything is a waste of cpu cycles, it would be that.
Now observe Roberto’s answer. It may not even come to his mind to say Yes, because it would be a stupid thing to do, as it is already possible with the best “query” possibilities – those which PRT provides natively. In this case the Highest and Lowest functions.

I don’t keep track of anything, but the past year I read something like the most basal functions (i.e. all of them) like Average, can not be let lose on arrays YET. This read like “in due future it will be possible”. And why not. Maybe this is now already. And a bit difficult to try for me because I don’t know the syntax which can be daunting to guess in this (array-)case.
Thus without such functions being possible for arrays, we’d probably need to do everything ourselves. And if *then* you are as “smart” to store the signals directly like I tried to work out in the very first example of my first post, well, then I can’t cope. So *then* this literal query language would be a requirement, that even combining the two required arrays (because only one dimension possible) without hassle for the user (us).

GraHal wrote:

Above is how I imagined / hoped Arrays worked.  I can see now that Arrays have limited use in PRT, partly (or mainly) it seems, due to the slowing down of the already tortoise like speed of backtesting etc.

GraHal – Well, you are not even referring to the fact that using arrays in general slows down significantly. That is, I don’t recall me mentioning this yet. OK, when accessing undefined elements, yes, but in general using arrays indeed is way slower as well in comparison to using normal variables.
I think I implied that by telling about the general setup regarding, for example, the unnecessity to Dimension arrays in advance. Or that you can randomly assign an element a value instead of subsequently (in a row). So these issues are not really the fault of how we (the user) use it but how it under the hood has to work (I think I referred to mimicking), which in itself (IMHO) is smart but slowww.