Arrays and how they can not be used (?)

PeterSt · Wed, 19 Jan 2022 15:33:34 +0000

This is merely a (too long) brainstorm. Thus please feel free not to read at all. 🙂 GraHal wrote: With above, I just did a quick mash up of an existing code, so not a valid test of Arrays. My understanding of Arrays is (using an xls to illustrate) … imagine a spreadsheet with numbered rows; after each bar the Close value is entered into a cell then next bar / close value is entered under the last entry … repeat etc. When required, a query of the Array (e.g. filled cells as above) can give HH, HL, LL, LH value over x bars. The clever bit is to code into a strategy to determine if price has passed the Highest High and price is now hitting a Lower High (a High lower than Highest High). And so on, so as to determine if the uptrend has ended. Anybody agree / disagree with above? Can any wizard code it? 🙂 (emphasis is mine) If it only were true that it would work like that. But it doesn’t and I think it can’t. What you suggest/imply there, @GraHal, is that the array lives over bars. As in Once $MyArray // Syntax error Once $MyArray[0] // Syntax error Once $MyArray[] // Syntax error On a side note : and *if* it would work like that, you’d get completely crazy of the fact that arrays can be one dimensional only. Thus, whatever you do, you would need two arrays or more. Think like (not tested for syntax, thus mere pseudo code) : If Highest(Close[20]) then // whatever this does. ElementToSet = ElementToSet + 1 $BarReference[ElementToSet] = BarIndex $HighReference[ElementToSet] = Close // Register highest price of last 20 bars at this moment in time. endif For i = 0 to $BarReference[LastSet] HighestBar = $BarReference[i] HighPriceBar = $HighReference[HighestBar] HighPrice = Close[HighestBar] next // Anyone who can understand this is crazy. // Anyone who can code this properly *and* understands himself, is ready for the mad house. The worst part is that this can easily be done without array, and then we suddenly understand. But meanwhile notice what I attempt there : Writing (for me) useful values in array elements, without consuming an 1M-elements array. This a. Makes it incomprehensible b. suggests that the array lives across bars (which it does not). Conclusion of that : There is no use to “save” array elements like I suggest doing there, and having them 1:1 with real bars (BarIndex) would – make it understandable indeed – make it a useless exercise because we already have that array (it is named Close[]). Working with arrays in PRT is mind boggling, or it is me who just can’t cope. Anyways … I personally use arrays everywhere and my whole life, but not within ProRealCode because there is no need to for me. This could also relate to actually everything already being in an array (or being presented as one), so that stuff like // Preserve the last 4 bars' gain. Gain4 = Gain3 Gain3 = Gain2 Gain2 = Gain1 Gain1 = Gain Gain = ThisBarsGain should better be done in a nice array, a small loop taking care of this. But this too is useless once you learn that this does all tricks already : If Gain > Gain[3] then // Gain larger than the gain 3 bars back ? // All still OK. endif So no definitions because PRT does this internally for you, and all is accessible in “array fashion”. Summarised : I am not debunking the array feature, but its real use-cases for me would occur when they live across bars; they are at least two dimensions. The latter is important so we could attach the value we want to preserve to the bar where it occurs in one go, which combines with not maintaining array elements per (history) bar. If Highest(Close[20]) then // whatever this does. ElementToSet = ElementToSet + 1 $HighReference[ElementToSet,1] = BarIndex // In case we want to know the bar where this happened. $HighReference[ElementToSet,2] = Close // The price where we detected this (whatever) High. endif // Find all the "highest" in subsequent elements of the array. For i = 0 to $BarReference[LastSet] HighPriceClose = $HighReference[i,2] // A high price occurred. HighBar = $HighReference[i,1] // Use this at convenience. For example : PricePriorToHigh = Close(HighBar - 1). // This happened right before this high. next So never mind this makes sense or not, at least I am now able to be creative and let it do something without being continuously disturbed by thinking of what I’m actually doing or even trying to do. We should not hallucinate over arrays going to live across bars (hoping that PRT is going to develop that) because I foresee all kind of hurdles regarding referencing bars in the past which are of a too far away history. Thus, if I would be able to store “events” (like my Highs) in an array in subsequent elements and once per month such an event occurs with my 1 second TimeFrame, then personally I would noy know what happens when my Live Strategy is 5 months in, and I still refer to a 1 second bar of 5 months in the past. I know, this is a general problem already, and the topic with the question “I save a barindex in a variable in order to refer to it as “Close[myBarFromDec2019]”, I postponed for later – but sure is due. IMO these things can’t work and my Live Strategy will eventually break (error out) or possibly come up with 0 values which ever back were not 0. Anyway : An array would be useful for this, if I would only store the resulting values of Close[myBarFromDec2019] and never the index to bars. Thus $MyHistoryArray[element 773] = Close[0]. After 3 years I will surely be able to access element 773 and its stored value. That is, if my strategy still runs, which chances are small either because of error or me being broke in that portfolio. And restarting that Strategy will have lost the value(s). There is really more to it all and I think that PRT does not have the infrastructure to let this work as I like to. But I’ll survive. 🙂 Peter

This topic has 32 replies, 6 voices, and was last updated 2 years ago by PeterSt.

Viewing 15 posts - 16 through 30 (of 33 total)

← 1 2 3 →

01/20/2022 at 6:19 PM #186111

PeterSt

robertogozzi wrote:

About history for arrays, I think it’s impossible to achieve. In a 200K backtest (not to use 1M) with just, say, a 100-element array, more than 20M (32+ bit) memory slots would be allotted to it, which is impossible to achieve.

Yes, I agree with your general stance. However …

This assumes the most-normal PRT procedure that we would want something like $MyArr[30[200]] (element 30, 200 bars back). But this approach IMO is just not how arrays should be used. Or maybe better : this is not how you’d want them to be used, if they only would live normally across bars *plus* if we’d only use them “normally”. Thus :

LastUsedElement = LastUsedElement + 1
$MyArr[LastUsedElement] = BestSignalForThisDay   // Occurs once per day.

1 2	LastUsedElement = LastUsedElement + 1 $MyArr[LastUsedElement] = BestSignalForThisDay // Occurs once per day.

implies one new element to occupy and with the already allowed array size of 1M, it would last 1M days, right ?
However, PRT-habit(-coding) would be something like :

LastUsedElement = LastUsedElement + 1
$MyArr[LastUsedElement] = Close   // Occurs each bar.

// From here sort out the highest occurrences per day which ...
// ... which already can be done without arrays.

LastUsedElement = LastUsedElement + 1

$MyArr[LastUsedElement] = Close // Occurs each bar.

// From here sort out the highest occurrences per day which ...

// ... which already can be done without arrays.

Please keep in mind : I made this array to live across bars (in my mind).

… And *that* thus does not work, because even without the possible [30[200]] desire, it already goes wrong after 1 month and the 1 second TF.

In my first post I said “there is much more to it”. Actually so much so, that this won’t ever happen. Still nothing is lost when the [30[200]] syntax just would not be allowed. And well, currently that is not allowed already (because the array lives the current bar only to begin with).

In the end all relates also to this (I referred to that in my first post as well) :

Once BarCount
BarCount = BarCount + 1

If Close[Barcount] > close then
  // I apparently now have a lower price than when my Strategy started.
endif

Once BarCount

BarCount = BarCount + 1

If Close[Barcount] > close then

// I apparently now have a lower price than when my Strategy started.

endif

Can someone tell me what will happen with that when LIVE ?
OK, people from PRT Development will be able to tell. But not “us” because we did not even attempt it (me neither, afraid of what might happen after … how many bars actually ?).
But possibly dev people thought of this and it might even work. Mind you, the backtest data of 1M bars is only a commercial thing. The data is just there. Especially the data which builds up during the strategy running. … But it conflicts with our thinking and (about) backtesting. Hey, we have 1M bars there, not more. … But in Live we logically have already because our code passed them all (if my s1 code runs for 12 months it has seen 12M bars).

On a side note and thinking “machine learning”, indeed I am building up my own averages (not moving averages but normal averages). Or say “statistics”. Just because I can predict it won’t work otherwise … Say lowest lows an highest highs; possibilities to take into account. Thus for example, what would the S&P500 report for all time highest high ? Probably nothing much with my 1 second 1M bars. But my program would have known it, if it only keeps on registering the all time highs. And maintain that in an array as how I would propose it ? how many elements would it consume for the past year ? maybe 20 ?

PeterSt

Participant

Topics: 71

Replies: 2304

Been thanked: 698 times

01/20/2022 at 6:29 PM #186112

GraHal

PeterSt wrote:

it will always lose its content at the next bar’s call.

So are you saying (above) that no data can be retained in an Array (on PRT) from one bar to the next bar??

Apologies if my questions are simplistic, considering all you have written! 🙂 An answer will likely begat another question. I’m trying to decide if it’s of any use trying to understand Arrays or if it’s a waste of my life!? 🙂

GraHal

Participant

Topics: 204

Replies: 8419

Been thanked: 2044 times

01/20/2022 at 6:41 PM #186114

phoentzs

I believe the usefulness of arrays is either not there for us or not usable. Or doesn’t make sense to want to use it. That’s the conclusion I’ve come to so far.

1 user thanked author for this post.

GraHal

phoentzs

Participant

Topics: 84

Replies: 917

Been thanked: 170 times

01/20/2022 at 6:43 PM #186115

PeterSt

GraHal wrote:

So are you saying (above) that no data can be retained in an Array (on PRT) from one bar to the next bar??

GraHal, yes, correct. That is what I’m saying.
(don’t shoot the messenger :-] )

PeterSt

Participant

Topics: 71

Replies: 2304

Been thanked: 698 times

01/20/2022 at 7:10 PM #186120

robertogozzi

Close[Barcount] will always refer to the same datum, as Barcount grows each bar, so when it’s 0 you refer to CLOSE[0], when it’s 1 you refer to CLOSE[1], which is the same data you referenced the previous bar, then it will become CLOSE[2], CLOSE[3], till, say CLOSE[11467] and so on.

1 user thanked author for this post.

GraHal

robertogozzi

Moderator

Topics: 69

Replies: 16475

Been thanked: 3643 times

01/20/2022 at 8:05 PM #186126

GraHal

Using, for example, the last 60 x 5 sec TF bars; is it possible to set up an array (via the code) that allows the array to be scanned / queried (correct term?) to provide the value of the highest[60]close and / or the lowest[60]close?

I know above can be done without arrays, I am just trying to understand what can and can’t be achieved with arrays.

GraHal

Participant

Topics: 204

Replies: 8419

Been thanked: 2044 times

01/20/2022 at 8:16 PM #186127

robertogozzi

You don’t need an array for that, just use:

HC = highest[60](close)
LC = lowest[60](close)

1 2	HC = highest[60](close) LC = lowest[60](close)

robertogozzi

Moderator

Topics: 69

Replies: 16475

Been thanked: 3643 times

01/21/2022 at 4:51 AM #186132

phoentzs

@Grahal I don’t know exactly how it works, but your system is winning so far.

1 user thanked author for this post.

GraHal

phoentzs

Participant

Topics: 84

Replies: 917

Been thanked: 170 times

01/21/2022 at 6:06 AM #186133

PeterSt

GraHal wrote:

Using, for example, the last 60 x 5 sec TF bars; is it possible to set up an array (via the code) that allows the array to be scanned / queried (correct term?) to provide the value of the highest[60]close and / or the lowest[60]close?

You hit the nail on the head by using the term “query”. Or, what I would like in these modern times, a literal query as how it could be used in the SQL language to dig out certain values and elements from the array. But *that* is not going to work, because it is too far out of line for PRT. So what remains is :

Sure GraHal, that is possible, but by means of scanning the array (looping through all of it, you remembering the Highest etc. you run into). But if anything is a waste of cpu cycles, it would be that.
Now observe Roberto’s answer. It may not even come to his mind to say Yes, because it would be a stupid thing to do, as it is already possible with the best “query” possibilities – those which PRT provides natively. In this case the Highest and Lowest functions.

I don’t keep track of anything, but the past year I read something like the most basal functions (i.e. all of them) like Average, can not be let lose on arrays YET. This read like “in due future it will be possible”. And why not. Maybe this is now already. And a bit difficult to try for me because I don’t know the syntax which can be daunting to guess in this (array-)case.
Thus without such functions being possible for arrays, we’d probably need to do everything ourselves. And if *then* you are as “smart” to store the signals directly like I tried to work out in the very first example of my first post, well, then I can’t cope. So *then* this literal query language would be a requirement, that even combining the two required arrays (because only one dimension possible) without hassle for the user (us).

PeterSt

Participant

Topics: 71

Replies: 2304

Been thanked: 698 times

01/21/2022 at 9:33 AM #186136

GraHal

PeterSt wrote:

how it could be used in the SQL language to dig out certain values and elements from the array

Above is how I imagined / hoped Arrays worked. I can see now that Arrays have limited use in PRT, partly (or mainly) it seems, due to the slowing down of the already tortoise like speed of backtesting etc.

It seemed for years we were so expectant that Arrays would be the turning point and our Algo effectiveness would take a massive step change!? Oh well, back to the drawing board! 🙂

GraHal

Participant

Topics: 204

Replies: 8419

Been thanked: 2044 times

01/21/2022 at 9:47 AM #186137

PeterSt

GraHal wrote:

Above is how I imagined / hoped Arrays worked. I can see now that Arrays have limited use in PRT, partly (or mainly) it seems, due to the slowing down of the already tortoise like speed of backtesting etc.

GraHal – Well, you are not even referring to the fact that using arrays in general slows down significantly. That is, I don’t recall me mentioning this yet. OK, when accessing undefined elements, yes, but in general using arrays indeed is way slower as well in comparison to using normal variables.
I think I implied that by telling about the general setup regarding, for example, the unnecessity to Dimension arrays in advance. Or that you can randomly assign an element a value instead of subsequently (in a row). So these issues are not really the fault of how we (the user) use it but how it under the hood has to work (I think I referred to mimicking), which in itself (IMHO) is smart but slowww.

1 user thanked author for this post.

GraHal

PeterSt

Participant

Topics: 71

Replies: 2304

Been thanked: 698 times

01/21/2022 at 10:03 AM #186139

PeterSt

PeterSt wrote:

Oh well, back to the drawing board! 🙂

A bit reluctant to post this :

This is something else, which massively disturbs me. It appears completely impossible to get in contact with developers – or their representative. This is just no communication between us and them. Only Nicolas is able to, and since it is not his job to do so, why should he. But :

This community is all (and only) about ProRealCode hence AutoTrading. Well, can anyone tell me what actually happened around this since 2018 and V10.3 ?
I see no-thing. Ah, arrays. Ah, some small function adjustments and a few new ones (and indicators). But in the development environment (ours) – nothing that I ever noticed. And that while the bugs and annoyances are countless which are mentioned countless times not only here, but especially and maybe foremost by myself and the seemingly cooperating local support (I don’t even dare to mention our Bug and Want List – grrrr).

Back to the Drawing Board, yes. But nobody listens so there is nothing much to draw. This, while PRT people monitor the “ProRealTime Platform Support” board. You can see/notice they do that indeed. But they pick what they like to comprehend or quickly can deal with or whatever it is.
Believe it or not, I even offered to pay to solve bugs. In the end I just can’t het through (no matter how much I thought I could, 1 year ago by now). It is terrible.

If only PRT weren’t so addictive …

PeterSt

Participant

Topics: 71

Replies: 2304

Been thanked: 698 times

01/21/2022 at 10:45 AM #186145

GraHal

I feel the same and in the past have been vociferous about it, but it got me nowhere (as in nothing changed) and I even thought maybe I had become a ‘marked man‘!

I hate being ignored, even in normal everyday life, but where money is involved the bad feelings are multiplied 10 times!

I decided to ‘shut up or ship out’ and so for several years now I have managed to keep schtum! Still it gets me nowhere … witness my 14 days waiting for 89 Systems to be restored!

I also would be willing to pay some fee for my 89 Systems to be restored, after all time is money if I had to retrace and reproduce the 89 Systems back to Live running! I likely never could anyway!

If only PRT Developers would work on the – Suggestions we make and the Bugs we hate – instead of enhancing colours availablity and List variations etc etc??
If only PRT would do a Customer Survey where we are asked to tick for – High Priority through to Zero Priority – before they spend / waste Developer time on Zero Priority functions!?
If only PRT replied to emails … not hard is it and is the most basic Customer Service?

I’d best finish else I will unleash my monster again!!!

GraHal

Participant

Topics: 204

Replies: 8419

Been thanked: 2044 times

01/21/2022 at 12:16 PM #186158

PeterSt

At least I am learning a lot of new English words. Haha.

GraHal wrote:

If only PRT Developers would work on the – Suggestions we make and the Bugs we hate – instead of enhancing colours availablity and List variations etc etc??

You know what my very first response was when I learned about PRT working on V12 ?
… that I feel scoffed …

So I told them, but the response literally was “nah, no problem, that’s another team working on V12”.
And why would I not believe that. But still the priorities seem wrong. That is, if nothing happens on the issues …

Maybe you’ll believe me if I tell you that I find new bugs on a daily basis. Really so. The last one is from the past 10 minutes. Sorting out how to solve it right now. See 1st attachment. Nice one eh ? Baby pips work out. It’s only that I have no clue which of the results is which.

In attachment 2 a secret;
That’s my in-box for PRT responses (past 4 years). Only 449. Not so many lately, because I really gave up after learning about V12. With some obfuscation you can see a few different respondents. These are from 2nd line support, mostly springing from Technical Reports. So it is not that I can’t reach them. It is also not that they don’t try to sort out issues. They spent ages on me. But the issues solved can really be counted on one hand. Wait, let me be on the safe side – on on hand over the course of the past year. But in more reality I recall only one solved issue. Dozens and dozens, if not 200+ pending. And most pending for years already.

GraHal wrote:

Still it gets me nowhere … witness my 14 days waiting for 89 Systems to be restored!

Most of my time I spend on helping people by phone. Let’s say that it is my (working) joy for life.
In seriously ZERO occasions throughout life I myself consult other instances for whatever issue or even a question (you don’t see me doing that in here as well).
The combination with that quote (above) and what I am talking about now, *and* apparently 449 whatever emails which all DO ask for help (indeed, this is about $), I now am in that “mood” (as announced in your 89-story topic).

Something has to be done.

PS: For weeks I did not dare to create a topic for this – and I still don’t. So if this is hidden a bit, is good enough for me. Venting may help. Collaboration helps too. Sorry for ranting.

PeterSt

Participant

Topics: 71

Replies: 2304

Been thanked: 698 times

01/21/2022 at 12:38 PM #186162

PeterSt

OffTopic, but since I mentioned it anyway :

PeterSt wrote:

Nice one eh ? Baby pips work out. It’s only that I have no clue which of the results is which.

LOL, see below for how to deal with this.

Thus, could work around it myself – it’s the xxxth bug I won’t even announce – and may cause troubles for others eventually – but … this is how it goes continuously.
Maybe I am too demanding.

PeterSt

Participant

Topics: 71

Replies: 2304

Been thanked: 698 times

Author

Posts

Viewing 15 posts - 16 through 30 (of 33 total)

← 1 2 3 →

Find exclusive trading pro-tools on

Arrays and how they can *not* be used (?)

1 user thanked author for this post.

1 user thanked author for this post.

1 user thanked author for this post.

1 user thanked author for this post.

Arrays and how they can not be used (?)