Pattern matching price movements in PRT – machine learning

Forums › ProRealTime English forum › ProBuilder support › Pattern matching price movements in PRT – machine learning

Viewing 15 posts - 1 through 15 (of 19 total)
1 2
  • 02/03/2017 at 5:45 PM #23818
    Wing

    I am wondering if anyone out there has experimented with pattern matching, in PRT, or using another platform/language.

    By pattern matching, I mean for example: comparing the last 20 bars, trying to find a sequence of 20 bars in previous history that best match it. From this you can make predictions on how such a similar event will fall out. You compare two price sequences by finding a common low and/or high point as your base(s), and then compare each close. The difference between each close is then squared and a total score is produced for the sequence. The sequence with lowest score is the best match. Advanced algorithms would compare highs and lows as well.

    Now, the problem is that PRT has a limitation on the number of iterations (for loops/if statements) that you can include in a system. This is probably to reduce server load. It also counts all indicators used in a system, so the max iterations are applied to the whole system. I believe this limit is 150 iterations per bar or something. It is possible to check for example this day’s price movement from 9;00 to 11:00 as compared to the same time period for each of the previous 100 days or so. To me, this is not enough. I have tried this out, but I am still unsure whether pattern matching is a viable strategy. In what markets/timeframes do you know/believe it performs well, or at all?

    To battle this limit on PRT iterations, I have a few ideas in mind:

    • Create multiple systems/indicators, that each check a certain time period every day.
    • Build a smarter pattern matching algorithm that relies on high/low points instead of brute-forcing and checking every bar in every sequence. Does anyone have any ideas on this?
    • If I can only compare a 15-bar sequence to 100 other 15-bar sequences, then create a filter that only compares it to good sequences. For example: sequence has a minimum range or is in a down/up trend.
    • Reducing the number of iterations by storing/checking information in variables in a better way. Why, oh why, are there no arrays (except indicator arrays) in probuilder/proorder?
    • If the above fails (which may be likely), build this pattern matching algorithm in another language.

    Would appreciate any ideas on how to accomplish this. Ready to explain my ideas further if there are questions. I might post some code soon of my experiments so far.

    I will also add that a bar sequence can be compared in two ways: with or without relativity. To compare two price movements that look similar, but where one has a range of 100 points and the other 50 points, the large one will be compressed to more easily be compared with the smaller one.

    3 users thanked author for this post.
    avatar mr blue, avatar Pepsmile, avatar Maz
    02/08/2017 at 7:32 AM #24281
    Wing

    Since I have no replies I will share some code and promising results I have. This code is a proof-of-concept indicator that uses pattern matching to analyze for example the daily open of DAX30 on a 15m or 30m timeframe. I have included an image also that shows that this indicator can be used to create an edge. Since neither the indicator or system is finished, I don’t want to publish it in the library yet, but feel free to experiment with the indicator below. If people find this prototype of use it can be published in the library.

    Indicator name: Wing’s Prophecy Boot version 0.1

    Indicator description: Every day at a set time (time of open for example) indicator returns a score of positive or negative. This is the indicators prediction of how price will move. Bigger score means more probability.

    How it works: At set time (example: 090000) check the 10 preceding bars. Compare these 10 bars to the exact same sequence of bars, in each of the approx. 100 previous days. Find the three best matches. Look how these best matches did, in terms of price change n bars later. Based on these previous movements, and their build-up’s similarities to this day’s price movements (accuracy of the pattern matching), form a ‘prophecy’ for how this day’s price will move.

    Redommended timeframe: 15m or 30m.

     

    5 users thanked author for this post.
    avatar laurenzo, avatar Francesco78, avatar Eric, avatar Robin, avatar Nicolas
    02/08/2017 at 9:07 AM #24287
    Nicolas

    That’s what I talked about yesterday when I was talking about interesting thread I have lost in the forums.. 🙂 I’m subscribing!

    Arrays should have been useful here you are right, and I’m impressed by your work to fill the void of it.

    Your approach is a kind of Goertzel algorithm? Find an already known frequency in a larger spectrum?  Should be a great idea to comment a bit your indicator’s code, otherwise you’ll lost a lot of people..

    Many statistical studies have concluded that the ORB (open range breakout), the direction of the price at the opening of the market is significant for 30% of the sample.

    1 user thanked author for this post.
    avatar laurenzo
    02/08/2017 at 11:25 AM #24300
    Wing

    I’m glad you like it. It took some thinking to figure out a good approach.

    I have not heard of the Geortzel algorithm previously, but since it is used in tone detection, it should be fairly similar.

    I can’t edit the code post to add comments, so I will do it below:

     

    02/08/2017 at 11:29 AM #24304
    Wing

    I will also add that there is the possibility to create a REVERSE pattern matcher. I have that code somewhere, and I might post it later if I find it.

    A reverse pattern matcher creates composite images of all patterns that preceded event A. For example, event A could be price moving over 1 % upwards within a span of 10 hours. The average of all preceding price movements to event A can then be compared to the current price movement and maybe predict when new such events will occur.

    This approach is much less taxing on the PRT servers, and therefore let’s you take more data into account, compared to the code posted above. However, it is less versatile and accurate.

    02/08/2017 at 12:03 PM #24308
    Nicolas

    I changed the bars comparison sequence with a loop to have more bars than 10:

    I fell into the infinite loop bug of the platform if I set “barstocompare” to 20, only tested 11, currently testing more bars. Is it still accurate for you? Maybe adding this code into an external indicator as a function would reduce the phenomena.

    edit: 15 bars is workings ok.

     

    02/08/2017 at 2:26 PM #24336
    Wing

    Nice loop. My goal is of course to increase the amount of bars per sequence, and days checked.

    I have no idea what the maximum bars are, but the more the merrier. The maximum iterations limit counts everything called in the program I believe.

    And yes, adding things as external indicators and calling them in an indicator or system sometimes helps.

    02/08/2017 at 4:01 PM #24367
    Nicolas

    So in your own development, you were trying to compare pre-open price behaviour to predict the whole daily movement or only for a short period of time?

    Did you try to compare the London session instead to predict price direction of the NY one?

    02/08/2017 at 4:43 PM #24381
    Wing

    My system where I have tried this indicator, I check price change for the whole daily movement. From open to close. Optimal prediction depends on the length of sequence, time of day, market, timeframe etc.

    You do have a point with your example of London and NY. Pre-open movements are less reliable. It is something I have to look into.

    02/16/2017 at 5:04 AM #25237
    Wing

    So far I am unsure if pattern matching works. On the 15/30m timeframe I can create an edge in DAX. But the system has many moving parts (variables) that make biases and over-optimization a risk. It seems it works best on indices though. My next move will be to investigate if it can perform in equities and maybe on new timeframes.

    03/01/2017 at 5:52 PM #26829
    Wing

    I have abandoned the approach of price pattern matching, since I couldn’t produce any compelling strategies. An edge exists, but not enough to cover spreads and drawdowns. I am however experimenting with indicator movement matching. For example, the movement pattern of an momemtum indicator compared to previous days, combined with also pattern-matching movement patterns for trend indicators etc. This is an attempt to find a similar price movement, but not by comparing price, but by taking other perspectives into account (different indicator types and periods, etc). This approach could be even more complex than the previous one and therefore less reliable and understandable, but I will post more if it turns out to be good.

    03/01/2017 at 6:26 PM #26833
    Nicolas

    Your code is an interesting approach and be sure I can help if you need assistance on any project related to pattern matching.

    You are right about the existence of an edge somewhere in pattern matching, but reducing it only to candlesticks patterns is almost like trading market noises IMO. I’d like to train an algorithm to find every combination of short term market’s “pause”, sounds tricky though..

    05/17/2017 at 11:08 AM #35812
    Maz

    This topic is my bag…. Can we discuss offline?

    Good job for getting this far with top-down code without arrays @Wing, I am impressed. So just some background – we have systems using multi layer SOMs, running in OpenCL on some very custom hardware. With arrays in PRT this could also be attempted there. The top-down structure is not really adept for this kind of work – you need parallel processing to harness the power of it.  But seriously good work. Would like to get in touch.

    05/17/2017 at 12:28 PM #35817
    Francesco78

    Thank you Wing for your outstanding work, I have been wandering wheter machine learning algos could be used  on PRT, I’ll do my best to help on this

    05/17/2017 at 3:08 PM #35853
    Wing

    I have a system based on another form of machine learning I can maybe show if you contact me. I have planned to post it publicly but have not had time to do so yet.

    1 user thanked author for this post.
    avatar Francesco78
  • Author
    Posts
Viewing 15 posts - 1 through 15 (of 19 total)
1 2

Find exclusive trading pro-tools on ProRealCode Marketplace

Similar topics:
prt personal account vs prt ig account

Create your free account now and post your request to benefit from the help of the community
Register or Login