Pattern Matching Concept EA | Page 2 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Trading Equity Millipede Concept by Concept 99 replies

pattern matching 97 replies

Two matching highs trend breakout? 3 replies

Matching metatrader times to alerts 0 replies

Oanda's NOT a matching engine...? 4 replies

Attachments: Pattern Matching Concept EA
Exit Attachments

Pattern Matching Concept EA

  • #21
  • Mar 28, 2010 3:51pm Mar 28, 2010 3:51pm
There are only 6 total ways to analyze an individual bar:

 

  1. High - Low
  2. High - Open
  3. High - Close
  4. Open - Low
  5. Open - Close
  6. Low - Close

(4C2 for those who remember combinations)

First off, can any of the above be removed because it's too redundant?

As for the total number of characters...

 

  1. A
  2. B
  3. C
  4. D
  5. E
  6. F
  7. G
  8. H
  9. I
  10. J
  11. K
  12. L
  13. M
  14. N
  15. O
  16. P
  17. Q
  18. R
  19. S
  20. T
  21. U
  22. V
  23. W
  24. X
  25. Y
  26. Z
  27. a
  28. b
  29. c
  30. d
  31. e
  32. f
  33. g
  34. h
  35. i
  36. j
  37. k
  38. l
  39. m
  40. n
  41. o
  42. p
  43. q
  44. r
  45. s
  46. t
  47. u
  48. v
  49. w
  50. x
  51. y
  52. z
  53. 1
  54. 2
  55. 3
  56. 4
  57. 5
  58. 6
  59. 7
  60. 8
  61. 9
  62. 0
  63. ~
  64. `
  65. _
  66. -
  67. +
  68. =
  69. {
  70. [
  71. }
  72. ]
  73. |
  74. :
  75. ;
  76. "
  77. '
  78. <
  79. ,
  80. >
  81. .
  82. ?
  83. /
  84. !
  85. @
  86. #
  87. $
  88. %
  89. ^
  90. &
  91. *
  92. (
  93. )


We actually only have 93 characters.

Given the bar equations above, maybe there is some algorithm which we can use to classify the bars automatically? That would make our jobs a lot easier.

  • #22
  • Mar 28, 2010 3:59pm Mar 28, 2010 3:59pm
Quoting Ronald Raygun
There are only 6 total ways to analyze an individual bar:

 

  1. High - Low
  2. High - Open
  3. High - Close
  4. Open - Low
  5. Open - Close
  6. Low - Close

(4C2 for those who remember combinations)

Why you want only pick 2 out of 4 values to describe a bar?

Quoting Ronald Raygun
First off, can any of the above be removed because it's too redundant?

As for the total number of characters...

 

  1. A
  2. B
  3. C
  4. D
  5. E
  6. F
  7. G
  8. H
  9. I
  10. J
  11. K
  12. L
  13. M
  14. N
  15. O
  16. P
  17. Q
  18. R
  19. S
  20. T
  21. U
  22. V
  23. W
  24. X
  25. Y
  26. Z
  27. a
  28. b
  29. c
  30. d
  31. e
  32. f
  33. g
  34. h
  35. i
  36. j
  37. k
  38. l
  39. m
  40. n
  41. o
  42. p
  43. q
  44. r
  45. s
  46. t
  47. u
  48. v
  49. w
  50. x
  51. y
  52. z
  53. 1
  54. 2
  55. 3
  56. 4
  57. 5
  58. 6
  59. 7
  60. 8
  61. 9
  62. 0
  63. ~
  64. `
  65. _
  66. -
  67. +
  68. =
  69. {
  70. [
  71. }
  72. ]
  73. |
  74. :
  75. ;
  76. "
  77. '
  78. <
  79. ,
  80. >
  81. .
  82. ?
  83. /
  84. !
  85. @
  86. #
  87. $
  88. %
  89. ^
  90. &
  91. *
  92. (
  93. )


We...

What about all other character, one can use when using one byte...? For example it is possible to use even ChrToStr(233)... So we would be able to use 255 different candle patterns ...
  • #23
  • Mar 28, 2010 4:01pm Mar 28, 2010 4:01pm
Quoting nightyhawk
Why you want only pick 2 out of 4 values to describe a bar?
You're right, I forgot about that .

[edit]: Combined, is there any other way to make a unique bar based off of those 6 (or fewer?) equations?

E.G.

High - Open
High - Close
Low - Open
Low - Close

Assuming I turn each answer into a string, and then combine the strings, any other bar that matches that unique numerical string should look exactly like the original bar.

[End Edit]

Quoting nightyhawk
What about all other character, one can use when using one byte...? For example it is possible to use even ChrToStr(233)... So we would be able to use 255 different candle patterns ...
I didn't realize MT4 had that. But this makes things a little easier.


Between 1/1/2000 and the present, there were ~3 million minutes. => 3 million potentially unique bars. I'm gonna try and figure out all the unique bars from this dataset. Might take awhile
  • #24
  • Mar 28, 2010 4:08pm Mar 28, 2010 4:08pm
rr,

if the mm is false, what tp and sl will it use? how will it close a trade?

---

wrong question i think

if the tp and sl is false, how will it close trade? (",)
  • #25
  • Mar 28, 2010 4:11pm Mar 28, 2010 4:11pm
Quoting mistersimple
2 trades 2 won...(+60 and +2 )

for old dogs = 1 trade 1 won (+100)
my friend,

what settings are you using?

i just saw this thread and applied the EA to m15 and m30 GU using default except changing the numberofbars and the threashold to 10, and 75 respectively.
  • #26
  • Mar 28, 2010 4:14pm Mar 28, 2010 4:14pm
Quoting outloud20
rr,

if the mm is false, what tp and sl will it use? how will it close a trade?

---

wrong question i think

if the tp and sl is false, how will it close trade? (",)
After x bars as defined in BarsClose
  • #27
  • Mar 28, 2010 6:54pm Mar 28, 2010 6:54pm
Inserted Code
2010.03.28 19:48:44    Bar Identifier GBPUSD,M1: There were 3074271 bars tested and 20092 unique bars found.

Notes:
The test was from 2000 to the present.

Surprisingly enough, 90% of the unique bars happened between 2008 and the present. Could this signify a market change?
  • #28
  • Mar 29, 2010 12:42am Mar 29, 2010 12:42am
I'm looking through clustering solutions: http://en.wikipedia.org/wiki/Cluster_analysis
  • #29
  • Mar 29, 2010 1:27am Mar 29, 2010 1:27am
Here is one potential solution.

Each bar generates a point in four dimensional space (each of the four equations is one coordinate). In order to get the entire range of bars, I would need to find the two points in the entire array with the largest distance between each other.

Once I have the largest distance, I simply divide the distance by 255, and the resulting ranges will show where each bar will be grouped, so there is some similarity between bars.

If there is a way to reduce the number of formulas to determine a single bar from four to three, visualizing the data would be far easier to do, and I would most appreciate anyone to post their answer.
  • #30
  • Mar 29, 2010 2:52am Mar 29, 2010 2:52am
Quoting Ronald Raygun
You're right, I forgot about that .

[edit]: Combined, is there any other way to make a unique bar based off of those 6 (or fewer?) equations?

E.G.

High - Open
High - Close
Low - Open
Low - Close

Assuming I turn each answer into a string, and then combine the strings, any other bar that matches that unique numerical string should look exactly like the original bar.

[End Edit]
Could you clearify that with an example? I'm not sure what you mean with equation "High - Open" and so on...

As I stated, we have 255 possible characters let me evaluate this fact:
a) one could expand this by using 2 byte for each bar --> pattern algorithm should work on that as well, which means we have potential 65k Bar-Types
b) thinking about pt. a) my opinion is, that one never should use the maximum of 255 at all.... the more differences in bartypes you look for, the more "unique" bars you find... all in all it might become some unlikely to find any suitable pattern, when having so much choice....
  • #31
  • Mar 29, 2010 9:38am Mar 29, 2010 9:38am
Quoting nightyhawk
Could you clearify that with an example? I'm not sure what you mean with equation "High - Open" and so on...

As I stated, we have 255 possible characters let me evaluate this fact:
a) one could expand this by using 2 byte for each bar --> pattern algorithm should work on that as well, which means we have potential 65k Bar-Types
b) thinking about pt. a) my opinion is, that one never should use the maximum of 255 at all.... the more differences in bartypes you look for, the more "unique" bars you find... all in all it might become some unlikely...
High - Open returns a pip value, let's call that HO
High - Close returns a pip value... call that HC
Low - Open = LO
Low - Close = LC

If I were to generate a string using these values. I'd use StringConcatenate(HO+","+HC+","+LO+","+LC)

The resulting string is a 4 coordinate point which can be plotted in 4-d space. Each point represents one bar exactly, no other bar has that coordinate.

Regarding part a), I only found 20k unique bars against 3 million, I'm pretty sure bars repeat themselves.

As for part b) The current pattern finding EA uses 3 classifications for bars (Long, short, doji). That obviously was too little. I'm trying to develop something to classify 255 types of bars. Against my original 20,000 unique bars, each of the 255 characters represents ~78 bars. I figure that is a safe enough range to uniquely identify reasonable patterns. If not, then I guess I'll have to add another byte.

I'm still trying to find a way to remove one coordinate while still uniquely identifying each bar, that way I can at the very least visualize the data.
  • #32
  • Mar 29, 2010 12:05pm Mar 29, 2010 12:05pm
Ronald, what do you think about this set:

1. High - Open
2. Close - Low
3. Abs(Close - Open)
4. which is greater Open or Close (0 - Open > Close; 1 - Open < Close)

You can use only 1 bit to represent no. 4.

What is more visualization can also be easier. Lets say we have following information about the bar (this way we store the data):

S1 = High - Open
S2 = Close - Low
S3 = Abs(Close - Open)
S4 = 0 IF Open > Close; 1 IF Open < Close

Before visualization we do something like this:

IF S4 == 0 THEN S3 = -S3

And use S1, S2 and S3 to visualize the bar.

Of course we can also just store no. 3 as Close - Open (with sign) and do not use no. 4 at all...

I hope it makes any sense
  • #33
  • Mar 29, 2010 12:14pm Mar 29, 2010 12:14pm
IMHO, you want to be careful about trying to identify too many distinct patterns (for example: 255). The reason is that you want to have statistical validity before you start pulling the trigger on trade decisions. To achieve statistical validity, you must have a sizable number of occurrences for each pattern before you can draw any meaningful conclusions from it. As you start to slice each price section into tinier pieces of the statistical pie, it takes a long time before you see enough occurrences of any particular slice to identify any valid trends.

For example, let's imagine that after a significant period of time you've found 6 occurrences of (255)(202)(041)(132) and of those 6 occurrences, 5 times the price increased by 50 pips thereafter, and 1 time it decreased by 20 pips thereafter. What does this tell you about the validity of the pattern? Unfortunately, little - if anything - useful. You need dozens (and ideally, hundreds) of occurrences of a pattern before you can ensure, statistically, that what you're seeing is not just a random blip - nascent "patterns" arising out of the background noise, only to vanish before they can be profited from.
  • #34
  • Mar 29, 2010 1:15pm Mar 29, 2010 1:15pm
Quoting Bytebodger
IMHO, you want to be careful about trying to identify too many distinct patterns (for example: 255)....

In statistics, when calculating direct numbers, you need a minimum sample size of 30 in order to achieve the minimum normal curve.

When comparing proportions, you need a minimum of 15 to determine the minimum normal curve.

You're right I do need a sizable sample, and there's no reason why I can't build that in as a filter to the EA. Remember, I'm not interested in every single pattern, I'm only interested in those predictably occurring.
  • #35
  • Mar 29, 2010 1:15pm Mar 29, 2010 1:15pm
Quoting kapaks
Ronald, what do you think about this set:

1. High - Open
2. Close - Low
3. Abs(Close - Open)
4. which is greater Open or Close (0 - Open > Close; 1 - Open < Close)

You can use only 1 bit to represent no. 4.

What is more visualization can also be easier. Lets say we have following information about the bar (this way we store the data):

S1 = High - Open
S2 = Close - Low
S3 = Abs(Close - Open)
S4 =...
Remove the absolute value in #3 and you don't need #4, right?
  • #36
  • Mar 29, 2010 1:42pm Mar 29, 2010 1:42pm
Quoting Ronald Raygun
Remove the absolute value in #3 and you don't need #4, right?
Right.

The only problem is memory usage. But I'm not sure if there is anything shorter in MQL4 then a 4-byte integer number.
  • #37
  • Mar 30, 2010 3:19am Mar 30, 2010 3:19am
Quoting Ronald Raygun
High - Open returns a pip value, let's call that HO
High - Close returns a pip value... call that HC
Low - Open = LO
Low - Close = LC

If I were to generate a string using these values. I'd use StringConcatenate(HO+","+HC+","+LO+","+LC)

The resulting string is a 4 coordinate point which can be plotted in 4-d space. Each point represents one bar exactly, no other bar has that coordinate.

Regarding part a), I only found 20k unique bars against 3 million, I'm pretty sure bars repeat themselves.

As for part b) The current pattern finding...
Ok, thats clear now... Using this way you describe bars by leaving out their absolute position in the chart, which is given by OHLC... Nice way... Nevertheless you said, that you will normalize each value, that it fits to 255 possible values... that means:
HO - maximal 255 possibilities
CL - maximal 255 possibilities
OC - max. 255 poss.

All in all we would have about 16M different bar types which eventual differ in only one bit (which means candle x is same as candle y but slightly longer)... Now imagine, every bar is uniformly distributed (don't know wether my translation is correct - I mean, the chance for every bar to appear is equal to any other bar) that means you have to wait 4 weeks wihtin your tick database to see all possible bar...
Ok, the approach to this project is even that there is no uniform distribution - nevertheless with this deep classification you have to regard billions of ticks to find some suitable patterns...
  • #38
  • Mar 30, 2010 3:36am Mar 30, 2010 3:36am
Quoting nightyhawk
Well 127 (or to say it better 128) means we have 7 different boolean values to use....

Imagine we'll take the pattern I described above as a base, then we have 4 Bits out of 7 used... so one could use further 3 bits to classify e.g. the length of the candle depending of ATR or something like that...
Using the patternscheme from http://www.forexfactory.com/showpost...1&postcount=20 and do not regard volatility at first EURUSD would generate for the last 1000 days the following string:

1325627650682124527642462268166342672624224676226534324625322866507255476753621227547261768524416511272562341642466303532075685832356676162663118536252365454287662223525375334162516334526631566700540636010266246473613322665741266216066576051642642156525564625312427781706712112006632160404661853168645444327232654665262322206586762531840366666414616756576225602377171663260660062626765267531601565022762036571243127666626641756275245260626164572124461671606706664743654863385824066145230608656656116322272321672063205626623455231556622621673562766357621316361823162217721828255283422831621175521022242064268362884125242775148515522321368256427528617455663272260316166366652886757182164212416226106882266256414170257356271153272763373662126521706022020621122736623657324462753236666002667624216562276572626735621351613463164157452366142254462266662446532683766383366106554117246716266631313052526271645615267160160620465821274272524556775083532664742622474222263521561112622355807136235133626522322563
  • #39
  • Mar 30, 2010 6:35am Mar 30, 2010 6:35am
Although I can definitely see value in the normalization of the bars, I also think that there is valuable information to be had in the relationship between a bar and its preceding bar. I'm not sure if you want to include this in your calculations, but I will tell you that I was inspired to run my own pattern analysis against my internal database of currency prices yesterday (using previous-bar relationships) and I think I came up with some promising leads.

I assign a number of values to each bar depending upon whether it closed up/down, whether it closed high/low, whether it had high/low range, etc. But I also specifically assign one of four values to each bar based upon its previous bar:

 

  1. Rising bar - this bar's high is higher than the previous bar's high AND this bar's low is higher than the previous bar's low
  2. Falling bar - this bar's high is lower than the previous bar's high AND this bar's low is lower than the previous bar's low
  3. Engulfed bar - this bar's high is lower than the previous bar's high AND this bar's low is higher than the previous bar's low
  4. Breakout bar - this bar's high is higher than the previous bar's high AND this bar's low is lower than the previous bar's low

  • #40
  • Edited 8:47am Mar 30, 2010 7:53am | Edited 8:47am
Quoting nightyhawk
Ok, thats clear now... Using this way you describe bars by leaving out their absolute position in the chart, which is given by OHLC... Nice way... Nevertheless you said, that you will normalize each value, that it fits to 255 possible values... that means:
HO - maximal 255 possibilities
CL - maximal 255 possibilities
OC - max. 255 poss.

All in all we would have about 16M different bar types which eventual differ in only one bit (which means candle x is same as candle y but slightly longer)... Now imagine, every bar is uniformly...
Not quite. It's 85 possibilities for each axis. (255/3)
I've found only 20k unique bars using the above coordinate system from 2000 to the present on GU 1M. I suspect that value shouldn't go much higher.


[edit] You could change the distribution of the 255 'characteristic slots' so one feature is more weighted than another.
0 traders viewing now