Fractals, ZZ & the Pissing Dog | Page 39 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

ECN && STP && Scalping && Hedging - Broker Review - Oct 2009 24 replies

Alert on Fractals (MT) 2 replies

Zigzag & Fractals 10 replies

EA wanted for sound alert depending on fractals condition 2 replies

Fractals Edge Fx 3 replies

Attachments: Fractals, ZZ & the Pissing Dog
Exit Attachments

Fractals, ZZ & the Pissing Dog

  • #761
  • Jun 23, 2014 6:12pm Jun 23, 2014 6:12pm
Quoting Kilian19
I am curious how gg managed to create this algo straight from his head. Yes creativity is good and it's enjoyable to try out new things but if you are in the 20th version of your indicator and the results are more or less random and no one has ever done it before how did he keep himself motivated to move along? I don'T think he managed to get everything correct on the first try.
You raise a good point!

But some people already have past knowledge that may remain dormant for some time and eventually have a light bulb moment to put two and two together. Which may have happened here.
  • #762
  • Jun 23, 2014 8:41pm Jun 23, 2014 8:41pm
i think i made more than 100 versions but still not tradable.
i thought gg53 algo is very simple because he presented forex gt arrow at early stage of DNA idea.
but it seems not that simple (my opinion).

gg53, please give us more advice.
  • #763
  • Edited 10:12pm Jun 23, 2014 9:00pm | Edited 10:12pm
Quoting Kilian19
I am curious how gg managed to create this algo straight from his head. Yes creativity is good and it's enjoyable to try out new things but if you are in the 20th version of your indicator and the results are more or less random and no one has ever done it before how did he keep himself motivated to move along? I don'T think he managed to get everything correct on the first try.
He didn't go into details, but he did mention using R (EDIT: or MATLAB) to dig into the "DNA". It's near impossible to duplicate without discovering or knowing which R functions/tools he may have used to develop the system. I'd love to know more. ;-)

The bar DNA is based on n-grams which are useful in probabilities. From what we can gather/know so far is that the fractal DNA is only based on approximate matching of the pattern and length. Recently I'm incorporating a Levenshtein Distance algorithm for this approximate pattern matching. GG mentioned removing/replacing one fractal at a time.

Someone please correct me if I'm wrong, and this is probably obvious but I'll say it anyway, but the real "trick"/difficulty is the approximate fractal pattern and length matching algorithm and it's ability to determine the ZZ turning points based on the current pattern/length compared to the historical patterns/length. So far this is either more complicated than the algorithms that have been provided by several folks in this thread and/or there is a publicly known algorithm that we're just not aware of or it hasn't been mentioned that could be used. GG described this with his "three door puzzle" analogy and in the previous thread he compared it to a family taking a vacation and the kids repeating "are we there yet? are we there yet? are we there yet?".
  • #764
  • Jun 23, 2014 9:05pm Jun 23, 2014 9:05pm
Quoting jimsterk
i think i made more than 100 versions but still not tradable. i thought gg53 algo is very simple because he presented forex gt arrow at early stage of DNA idea. but it seems not that simple (my opinion). gg53, please give us more advice.
There may be a grey area, when you have your "in sample data" (Jan 1st 2011 - Dec 31st 2012), by logic you have to run it from Jan 1st 2013 till now so that the database will not have any spaces/gaps without it being updated, this is why I believe gg kept saying "my database may be different from yours".

It will help if we all had the same date range and the same predictions errors in our database first, then we will be able to move forward.
  • #765
  • Edited 3:07am Jun 24, 2014 2:52am | Edited 3:07am
Quoting Madmoney
{quote} There may be a grey area, when you have your "in sample data" (Jan 1st 2011 - Dec 31st 2012), by logic you have to run it from Jan 1st 2013 till now so that the database will not have any spaces/gaps without it being updated, this is why I believe gg kept saying "my database may be different from yours". It will help if we all had the same date range and the same predictions errors in our database first, then we will be able to move forward.

Yes that is a huge issue as well as makes it impossible to compare signals. I tried to implement the different parts of the algorithm step by step for itself.

Pattern Match. If that does not give me a potential signal at the end of the leg there is no reason to continue developing the length match.
Length Match. Same ...
....

If I look up fractal analysis he hit the nail spot on. At least I can relate to some things those guys do there.

http://en.wikipedia.org/wiki/Box_counting
http://en.wikipedia.org/wiki/Lacunarity
  • #766
  • Jun 24, 2014 3:56am Jun 24, 2014 3:56am
Quoting Kilian19
{quote} Yes that is a huge issue as well as makes it impossible to compare signals. I tried to implement the different parts of the algorithm step by step for itself. Pattern Match. If that does not give me a potential signal at the end of the leg there is no reason to continue developing the length match. Length Match. Same ... .... If I look up fractal analysis he hit the nail spot on. At least I can relate to some things those guys do there. http://en.wikipedia.org/wiki/Box_counting http://en.wikipedia.org/wiki/Lacunarity...
A good hint to see if you have got the different mechanisms correct within the algo and the probabilities correct is to run the algo as "in sample", if your algo is incorrect for "in sample" you are far off the mark, if it runs as it should do then you are close.

The only to replicate gg's algo is to follow what he has done to the tee, I believe no-one has followed him like for like, so no-one can expect the same results. I'm following everything he's done and I know what is missing from my algo, just need a bit more time to implement the missing features and then I'll know.
  • #767
  • Jun 24, 2014 5:26am Jun 24, 2014 5:26am
Quoting Madmoney
{quote} A good hint to see if you have got the different mechanisms correct within the algo and the probabilities correct is to run the algo as "in sample", if your algo is incorrect for "in sample" you are far off the mark, if it runs as it should do then you are close.
When I was halfway through the development, I did that! Results were great as expected, I could tweak out ridiculous win rate and trades but nonetheless off-sample performance didn't better from it.

I had a couple of ideas, which might make a mark, but I'm still recharging my batteries to take this thing on again. Bloody hell, it'll soon be 1 year since the start of this development.

Has anybody figured out the "one thing about this "Fractal" based system that almost everyone, amazingly, fail to see. It has nothing to do with math, statistics or algorithm. It's in plain sight on the chart"?
  • #768
  • Jun 24, 2014 6:23am Jun 24, 2014 6:23am
Hey guys,

I would like to invite a couple of you to work together with me on this. Please PM me if you are interested. You must have some programming experience (meaning be able to program this yourself and must have a working version) and some understanding of what the heck this is. This way we can tackle this as a team. I have a Node.js version that's pretty easy to setup and code (in Javascript). Once we get the group together, I will share with you my code and instructions on how to get it up and running. I'm looking to form a team of 4-5. We should be able to figure this out if we work as a team

Let me know what you think.
Focus, Patience, Determination & Order in chaos
  • #769
  • Jun 24, 2014 6:31am Jun 24, 2014 6:31am
Quoting Madmoney
{quote} A good hint to see if you have got the different mechanisms correct within the algo and the probabilities correct is to run the algo as "in sample", if your algo is incorrect for "in sample" you are far off the mark, if it runs as it should do then you are close. The only to replicate gg's algo is to follow what he has done to the tee, I believe no-one has followed him like for like, so no-one can expect the same results. I'm following everything he's done and I know what is missing from my algo, just need a bit more time to implement the...
Hi Madmoney,

Can you or someone with a good understanding please write a list of things that we should implement in order?

For example, what should the first working app should have? What should we implement next? And then what should we implement last? And what should go into each. I feel that a good summary is needed. Tomorrow or later tonight I will write up a summary of what I understand.

Thank you!
Focus, Patience, Determination & Order in chaos
  • #770
  • Jun 24, 2014 9:10pm Jun 24, 2014 9:10pm
Quoting jimsterk
{quote} hi madmoney, i am trying to put everything simple, so no adaptability, no error prediction, no 3-doors, and no atr5. and this is next day prediciton. blue rectangles are where i choke myself {image} possibly good mm can bring some $$ if i real-trade with this algo, but i dont like gambling. i stop posting current algo, i will create new one. has anyone got acceptable result? thank you,
Hi jimsterk,

By comparison here is my algo over the same period. It has some of the same problem points that yours has, with a few additional. Back to debugging...
Attached Image (click to enlarge)
Click to Enlarge Name: EU_Jan_2013_3.png Size: 39 KB
  • #771
  • Jun 24, 2014 10:12pm Jun 24, 2014 10:12pm
Quoting Madmoney
{quote} A good hint to see if you have got the different mechanisms correct within the algo and the probabilities correct is to run the algo as "in sample", if your algo is incorrect for "in sample" you are far off the mark, if it runs as it should do then you are close.
Hi Madmoney,

Yes I think that's a good way to judge initial algo performance. Generalizability to new data becomes more important once the in-sample issues have been hammered out..

Quote
The only to replicate gg's algo is to follow what he has done to the tee, I believe no-one has followed him like for like, so no-one can expect the same results. I'm following everything he's done and I know what is missing from my algo, just need a bit more time to implement the missing features and then I'll know.
I think a lot of people are/were looking for a recipe or step-by-step instructions. But instead what has been provided is more of a shopping list with a few topics explored in more detail. You still have to bake the cake on your own and determine how it all fits together within your code. It's a fascinating challenge but no trivial pursuit!
  • #772
  • Edited 1:19am Jun 25, 2014 12:27am | Edited 1:19am
Quoting saver0
{quote} Hi Madmoney, Can you or someone with a good understanding please write a list of things that we should implement in order? For example, what should the first working app should have? What should we implement next? And then what should we implement last? And what should go into each. I feel that a good summary is needed. Tomorrow or later tonight I will write up a summary of what I understand. Thank you!
Quoting FXEZ
{quote} Hi Madmoney, Yes I think that's a good way to judge initial algo performance. Generalizability to new data becomes more important once the in-sample issues have been hammered out.. {quote} I think a lot of people are/were looking for a recipe or step-by-step instructions. But instead what has been provided is more of a shopping list with a few topics explored in more detail. You still have to bake the cake on your own and determine how it all fits together within your code. It's a fascinating challenge but no trivial pursuit!

You guys are right, it is not all in order. The golden nuggets are scattered all over the place, but I have read the old and new thread over 25+ times and I am confident of what needs to be done (thanks to gg53, Crodzilla, burnss, Kilian19 and many others) it's just my time constraints.

I'm developing this algo to run in C/C++ via DLL to MQ4.

The first most in important thing is to make sure the data you have is CLEAN and RELIABLE.

This is the free tool I use to get my RAW data - http://www.strategyquant.com/tickdatadownloader/#programs

You can select the date range (1st Jan 2011 - 31st Dec 2012 or 1st Jan 2012 - 31st Dec 2013)

Then you can choose to export that data into 5, 15, 30, 60 minute csv files just as gg did. (EURUSDM5, EURUSDM15, EURUSDM30, EURUSDH1)

From then on I made a module that parses all this data file by file into SET1, SET2 and SET3. (EURUSDM5_dataset1, EURUSDM5_dataset2, EURUSDM5_dataset3)

gg said this- "As to historical raw data - It's VERY important that you use only CONFIRMED Fractals and ZigZag end points. It's easy to do in MT4 - just export to CSV/Text file with some delay (-xx bars from current bar)".
  • #773
  • Jun 25, 2014 3:22am Jun 25, 2014 3:22am
Quoting FXEZ
{quote} Hi jimsterk, By comparison here is my algo over the same period. It has some of the same problem points that yours has, with a few additional. Back to debugging... {image}
FXEZ,

What date range are you using?
  • #774
  • Jun 25, 2014 3:55am Jun 25, 2014 3:55am
Quoting Madmoney
{quote} FXEZ, What date range are you using?
Madmoney,

I'm using 2011-2013 at the moment (3 years) for my in sample period. I have high quality data outside MT4 that I use to create the database. I'm working on adaptability now.
  • #775
  • Edited 4:17am Jun 25, 2014 4:06am | Edited 4:17am
Quoting FXEZ
{quote} Madmoney, I'm using 2011-2013 at the moment (3 years) for my in sample period. I have high quality data outside MT4 that I use to create the database. I'm working on adaptability now.
Oh, what made you use three years instead of two?

Also, are you planning on running the algo from the beginning of it's "out of sample" date once you have implemented the adaptability/error handling?
  • #776
  • Jun 25, 2014 5:08am Jun 25, 2014 5:08am
Quoting Madmoney
{quote} Oh, what made you use three years instead of two?
Madmoney, when I started out I was having trouble with generalizability and I didn't have any way to deal with patterns that aren't in SET2, so more data helped reduce unfound patterns. But now that I've improved my pattern matching (logic1), this isn't so much of a concern. Going forward I may opt for less data on in-sample depending on how my testing goes.
Quote
Also, are you planning on running the algo from the beginning of it's "out of sample" date once you have implemented the adaptability/error handling?
I'm not sure exactly how I'm going to deal with updating/changing the database. Right now I have a separate "create database" sequence and a "run model" sequence. I like the flexibility of not having to re-create the database every time I make a tweak to the model. But to implement adaptability requires updating the database incrementally as data is received. hmm... It's also not possible to adapt to data before it has occurred so it must be incremental. I'm thinking of a way to structure the data so that it is more amenable to this process and requires no rebuilding, just data addition - still in concept.

So, when do we get to see your model?
  • #777
  • Jun 25, 2014 5:22am Jun 25, 2014 5:22am
Quoting FXEZ
{quote} Madmoney, when I started out I was having trouble with generalizability and I didn't have any way to deal with patterns that aren't in SET2, so more data helped reduce unfound patterns. But now that I've improved my pattern matching (logic1), this isn't so much of a concern. Going forward I may opt for less data on in-sample depending on how my testing goes. {quote} I'm not sure exactly how I'm going to deal with updating/changing the database. Right now I have a separate "create database" sequence and a "run model" sequence. I like the...
gg stated that he has Jan 1st 2011 - Dec 31st 2012 and used the whole of 2013 as "out sample", meaning he had no gaps of the algo not updating itself incrementally with prediction errors, that is why his algo always behaves the same and "adapts" to whatever the market gives it.

I also had that idea of setting aside a seperate folder just on "errors" (SET2, filtered by SET1 to store with SET3) with the time and date.

My model is not ready for public display , as I know what is missing. When I finish all features I will show it, It just takes me a long time to implement things as I'm not skilled in coding, I am very green.
  • #778
  • Jul 5, 2014 8:43pm Jul 5, 2014 8:43pm
Quoting Madmoney
{quote} {quote} You guys are right, it is not all in order. The golden nuggets are scattered all over the place, but I have read the old and new thread over 25+ times and I am confident of what needs to be done (thanks to gg53, Crodzilla, burnss, Kilian19 and many others) it's just my time constraints. I'm developing this algo to run in C/C++ via DLL to MQ4. The first most in important thing is to make sure the data you have is CLEAN and RELIABLE. This is the free tool I use to get my RAW data - http://www.strategyquant.com/tickdatadownloader/#programs...
Hi Madmoney,

I have also read ALL of Eurusdd's & GG's thread more than 30 times each, i believe the puzzle pieces are there, still need time to put each piece to its right place ..... I think i must change Smallcat's script (after i read the explanation from GG) : "The end of zigzag must be a fractal exists" ..... this is not moving the fractal to zigzag end point or moving the zigzag end point to where the fractal appear ..... but just simply "ADDING" a fractal at zigzag end point .... , still trying to make a small change to it ....

About access to database .... I am also still young in programming, and still have problem to create a DLL based on C++/C#/Pascal etc .... is there some one here that use Sql wrapper (sqlite wrapper / mysql wrapper) successfully for this purpose ? I have tried SqliteWrapper (the new one for MT4 build 6xx), but some times i got some error like: error code 1, 5, etc .... Error 1 = "SQL error or missing database " ... Error 5 = "SQLITE_BUSY - The database file is locked" Any idea is welcome.
Thank you in advance ....
  • #779
  • Jul 5, 2014 9:09pm Jul 5, 2014 9:09pm
Quoting elen21168
{quote} is there some one here that use Sql wrapper (sqlite wrapper / mysql wrapper) successfully for this purpose ? I have tried SqliteWrapper (the new one for MT4 build 6xx), but some times i got some error like: error code 1, 5, etc .... Error 1 = "SQL error or missing database " ... Error 5 = "SQLITE_BUSY - The database file is locked" Any idea is welcome. Thank you in advance ....
I got the same errors some time ago.

Try this :

Error 1 => i found that this is not missing of database, but there was mql error at my scripts . try to print out the error message at some points of your script to catch where is actually the problem exist

Error 5 => you tried to access database with different access type (Select/Delete/Update) at the same time. I got this problem as i tried to UPDATE data i got from SELECT query. Solution : i used "sqlite_free_query (handle); " before i call Update query ...
Similarity can change your life !
  • #780
  • Jul 6, 2014 4:30am Jul 6, 2014 4:30am
Quoting elen21168
. Error 1 = "SQL error or missing database " ... Error 5 = "SQLITE_BUSY - The database file is locked" Any idea is welcome. Thank you in advance ....
Error 1 could be a syntax mistake in your sql if I remember correctly.
Error 5 you either have to free your handles or you have your database open in a different program to view the data.

I gave up to manipulate the database using the mql4 wrapper for sql since it is highly inefficient and once you make multiple call's it takes minutes just for the queries.
0 traders viewing now