DATFRA - Mt4 EA Builder & Walk Forward Analyser

Thanks for the bug report and the feedback, I will fix that tomorrow. Note that the "real" (not the simple) builder can do whatever you want, including trailed exits
However, a trailing stop is already included in the options.

-Darwin

Nothing to thank, I thank for the amazing work you´ve done and doing!

First "simple building" is now running 24 hours, but hasn´t found any system, though I understand it´s going to be advanced to make it more flexible. Meanwhile I´ll simply let it run until the next update, the server it´s on is unused anyway, so it can do whatever it wants there

Oh, it is flexible already, what you see is just an interface that makes the configuration simple, the underlying builder is the powerful and flexible one I mentioned already. However, it takes some tutorial so everyone understands how to use it, therefore I did not yet release it.

Also, yesterday I had some nice idea, and in the next week I want to release a paper-trader for that So a tool where you can play-back the market as fast as you want, to learn and test manual trading systems.

Also, a simple tool that takes an existing system (EA or paper-traded) and checks for various filter and exit criterias that might improve it

But more on that soon!

-Darwin

EDIT: Try to reload the database tree, perhaps it already found some candidates. It should find a few per day, however, most of them suck in out-of-sample, which is then the real challenge for the systems!

EDIT2: To thank me, just share it The more the community uses this, the more time I will put into this

That sounds very great, looking forward to the extended builder then

As for the paper-trader, I think many people will like it, but for me it´s nothing, I have no discipline for manual trading, always f*cked it up, I am a 100% automated systems trader only, otherwise I wouldn´t be profitable at all However, the extra filter / checks thing for existing systems sounds like a very good idea!

I´ve reloaded the database, no system there (but many dismissed ones, so it seems to work at least), but it might be because I am letting it work at max of M15 for the indicators and testing on M5 data. All my profitable systems are M5 or M1 based (always on the 13 years backtest with a as much as possible straight equity curve - have developed a custom fitness function for MT4 for that), I am not a fan of the higher timeframes as they usually give to few trades for statistic relevance and the higher the timeframe, the less likely it becomes that the inefficiencies the system exploits do repeat in the future as the situations that are traded become to "special". On M5 (and M1), I know that if I have a found a system that has 4000 to 10000 trades and works well in the backtest most likely will continue to do that in the future (and they did so far for the last 2 years). Any higher timeframe systems always got killed going forward for me or had to long drawdowns for my taste.

Anyhow; I will spread the word for sure and I am sure this thread will do it too if it stays active. Thanks again.

This is a great thread! Subscribe, i will learn and follow. Darwin-FX Best regards

@geektrader: But be carefull, the consense goes against what you say, at least for algorithmically created systems. The higher timeframes are not as noisy and therefore more predictable with a lower chance for systems that have nice backtest by chance. Additionally you run into broker dependency, slippage and other similar simulation traps with that timeframes.

So I would be very carefully to build systems on that lower timeframes, however, I have no hard numbers on that, so I might be wrong.
If the builder does not find systems, try the >= M30 timeframes and see if its better It should, that was at least the ones I tested.

Regarding HTML_REPORT DOES NOT EXIST, it might be that the backtests just take too long so a threshold is reached where DATFRA assumes MT4 to hung up. I will increase or remove that threshold in the next update, which will be sometimes today. I will post when it's uploaded, however, DATFRA should recognize the new version and ask for an update anyways

-Darwin

Update available! (DATFRA should update automatically)

HTML_REPORT DOES NOT EXIST should now be fixed. Also, an alpha of the papertrader is included, but its not yet an official release of that feature.
For a quick intro on how to use it, view the Changelog Tab in DATFRA's main window.

When I finished it I will release it with an tutorial!

-Darwin

Ooops, there was a small mistake in the last update. But now there should not be any time-thresholds anymore, so it should now work with M1 and M5 systems as it is meant to.

-Darwin

Hi Darwin,

you work very proficient! Great and thanks, I will look into the update

Thanks for your tips, I get your points in theory about lower timeframes and that´s also what I always thought before finally going to the lower timeframes after being unsucessfull to find a good system for years on the higher timeframes. Broker and feed dependency applies for any timeframe, it does not really matter is what I have found. It´s actually even worse if you miss signals on the H1 as the system can´t get the trade on the next 1H bar anymore as it´s very likely that the scenario doesn´t exist anymore that is needed for an entry. Contrary on the lower timeframes the chance is higher that the setup still exists on the next M5 or M1 bar. Anyhow, what you say and what most others say is exactly what has not been working for me and I had to find out with my own money One day I tought: let´s f*ck it, do it completely different than anyone *THINKS* it should work, since in the end all these guys just have a theory but no proof since they don´t have any profitable system at all too and just selling their advise which though was never proven and just is a theoretical approach of what *SHOULD* work.

So I was going to the lower timeframes, M5 an M1 and created systems and without using WFA or any out of sample testing, just optimizing on the whole data and re-optimizing from time to time, but also on the whole data again, just adding the new data of the last months since the last optimization (so the data that is being optimized on is always extended). And voila, the system is now working LIVE and is profitable since near 2 years with that approach and that even on a MAM with many investors and I am making a living from it.

Anyhow, that´s just my personal experience and it works for me, so that´s what I am using as it makes me money, while all the theory that this is the wrong approach does not and is just gray theory

Little update: I´ve downloaded the new version and am again getting (after allowing 365 to 4000 days for the initial backtest):

MT4 IDLES, DAMN! cpuusage = 0; mem = 36352; caches = 0. Its ok when this happens from time to time.

When it happens often, please start the MT4 from which you build the clusters, let it update, close it and rebuild the mt4-cluster!

Same thing when 'User Account Control' pops up sometimes.

And then subsequently because of above error:

HTML_REPORT DOES NOT EXIST!

mt4_instance::backtest ERROR XYZ_DATFRA_8181740166477

could not execute mt4, err = 11; PARTLOG =

HTML_REPORT DOES NOT EXIST!

ERROR! ATTENTION! BACKTEST REPORT DOES NOT EXIST!

MT4 ERROR NUM == 13

Hello Darwin-FX, very exciting piece of software you have here! I am trying it out right now, but i have run into some trouble.


When creating systems with the "Add from builder (simple)..."-function, the images keeps ending up in my "E:\DATFRA_FREE\bin\" directory instead of "E:\DATFRA_FREE\bin\..\ROOT_DIRECTORY\builder-preview\" as it should, according to the "Configuration"-tab.

I´ve tried to change the path to something else just to try, but when i change the directory path in the Configuration tab i cant find an option to save the change. When i restart the software and check the configuration tab again all the values are back to default.

I´ve re-installed the software and double-checked permissions on all folder and tried running the software as administrator, but still no luck.

Same thing seems to happen when running WFA, the images with the result doesn't show up in the software, so i guess they to are stored in an incorrect folder.

How do you utilize any of these algorithms when using metaquotes? In the MT4 Strategy Tester when performing optimization you just set the min/max values for each parameter and the step value so how do you go about using these better approaches, CMA-EA, BSA or otherwise in MT4 or Darwin's new software?

Hi DarwinFX,

Many thanks for starting this thread & contributing your amazing efforts for free, the Manual alone must have taken ages to compile. I only wish I were a coder myself so I could make a meaningful contribution.

Just one question, I couldn't find any reference in the Manual and Forum articles, but will DATFRA work with volume bars such as MeanRenko, Range Bars etc?

I confirm this "bug" too, the equity curve pictures are in the main (/bin) directory for me too. This might be the reason why it still hasn´t found one system for me.

Hi Darwin-FX,

Thank you for your precious work.

I stuck with the installation process because in the manual where the text says download FROM HERE there are no links to download from, at least in my case. Can you upload the latest version of the mql.exe please? I couldn't find it anywhere else.

Thanks.

Yes, he forgot the link in the manual. But it´s here:

http://files.metaquotes.net/metaquot...rp/mt5/mql.exe

Thank you very much geektrader.

@Missing mql.exe link
I just checked, and in the manual there is a link to mql.exe?!
http://s7.directupload.net/images/140729/sg353opu.png
Could you guys please double-check if you have a missing link there? And if so, what browsers do you use?
When I am home again on thursday, I will however add a more visible link.
Do you guys at least see the gnuplot link??

@Valkyry:
Well, the Parameterspace analysis will not. However, in the WFA, Metatrader's optimisation is used and then the single trades, from the "forward trading" backtest are parsed. So, if the renkos are set up properly in the Metatrader installation that you use to create the cluster from it *should* work.
But I dont know, try it, and if it does not, perhaps we can find a workaround
Attention: forextester.com data does not have proper volume informations! So you should import MT4 hst files or MT5 csv files (as in the manual).

@Problem with equity pictures in bin/
This is the local storage, so when DATFRA displays equity curves, they are stored there, as QT allows to display images from the filesystem very easy. So its ok when they are there.
The thing is, if there are nice equities in there, and you do not have some in the builder_preview, its because these systems (which are simple backtests), did not meet the filtering criterias (this is the case if DATFRA does not even start a WFA for them) or they failed in the WFA (this is the case if DATFRA did an WFA for them). You can see if it starts an WFA if you see the WFA window

@Problem with resetted config values:
Its by design, as these are internal directories and they should not be changed!

@Freefox:
One could only use MT4 to backtest the optimisation-candidates, then do the optimisation algorihms internally and create the next generation and then again only use MT4 to test them. So as simulation engine only, with the actual algorithm in a 3rd party program!

@PontusLSD & geektrader:
What timeframes did you guys use? I never tested the simple config with the low timeframes! So that might be a problem and you will have to wait for the actual extended builder.
It might also be that genetic building is not really good. You know, creating an M1 system is one thing, creating one that survives the Walk Forward Analysis is .. really hard I'd say.

@geektrader:
When that error happens, open one of the MT4s that are spawned (i mean, just get it to the foreground), and press Ctrl R to open the backtester. Then go to journal tab, and see if there is an error reported there. If so, please show me. If not, please tell me and we will seek on

Perhaps I should make a builder that does not use WFA, but only create systems, backtest them, and then store the backtest?!
EDIT: Fuck that shit, I will make that! I will save the initial backtest and allow the user to choose if a WFA should also be done



THANKS FOR THE FEEDBACK; This is really valuable for me, as I can never find all bugs as a single person
So, if you guys want to try it, and it does not work, dont hestiate and POST HERE OR CONTACT ME!!

- Darwin

We thank for your great work Darwin!

The manual he is referring to is most likely the included PDF file, not the online help. In the PDF the link is missing.

As for the "time outs", they only happen if I set the optimization days to high. If I just allow 365 to 1000 days it´s all good, if I allow 365 to 4000 days, the problems begin. The actual MT4s are working though and the FXT files in /tester are also created fine as I saw. It just seems that DATFRA wants the results a bit to quick and then decides MT4 must be hanging and decides the party is over If you want, you can have a look by yourself at the VPS I am using for this. Just let me know and I will send you the login details via PM.

As for the "it doesn´t find any systems on M15", yes, that might be the case then and we should wait for the more advanced builder.

Great work anyway, this has SUPERB potential Darwin!!

Hello Darwin,

Just wanted to say absolutely fantastic job you've done here! The sheer amount of work load you've put into this project is astounding!

it will take me a while to get my head around everything, but I'll be sure to keep glued to this thread for further development!

Cheers!
Sunrisetofu