DATFRA - Mt4 EA Builder & Walk Forward Analyser

Hi again,
I am sorry if my question sounds silly, but is there any way to improve calculation speeds while using DATFRA? It would take for my intel i5 processor about 5 days to complete Meta Analysis for PSA for my trading system, which has 500 000 parameterspace size. It should be real problem if you want to do calculations on multi timeframes, multi pairs and multi systems.

I do not know much about "gpu hardware acceleration", but I know that GPU greatly accelerates computing, since bitcoin miners using it Any chance to apply this practice in our calculations? Or is there mt4 limitations or other obstacles?

Mt4 IS the limitation here, as I use it for the simulations That means, I can not port it to GPU, not if I do not want to re-implement the whole mt4 backtesting capabilities, which I do not plan

The problem above can be solved simpler, just do not analyse the whole parameterspace. My algorithm uses random sampling, so each iteration takes a random parameterset in the parameterspace, which means even something like 10.000 or 50.000 iterations should give a quite good approximation of the full parameterspace.

Keep in mind that in the end (when you view the psa results) you mostly end up with analysing like 1 parameter in reference to forward trading profit. And this one parameter mostly has a few hundred or thousand states maximum, which means if you analyse 10k parametersets you get 10k samples (*the amount of psa-windows, which is mostly like 100) just for this one parameter alone. And that is enough 99% of the cases.

Hope I could help!

-Darwin

hi Darwin,

try to install the software, but get error information: the version of file is not compatible with the version of window your are running. I use windows 7 ultimate version6.1. 32bit operation system.
also after download mt4, can not find the mql.exe file.

need help, thanks

Sorry but DATFRA is limited to 64bit windows versions, as it has a very high memory consumption when it generates PSA data, which 32 bit systems are not able to adress properly.

You might try DATFRA-M, tough

-Darwin

Hey again,
Thank you for your last answer, it helped!
I want to ask about spread and commission. I am using metaquotes 5 digit data, downloaded from my ECN broker and inserted to DAFRA how instructed. I guess that mt4 is doing simulations and WFA without commission? So I need to add commission to the spread settings. I am right? I get about 0.3 pip for EURUSD as average, so I shoud add about 0.5 pip for commission, and write to DAFRA 8, becouse for 5 digits it would be 0.8 pips Thanks in advance.

Have a nice day

Hey

Yes this is all correct and should work that way!

-Darwin

Hi,

I have problems running DATFRA it starts to crash after stopping meta analysis, and trying to run it again I get "not responding" and software stops working before any MT4 is opening, however WFA is still working. I tried: to restart my pc, to change .exe file to fresh one, delete all the systems in systems database, to change mql.exe to fresh one, to rebuild mt4 cluster, to change systems .mq4 files, however problem remains. I think that only full reinstall of DATFRA helps, that actually helped. How can I fix this crash?

Forgot to mention that I am using Win7, and debug_log_utils.txt shows nothing, except default few lines.

Thanks

Sorry for the late answer, I got ill (and still am) so I did not work..

Hmm that is very strange as DATFRA does not keep it's state after a restart.. So I can only assume there is some problem with something else, and you should really try to install it again.
You can take over your database in such a case, you can find the directory in ROOT_DIRECTORY\intern\json-database

Feel free to ask again if that does not help!

-Darwin

Notice: The server, and therefore the tool, will be offline quite some time today! Please excuse this..

-Darwin

Ok guys, I guess I will just go to a new hoster/server that makes less problems in the coming days.. So expect this project to be offline in the next week or so, I will post eventual updates here

-Darwin

Goodmorning Darwin,

first congratulation for your amazing and incredible work !
I think it should be one of the most important tribute of the history at ForexFactory ...
I'm waiting that your web hoster will return online ( this morning I visited your webpage at http://darwins-fx-tools.com/, off line.) so that I will test your software...
C U soon

Thank you for the nice feedback

Yes, the hoster is causing trouble since too long now, so I really think about going somewhere else, and this could take a while. :\
At the moment it is really bad, so yeah it might be best to wait with testing

-Darwin

Hi Darwin.
I started to learn your software first from the 32 bit version DatFra M. After i will try 64 bit version with another pc.
I made a mistake choosing wrong Mt4 terminal.exe linked to DatFraM (beetwen a lot of mt4 installed on my OS.
How can i change the exe mt4 file?
I tried uninstall and install datfram again but the first wrong choice still remain.
Greetings

Hey sardinian, you can just click the "Sync with Metatrader" button in DATFRA-M's home screen to choose a different mt4 installation



UPDATE REGARDING HOSTER PROBLEMS

I spent the last few hours to set up some monitoring stuff in order to be able to find bottlenecks in the server performance, if there are any.
So, if there is a problem with my server, instead of the hoster's infrastructure, I should be able to find it now.

Long story short: This is under active debugging! Hope it will run as smooth as it does at the very moment in the future

-Darwin

Hi Darwin, I tryng install DatFra free 64 bit today.
I'm using Windows 7 64 b. with last upgrades, monitor 1366 x 768.
Let start with issues :-)


1) I met some issue as member Mantise (above), but I resolved reading the thread...


2) At http://files.metaquotes.net/ the mql.exe file is not avaible. (403 Forbidden)


3) I found another mql.exe file from the web, and i use it but it has different size...


I had downloaded the new file DATFRA (4684 kb version)


4) I met an auto-update of indicators, but not possible to see the botton OK ( i change monitors setting without good results..so i pushed Return).


I try rebuilt the cluster with the new folder where i've installed a new mt4, but i met always the message " mql4 not found "
Tested different disk without good results


In Configuration I see always the same default cluster of DATFRA and not the new folders where is installed new-never-opened mt4. It this normal?


So I tried to rebuilt the cluster with an existing older mt4 folder, and i met the cmd.exe's windows open showing message "symbolic link created for D...the folder where is installed DATFRA.
After 10/15 minutes of work of cmd.exe i met message ATTENTION...FIRST EXECUTION OF MT4 WILL...etc... i wait 15 minutes...but nothing
I try start MT4 with comand Open:metatrader 4 , but a flash-black window opens and closes suddenly...

Excuse the late answer. Try to press the left button first, and then return, when it asks you to update its files That should do the trick!

-Darwin

Hi,
Is there any way how to change mt4/datfra commission/spread settings like using ECN broker? I want to make same settings like my broker has for realistic simulations. At the time there is no commission calculation in default mt4/datfra. We should include commission in the spread, let's say I get for EURUSD average 3 points spread on my 5 digits ECN broker, and I pay about 5 points for commission, so I should write in spread settings 8 and think that's the same thing. However it's not the same, sometimes bigger spread effect exit points, and it would be better to pay commission than to have bigger spread. Sometimes due higher spread I do not get take profit exit at all and market turns against me. Same for stop loss exit too, due higher spread sometimes it gets triggered too quickly and market turns to right direction without my position. This of course do not have big effect on higher timeframes and big stop loss and take profit exits, but on M1 scalping systems and it means a lot.
Below are example of EURUSD M1 scalping system with the same EA settings and same period, but different spread/commission calculations.
Have a nice day

Hi again,
If you don't mind I have one more question to ask
How do you decide proper size of "in sample" and "out sample" periods?
It's clear that same strategy on different timeframes generates different trade amounts, so logicaly "in" and "out" sample periods shoud differ too. I have read that about 25-50 trades for each optimizable parameter is OK, so if I have strategy with 3 parameters I should have at least 25-50 * 3 = 75-150 trades for each "in" sample optimization window. What is your view about this point? How do you calculate these periods for PSA and WFA? Thanks in advance.

Hi Darwin,
I think I found a bug. I had problems running PSA on M1 data (mt4 data, m1.hst file weights about 300mb). For other timeframes everything is working great. I was getting error:


Rebuilding cluster with smaller "Max possible simultaneous metatrader instances" helped. I think that's something to do with RAM, because my PC gets very sticky and all RAM is used, probably due higher m1.hst file sizes. For me max instances is 6 and after rebuilding it starts to work again. I have 8 Gb of RAM.
Hope it helps for someone who is experiencing same problem while working in m1 tf.

Have a nice day

Hi Darwin,

As others have said, thanks so much for the program and the detailed instructions and explanation of these concepts! I have a question about PSA vs. WFA and what information is displayed in the charts provided in the program. As well, I received some error messages during execution of the builder (simple).

From my understanding, a normal WFA optimizes on an in sample data set and forward tests the best performing set of parameters on the out of sample data set. The advantage of PSA is that forward tests are performed on every set of parameters (or at least a subset), not just the best performing set found during in sample optimization. Is this accurate?

The reason I ask for clarification is mainly because I don't see where the PSA data is being used or presented anywhere (other than a forward test of the average of the top 50 parameter sets). The view backtest window I believe shows WFA data only, minus the aforementioned top 50 average forward test. Is this correct, or am I missing something?

There were many different PSA charts presented in your pdf manual that caught my attention. Are these charts accessible from the program someway, or is there a database I can access and create these with gnuplot?

Lastly, I received the following errors repeatedly during builder (simple):
mt4_instance::backtest ERROR, could not compile XYZ_DATFRA_6672766610481
mt4_instance::backtest ERROR, could not compile XYZ_DATFRA_6672766610481
ERROR! ATTENTION! BACKTEST REPORT DOES NOT EXIST!
MQL4/MQL5 Compiler build 1035 (11 Dec 2014)
Copyright 2001-2014, MetaQuotes Software Corp.
XYZ_DATFRA_6672875074662.mq4(89,111) : error 256: 'cmp_2FQ' - undeclared identifier
Result: 1 error(s), 4 warning(s)
Time: 31 ms

It appears to be an insignificant error message, because I can access the backtest reports through MT4 now that I stopped the builder. Do you know whats going on here?

Sorry for the long message... lol,
VVizlan

P.S. Thought you might also want to know that the first time I went to build the cluster (not really sure I know what this means... lol) the MQL4 folder did not exist. It was not created during installation of MT4. So, I simply created an empty folder called MQL4. Everything seems to be working, but perhaps I shouldn't have done this?