EA31337 released for free

Is it possible to make a Docker container including EURUSD data 2007-current?

I don't think it makes sense to use earlier data but some people go back to 2003.

Maybe there is a file size limitation of the containers.

This was the first thought I had while reading your code.

Not only could it optimize the parameters but also enable/disable indicators and strategies based on performance during different market conditions.

You can use -y option for any year which is listed in FX-Data-EURUSD-DS repo (check the Branch select list, or tag release files), and it'll be downloaded automatically from this repo. There is data available between 2007 and 2017. Usually I upload the current year once it's finished, but I can also upload 2018 partially before that. There are also few combinations of multiple years such as 2011-2015, 2015-2017. I'll publish containers including the data inside soon, so downloading the container will have everything included. Using -m is optional, you can use -m1-2 for the first 2 months, and when you don't specify -m, the whole year is tested.

The actual result files can be generated and published on your host machine, or uploaded anywhere. I'm working on it to test and improve this process as well.

Using containers, you can run backtests/optimization tests non-interactively and in parallel on the cloud (any cloud that can support Docker containers, and most of them do). So it's very powerful tool. Something like MT5 got the optimization on the cloud, but this method gives you more flexibility. Potentially it can support MT5 backtesting as well.

To answer your question from the different point. To test the full period since 2007, it requires a lot of memory, processing and time. Initially, I was using Vagrant (similar to Docker) to automate the tests. Testing 2 years in a row was challenging as it was very slow and memory hungry given the number of ticks (~150mln ticks), then I've advanced to 3 years (~350mln ticks), which also takes very long time and MT4 is using a lot of memory. I've already created dataset for 2011-2015 (4 years), but I think there was a problem running it, but I'll need to check again. The first limitation probably is with the platform, as far as I remember, the FXT files cannot be bigger than few GB (2 or 4). If you pass that limitation, it still can go out-of-memory, so most likely MT4 platform won't run the test. So for easier testing and to avoid headaches, I prefer to backtest each year separately, also to not rely on the success and higher balance from the previous periods, and to test the different start. But I can also provide backtesting for the 4-10 years in a row, if that will work.

1.073 has begun to trade, though totally different from 1.072 which was a wild beast that needed taming :-)

How do you start an optimization?

Maybe this is not available yet?

I guess one should enable only part of the profitable strategies first and then optimize each one like you already explained.

MQL5 Ltd does have some biased opinion when reports come to them.

I also have seller's account at MQL5; FYI.

I am here now because my EA, though passed preliminary check, it kept failing when moderator tested it.

It never fails to run what-so-ever using the terminal which i used in many different brokers and does work flawlessly.

I finally gave up and now doing some research to improve my strategy for the sake of long term running, which leads me to this thread.

Just to say it is nice of you to do this. Keep up the work man.

It's supported, all the optimization was done using the scripts. You need to pass -o param to run_backtest script to activate the optimization mode. Here is the example syntax:
run_backtest.sh -x -o -t -v -I TestModel=0 -S ValidateSettings=0,WriteReport=0 -E maxdrawdown_enable=1,maxdrawdown=50 -f $SET_PATH/EA31337.set -e EA31337 -c USD -p EURUSD -d 10000 -s 10 -y 2017 -m 1-12 -D 5 -P M30 -b DS -M 4.0.0.1010 -i $RULES_PATH/Common/.init.rules.inc -i $RULES_PATH/$RULEFILE.rule -r ReportName -O /opt/results

Most of the params are optional, or you can adjust to your needs. Most important is to pass set file with -f. Either you can prepare SET file ready for optimization, or you can use rules files which will parse the SET and prepare the file according to which param you want to optimize for MT4 optimization process (e.g. to optimize MA profit-take load the Strategy-MA-All-MA_TrailingProfitMethod.rule file). Then the results are generated in the specified output dir. With Docker, you can mount your external dir by -v (e.g. -v /LOCAL/PATH:/opt/results:rw) so your results are generated on your local folder. I haven't tested it with Docker yet, but it should just work, and I'll be improving the process further to simplify it.

When you'll use the provided rule files, they're automatically designed to disable all the other strategies (by modifying the SET file) and optimize only one specific strategy at a time with the fixed lot size and other strict rules, to only focus on the specific param.

They're very bad at doing good. I had also problems when accepting each version of EA, so important fixes took few weeks to accept. They've added preliminary check which was always finding some errors and issues which were not reproducible, so I had to put some blind fixes with trial and error to satisfy their checks.

After I've reported scam brokers in their official app (which are still there), and CDN servers using the incognito account (as I knew this is going to end bad), then they kept removing my few forum posts (here is the archived copy), like they wanted to hide it from the public. In short, I wanted to report that they've dozens of MT4 broker's platforms on their CDN server (these servers are officially listed in their app) which are pointing to the same fake/scam servers (where people believe it's a live server, but in fact, it's the demo), so scammers get their money directly to their bank, your money never touches your real account and they make you believe you've lost your money (which are the numbers of the screen). After I've reported this problem in details at their Service Desk, they've tracked my seller account via IP address and removed both accounts, they didn't want to give back my $2000USD balance which I had deposited, so I had to scare them with the court, so they gave me back part of it. They've also removed all my effort which I put on reporting 20+ bugs against their MT4/MT5 platform, and removed unfinished jobs, so I couldn't pay the developer. So basically no comment. I don't think MQL5 Ltd will stay on the market for the long time given how they operate and what decision they make.

I've described this scam method in details in the following post. This is the scam on the global scale finding their victims worldwide. Basically, MT4 is the only platform which is scam friendly.

Hi all,

I was doing backtest for V1.075 few days ago but encounter some problem.
1. Why the maximal drawdown value is different in the backtest report and journal details?
The backtest report show maximal drawndown is $111.62 But the journal is written as $32.5 ? Which 1 is correct?



2, What is the range for the signal level?


Thanks all

Awesome results you've got, especially for such a high spread.

The signal level range depends on the technical indicator which strategy is using, so there are no fixed ranges, and it'll be difficult to standarize all of them into fixed range. It's basically the shift from the value which is produced by the given technical indicator. The easiest would be to check the code logic in corresponding Trade_... function.

It's not clear which strategy you're talking, but let's give as the example the RSI, and the following logic:
case ORDER_TYPE_BUY: result = rsi[period][CURR] < (50 - signal_level);
case ORDER_TYPE_SELL: result = rsi[period][CURR] > (50 + signal_level);
So the reasonable signal level is starting from 20 (so you will get 30 for buy, and 70 value for sell), then increasing it to 40 makes 10 vs. 90, which is already extreme, so making it 50 and beyond, you won't get any trades, as RSI technical indicator operates between 0 and 100. And setting it too low, like between 1-20 most likely will produce too many trades. That kind of range is similar to CCI and WPR.

For other strategies where base indicators are like MA or Alligator, where the indicator values corresponding to the pair prices, the signal level is basically the distance in pips. So 0 means the original exact signal condition, 0.5 means half a pip, 1.0 means a pip away from the original value. Negative values mean the opposite (like triggering the signals right before the MA cross happens).

So basically, for MA strategy/indicator, it's the distance between MA Fast and Medium on cross condition, so positive values will trigger signal after the cross happened, and negative values mean the opposite, will trigger the signal right before the MA cross happens.

For Alligator, it's similar, so it's the distance between Lips/Teeth lines, and Teeth/Jaw lines. For MACD, it's the the distance between main and signal line in pips (signal level set to zero, means no extra distance, therefore the original, usual condition). E.g.
double gap = signal_level * pip_size;
case ORDER_TYPE_BUY: result = macd[period][CURR][MODE_MAIN] > macd[period][CURR][MODE_SIGNAL] + gap; // MACD rises above the signal line.
case ORDER_TYPE_SELL: result = macd[period][CURR][MODE_MAIN] < macd[period][CURR][MODE_SIGNAL] - gap; // MACD falls below the signal line.

For some strategies like Bands, Envelopes, the signal level is not implemented yet. And for some, like Fractals, the param is not compatible, due to lack of dynamic values, so some other similar value needs to be used.

So in general, some signal levels should be increased with smaller steps like (0.1-0.2), some with integers (1-2), so while optimizing, you should know by trial and error how much it changes the results.

One of these drawdowns isn't correct, so it's most likely the bug of the calculation function (as it's calculated independently from MT4) or due to some other factors. The logic is in CalculateSummary() (SummaryReport class).

great result bro ...
may i know your set ?

thanks
pintubesar1

The EA looks very promising - many thanks to the author for sharing it and also for sharing the source code!

Despite following the instructions for downloading and placing the sources into the right MT4 directories I am struggling to compile the EA in the MQ4 editor (version 5.0, build 1601) and would be thankful for any advice. Here are the errors I am getting:

'MA_SLOW' - undeclared identifier EA31337.mq4 187 44
'[' - invalid index value EA31337.mq4 187 43
'[' - invalid index value EA31337.mq4 202 48
'=' - object required Tests.mqh 42 20
'=' - structure have objects and cannot be copied Trade.mqh 60 18
'S_IND_NONE' - undeclared identifier EA31337.mq4 741 29
'[' - invalid index value EA31337.mq4 741 28
'S_ALLIGATOR' - undeclared identifier EA31337.mq4 779 10
'S_ALLIGATOR' - constant expression required EA31337.mq4 779 10
'S_BANDS' - undeclared identifier EA31337.mq4 814 10
'S_BANDS' - constant expression required EA31337.mq4 814 10
'BANDS_BASE' - undeclared identifier EA31337.mq4 821 25
'BANDS_UPPER' - undeclared identifier EA31337.mq4 822 25
'BANDS_LOWER' - undeclared identifier EA31337.mq4 823 25
'BANDS_BASE' - undeclared identifier EA31337.mq4 825 42
'S_CCI' - undeclared identifier EA31337.mq4 845 10
'S_CCI' - constant expression required EA31337.mq4 845 10
'S_DEMARKER' - undeclared identifier EA31337.mq4 852 10
'S_DEMARKER' - constant expression required EA31337.mq4 852 10
'S_ENVELOPES' - undeclared identifier EA31337.mq4 861 10
'S_ENVELOPES' - constant expression required EA31337.mq4 861 10
'UPPER' - undeclared identifier EA31337.mq4 875 29
'LOWER' - undeclared identifier EA31337.mq4 876 29
'S_FRACTALS' - undeclared identifier EA31337.mq4 889 10
'S_FRACTALS' - constant expression required EA31337.mq4 889 10
'LOWER' - undeclared identifier EA31337.mq4 891 28
'UPPER' - undeclared identifier EA31337.mq4 892 28
'S_GATOR' - undeclared identifier EA31337.mq4 896 10
'S_GATOR' - constant expression required EA31337.mq4 896 10
'LIPS' - undeclared identifier EA31337.mq4 899 25
'TEETH' - undeclared identifier EA31337.mq4 900 25
'JAW' - undeclared identifier EA31337.mq4 901 25
'JAW' - undeclared identifier EA31337.mq4 903 42
'S_ICHIMOKU' - undeclared identifier EA31337.mq4 905 10
'S_ICHIMOKU' - constant expression required EA31337.mq4 905 10
'S_MA' - undeclared identifier EA31337.mq4 915 10
'S_MA' - constant expression required EA31337.mq4 915 10
'S_MACD' - undeclared identifier EA31337.mq4 937 10
'S_MACD' - constant expression required EA31337.mq4 937 10
'NEAR_ZERO' - undeclared identifier EA31337.mq4 938 171
'S_MFI' - undeclared identifier EA31337.mq4 947 10
'S_MFI' - constant expression required EA31337.mq4 947 10
'S_MOMENTUM' - undeclared identifier EA31337.mq4 953 10
'S_MOMENTUM' - constant expression required EA31337.mq4 953 10
'FAST' - undeclared identifier EA31337.mq4 955 28
'SLOW' - undeclared identifier EA31337.mq4 956 28
'SLOW' - undeclared identifier EA31337.mq4 958 45
'S_OBV' - undeclared identifier EA31337.mq4 960 10
'S_OBV' - constant expression required EA31337.mq4 960 10
'S_OSMA' - undeclared identifier EA31337.mq4 966 10
'S_OSMA' - constant expression required EA31337.mq4 966 10
'S_RSI' - undeclared identifier EA31337.mq4 972 10
'S_RSI' - constant expression required EA31337.mq4 972 10
'UPPER' - undeclared identifier EA31337.mq4 978 46
'LOWER' - undeclared identifier EA31337.mq4 979 46
'S_RVI' - undeclared identifier EA31337.mq4 986 10
'S_RVI' - constant expression required EA31337.mq4 986 10
'S_SAR' - undeclared identifier EA31337.mq4 995 10
'S_SAR' - constant expression required EA31337.mq4 995 10
'S_STDDEV' - undeclared identifier EA31337.mq4 1003 10
'S_STDDEV' - constant expression required EA31337.mq4 1003 10
'S_STOCHASTIC' - undeclared identifier EA31337.mq4 1010 10
'S_STOCHASTIC' - constant expression required EA31337.mq4 1010 10
'S_WPR' - undeclared identifier EA31337.mq4 1019 10
'S_WPR' - constant expression required EA31337.mq4 1019 10
'S_ZIGZAG' - undeclared identifier EA31337.mq4 1028 10
'S_ZIGZAG' - constant expression required EA31337.mq4 1028 10
'S_AC' - undeclared identifier EA31337.mq4 1536 19
'S_AC' - cannot convert enum EA31337.mq4 1536 19
'S_AD' - undeclared identifier EA31337.mq4 1589 19
'S_AD' - cannot convert enum EA31337.mq4 1589 19
'S_ADX' - undeclared identifier EA31337.mq4 1643 19
'S_ADX' - cannot convert enum EA31337.mq4 1643 19
'S_ALLIGATOR' - undeclared identifier EA31337.mq4 1698 19
'S_ALLIGATOR' - cannot convert enum EA31337.mq4 1698 19
'LIPS' - undeclared identifier EA31337.mq4 1706 33
'TEETH' - undeclared identifier EA31337.mq4 1706 65
'JAW' - undeclared identifier EA31337.mq4 1707 66
'S_ATR' - undeclared identifier EA31337.mq4 1775 19
'S_ATR' - cannot convert enum EA31337.mq4 1775 19
'S_AWESOME' - undeclared identifier EA31337.mq4 1829 19
'S_AWESOME' - cannot convert enum EA31337.mq4 1829 19
'S_BANDS' - undeclared identifier EA31337.mq4 1882 19
'S_BANDS' - cannot convert enum EA31337.mq4 1882 19
'BANDS_LOWER' - undeclared identifier EA31337.mq4 1891 50
'BANDS_BASE' - undeclared identifier EA31337.mq4 1897 72
'BANDS_UPPER' - undeclared identifier EA31337.mq4 1898 72
'BANDS_BASE' - undeclared identifier EA31337.mq4 1899 81
'BANDS_BASE' - undeclared identifier EA31337.mq4 1900 84
'BANDS_BASE' - undeclared identifier EA31337.mq4 1901 103
'BANDS_UPPER' - undeclared identifier EA31337.mq4 1908 51
'BANDS_BASE' - undeclared identifier EA31337.mq4 1914 72
'BANDS_BASE' - undeclared identifier EA31337.mq4 1916 80
'BANDS_BASE' - undeclared identifier EA31337.mq4 1917 84
'BANDS_BASE' - undeclared identifier EA31337.mq4 1918 103
'S_BPOWER' - undeclared identifier EA31337.mq4 1943 19
'S_BPOWER' - cannot convert enum EA31337.mq4 1943 19
'S_BWMFI' - undeclared identifier EA31337.mq4 2030 19
'S_BWMFI' - cannot convert enum EA31337.mq4 2030 19

You need at least MetaEditor build 1745, as 1601 is too buggy for the code. For further details, check posts #31, #68 and beyond. If you still got the issues, try compiling the tag release branch (v1.075) as the master branch could be not stable right now.

Is there EX5 files? I only see EX4 on your releases page
I'm trying to run it on MT5 and not sure how.

Many thanks for the quick and helpful response! I followed your advice and all errors but the copy struct disappeared:

'=' - structure have objects and cannot be copied Trade.mqh 60 18

To solve this last issue, I changed the code in Trade.mqh to copy the structure parameters one by one. Here is my solution it just in case somebody has got the same issue:

void Trade(TradeParams &_params) {
// trade_params = _params; -> this didn't work on my MQ4 version, so copying the parameters one by one
trade_params.account = _params.account;
trade_params.chart = _params.chart;
trade_params.slippage = _params.slippage;

trade_params.account = (CheckPointer(trade_params.account) == POINTER_INVALID ? new Account : trade_params.account);
trade_params.chart = (CheckPointer(trade_params.chart) == POINTER_INVALID ? new Chart : trade_params.chart);
}

Now the EA successfully compiles.

Version for MT5 requires further work, so it's not available yet. Most of the classes should compile and work in MT5 as well, but the trading functions need to be fixed/refactored. The aim is to create the EA which will work on both MT4 and MT5.

Lately I was doing some research on the subject as a prep for converting my own EAs to MT5. The most challenging part is the trade management.

Luckily for late comers like me, others have done some of the work already and there is a library out there to help with the conversion: https://www.ea-coder.com/convert-mql4-to-mql5/

I believe you're still missing the MetaEditor build 1745 requirement, as the code should compile without any changes, so please double check which build you're using.

EA got its own libraries/classes/methods implemented which work for both MT4 and MT5 at the same time (using preprocessor conditional compilation directives), so no need for extra conversions or external libraries. Support for MT5 code is already in the advanced stage, so in this case, the general articles won't help much. These are only for some simple conversion.

The current challenge is to implement asynchronous trades (MT5) along with the synchronous trades (MT4). Most of the forum posts say it's not possible, but EA aims at implementing something despite what other people claim, but it takes time.

Not that I've tried but I see no reason for why this would not work. All is implemented in MT5 for TradeAsync. For HFT this is a must.