Roffild's Library - Library for MQL5 (MetaTrader) with support

Roffild’s Library for MQL5 (MetaTrader) with Java, Apache Spark, AWS

I am known to the MQL5 community by the name of Roffild and this is my open source library for MQL5.An attempt to implement the features on MQL5, which have long become the standard for popular programming languages.Each file has one idea. The library is replenished as needed for new capabilities.

Few people tried to put the project in Github. There is no single standard.MetaQuotes does not use version control systems when creating a project.For some reason, programmers from MetaQuotes believe that the project should be of the same type.For small projects that are published in CodeBase on the MQL5.com site, this division is justified.For medium and large projects, it is not possible to select one type of project.

I experimented with different structure of the project.To use Git, I had to take the files out of the standard folder structure that MetaQuotes adopted.Create a link to the staging folder (in this library, the folder "Roffild") - the best option.

MetaEditor can save code in UTF-16, but encoding UTF-8 with BOM is also supported.To convert a file with the source code, you need to use a third-party editor (I recommend Notepad++).

This library can be divided into interests:

1. common tasks (ArrayList, Log4MQL, ToIndicator, etc.);
2. experiments with AlgLib in machine learning;
3. using Apache Spark with Amazon Web Services (EC2 and EMR), when the capabilities of AlgLib ceased to be enough.

Documentation

MQL5
Java

Links

Roffild.com
Github
MQL5.com: topic for discussion in English
MQL5.com: тема для обсуждения на Русском

1. Experts/Roffild/
   1. AmazonUtils - Can be used as an example of developing a project in Java.
   2. Alglib_MultilayerPerceptron.mq5
   3. Alglib_RandomForest.mq5
   4. Examples/
      1. ToIndicator_Example.mq5

2. Include/Roffild/
   1. MLPDataFile.mqh - Data format for Alglib_MultilayerPerceptron and Alglib_RandomForest. MLPDataFile = CSV in a binary format.
   2. ArrayList_macros.mqh - This variant is still used because of poor template support by the code editor.
   3. ArrayList.mqh - ArrayList from Java.
   4. ArrayListClass.mqh - ArrayList from Java for Class only.
   5. ForestSerializer.mqh - Save and load data for the class CDecisionForest (Alglib).
   6. Log4MQL.mqh and Log4MQL_tofile.mqh + module (download) for LogMX
   7. OrderData.mqh - Simulation of orders with attached data for research.
      1. OrderSql.mqh - Record data of simulated orders (COrderData) in a file format MySQL.

   8. SqlFile.mqh - Write data to a file format MySQL.
      1. CsvFile.mqh - Write data to a file format CSV.

   9. Statistic.mqh - Counting data and printing out the accumulated information.
   10. TesterSql.mqh - Record optimization results in SQL and CSV format files.
   11. ToIndicator.mqh - Displaying data from Expert or Script using functions for indicators.
   12. UnitTest.mqh - Base class for UnitTest.
   13. Serialization.mqh
   14. RoffildJava/
       1. AmazonUtils
       2. RoffildLibrary
       3. Spark - Reading from MLPDataFile.
       4. aws_ubuntu_user_data.sh - Working script for raising test agents on Ubuntu 14 in AWS.

3. Indicators/Roffild/
   1. ToIndicator.mqh
   2. ToIndicator.mq5
   3. ToIndicator_window.mq5

4. Scripts/Roffild/
   1. MLPDataFileSparkTest - Example project for Spark and MLPDataFile test.
   2. UnitTests

Installation
(Optionally)

mklink /j link where - does not require administrator rights.

It makes sense to move the %APPDATA%\MetaQuotes folder to the root of the partition or to a larger partition.Windows has a limit of 255 characters to the file path. The full path to the MQL5 folder I have is 88 characters.When testing, the terminal copies history by the number of local agents that increases the size of this folder by several gigabytes.

1. Move the %APPDATA%\MetaQuotes to D:\MQLProjects
2. mklink /j %APPDATA%\MetaQuotes D:\MQLProjects
3. mklink /j D:\MQLProjects\Terminal\D0E8209F77C8CF37AD8BF550E51FF075\MQL5\ D:\MQLProjects\MQL5

(Important)

Run the create_links.bat from the MQL5\MyProjects\RoffildLibrary folder after cloning the project.

Code style

Google Java Style

Tab = 3 spaces

Column limit = 110

License

Apache License 2.0

Russian

Added documentation.

https://roffild.com/

https://github.com/Roffild/RoffildLibrary

https://gitlab.com/Roffild/RoffildLibrary

https://bitbucket.org/Roffild/roffildlibrary/

The script for running Test Agents on Amazon servers aws_ubuntu_user_data.sh now uses the Role and internal disks.

Running Agents on Amazon servers.

MetaTrader 5 Agents & AWS EC2 (VPS/VDS)

Using MetaTrader 5 (MT5) Agents on Amazon servers VPS/VDS.
This is an excellent option for checking the trading strategy for the currency and stock markets Forex, CFD and Futures.

MetaTrader 4 (MT4) does not have Test Agents.

The difference from the use of the Agents cloud:

1. Fixed price per hour and a fixed number of processor cores.
2. Ability to use local files from FILE_COMMON.
3. Ability to use OpenCL on the CPU (and GPU, if the instance allows).
4. Upload large files needed for testing, you can through the bucket S3.
5. Possibility of using dynamic libraries (DLL) is possible.

Agents are run with the flag "/local" as with local use.
The cost of using Amazon servers can be much more economical than using the Cloud of Agents.
Calculation of the cost of the Cloud occurs by a mysterious formula, so it is very difficult to estimate the cost of full optimization.

Read more here...

Added MLPDataFile class for Python to the library.

buildall_and_tests.py - The script builds all components of the library and run tests.

MetaTrader 5 (MQL5) + Python 3 DLL

Do you have any future plan to connect Azure machine learning library to mql5 ör mql4?

Update PythonDLL

Switching to simple GIL, because many popular Python libraries are not compatible with the sub-interpreter.

If necessary, you can build with PYTHONDLL_SUBINTERPRETERS.

Update:

MetaTrader 5 Agents & AWS EC2 and aws_ubuntu_user_data.sh

MLPDataFile can now be converted to Pandas and Numpy.