Script to unscramble binary (e.g. FXT, HST) files into a readable format
The attached MQL4 script is capable of unscrambling any binary file in the …./MQL4/Files folder into an easily readable format. I created it primarily for viewing the FXT and HST files that are used by MT4.
Each binary file may optionally have one header record; and then any number of data records. The data specifications for each field must be stored in a descriptor file, also in the …./MQL4/Files folder. I have created two descriptor files FXT.DES and HST.DES, for unscrambling and viewing the content of FXT (MT4 tester history files) and HST (MT4 price history) files, respectively. You can create your own descriptor files for viewing files of other formats. The content of descriptor files is explained below.
All you need to do is copy the file you want to view into the …./MQL4/Files folder, and then run the script, specifying its name in the FileName parameter. The script unscrambles this file, and writes a readable version to the output file that you specify, also in the …./MQL4/Files folder. You can then view this file using a text editor like Notepad. A sample output from a FXT file is also attached.
INSTALLATION —
Unzip the files into exactly the same folders that they reside in, in the attached ZIP file.
PARAMETERS —
FileName — the name of the binary file (i.e. input file) you want to have the script unscramble. If it doesn't already exist in the …./MQL4/Files folder, then you must first make a copy of it there.
DescriptorFileType — specify FXT, HST or any custom file (e.g. XXX) that you’ve created yourself. The script will gain the field descriptions for both the header and data records from the FXT.DES, HST.DES, or XXX.DES file, in the …./MQL4/Files folder.
OutputFile — the name of the file, created in the …./MQL4/Files folder by the script, containing the unscrambled data, for you to view. If you leave this blank, the output file will be named 'Unscrambled [input file name]'
StartRecord, NumberRecords — this specifies which data record to start output at, and how many following records to unscramble and output. Large FXT and HST files may have several thousand records, hence this option allows you to choose whichever range of data records you wish to have output. The header record is always output.
DESCRIPTOR FILE —
The descriptor file is named XXX.DES (where XXX is anything you want to name it), and must exist in the …./MQL4/Files folder. There are two lines of text in a descriptor file:
--- The first line of the file consists of a string that contains the specifications for all of the fields in the binary file’s header record. If there is no header record, then leave the first line in the file blank.
--- The second line of the file consists of a string that contains the specifications for all of the fields in each of the binary file’s data records.
Both descriptor lines consist of a ‘triplet’ (three values separated by semi-colons) for each field in the binary file’s record.
The FIRST value in the triplet is the field name, which can be up to 20 characters long, and you can use any characters in the name except a semi-colon ( ; ).
The SECOND value in the triplet defines the data type and length. Valid entries are:
Sn — denotes a MQL4 string, where n is the length of the string in bytes.
I1 I2 or I4 — a 1-byte, 2-byte, or 4-byte MQL4 integer.
D4 or D8 — a 4-byte or 8-byte IEE floating MQL4 double value.
T — a 4-byte MQL4 datetime (integer) value.
C — a 4-byte MQL4 color (integer) value.
B — a 1-byte MQL4 boolean value (0 or 1).
The THIRD value in the triplet defines the output mask that will be used to format the value that is output from the binary field. You can generally leave this blank, in which case a sensible default mask will be used.
Hence the format is:
[name for field #1]; [data type/length for field #1]; [override mask for field #1]; [name for field #2]; [data type/length for field #2]; [override mask for field #2]; [name for field #3]; [data type/length for field #3]; [override mask for field #3] ...... etc
NOTES —
The FXT.DES and HST.DES files that I’ve created should be correct for MT4 build 427. If the formats have been changed in later builds, it’s a simple task to update these descriptor files accordingly.
Of course you can view the FXT and HST files in a hex editor, but it’s a painstaking job interpreting the binary data. (The best free hex editor I’ve found is HxD, although hexedit, XVI32 and frhed are also very usable).
I’ve posted this on the off chance that it might be helpful to somebody. However, I’m trying to leave the forums alone for a while, in order to get on with other projects, hence you’ll need to find answers to your own questions. I’ve posted full source code; it’s only 160 lines of MQL4, hence any competent programmer should be able to work it out.
Terms and Conditions
All code is supplied FREE of charge. It may NOT be sold or distributed commercially.
Software is offered on an 'as is' basis. I'm not offering a programming or troubleshooting service. If you want the MQ4 source code to compile without errors, you'll need to do this first (or you could try this version generously provided by danielkniaz).
There is no guarantee that the software is fit for purpose, or free of errors. Download and use the indicator(s) at your own risk; I accept no liability for computer damage or financial losses.
The attached MQL4 script is capable of unscrambling any binary file in the …./MQL4/Files folder into an easily readable format. I created it primarily for viewing the FXT and HST files that are used by MT4.
Each binary file may optionally have one header record; and then any number of data records. The data specifications for each field must be stored in a descriptor file, also in the …./MQL4/Files folder. I have created two descriptor files FXT.DES and HST.DES, for unscrambling and viewing the content of FXT (MT4 tester history files) and HST (MT4 price history) files, respectively. You can create your own descriptor files for viewing files of other formats. The content of descriptor files is explained below.
All you need to do is copy the file you want to view into the …./MQL4/Files folder, and then run the script, specifying its name in the FileName parameter. The script unscrambles this file, and writes a readable version to the output file that you specify, also in the …./MQL4/Files folder. You can then view this file using a text editor like Notepad. A sample output from a FXT file is also attached.
INSTALLATION —
Unzip the files into exactly the same folders that they reside in, in the attached ZIP file.
PARAMETERS —
FileName — the name of the binary file (i.e. input file) you want to have the script unscramble. If it doesn't already exist in the …./MQL4/Files folder, then you must first make a copy of it there.
DescriptorFileType — specify FXT, HST or any custom file (e.g. XXX) that you’ve created yourself. The script will gain the field descriptions for both the header and data records from the FXT.DES, HST.DES, or XXX.DES file, in the …./MQL4/Files folder.
OutputFile — the name of the file, created in the …./MQL4/Files folder by the script, containing the unscrambled data, for you to view. If you leave this blank, the output file will be named 'Unscrambled [input file name]'
StartRecord, NumberRecords — this specifies which data record to start output at, and how many following records to unscramble and output. Large FXT and HST files may have several thousand records, hence this option allows you to choose whichever range of data records you wish to have output. The header record is always output.
DESCRIPTOR FILE —
The descriptor file is named XXX.DES (where XXX is anything you want to name it), and must exist in the …./MQL4/Files folder. There are two lines of text in a descriptor file:
--- The first line of the file consists of a string that contains the specifications for all of the fields in the binary file’s header record. If there is no header record, then leave the first line in the file blank.
--- The second line of the file consists of a string that contains the specifications for all of the fields in each of the binary file’s data records.
Both descriptor lines consist of a ‘triplet’ (three values separated by semi-colons) for each field in the binary file’s record.
The FIRST value in the triplet is the field name, which can be up to 20 characters long, and you can use any characters in the name except a semi-colon ( ; ).
The SECOND value in the triplet defines the data type and length. Valid entries are:
Sn — denotes a MQL4 string, where n is the length of the string in bytes.
I1 I2 or I4 — a 1-byte, 2-byte, or 4-byte MQL4 integer.
D4 or D8 — a 4-byte or 8-byte IEE floating MQL4 double value.
T — a 4-byte MQL4 datetime (integer) value.
C — a 4-byte MQL4 color (integer) value.
B — a 1-byte MQL4 boolean value (0 or 1).
The THIRD value in the triplet defines the output mask that will be used to format the value that is output from the binary field. You can generally leave this blank, in which case a sensible default mask will be used.
Hence the format is:
[name for field #1]; [data type/length for field #1]; [override mask for field #1]; [name for field #2]; [data type/length for field #2]; [override mask for field #2]; [name for field #3]; [data type/length for field #3]; [override mask for field #3] ...... etc
NOTES —
The FXT.DES and HST.DES files that I’ve created should be correct for MT4 build 427. If the formats have been changed in later builds, it’s a simple task to update these descriptor files accordingly.
Of course you can view the FXT and HST files in a hex editor, but it’s a painstaking job interpreting the binary data. (The best free hex editor I’ve found is HxD, although hexedit, XVI32 and frhed are also very usable).
I’ve posted this on the off chance that it might be helpful to somebody. However, I’m trying to leave the forums alone for a while, in order to get on with other projects, hence you’ll need to find answers to your own questions. I’ve posted full source code; it’s only 160 lines of MQL4, hence any competent programmer should be able to work it out.
Terms and Conditions
All code is supplied FREE of charge. It may NOT be sold or distributed commercially.
Software is offered on an 'as is' basis. I'm not offering a programming or troubleshooting service. If you want the MQ4 source code to compile without errors, you'll need to do this first (or you could try this version generously provided by danielkniaz).
There is no guarantee that the software is fit for purpose, or free of errors. Download and use the indicator(s) at your own risk; I accept no liability for computer damage or financial losses.
Attached File(s)
