Need help deciphering HST file

I need help deciphering HST files.

Primarily, I want to use perl to extract the datetime, high, low, open and close prices directly from an HST file and produce my own graph from it. I (as well as the many FF people) will highly appreciate if anyone could show me how.

Thanks!

Hi Burger

My understanding is that each HST file consists of a single header record, followed by one data record for each time period, e.g. in the M15 file, there is one record for the OLHCV for each 15 minute period. The latter are added in real time, as each time period completes, and are chained one after another, without a separator/delimiter. The latest record gets appended to the end of the file.

The code below is taken from the periodcon.mq4 indicator (attached). It gives a good clue as to the data types and locations of each field. A hex editor (Google a freeware one like frhed or xvi32 if you don't already have one) will let you step thru a HST file and read the data.

LONG_VALUE is a standard 4-byte signed integer.
DOUBLE_VALUE is a standard 8-byte floating point value.

Character fields are right padded with nulls (ASCII 0).

File name:
Is the pair (e.g. EURUSD) followed by 1,5,15,30,60,240,1440,10080,43200 for M1, M5, M15, M30, H1, H4, D1, W1, MN respectively.
Extension = HST

Data format - for Header record:
i) Version = 400 (4-byte LONG_VALUE)
ii) Copyright label = "(C)opyright 2003, MetaQuotes Software Corp.", right padded with nulls to a length of 64 bytes.
iii) Symbol = something like EURUSD, right padded with nulls to a length of 12 bytes.
iv) NewPeriodOutput (4-byte LONG_VALUE) = 1,5,15,30,60,240,1440,10080,43200 for M1, M5, M15, M30, H1, H4, D1, W1, MN respectively.
v) i_digits (4-byte LONG_VALUE) = number of decimal values to a pip, e.g. 4 for most currencies, except 2 for JPY (at least that's how I think it works)
vi) imesign, lastsync = (both 4-byte LONG_VALUEs) sorry, I'm not sure what these represent.
vii) i_unused = 52 bytes worth of nulls (13 x 4-byte integers).

Data format - for each Data record:
Consists of Date/time (4-byte LONG_VALUE), Open, Low, High, Close, Volume (all 8-byte DOUBLE_VALUEs), repeated for each time interval:
i) The date/time value (i_time) in the data record is the number of seconds since midnight on 1 Jan 1970.
ii) Open, Low, High, Close, Volume are floating point values, contain exactly what you'd expect (note that Low is stored before High though).

Hope I've understood your question, and that this helps to get you started.

David
___________________________________________________

int WriteHistoryHeader()
{
string c_copyright;
int i_digits = Digits;
int i_unused[13] = {0};
int version = 400;

if (FileHandle < 0) return (-1);
c_copyright = "(C)opyright 2003, MetaQuotes Software Corp.";
FileWriteInteger(FileHandle, version, LONG_VALUE);
FileWriteString(FileHandle, c_copyright, 64);
FileWriteString(FileHandle, Symbol(), 12);
FileWriteInteger(FileHandle, NewPeriodOutput, LONG_VALUE);
FileWriteInteger(FileHandle, i_digits, LONG_VALUE);
FileWriteInteger(FileHandle, 0, LONG_VALUE); //timesign
FileWriteInteger(FileHandle, 0, LONG_VALUE); //last_sync
FileWriteArray(FileHandle, i_unused, 0, ArraySize(i_unused));
return (0);
}
___________________________________________________

static double d_open, d_low, d_high, d_close, d_volume;
static int i_time;

void WriteHistoryData()
{
if (FileHandle >= 0) {
FileWriteInteger(FileHandle, i_time, LONG_VALUE);
FileWriteDouble(FileHandle, d_open, DOUBLE_VALUE);
FileWriteDouble(FileHandle, d_low, DOUBLE_VALUE);
FileWriteDouble(FileHandle, d_high, DOUBLE_VALUE);
FileWriteDouble(FileHandle, d_close, DOUBLE_VALUE);
FileWriteDouble(FileHandle, d_volume, DOUBLE_VALUE);
}
}

If you need the datetime, high, low, open and close, the periodcon.mq4 has an optional input as below:

OutputCSVFile = 0 / no CSV is output
OutputCSVFile = 1 / both HST and CSV are output
OutputCSVFile = 2 / Only CSV is output

Choose 2 may well solve your problem, if I understand you right.

Cheers.

Thanks for the reply; the sample mq4 file will help understanding the format too. should I find any problem, i will let you all know.

I wonder how 8 byte floating data used by hst files differ with php/perl's floating numbers?