Backtesting | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Backtesting programs 20 replies

Best backtesting Period 3 replies

good backtesting software 5 replies

Custom backtesting code 9 replies

Backtesting vs. Thought Process 10 replies

Backtesting

  • #1
  • First Post: Oct 28, 2006 4:24pm Oct 28, 2006 4:24pm
As some of you know, backtesting in Metatrader is done using interpolated ticks. That is bad and it pretty much means that it's quite unreliable. The good part is that Metatrader supports, let's call them hand-made for want of a better word, history files. What I mean by history files is not the HST things that everyone downloads and uses and that have at best M1 OHLC data, but FXT files which can actually contain tick data. In order to get as much accuracy from the data as possible, I downloaded the Gain Capital tick files, ran them through some filters to remove the duplicates, stored everything in a MySQL database for easier access then wrote a program to export it in FXT format for MT4. The result was tick-by-tick data for whatever timeframe I chose, which, while a lot more accurate than the Metatrader interpolation, is still plagued by what I call 'holes' - missing periods of data, ranging from a few minutes to a few hours.

I'm willing to share the data, but the problem is that it's rather large, for example USDJPY 2005-2006 has over 200 MB unpacked and around 14 MB zipped and that is for a single timeframe. The forum doesn't allow such filesizes, but if anyone has some webspace for it, I can upload a few files for the public.

In addition, if someone can point me to a data source better than what I used, I'd gladly convert it to FXT.
  • #2
  • Oct 28, 2006 5:17pm Oct 28, 2006 5:17pm
Nice job!, where did you get the FXT format from?
With the holes in the gain capital data, I remeber they ranged from 5-15 minutes a couple of times a week, depending on the timeframe of your stratergy this should not matter to much I would have thought, have you found different?
The breaking of a wave cannot explain the whole sea.
  • #3
  • Oct 28, 2006 6:17pm Oct 28, 2006 6:17pm
The FXT format is described in detail in the help file, under "History Files in FXT Format". The only thing they don't mention is that the doubles are aligned to 8 in the header.You're right, most of the gaps are only a few minutes and as far as I'm concerned they can be the equivalent of a short disconnect.
  • #4
  • Oct 28, 2006 7:31pm Oct 28, 2006 7:31pm
Interesting, the FXT defination seems to say nothing of ticks, only bars.



#pragma pack(push,1)
struct TestHistory
{
time_t otm; // bar time
double open; // OHLCV values
double low;
double high;
double close;
double volume;
time_t ctm; // the current time within a bar
int flag; // flag to launch an expert (0 - bar will be modified, but the expert will not be launched)
};
#pragma pack(pop)

Am I looking at the right thing? if so where does the tick data go?
</pre>
The breaking of a wave cannot explain the whole sea.
  • #5
  • Oct 29, 2006 2:50am Oct 29, 2006 2:50am
Sure you're looking at the right thing...

Inserted Code
time_t ctm; // the current time within a bar

That allows you to specify the time within the bar; you can have as many ticks as you want within a bar, all that has to be adjusted is the high/low and close of the bar for every tick. The major annoyance is that it doesn't allow specifying both bid and ask, thus not letting one have variable spread, but I guess that's not such a big problem.
  • #6
  • Oct 29, 2006 6:26am Oct 29, 2006 6:26am
Excellent!

How'd you go about getting the Gain tick data though?

I heard it was available to clients but have never found it.
Bundy's status today: "Waiting..."
  • #7
  • Oct 30, 2006 12:43am Oct 30, 2006 12:43am
I just Google'd and found a link to it, apparently it's available to anyone who wants to download it. However, since you mention they are saying it's only for their clients, the link wasn't probably supposed to be publicly posted, thus I'm not going to post it here.
  • #8
  • Oct 30, 2006 12:56am Oct 30, 2006 12:56am
Hi Cristid,

The MQL documentation is crap to say the least, how do you know the Metatrader program interprts the data in such a fashion? in fact how do you even know if the ctm variable is absolute or relative to the current bar. Have I missed some piece of doco explaining all this? I'm sure I remember reading on the MQL website that Metatrader could not cope with tick data.
The breaking of a wave cannot explain the whole sea.
  • #9
  • Oct 30, 2006 6:58am Oct 30, 2006 6:58am
ctm is absolute, I also wrote a couple programs to dump portions of data from pre-made FXT files to find out some of the information. And it's really easy to verify, just use Visual Mode with speed at the minimum, pause it every now and then, check against the data in the tick file. The only problem was that when I imported the data, I didn't consider the GMT offset so my FXT files are in sync with the tick data + my local GMT offset.

In addition, I found a reference to http://codebase.mql4.com/512, used babelfish.altavista.com to translate it and also gathered some useful information from there. One could probably even use that script to convert, if the input files were merged and cleaned up.
  • #10
  • Oct 30, 2006 11:30am Oct 30, 2006 11:30am
Nice detective work, they sure don't make it easy!!
The breaking of a wave cannot explain the whole sea.
  • #11
  • Oct 30, 2006 11:55am Oct 30, 2006 11:55am
You can post files up to 100 MB for free here: www.rapidshare.com

Thank you for your effort.

D.


Quoting cristid
As some of you know, backtesting in Metatrader is done using interpolated ticks. That is bad and it pretty much means that it's quite unreliable. The good part is that Metatrader supports, let's call them hand-made for want of a better word, history files. What I mean by history files is not the HST things that everyone downloads and uses and that have at best M1 OHLC data, but FXT files which can actually contain tick data. In order to get as much accuracy from the data as possible, I downloaded the Gain Capital tick files, ran them through some filters to remove the duplicates, stored everything in a MySQL database for easier access then wrote a program to export it in FXT format for MT4. The result was tick-by-tick data for whatever timeframe I chose, which, while a lot more accurate than the Metatrader interpolation, is still plagued by what I call 'holes' - missing periods of data, ranging from a few minutes to a few hours.

I'm willing to share the data, but the problem is that it's rather large, for example USDJPY 2005-2006 has over 200 MB unpacked and around 14 MB zipped and that is for a single timeframe. The forum doesn't allow such filesizes, but if anyone has some webspace for it, I can upload a few files for the public.

In addition, if someone can point me to a data source better than what I used, I'd gladly convert it to FXT.
Quidquid latine dictum sit altum viditur
  • #12
  • Oct 30, 2006 12:17pm Oct 30, 2006 12:17pm
Here is somebody doing the same thing http://www.cubesteak.net/mt4-multi-b...-data-project/
The breaking of a wave cannot explain the whole sea.
  • #13
  • Oct 30, 2006 2:21pm Oct 30, 2006 2:21pm
http://rapidshare.com/files/1333387/EURUSD1_0.zip

Let me know if you have any comments. I'll upload more once I get some feedback and fix the bugs, if any. Well, there's the time offset problem, but that's not really such a huge issue, I think the data is GMT+3 right now.

By the way, most EAs that run with extremely good results on a short timeframe (M1 typically) on the interpolated data generated by MT4, will simply crash into the ground really bad on the real data or at best just hover around the same balance.
  • #14
  • Oct 30, 2006 7:18pm Oct 30, 2006 7:18pm
Got 95% 'accuracy', it's kinda akward how Metatrader is so keen to write over the file, just make sure you don't recalcuate!
The breaking of a wave cannot explain the whole sea.
  • #15
  • Oct 31, 2006 11:15am Oct 31, 2006 11:15am
That accuracy is not correct. I just wrote 95.9% in there... I could've written 100%, 72.37% or 101% although I don't know what effect the latter would have had on the result sheet.
  • #16
  • Nov 3, 2006 5:08pm Nov 3, 2006 5:08pm
In what ways is backtesting not so reliable? Are there certain timeframes or trends that are more reliable? I'm backtesting a theory based on entering trades based on on the close of a candle at the M30 scale, but it came back with something like 95% winners. Did I strike gold or is the system unreliable?
-JB
  • #17
  • Nov 3, 2006 5:15pm Nov 3, 2006 5:15pm
Quoting Dabellator
...I'm backtesting a theory based on entering trades based on on the close of a candle at the M30 scale, but it came back with something like 95% winners. Did I strike gold or is the system unreliable?
Even Christ himself couldn't answer that question with what you've provided. I would, however, suggest "zooming in" to a lower time frame (5 minute for example) and seeing exactly how price is behaving. You may be missing something.
  • #18
  • Nov 3, 2006 5:50pm Nov 3, 2006 5:50pm
Hi,

I'm new to EA's, been doing forex for a while, just wondering if you can give me some more detail/understanding as to why:

"backtesting in Metatrader is done using interpolated ticks. That is bad and it pretty much means that it's quite unreliable"

I dont understand the terminology and discussion thats going on but I am building my own EA and I would like to be able to back test it in the best possible way. My EA runs on 5 min charts, if that is at all important.

Thanks very much for your time,
Ah.
  • #19
  • Nov 3, 2006 6:13pm Nov 3, 2006 6:13pm
Quoting twinchell
Even Christ himself couldn't answer that question with what you've provided. I would, however, suggest "zooming in" to a lower time frame (5 minute for example) and seeing exactly how price is behaving. You may be missing something.
If you are speaking for Christ, I congratulate your infinite power. If you are asking me to clarify my question, perhaps you could simply ask. I'm trying to find out if the strategy test is accurate. When it goes through it's records and comes back claiming a strategy will win 95% and double one's initial investment, is it because the strategy itself is remarkable or because the MT4 backtesting is unreliable? If anyone has experience with similar results, please advise. If anyone wants to suggest looking closer, I'm fine. Thanks!
-JB
  • #20
  • Nov 3, 2006 6:25pm Nov 3, 2006 6:25pm
The backtesting is typically unreliable, it just gives a little indication on things. the best way to see if your strategy is worth anything, would be to forward test it (on a demo of course). and then go from there. goodluck with it.
-i trade currencies at the airport-
0 traders viewing now