It's been a long, tiring day...

I've been huddled over my laptop all day struggling with MetaTrader and MetaEditor and I was just about to throw in the towel for the day when I made a random change to my TerryTibbs EA I'm working on... and I've had a breakthrough.

I'll tell you...Cracking the Forex Code ain't easy and when I backtested my EA on this month (june) and saw a positive result I was preparing for the worst (and had my fingers crossed) when I ran it on data ffrom Janurary to June..

But I was pleasently surprised....

Then I ran it on all 2008 & 2009 (so far)...

And again I was pleasantly surprised.

It's only 25% modelling quality but they're big trades so it doesn't matter that much. as they said in "300" after defeating the first wave of persians...

HELL OF A GOOD START!

Sure the graph isn't that smooth but for a very simple gerenal EA with les than 100 lines of code (if you remove all the comments and crap MetaEditor adds), and the best thing of course is theres no ugly green line dipping down south

Todays graph ... slight improvement

90% or NOTHING!!

~1200 trades and 25% modeling quality. That's a severely distorted result.

I'd appreciate a good tutorial for getting 90% quality cos so far it eludes me....

Easy if you use every tick method you MUST have ALL Data for ALL timeframes you use. If you use H1 you need M30, M15, M5 and M1.
If your M1 only goes till 2. Feburary you won't get 90% when you start you backtest in January because the M1 is missing -> <90% MQ.
That simple.

Yup totally agreed

Now I'm even more confused.

How do they arrive at the figures 25% and 90% anyway?!

I'd say "slightly distorted" not "severely distorted" as the trades have a 60 pip SL and 60 pip TP. Unless the price jumps (or falls) by 60 pips in a matter of a few minutes its not going to have much bearing on the results.

But anyway I soldier on...

Next step is to optimize this beast, get 90% quality and make it work on all currency pairs

What the Numbers in the Expert Testing Report Mean

One-Minute Data Modelling Quality Rating

Strategy Tester: Modes of Modeling during Testing

Now, some explaining.

Metatrader's history allows you to import data with 1 minute resolution. What that means is the smallest point of data is the Open, High, Low, and Close for a one minute collection of ticks. The problem is, during that one minute bar, we have no idea what happened. We do not know the progression of ticks during that minute.

In one scenario (This is an example only, this is not meant to demonstrate realism):

Open: 1.0000
High: 1.5000
Low: 0.5000
Close: 1.1000

Let's say at the start of the bar, you open a trade at 1.0000. you have a take profit at 1.5000 and a stoploss at 0.5000. We do not know whether or not the high or low was reached first. Had the high been reached first, then the trade would have closed in a profit, but had the low been reached first, the trade would have closed in a loss. The strategy tester only has the minute data, so it guesses. It has no way of knowing whether or not it's right or wrong, so for all we know, trades affected by this interpolation and guessing really cannot be counted because we have no idea if the underlying data is legitimate.


Your backtests look like they were run off of the 1 minute timeframe. That means you were using the minute bars in your backtest. Now answer this:

Do you have a minute bar in your history for every minute of that backtest?

My suspicion is that you do not, which means the strategy tester has to guess what the values of the Open, High, Low, and Close of the missing minute values are. The larger a chunk of missing data, the higher the distortion. Regardless of your TP or SL, the fact that the majority of your data the strategy tester had to guess, tells me that the potential for distortion is great.

Now, the model used to guess what the values of the missing bars were is well developed, so it's possible that the results will not change a lot between 25% modeling quality and 90% modeling quality. However, by getting as close to 90% modeling quality as you possibly can, you can eliminate the modeling quality as something which can distort your results.

well I didn't check every single minute (cos theres a lot of them) but I opened the history center and the 1M data goes back as far as 1999. and if I scroll it to random points there seems to be data for each minute

Edit: my computer is a piece of crap but I'm back testing it from 2005-2009 at the moment (should take about 10 mins) but I'll post the chart here if it looks promising

Scan through it. I'm almost certain there is a missing chunk somewhere. Given your lack of a rebuttal to the other points, I'll assume you agree with me on them, and at least now you know why it's a good idea to make sure you have complete data.

As for number of minutes:

1440 in a day
5 days in a week
52 weeks in a year
etc.

See if the number matches.

Well heres the 2005-2009 graph. Not so pretty but consider this:
It's trading with the exact time TP and SL and the exact same hard-coded rules every time. Not bad for a EA that's so "Dumb".

All I need to do is introduce some kind of dynamic adjusting algorithm for it to adapt to market fluctuations etc and I might , jut might be on to a winner

We call that curve-fitting. Nothing wrong with it, but use sparingly.

From the Strategy Tester report:
Bars in test:1561149
First trade made on:2005-01-06
Last trade made on:2009-06-30

That gives 1440 times 5 times 52 = 374400 (number of bars(minutes) in a year)

4 complete years covered (2005-2008) which equals 1,497,600 mins

Plus 2009 which were exactly halfway through = 187,200

1,497,600 + 187,200 =1,684,800

so it's missing 123651 bars (about 85 days worth of data)

now whether this 85 days is in one big chunk or several smaller chunks or sprinked evenly throughout the data I don't know and I sure as hell ain't going to waste my time checking

I'd like to know a bit more about curve fitting, If you have any info or anything I'd appreciate it

articles.mql4.com

Mostly written by russian authors, but the translation is sufficient.

Thanks I'll take a look.

I can get 90% on M5 timeframe (just for april-july) but still only 25% on M1.
Ran period convertor on an offline chart for 5,15,30,60,240 and 1440

any ideas?

90% Tutorial - http://alansforexblog.com/2008/02/27...pert-advisors/
FREE Data - http://forextester.com/datasources.html