Menu

Options

Bookmark Thread

First Page First Unread Last Page Last Post

Print Thread

Similar Threads

MT4 Backtesting tutorials 2 replies

MT4 Backtesting website experiment... 4 replies

MT4 Backtesting and spreads 0 replies

Attachments: MT4 Backtesting Threads
Exit Attachments

MT4 Backtesting Threads

  • Post #1
  • Quote
  • First Post: Edited Jun 21, 2006 10:10am Jun 20, 2006 5:49pm | Edited Jun 21, 2006 10:10am
Use this thread to discuss MT4 backtesting stuff.


Index of MT4 backtesting information:

 

  1. Backtesting reliability: http://www.forexfactory.com/forexfor...06&postcount=2
  2. Historical data: http://www.forexfactory.com/forexfor...07&postcount=3
  3. Manual backtesting: http://www.forexfactory.com/forexfor...10&postcount=4
  4. Coders Guru documents on strategy testing: http://www.forexfactory.com/forexfor...0&postcount=15

-soso
  • Post #2
  • Quote
  • Edited Jun 21, 2006 4:48am Jun 20, 2006 5:55pm | Edited Jun 21, 2006 4:48am
It is a known issue that MT4 reliability of backtesting is questionable. How can one improve the reliability of MT4 backtesting?


mcboogs writes at http://strategybuilderfx.com/showthread.php?t=15309:

Quote
I decided to write up this little tutorial, because backtesting different systems comes up very often in threads on this forum. There seems to be a lot of confusion about reliability issues and how to go about achieving the most accurate possible results. I am not a programming or trading guru, but I believe I can provide a helpful little FAQ on backtesting using MT4.
Good backtesting is important when considering a system-trading approach, because you want to have some idea of the feasibility of your idea before you go live with it [at least I do]. If you're backtesting with a 50% model quality, eh... you can't really be sure what's going on. If you have a 90% modeling quality, you can have more confidence on how your system actually would have performed.


+=========================+
|MCBoogs' MT4 Backtesting FAQ v1.0 |
+=========================+

Contents:
- Section 1: Is MT4 Backtesting Reliable?
- Section 2: Downloading/Importing/Converting 1M Data
- Section 3: Configuring the Backtester
- Section 4: Other Issues


Section 1: Is MT4 Backtesting Reliable?

This question often gets pretty heated and people even get to the point of flaming each other about it. Backtesting in MT4 can be reliable, but its reliability is contingent upon the data you are backtesting on. Demo account data that is streamed in through a demo account broker has gaps, holes, and is basically not suitable for testing.
When backtesting, you want to use the EVERY TICK MODEL and have accurate 1M data to get the most accurate test possible. The 1M data is important, because the EVERY TICK MODEL uses whatever the smallest available timeframe available is and "fakes" the movement of price within the smallest available bars. Having 1M data allows for the fractal interpolation within bars to occurs only within the very narrow range of 1M bars.
The easiest [and only] solution to this is to use good 1M data. The most complete data you can get [at least for free] is from Alpari's Databank. They have data in MT native format, on the 1M timeframe back through mid 2004. However, setting up the data for use requires some doing.

---------------------------------------------------------------------------


Section 2: Downloading/Importing/Converting 1M Data

(1) You need to modify MT4 to allow for more bars. Go into the Tools Menu, then go to Options [or just hit C+O]. Go into the charts tab and put in 9999999999999 for bars in history. MT4 will default to whatever it's maximum is.

[Note: The reason MT4 has a limited bar count to begin with is because more bars (particularly when used in backtesting models) means MT4 is going to eat up more HD space.]

(2) Download the 1M data from Alpari's Databank in whatever currency[ies] you're going to test on.

(3) Import the data into MT4 using the History Center. Go to Tools => History Center [or push F2]. Make sure you import it in proper currency and in the M1 timeframe. You don't want EURUSD data being important into USDCAD for instance.

(4) Convert the data using the period converter script included in MT4 [you only have 1M bars right now]. You have to open offline charts to do this.
-Go to the File Menu, then Open Offline, select the 1M data of the currency you need to convert. A chart will pop up with that data.
-Then drag & drop the period_converter script onto the offline chart. The ExtPeriodMultiplier int that you can modify is the multiplier you are applying to the chart. So making it 5, will convert 1M data into 5M data.
-For simplicity's sake, you need to run the period converter with the following integers to get all the backtesting timeframes: 5,15,30,60,240, and 1440.

[NOTE: you can also convert 1M data to timeframes not native to MT4 if you want to do some indicator analysis or something on another timeframe.]

Congratulations, you have now imported and converted data into MT4. Now, for the sake of illustrating one of my earlier points, open up a currency you have imported data on. Look at the difference in the bars from the downloaded data as opposed to data streamed in from a Demo broker [So, if you downloaded 1M data from July 04 to August 05, look at the chart at August 05's end and September 05's beginning]. You will notice that the bars (on every time frime if you have converted them properly) from your downloaded time period will be more complete.


------------------------------------------------------------------


Section 3: Configuring the Backtester

Now that you've succesfully imported complete data, there are a few more things you need to do to run a reliable backtest.

(1) Check the recalculate option the next time you run a backtest, because you need the backtester to utilize your shiny new happy data (which it won't do unless you tell it). Anytime you import new data, you need to recalculate (I recalculate every few tests just to feel safe, maybe its a reflection of internal confidence problems, but that's for another FAQ).

(2) Check the use date option and set the date range only over a time period where you have good reliable data. This way you're only backtesting the good stuff. It will be reflected in the modeling quality percentage.

(3) Make sure the model is set to EVERY TICK. If you're not, all this hard work we just did was for nothing. I addressed why we do this earlier in the FAQ.


------------------------------------------------------------------------


Section 4: Other Issues

MT4 is a work in progress, sometimes there are strange bugs that crop up in backtesting. However, usually when you think you have a bug on your hands, there is something wrong with your code. I can't emphasize enough how important debugging is. If you have problems, check your code first because it's probably the problem. If you really think you have a legit bug on your hands, post it to the MT4 forums.
Because you are not actually backtesting on every tick that happened [you are dealing with an interpolation on 1M data], it is still not a perfect reproduction of what actually happened in the markets. Because of this, 1M and 5M scalping EAs that get in and out of trades really quickly will run into some problems just because of this limitation. The longer timeframe you are trading on, the less likely your testing is to be hampered by this.
Well, that's all I can think of now. I read this over, I think I made everything clear and have the steps outlined correctly. If you notive a mistake, let me know, and I'll correct it in my next version of the MT4 Backtesting FAQ.


Acknowledgements:
I learned most of what I know about MT4 and trading in general from these forums and others like it. Thanks to all the people who contribute that have provided me with useful tidbits of information. There are too many names [and some of them are weird, have lots of numbers in them, etc.] to list, but a serious thanks to all the StrategyBuilder contributors out there.
Best of luck in the markets everyone.


In another thread here on FF mackdodger discusses and provides solutions on MT4 backtesting reliability:

http://www.forexfactory.com/forexfor...ead.php?t=5585
-soso
Alpari provides MT reliable data going a few years back. If you're serious abotut backtesting then you'll likely need it. Here's the link:

http://www.alpari-idc.com/en/dc/databank.php
-soso
It is possible to perform manual backtesting (bar by bar). Here's how to do it:

- first disable chart autoscroll (right click on chart window -> Properties -> Common tab -> Uncheck chart autoscroll option.
- scroll back the chart to the point where you want to start. Don't peek while scrolling
- now you can play the chart bar by bar using F12.
-soso
Quoting soso_beton
It is possible to perform manual backtesting (bar by bar). Here's how to do it:

- first disable chart autoscroll (right click on chart window -> Properties -> Common tab -> Uncheck chart autoscroll option.
- scroll back the chart to the point where you want to start. Don't peek while scrolling
- now you can play the chart bar by bar using F12.
I have always wondered how to do that. And it had to be that simple too.
JOHN 3:16
Ok, here is a dumb question I am sure. Do I need an Expert Avdisor for the backtesting. I have read through things explaining how to do it and I am totally lost. I would really like to learn it though.
JOHN 3:16
Yep, you need to do one. You need to know programming. Do you have any programming knowleddge? Not necessarily MQ4 or trading related.

Anyhow, there will be a thread strictly dedicated to MQ4 programming. If requested then I'll look to put general programming resources for true beginners too!

Later edit: Actually here's a link to a thread that will code the Expert for you: http://www.forexfactory.com/forexfor...ead.php?t=5990
-soso
I have recently started programming my EAs in for testing. I cant figure out how to get MT4 not to give me profits on spreads.

Say i start usdjpy at 114.00 with limit of 20 and stop of 20 and 1 lot. it will say i got $200 if it hits limit, but i have a 2pip spread it should be $180.

This makes a huge difference when you add money management to the test (eg. i want to risk 1% aviablemargin a trade).
You explained how to turn off the auto scroll. Now, how far back in time can a person go? I have up a 1 hr chart and I can only go back to March is that normal?
JOHN 3:16
Keep pulling of the chart. After a little delay you'll see that it scrolls, basically it reads more historical data from the MT server you are connected to. It can go a few years back.

Quoting renasdad
You explained how to turn off the auto scroll. Now, how far back in time can a person go? I have up a 1 hr chart and I can only go back to March is that normal?
-soso
Quoting soso_beton
Keep pulling of the chart. After a little delay you'll see that it scrolls, basically it reads more historical data from the MT server you are connected to. It can go a few years back.
i don't have the time but somebody needs to write a simple article on historic data and period conversion.

on alpari you can find 1min data for last 2 years, you need to convert it to 5min, 15min, 1h and then you got yourself a chart. and so on..

Update: actually somebody just did it in this thread:
http://www.forexfactory.com/forexfor...ead.php?t=5585

haven't read it, though.
Quoting soso_beton
Keep pulling of the chart. After a little delay you'll see that it scrolls, basically it reads more historical data from the MT server you are connected to. It can go a few years back.
When I try to go back in time, I scroll with the mouse. It goes back so far and it will stall out. Then it starts back at the beginning and I do it again. It keeps on going back to the same spot.
JOHN 3:16
Quoting renasdad
When I try to go back in time, I scroll with the mouse. It goes back so far and it will stall out. Then it starts back at the beginning and I do it again. It keeps on going back to the same spot.
you need to import historic data to go futher back. take a look at my previous post.
Quoting piccolo
you need to import historic data to go futher back. take a look at my previous post.
Thank you,
Will have a look when I get home.
JOHN 3:16
Coders Guru documents on strategy testing
Attached File(s)
File Type: pdf Coders Guru - MetaTrader Strategy Tester (Part1).pdf   397 KB | 5,003 downloads
File Type: pdf Coders Guru - MetaTrader Strategy Tester (Part2).pdf   657 KB | 5,126 downloads
-soso
Here's an interesting point of view of Joe Ross about backtesting:


It’s our job to trade "Futures" not "Histories"

"Throughout the years I’ve been trading and writing I've often written about mind set—having the right frame of mind for your trading so you become a winner.

I've stated that it is our job to trade "futures," not "histories."

The future is the next bar on your chart. You can't possibly know how it will develop, how fast prices will move, or where it will end up. Since none of us know where the very next tick will be, it's impossible to know where the tick after that will be, or the tick after that, etc. All we know at any one time is what we're seeing. Interestingly, what we're seeing may not be true.

If we are daytrading, we are not sure that what we're seeing is a bad tick, especially if it is not too far astray from the price action.

The daily bar chart doesn't always tell the truth, either. The open may not be where the first trade took place. The close is merely a consensus, and may be quite a bit distant from where the last trade took place. The high may not have been the high, and the low may not have been the low. If you don't believe that, then I challenge you to pick up any newspaper and take a look at some of the back months.

For example if the exchange has reported that a back month they opened at 9755, with a high of 9802, a low of 9760, and a close of 9784. Does that make any sense? How can the low be higher than the open? How can the close be higher than the high? Yet that's the kind of garbage we have to put up with in this business.

Now you know the problem with back testing. Back testing and simulated testing are based on nothing but lies. That's why they don't work when you actually put them to the test with real data.

In fact, there are many reasons why back testing and simulation won't work, and I may as well dump them in your lap right here.

Because you don't really know where the high or low were, or if the market ever really traded there, you don't know if your simulated stop was taken out or not.

If you say you have a system in which if you get three up days followed by a down day, the market will be up twelve days from now 82% of the time, then your whole statistical universe may have been based on what is not true.

Have you ever watched cocoa from the open to the close? You can clearly see it trading at the open, but by the time the market closes, the open will at times be placed opposite the close. That might be fifty or more points away from where you saw it open and trade, and also as born out by a report of time and sales.

The way they report cocoa prices is going to give a fit to a lot of candlestick traders. Why? Because they are going to see far too many "doji's" (open=close), more than are really there. Cocoa is not the only culprit, but historically, it is certainly one of the worst

When you see a completed bar on a chart, you have no idea which way prices moved first. You don't know if they moved down first or up first. You don't know whether or not prices opened and then moved to the high, went down to the low, and then traded in the lower half of the price range until the close, at which time prices soared up to the high and closed there. You have no idea of the overlap. I've seen prices trade from one extreme to the other more than once at each extreme.

In any of those instances, your protective stop could have been taken out intraday.

You know nothing of the market volatility on any given day, once you see a completed price bar. Were prices ticking their normal, exchange minimum tick, or were they ticking two or three times the minimum every time prices ticked?

Even if you purchased tick data for your simulation, showing every single tick the market made, you don't know what the volatility was. For instance, you don't know if the S&P was ticking five minimum fluctuations per tick or twenty-five minimum fluctuations per tick, and if it was doing it quickly or slowly. You don't know and you can't know, and anyone who tells you their simulated system works, based on such phony baloney, is a liar.

Not knowing how fast the market was means you can't really know what the slippage might have been. The faster the market, the greater the slippage. You can sit there and say that you would have gotten in at a certain price or that you would have exited at a certain price, but if you don't know the market volatility, and how fast the market was, you do not know enough to say that you would have done such and such. Not knowing how fast the market was, you have no way of knowing how much slippage there would have been on your entry or your exit. Without knowledge of slippage, you can't possibly know the risk.

That is also true of volatility. Volatility is made up of range of movement, speed, and tick size. If you don't know the extent of slippage, you will not know the extent of the risk you would have encountered.

As if that's not bad enough, you also don't know how thin the market was at the time you would have traded it. If you are position trading, you can't go by the reported daily volume (which is always too late to do you any good), because there is no way to know what the volume was at the time your price would have been hit. So here again you have no idea of what slippage you might have encountered, and once more you would not have known the risk.

If you want to spend your money on trading systems based upon the unknown, then you must assume the risk of doing so. Since this is a business of assuming risk, you are entitled to insure prices in any market that you care to.

Insurance companies spend a lot of money to make sure that the risks they take are actuarially sound. That is the equivalent of finding good, well-formed, liquid markets to trade in. But any market can become totally chaotic. Markets can become extremely fast, and they can become quite volatile. So even if your system was back-tested in a liquid market, when that market becomes fast and/or volatile, your back-tested, simulated system will not be able to cope with it and you will lose. It's like going out to write life insurance on a battle front.

If your back-tested, simulated system does factor in some room for fast and/or volatile markets, then, when you will be trading in slow, non-volatile markets with the built in factor, you will be utilizing a system that is totally inappropriate for the slow, non-volatile market you are in. The best you can hope for is an "optimized" system. How can you possibly expect to compete with traders who are acting and reacting to the reality that is at hand at the time?

Extensive back-testing is for historians, not traders. It is the wrong view of the markets. Your trading must be forward looking without being ridiculous about seeing into the future.

If you don't know where the next tick is, how can you possibly know where the next market turning point will be? Can you see into the future?

Maybe you like to trade astrologically. Those people are always trying to peer into the future.

In the auto business they have a saying, "There's an ass for every seat." Likewise, there's a fool for every fortuneteller who claims he can see into the future.

I guess you can always go out to your local coven and hire a witch to tell you what beans will do tomorrow. She may even be right from time to time.

You could always do as one charlatan did and run the biorhythm for each market based on the day it first started to trade. Or, you can cast the markets horoscope based on the same date. With the biorhythm, you'll know what time of day the market should be on its highs, and what time of day it will be on its lows.

You'll know which day the market will be ecstatic and reach a new high, and which day it will be down in the dumps and make a new low. However, you'll find that from time to time the market will reach new lows on the day it was supposed to reach new highs. Well, that's easy enough to explain. You can tell everyone "We've had an inversion. Until the market inverts again, the lows will be the highs, and the highs will be the lows!"

Source: http://www.fxstreet.com/nou/content/...menu=knowledge
does the backtester work with an ea that uses two time frames? for example, a M5 EA that uses an indicator in a M1 time frame.
Quoting soso_beton
Alpari provides MT reliable data going a few years back. If you're serious abotut backtesting then you'll likely need it. Here's the link:

http://www.alpari-idc.com/en/dc/databank.php

Hi

This is amazing. Could you just let me know how to install it. I downloaded one of the symbols and placed it in "tester" folder but the strategy tester does not use it.
Hi,

It is not enough to just download, follow the instructions from section 2 of the post http://www.forexfactory.com/forexfor...06&postcount=2 and it should work.

Quoting aparsai
Hi

This is amazing. Could you just let me know how to install it. I downloaded one of the symbols and placed it in "tester" folder but the strategy tester does not use it.
-soso
Quoting soso_beton
Alpari provides MT reliable data going a few years back. If you're serious abotut backtesting then you'll likely need it. Here's the link:

http://www.alpari-idc.com/en/dc/databank.php
This link seems to be dead, worked a week or two back.

Anyone know where to find this data now?

Thanks,
Ah.
0 traders viewing now