Weird MT4/ Strategy tester behavior observed

Hey folks

I have recently had a very peculiar experience related to mt4 and thought I will add it here for expert comments by peers, and also hopefully prevent any similar mishap to other users.

I suspect that this might become quite drawn out so am starting a new thread.

Here we go...

I was testing an EA that used Constant Range Bars (using a commercial script).

Now I never rely on mt4 strategy tester back tests if an EA places market or limit orders as this would be different under live market conditions.
Since this particular EA placed only buy and sell stops at several pips gap, I thought the strategy tester should yield some reliable stats.

I ran the backtests and got excellent results - v excited to try it out live using micro lots on a live account.

Initially, it ran a winning streak, but eventually started loosing, which was totally against what backtests had shown.

I was mystified, to say the least. I was seeing the CRB candles form in front of my eyes during back test visual mode - and except for the odd loosing positions, the ea mostly came out the winner. It seemed as if somehow by some strange anomally the live trading candles were not forming as these were in the strategy tester.

After one of my loosing nights, (NY session) I decided to play back exactly the same data of the previous (loosing) night to try and see what the hell was going on. I took the same 1M file and put it in the offline folder to run the strategy tester. Guess what? All winners!! Where as in real trading, it had all been losses. Total of six trades, no room for coincidence here.

Suspecting an anomaly with the range bar script, I contacted the vendor of the script. He mentioned something that seemed to make sense:

Now you all know, when we test CRB based EAs in strategy tester we have to create an offline file with the range we want but name as m5 or m15 etc so strategy tester can access this... - the script vendor told me to save the offline crb file as a higher timeframe (such as h4 or daily). The reason he said was that mt4 used some kind of averaging of ticks within each candle (apparently hard coded), and that the strategy tester used these averages to rebuild candles - these averages were based on the standard mt4 timeframes, so - the larger the time frame, the more ticks would be recorded, etc.

So instead of running the strategy tester on 50 range bars saved as m5 offline chart, I did this as a 50 range bar saved as daily time frame, hoping that this would give me the real life scenario with ALL of the ticks as recorded in live data.

Result => more ticks in each candle, but still the same scenario - all winners in the back test compared with all loosers for the same period live trading.

So I called up mt4 support to try and find an answer. They cooly told me they did not support end clients, and that I should route my support request through my broker.

I have since done that (about 4 days back) with all supporting screen dumps etc. - and awaiting an explanation.

This is really a weird one. Facts:

1. We know that mt4 records live ticks (all of them) in m1 timeframe.

2. We construct crb candles from this m1 timeframe from a live trading account.

3. The candles build up differently during live trading vs playback through strategy tester even though the OHLC of the crb candles are exactly the same - so they look the same after the playback, but have been built (recorded) differently (i.e. the sequence of the ticks is misrepresented).

What do you all think? Deliberate conspiracy by metaquotes by having mt4 behave in such a way so as to lull us all into a false sense of security by having strategy tester present a rosy picture while the real life deal is not so sweet? Or perhaps, the brokers throwing in extra ticks into each bar which do not get recorded into the m1 timeframe some how?

Anyone here know inner workings of mt4 tick recording to throw further light on this?

P.S. - Please note that in all of the above, I have never used more than two months worth of data, which is the live data recorded with a live trading account running - so data quality issues can (I hope) be discounted.

I am adding the following for further clarity (hopefully), as I think the purpose of this thread is perhaps not being clearly understood by the readers:


Going by the posts here, it seems like I may not have been clear enough regarding the issue at hand here. Let me try again...

Firstly, the quality of data for backtesting is not at issue here. Some posts are suggesting ways of getting high quality tick data, which is fine, except irrelevant to the purpose of this thread.

Please re-read Post#1 to understand the issue.

All we are tring to establish here is:

1) Does MT4 record ALL in coming ticks in the m1 file, or not?

2) Is the data recorded in m1 reliable - in terms of being able to reconstruct, if required, the exact SEQUENCE of ticks (bid/ ask) that were coming in during a previous period (CAPTURED LIVE IN A 1M CHART).

3) Can the Strategy Tester playback the ticks recorded/ captured live in 1m file in a reliable manner in the EXACT SAME SEQUENCE as the ticks came in real time?

4) If the answer to any of the above is a 'NO' then is that a bug, a short coming, or a feature of MT4? Is this shortcoming an honest oversight or a deliberate attempt to tilt the playing field in favor of the brokers at a disadvantage to the retail traders?

5) In case he answer to any question in # 4 is a 'Yes' - for a product that possibly millions of users entrust with their hard earned money, shouldn't Metaquotes fix this and/ or at least disclose this shortcoming/ bias clearly?

6) If any of points 1 - 3 above are true, MT4 would fail even the very basic data integrity/ reliability criteria of an IT system?

7) Relating points 5 & 6 above, are the brokers aware of this issue, and are they turning a blind eye to the issue as it favors them in taking money from unsuspecting traders?

8) Finally, why is this a big deal? Read points 1 - 3 above - if this is an issue, then most strategy testing will provide skewed results, which shouldn't be acceptable to anyone of us.

We are lead to believe by Metaquotes that "m1 all ticks method" is the wholly grail of back testing your strategy in mt4 before trading live. If the 1m file integrity is at question, then any tests we do are unreliable and hence any trading we base on our strategy tests is flawed and will go against us. And how can you even trade live without reliably back testing the strategy?

Just an educated guess:

MT4 assumes that each change in price will be the result of buying/selling at least one volume-lot. Therefore if a specific candle has a volume of e.g. 100 it means, that price has changed within that candle 100 times.

MT4 knows the OHLC-data of each candle so now there must be a way from Open to Close via High and Low within exactly Volume steps.

MT4 tries to rebuild that way by random. The result is, when you have a doji MT4 can not know, what was reached at first - High or Low - it just makes a guess.

All information is primary related to M1, cause in each tick mode - every other candle is build by M1.

Hi Nightyhawk

Thx for your feedback. I am assuming that m1 does contain time stamp info for the ticks, so building a candle should be in sequence. I watched candles unfold through strategy tester (with substantial volumes) and it does quite a bit of dance from open to close - it goes up and down and up and so on - sometimes slowing down quite a bit, other times rapid. Doesn't appear random.

Though would be good to get a confirmation of this...

You need to use tick-generated history files. Search google for "Birt tick data."

If you consider for a moment, its not data quality we are concerned here with so much - we want data as is available in live account which is already there being (supposedly) recorded tick by tick in the 1m hst file.

So are you saying that a live account with a 1m chart open does not record ticks correctly in mt4?

When it not seems to be random then probably it is! Just have a look in any MT4 - M1 - History - File! There you'll only find the information I've mentioned above.

Seconded. Create a completely fresh MT4 install and use it for testing only. this can improve your backtesting results.

Also, you may want to consider investing some time in forward testing your strategies. Backtesting has its place but you may be surprised how often those results cannot be recreated in real time.

Back test -> forward test -> live test -> profit ->

Going by the posts here, it seems like I may not have been clear enough regarding the issue at hand here. Let me try again...

Firstly, the quality of data for backtesting is not at issue here. Some posts are suggesting ways of getting high quality tick data, which is fine, except irrelevant to the purpose of this thread.

Please re-read Post#1 to understand the issue.

All we are tring to establish here is:

1) Does MT4 record ALL in coming ticks in the m1 file, or not?

2) Is the data recorded in m1 reliable - in terms of being able to reconstruct, if required, the exact SEQUENCE of ticks (bid/ ask) that were coming in during a previous period (CAPTURED LIVE IN A 1M CHART).

3) Can the Strategy Tester playback the ticks recorded/ captured live in 1m file in a reliable manner in the EXACT SAME SEQUENCE as the ticks came in real time?

4) If the answer to any of the above is a 'NO' then is that a bug, a short coming, or a feature of MT4? Is this shortcoming an honest oversight or a deliberate attempt to tilt the playing field in favor of the brokers at a disadvantage to the retail traders?

5) In case he answer to any question in # 4 is a 'Yes' - for a product that possibly millions of users entrust with their hard earned money, shouldn't Metaquotes fix this and/ or at least disclose this shortcoming/ bias clearly?

6) If any of points 1 - 3 above are true, MT4 would fail even the very basic data integrity/ reliability criteria of an IT system?

7) Relating points 5 & 6 above, are the brokers aware of this issue, and are they turning a blind eye to the issue as it favors them in taking money from unsuspecting traders? Wouldn't this behavior then be criminal? Should the brokers and/ or Metaquotes be allowed to get away with this nonsense?

8) Finally, why is this a big deal? Read points 1 - 3 above - if this is an issue, then most strategy testing will provide skewed results, which shouldn't be acceptable to anyone of us.

We are lead to believe by Metaquotes that "m1 all ticks method" is the wholly grail of back testing your strategy in mt4 before trading live. If the 1m file integrity is at question, then any tests we do are unreliable and hence any trading we base on our strategy tests is flawed and will go against us. And how can you even trade live without reliably back testing our strategy?

1) No
2) No
3) No
4) Shortcoming. Honest oversight
5) I'm sure this shortcoming has been discussed many times on multiple forums
6) Then you should probably use something else.
7) I'm sure the brokers are aware of this issue. No, it definitely not criminal. They are not "getting away" with anything.
8) There are multiple ways to get accurate backtesting results in MT4 using tick data. Many of these ways have been well documented on this and other sites.

Nobody said life was easy. Sometimes you need to overcome adversity and adapt.

1 - No
2 - Maybe
3 - No - see #1
4 thru 8 - Have you thought of other possibilities such as incompetence or disinterest or the pressure that comes from unrealistic deadlines. That to me is more common in an organization than one that can organize and carry out a conspiracy.

I am going to quote from this link http://articles.mql4.com/446

The number of price changes on a minute bar is equal to the number of candlestick changes during testing. So, at each point of time the tester sees a correct price Open (which is fixed and is equal to Open[0]), correct maximal and minimal prices (which may change during the period of bar progress), equal to the current High [0] and Low[0], and the last known price Bid, equal to the current price Close[0], which changes before the bar closes. Volumes are also modelled maximally correctly which is obvious from the growing green histogram of volumes, like on a simple graph. The red broken line stands for the indicator Moving Average (simple moving average) with the period 1. This moving average shows the position of Close at each moment of time during testing.

My interpretation is that ticks aren't recorded in the 1M data. I have read others who use the term "interpolated 1M data".

Finally! Getting some relevant responses - thx zznbrm and code meister

Lets get some more view points - Meanwhile, I have asked these same questions from Metaquotes support through my mt4 brokers - I'll post their say on the matter.

1) No
2) No
3) No. The strategy tester can play back the exact tick sequence... but not with an M1 data file.
4) Intentional short coming. Tick data files are large and resouce intensive, on top of that, not everyone needs them. If you need them, invest the time and energy required to get them.
5) They do disclose this by clearly stating that this is M1 data... not tick data.

6+7+8) Dont sweat it. Now that your eyes are open google "Birt tick data" and get yourself set up.

Also, please consider forward tesing as part of your system analysis.

After re-reading your original post I think the only person/corporation with a possible mal-intent is the vendor of your script who is obviosly perfectly aware of the capabilities and pitfalls of MT4 and Metaquotes data.

I forgot to point out in my earlier reply that the shortcomings of the MT4 Strategy Tester usually aren't the source of most of my backtesting problems - its the broker historical data. The quality differs amoung brokers from usually adequate to usually bad. The main problem is missing data which show up as obvious gaps. The other problem with some brokers is spurious spikes which weren't in the original data and aren't present on other broker's data. You cannot depend on seeing the same results in backtesting from one broker to another and I'm not talking about the slight differences in pricing and spreads. This makes it tough to support software if the client is using a different broker.

Thx for the feedback guys - just to remind everyone, the malfunction I observed does not relate to large volume data - just one night's worth; six 30 range bars only!

Over on the renko thread http://www.forexfactory.com/showthread.php?t=279397&page=11 I ran into some serious problems with the 1m live chart.
Our set up uses a template created offline through which live data is sent from a 1minute chart running in the background, minimized and unseen.
I was well aware of the discrepancies in the backtester on historical data and if I ever have to backtest I use tick data from Dukascopy but these are live charts we are using in renko.
I opened 2 different mt4 platforms from the same broker. USDCAD looked completely different in renko. The 2 charts were completely different in live time. http://www.forexfactory.com/showpost...&postcount=152
The underlying 1 minute charts were different. One had huge gaps and missing data, but the renko built on it looked fine just very different from the other renko. It was a shocker. How am I to know if my renkos are built on reality or if I am just playing a video game?
So I am drawing trend lines and looking for breakouts in outer space... trend line? what trend? The 1 minute is making it all up....
This is live, not backtesting,etc.........................

jiva34, clearly your history data is all messed up.

It can be a number of things ranging from internet connection problems to mismatched bar data.

For starters you should not use the same installation of MT4 for generating/backtesting renko charts and for live trading.

Install MT4 to a new folder.
Set max bars in chart/max bars in history to 99999999 in options
Close MT4
Reopen MT4
Go to history center, download M1 data,
Close MT4
Reopen MT4

Additionally, the installation of MT4 that you use for generating/backtesting renko should ALWAYS remain OFFLINE. If it connects to your broker then it will automatically download some history data and symbol info. This data will override and corrupt the renko chart data (if the renko is on a standard TF).

Which MT4 installation had the huge gaps in it? Was it the one that you used to generate/backtest the renko?
If this doesn't help, post a screenshot...

HERE IT IS HOW TO Back Test Range Bars using MT4's built in Strategy Tester.


Excellent video.

I'm also experiencing a MT4 strategy tester issue, ( me thinks )

Using the Daily time frame to test EA, the log shows that the tester is running the EA on each minute.

Is this normal? or is the EA opening positions each minute instead of each day on strategy tester?