MT4 EA - strategy tester - backtest results inaccuracies | Forex Factory

Options

Permalinks
Print Thread

Similar Threads

EA for manual backtest with MT4 strategy tester 267 replies

MT4 Backtesting Inaccuracies 8 replies

Huge differences in 15M/30M/1H results on MT4 Strategy Tester 0 replies

2009.03.11 15:41:15Tester memory handler: tester stopped because not enough memory 19 replies

Questionable Strategy Tester Results. 8 replies

Attachments: MT4 EA - strategy tester - backtest results inaccuracies
Exit Attachments

MT4 EA - strategy tester - backtest results inaccuracies

  • #1
  • First Post: Jul 24, 2007 3:31am Jul 24, 2007 3:31am
Hi

I've been testing my new EA in MT4.

The results that the backtest results produce in the strategy tester are dreadful!

I backtest using the "Every tick (based on all available timeframes with fractal interpolation of every tick)" model - which is supposed to be the most accurate model for backtesting within MT4.

On the chart, by hovering over the blue & red and yellow trade markets, we can see that MT4 automatically deducts the spread for each pair (eg cable = 3 pips, GBPCHF = 7 pips in alpari) from the open price of the candle of a trade signal (in an approximate sort of way - sometimes it seems to be 1-2 pips out +or-), which is not a BIG issue, because this may well happen with the broker anyway, when trading live.

However, the results produced within the results tab are often very inaccurate! it gives you the entry and exit prices after spread had been deducted. But the numbers don't add up!!

eg. buy (1 lot) GBP/CHF @ 2.3938. Close at 2.4093. Profit 1323.40. (Profit should be 155pips = 1550!!)
It seems to get a fairly high % of the profit calculation way wrong - some in your favour, & some not in your favour. But the bottom line must surely be that the backtest P/L numbers cannot be trusted in MT4 - right?


What is your personal experience regarding this issue - the accuracy of MT4 backtest results?

Many thanks.
  • #2
  • Jul 24, 2007 3:36am Jul 24, 2007 3:36am
Hello.

A few things. First, the "Test on all tics and interpolate to closest values" is only as good as the data you have stored in your MT4 database.

Each timeframe is stored seperately. If you do not have much M1 (1 minute) data in your database, your results will be off substantially.

MT4 does NOT deduct spreads from results, nor applies daily rollover premiums.

You need to read the sticky about backtesting for more detailed information.

Regards,
Tom
  • #3
  • Jul 24, 2007 3:57am Jul 24, 2007 3:57am

Hi, when trading gbp/chf your profits are in chf which is automatically changed to usd if your account is in dollars. 1550 profit / change rate usd/chf is equal to 13?? profit in dollars. Same is true for every cross. This is really the most basic for beginners. Before trying to find the holygrail in automatic trading, beginners, please learn the basics of the game. No offense.

  • #4
  • Jul 24, 2007 4:16am Jul 24, 2007 4:16am
Yes you should know that 1 pip = 10$ only for 1 lot of pairs xxx/USD.
All pips are not born equals

There is no problem in asking the question, the forum is done for that but please don't claim loudly about "MT4 backtest inaccuracies" because people may just read the title and get wrong ideas. Just ask before jumping to conclusions.
  • #5
  • Jul 24, 2007 5:05am Jul 24, 2007 5:05am
Quoting jlpi
Yes you should know that 1 pip = 10$ only for 1 lot of pairs xxx/USD.
All pips are not born equals

There is no problem in asking the question, the forum is done for that but please don't claim loudly about "MT4 backtest inaccuracies" because people may just read the title and get wrong ideas. Just ask before jumping to conclusions.
Thanks chaps!

actually i've done some backtesting on GBPUSD this morning and the results in the tester do seem accurate, so it must have been an issue with the GBPCHF - $ conversion.
  • #6
  • Jul 24, 2007 5:12am Jul 24, 2007 5:12am
Quoting tdion
Hello.


MT4 does NOT deduct spreads from results, nor applies daily rollover premiums.

You need to read the sticky about backtesting for more detailed information.

Regards,
Tom
Thanks.

i said -

Quote
On the chart, by hovering over the blue & red and yellow trade markets, we can see that MT4 automatically deducts the spread for each pair (eg cable = 3 pips, GBPCHF = 7 pips in alpari) from the open price of the candle of a trade signal (in an approximate sort of way - sometimes it seems to be 1-2 pips out +or-), which is not a BIG issue, because this may well happen with the broker anyway, when trading live.
What i meant is that lets say with GBPCHF alpari's spread is 7 pips.
If i get a long signal on the open of the new bar, at 2.4460, the backtest trade marker on the chart is showing that i got filled at 2.4467. It is consistently showing this 7 pip difference between the open of the bar (bid price) and my fill price. Therefore it seems to be accounting for the 7 pip spread on long trades (i.e. 7 pip difference between bid and ask price).
The same is happening on GBPUSD but with a 3 pip instead of a 7 pip spread.

Obviously on short/stopandreverse trades, you are selling, so get filled at or very clode to the chart bid price.

Are you saying that this is just a coincidence, and this is not in fact MT4 accounting for the spread?
If so, why is it showing this consistent difference "pip spread size" in the price the bar open at, and the price my long trade got filled at?

Thanks again.
  • #7
  • Jul 24, 2007 6:35am Jul 24, 2007 6:35am
It might be hardcoded into the EA. You can set the TP and SL to account for a spread, but the price won't always hit those values. (In other words, it isn't good practice.)
  • #8
  • Edited Jul 26, 2007 2:18pm Jul 24, 2007 7:09am | Edited Jul 26, 2007 2:18pm
I have a problem with build 207, tester generates ticks which are far away outside of bars, some times up to 26pips away. See attached images. I use tick-by-tick model, 90% modelling quality. Original Tester run on Daily TF. Tester produces execllent results but this is all far away from the real ...
Has anyone experiencing the same problem?

{images deleted, please keep to 600x600max}
  • #9
  • Jul 24, 2007 7:44am Jul 24, 2007 7:44am
Quoting tdion
It might be hardcoded into the EA. You can set the TP and SL to account for a spread, but the price won't always hit those values. (In other words, it isn't good practice.)
Thanks TD

so if the buy @ eg. 7 pips (the number of pips specific to the currency pair in question) above the candle open, as i described, is not alpari's MT4 taking account of the spread within the backtest, and is just a coincidence (seems like a very big coincidence to me), and lets say the spread is not accounted for in my ea, where do i make allowances for/set the spread?

Many thanks.
  • #10
  • Jul 25, 2007 1:02am Jul 25, 2007 1:02am
Quoting winters
I have a problem with build 207, tester generates ticks which are far away outside of bars, some times up to 26pips away. See attached images. I use tick-by-tick model, 90% modelling quality. Original Tester run on Daily TF. Tester produces execllent results but this is all far away from the real ...
Has anyone experiencing the same problem?
That can happen if the quotes you have loaded from the history are not the same than in live.
It happened to me in such a way:in one account I run a test based on historical data. Then I changed the account an run another test, then it still uses the previous data even if the data for the other account are different (I don't know if my explanation is clear)
Anyway to get rid of that usualy you only need to tick the box "Recalculate" and then run the test.
  • #11
  • Jul 25, 2007 1:10am Jul 25, 2007 1:10am
Quoting tdion
MT4 does NOT deduct spreads from results
I don't know what is meant exactly by "deduct" the spread and even if the spread is not deducted afterwards it is of course taken into account otherwise MT4 will be totallly useless.

OK, back to basics.
You buy at Ask and Sell at Bid (difference is the spread)

So jimbil, when you see 2.4460 on your chart it is the Bid, and if you buy, you buy at the Ask that is 2.4467 (normal if the spread is 7).

So MT4 tester take into account the spread. (nothing hardcoded in the EA)
  • #12
  • Jul 25, 2007 2:08am Jul 25, 2007 2:08am
Yes, I saw similar behavior when I loaded history to MT4 platform of the broker which uses different time, like Alpari history loaded to IBFX platform without time conversion. I use the same demo account for testing for about a year and did not load historical data from the history center for a long time as I often “Recalculate” data, so it gets downloaded automatically. Have a look attached tester results wich are far away to be true

Quoting jlpi
That can happen if the quotes you have loaded from the history are not the same than in live.
It happened to me in such a way:in one account I run a test based on historical data. Then I changed the account an run another test, then it still uses the previous data even if the data for the other account are different (I don't know if my explanation is clear)
Anyway to get rid of that usualy you only need to tick the box "Recalculate" and then run the test.
Attached Image (click to enlarge)
Click to Enlarge Name: StrategyTester.jpg Size: 23 KB
Attached File(s)
File Type: zip StrategyTester.zip   57 KB | 1,272 downloads
  • #13
  • Jul 25, 2007 2:18am Jul 25, 2007 2:18am
Quoting winters
Yes, I saw similar behavior when I loaded history to MT4 platform of the broker which uses different time, like Alpari history loaded to IBFX platform without time conversion. I use the same demo account for testing for about a year and did not load historical data from the history center for a long time as I often “Recalculate” data, so it gets downloaded automatically. Have a look attached tester results wich are far away to be true
I didn't spend time to look too much into details into the statement but I think that is the classical problem that we don't have tick data and even if you have 90% modelling quality, MT4 will "estimate" the ticks inside the M1 candle.
And if you have a scalping EA that is sensitive to 1 or 2 ticks then you will get wrong results.
Personaly, in order not to fall into this trap I use the following solutions:
- no scalping EAs with few pips targets
- use the open/close values of the candles (with 90% quality you have a reliable value every minute) and trigger your entries according to these points and not the MT4 estimates.

Hope it helps.
  • #14
  • Jul 25, 2007 2:31am Jul 25, 2007 2:31am
Quoting jlpi
I don't know what is meant exactly by "deduct" the spread and even if the spread is not deducted afterwards it is of course taken into account otherwise MT4 will be totallly useless.

OK, back to basics.
You buy at Ask and Sell at Bid (difference is the spread)

So jimbil, when you see 2.4460 on your chart it is the Bid, and if you buy, you buy at the Ask that is 2.4467 (normal if the spread is 7).

So MT4 tester take into account the spread. (nothing hardcoded in the EA)
Thanks jlpl

this is exactly what happens, and as you say, IT IS MT4 taking account of the spread.


I do wish FF members would check their facts before they reply to a question/query, as in this case i was right. Tdion was wrong and was making out that it was me who was in the wrong . Check your facts before trying to help/offer advice please .
  • #15
  • Edited Jul 26, 2007 2:19pm Jul 25, 2007 7:22am | Edited Jul 26, 2007 2:19pm
Quoting jlpi
I didn't spend time to look too much into details into the statement but I think that is the classical problem that we don't have tick data and even if you have 90% modelling quality, MT4 will "estimate" the ticks inside the M1 candle.
And if you have a scalping EA that is sensitive to 1 or 2 ticks then you will get wrong results.
Personaly, in order not to fall into this trap I use the following solutions:
- no scalping EAs with few pips targets
- use the open/close values of the candles (with 90% quality you have a reliable value every minute) and trigger your entries according to these points and not the MT4 estimates.

Hope it helps.
Thank you, your thoughs which are abolutely correct, I'll take them into account for the future developments. But it seems to me current tester issue might be related to internal bug. Please see image bellow to understand how it trades:

{image deleted, please keep to 600x600 max}
  • #16
  • Last Post: Jul 25, 2007 7:59am Jul 25, 2007 7:59am
Quoting winters
Thank you, your thoughs which are abolutely correct, I'll take them into account for the future developments. But it seems to me current tester issue might be related to internal bug. Please see image bellow to understand how it trades:
This bug was fixed by Metaquotes:
http://www.metaquotes.net/forum/3205/
0 traders viewing now