How does one get a good backtest? | Forex Factory

Options

Permalinks
Print Thread

Similar Threads

How Does One get the Last Open Price of the Last opened order 1 reply

EA does not run in demo and realtime but in backtest is normal 1 reply

BackTest EA with more than one currency, possible? 0 replies

Visual backtest result different from non-visual backtest 0 replies

two version of FF tunnel Chart, one say down, one say up, which one is correct? 3 replies

Attachments: How does one get a good backtest?
Exit Attachments

How does one get a good backtest?

  • #1
  • First Post: May 7, 2019 4:46pm May 7, 2019 4:46pm
Why the blazes can't we get a good backtest?

There are instructions on to get a 99% modeling quality, but wow, what a bunch of hoops to jump through!

I'm very disappointed and just want to rant. Have a great indicator, programmed an EA, it wins dozens of trades in a row, 80% wins, downloaded 1M data, it looked great, running optimization all weekend, today threw it on a small live account... then realized why my optimizations have not been working all this week....

During the weekend runs I used 2 for spread. Once the market was open, I used "current". The "2" is 2 points, 0.2 pips! Why on earth would there be a choice to use 0.2 pips as a spread? Needless to say, when using the correct spread, instead of doubling the account, it drains it.

MQ4 programmers were unbelievably smart people, they really know their stuff. So how did backtest turn out to be an absolute clusterbump unless it was done that way on purpose?

I know MQ4 is a toy, but that doesn't mean it isn't powerful, and there are EA's that make money. I've seen a few that do, but they all use no stop, price averaging, and have insane drawdowns.

Sheeyit.
  • #2
  • May 7, 2019 10:56pm May 7, 2019 10:56pm
Backtest is nothing else than useful to check that your EA code works bug free over long periods. That's all there is. Backtest on different brokers, platform, forward test methods etc and your results will always be different.

The backtest you mean is live trading on cent accounts. Define your system with proper documentation, than let it run on a cent account for weeks. Define another system, and do the same. You end up with many cent accounts and solid feedback. Take the best system on a ECN and you make profits.
  • #3
  • May 8, 2019 1:59am May 8, 2019 1:59am
Quoting Fx01ab
The backtest you mean is live trading on cent accounts.
Wouldn't that be called "forward testing"?
  • #4
  • May 8, 2019 3:20am May 8, 2019 3:20am
Quoting blueruby
Why the blazes can't we get a good backtest? There are instructions on to get a 99% modeling quality, but wow, what a bunch of hoops to jump through! I'm very disappointed and just want to rant. Have a great indicator, programmed an EA, it wins dozens of trades in a row, 80% wins, downloaded 1M data, it looked great, running optimization all weekend, today threw it on a small live account... then realized why my optimizations have not been working all this week.... During the weekend runs I used 2 for spread. Once the market was open, I used "current"....
The problem is the default backtesting tool of MT4. It is interpolating 1m data which is useless if you have to rely on true tickvalues for sensitive systems like scalpers for example. Still most people use it and waste their time on results provided by this crap.

This does not mean backtesting is essential for evaluating a promising strategy. Do not listen to the "only do forward tests" guys - you do not have enough lifetime to spend when it comes to observation timeframes of more than 10+ years.

You CAN backtest having variable spreads plus slippage simulation on MT4 (which is the only way of having reliable results by the way)!.
Unfortunately there are only paid solutions out there that allow you to do so afaik.

Birts Tickdata Suite is the one I am using and I am happy with it. These tutorials to get 99% manually are a waste of time in my humble opinion too, as you still do not get variable spread and slippage this way. The 99% are misleading here. You get what you pay for. Simple as that.
  • #5
  • May 8, 2019 10:01am May 8, 2019 10:01am
Quoting Drolph
{quote} The problem is the default backtesting tool of MT4. It is interpolating 1m data which is useless if you have to rely on true tickvalues for sensitive systems like scalpers for example. Still most people use it and waste their time on results provided by this crap. This does not mean backtesting is essential for evaluating a promising strategy. Do not listen to the "only do forward tests" guys - you do not have enough lifetime to spend when it comes to observation timeframes of more than 10+ years. You CAN backtest having variable spreads...
Thanks for that, Drolph. So, my indicator works quite well on the 15M, but I still consider it a scalp, looking for 5-10 pips. Is it safe to say that, if it doesn't work on MT4 backtest, then it isn't going to work any better doing a really good backtest? Or is that something I won't know until I do it.

Running the EA with 15M candles, is tick data really critical?

If there's a possibility that it's a good EA, it's surely worth paying to test it.
  • #6
  • May 9, 2019 8:16am May 9, 2019 8:16am
Basically I would say a backtest won't get any "better" using meaningful data plus slippage to produce a backtest. The default MT4 backtesting tool leaves out most of the negatively influencing parameters. But to put speculations aside I fear you have to do it by yourself.

As you already said, the costs for having a reliable result for more than 10 years are nothing compared to the time you are wasting with forward tests.

For example let your backtest show a positive result for 1-2 years only. Now imagine you are foward testing your approach exactly within that postive timeframe only to find out it is not working longterm. I can only repeat myself here: Lifetime is sometimes the most important ressource you should not waste. And a good backtest helps you to manage your time much more efficiently by truly knowing WHAT to concentrate on.. and what not.
  • #7
  • Mar 7, 2020 4:26pm Mar 7, 2020 4:26pm
The optimal variant in any situation is testing on a demo and at the same time constant competent statistics maintenance. I do not understand at all how one can evaluate the result in good or bad format. I think that in this situation only figures can help you to orientate yourself in the situation and at the same time identify your mistakes, which you make all the time. Because it is clear indicators that can help in this work. It's hard for me to say anything about robotic systems, because I'm not in principle a supporter of this format, because I think that for your money you have to be responsible for yourself, and every time you have to analyze the situation personally, so that you can protect yourself at the right time. Although I may be wrong...
  • #8
  • Mar 7, 2020 7:04pm Mar 7, 2020 7:04pm
It's relatively easy to simulate slippage in your system. Just subtract pips per trade execution based on current price velocity to get a more accurate result.
  • #9
  • Mar 8, 2020 10:30pm Mar 8, 2020 10:30pm
As has been stated earlier, MT4 interpolates the price action between the only four date points is stores, ie OHCL.
If you want a more accurate portrayal of how your system performs in live trading, try writing your EA to also only use these four data points.

Personally all my EAs are set up to only make trading decisions on the Open of a new bar. They do at time look back to see what the OHLC of previous bars were, but are not subject to the vagaries of MT4 data interpolation.

A side benefit is that back testing can be done more quickly as the model selected can be "Open prices only", rather than control points or tick data.
" check out The Traders Outpost "
  • #10
  • Last Post: Jan 15, 2025 2:18am Jan 15, 2025 2:18am
Hello guys, i can't post a new thread, so i just reply here. I have long backtesting EA i created, and many times i encountered this error where the candle range only for like ranging 1.25600 - 1.25500 but in the backtest, the EA can order sell or close buy at outside the range, like selling at 1.26400 or closed buy at that price. Can somone help me how to fix this bug ? Thanks
Attached Image (click to enlarge)
Click to Enlarge Name: Screenshot 2025-01-15 150845.png Size: 44 KB
0 traders viewing now