Does backtesting work?

Hi all,

Interesting topic and all of you gave very good answers and great resources on deciding how to handle backtests.

The purpose of using determines if it works or not.
1. I absolutely agree when you want to check the EA's methodology if it does what you expect from it then it is a great tool.
2. For checking the profitability of the strategy it is not so good, but not fully useless. If your strategy fails at backtest results it has a huge chance to fail on real trading. If backtest results show that it makes profit that means it worths the effort to see what will it do in real circumstances.

If you are ensured that your EA works properly and making the appropriate orders then it would be good idea to run it live and backtests parallely, to see the difference in the profitability.

Do you all use MT4 backtester module, or do you have more sophisticated program for backtesting?

backtesting works basically to check if the Ea functions properly and the code is correct. For the trading part of the testing I always prefer demo forward testing so I can see how the Ea reacts so I can see how it works and what needs to be changed or adjusted.

Agreed as it can be helpful to EA coders to make sure that the strategy they coded is working on the past trends however i think forex trading is purely risky where past results can't guarantee future performance.

I support this strategy of trading! Because, it saves traders time! In one month I practiced more than 15 trading strategies because of using backtesting system in my demo! But before starting back testing traders need to ensure their basic trading skill, don’t go for back testing directly after joining in demo!

In fact this back testing is to ensure either the developed strategy is working on the past results..

I never really believe in backtesting, I just focus on how the EA really work, not the profit/loss

Interesting topic. I do believe backtest could be viable in mt4, for measuring some probability of profitability. As explained, mt4 simulates tickdata inbetween open high low close for each candle. By having the EA only trade on next bar open based on previous candles signal, the tickdata problem should be eliminated. Other issues, as spreads and such would still be a problem. EAs that handles fixed levels and not trailing stops would be more accurate.

back testing is useful to see if the basic premise of the strategy works.. roughly how much profit for how much drawdown

its also very useful when you have been running the EA for an extended period.. say a year.. go back and backtest the same period.. see how the backtest performed against reality.. this can give some more confidence in your backtesting.

With an EA .. you have a worker.. a franchise.. you run them just like a worker.. you wouldn't flip burgers if you owned a burger franchise.. but you would check up on them to see how they are performing.. and intervene if required..So its the same with an EA.. you let it work for you and only intervene when there is a problem

With a well sorted EA portfolio.. you've just become a business owner.. congratulations

Forex back testing software is a program that uses historical data to recreate the behavior of trades and their reaction to a trading strategy. The resulting data is used to measure and optimize the effectiveness of a given strategy before applying it to real market conditions. Back testing in Forex works on the assumption that trades and strategies that have performed well in the past will perform well in the future.

If you want to test any trading strategy or system, in this case back test may work for you! Otherwise, it’s not! I personally used FT2 when I was a new Forex trader.

I do a lot of back testing, but I don't use my demo account very much. My back testing software can simulate a demo account pretty well. I think the only difference is that I can back test at whatever time of day I want, but demo Account trading is limited to normal trading hours. I prefer to do my real trading during my preferred trading hours, and experiment and try out new stuff on the back testing software during off hours.

Back testing can’t be alternative of the demo! Demo is the best way of learning and improving trading skill, but back testing can be helpful on strategies making! I think, in future, this sector will be more updated!

Some platforms, like NinjaTrader, offer a tick authentic Market Replay feature.

It really depends on your strategy. Backtesting for scalping is basically worthless because it does not include slippage and raised spreads. If your pip expectancy is 10pips or lower, you should not rely on back testing at all. I do think its a good indicator to show you how profitable your EA might be but you should still forward test it to confirm.

I think back testing can be useful, but it is often taken too far. It's more useful for system traders and traders that use indicators than traders who trade with price action alone such as myself. However, everyone backtests on some level though. Even if all that means is looking back at a few charts and seeing a pattern that has already occurred a few times. This is done to give the trader an idea that the pattern is capable of repeating some relative frequency. After all, you cannot gain an edge unless you have a positive condition that is prone to repeating itself after all. The problem with backtesting is that curve fitting is done way too often. To expand, the period of time under which the test is applied may or may not correlate to current market conditions. Many strategies can work under the right conditions, but no strategy works under all market conditions. Forward testing isn't fool proof for the same reasons. It doesn't matter whether it backwards or forwards. Once you trade live, you have no clue what kind of market conditions you will encounter.

Hi! I have a problem and i can't open new topic so please can you help me here.

I'm trying to test my strategy. I try to test it in a way as if I'm going to test robots just load my own template. It is important for me that the chart opens as in real life just faster. And there is a problem... The results are not the same as in ''real'' chart. The day shown in the ''real'' chart is like "smoothed" while that same day in the strategy tester is more rough and the indicator has more moves.
I'd like to know what the results are correct. The indicator is not important because I note the same problem in many of them.

Thank you!

Good point; but I got good result as a swing trader! By the way, I used this feature of trading after my demo.

I basically used my mt4, by the way; have any free software or platform?

Yes, backtesting works. What do you need to heed to? Too good to be true methodologies: Debunking good results.

Methods you use to try to "destroy" your strategy, in order to find your true alpha source or just debunk what is a false-positive result that sounds too good to be true.

For example, I have a strategy with the following metrics:

1. STD returns: 1.3
2. Beta to benchmark: -0.12
3. Maximum drawdown: 3.2%
4. Time frame: Daily, operates on last hour to closing with volume limits (<1% daily volume accounting possible impact in bps terms depending on the stock's average volume in the past X days)
5. Treynor Ratio: -15.97
6. Sharpe Ratio: 1.506
7. Information Ratio: 1.69
8. Excess VaR: 3.639
9. Omega ratio: 1177.28
10. Sortino ratio: 401.34
11. Kappa 3 ratio: 215.35
12. Gain loss ratio: 1178.2844
13. Upside potential ratio: 188.803
14. Calmar ratio: 133.81625

This strategy's framework uses the last 12 months excluding the last to calculate a metric that is tested in-sample month, then tested in the next out of sample-month with the same parameters. I've tried:

1. Analyzed individual trades
2. Adding extraordinary commissions and spreads for entry and exits
3. Changing datasets
4. Added random parameters
5. Added comparative s
6. Reduced trade size limit when trading near maximum permitted trade volume per % of stock's liquidity

It runs no optimization of parameters of sorts, but it does form an equation that is embed to the logic obtained from the warm-up 11 month period that precedes the testing month. Nothing changed the strategy's in a way that disqualifies its alpha, yet, extraordinary claims require extraordinary evidence.


Conclusion:

Look ahead Bias: How certain am I about timestamps. Has my data been backfilled? If you are using public fundamental data then it has probably been backfilled, and therefor useless.

Outlier Performance: Is the strategy only profitable because of a few outlier data points?

Have I simulated scenarios? History doesn't repeat. Strategies should be tested on various generated scenarios and synthetic data as well.

Do I understand where the alpha is coming from? Actual black-boxes shouldn't be traded with. Are the results expected, or am I justifying the results only afterwards.