How do you trust the historical back testing data | Forex Factory

Options

Permalinks
Print Thread

Similar Threads

Mismatch between live forward testing and back testing 4 replies

EA that computes swap losses (or gains) in historical testing? 0 replies

I have historical Bloomberg data back to the 1980's, here's what I need. 2 replies

What do you use for historical data/graphing? 0 replies

Historical chart to track historical news events. 0 replies

How do you trust the historical back testing data

  • #1
  • First Post: Jul 23, 2017 2:11pm Jul 23, 2017 2:11pm
Hello All,

I recently starting learning to program and test automated trading systems in NinjaTrader. I am using the free historical data that comes with the platform for back testing.

My trading strategy I try to develop are usually with RR>2. I do not develop scalping systems cause I have notice lack of intrabar granularity (incorrect price movement in the small time chart bars) can lead to incorrect profit or loss of the trade.

May I please have your honest and recommended opinion/advice to the following concern:

Questions:

1. How did you gain confidence, reliability and consistence in the backtesting historical data from your platform? Currently, i am developing intra days strategies (all positions closed by end of market session) and rely on platform free historical data to provide accurate back test results for strategies from the 3 min to 1 hour chart.

My Plan Test datafeed reliability and accuracy:
1. Choose a range (1-4 months) from the back test a strategy where there was lots of shorts and long trades, equally. About 100-200 trades. Document each trade.
2. Market Replay the range in step 1. Document each trade.
3. Manual Back test the range from step 1. Document each trade.
4. Compare each documented trades from the previous steps for accurate.

In conclusion, I hope for 90-100% accuracy between each documented performance. Likely, step 2 and 3 should be very similar.

What did you do to gain confidence that every time you click "back test", you had some belief in the performance report?

Please share your comments or thoughts.

Thanks
  • #2
  • Jul 23, 2017 2:15pm Jul 23, 2017 2:15pm
IMO, you don't. Backtesting is mostly useless since what happened in the past has no bearing on future events and it most often leads to curve fitting. Forward testing makes more sense. That's just my opinion.
  • #3
  • Jul 23, 2017 2:42pm Jul 23, 2017 2:42pm
Quoting Madinvestor
IMO, you don't. Backtesting is mostly useless since what happened in the past has no bearing on future events and it most often leads to curve fitting. Forward testing makes more sense. That's just my opinion.
Thank you Madinvestor for your comments.

Yes, forward testing does make more sense; however, most system developer depend +4 years of historical back testing using software simulation to have an idea if the strategies has odds of profitability, before moving to forward testing.

Forward testing may not produce enough trades for true evaluation of system future performance whereas back testing is more senseable.

Do you not back test at all, only forward testing?

Thanks you
  • #4
  • Jul 24, 2017 4:16am Jul 24, 2017 4:16am
Quoting goodoboy
{quote} Thank you Madinvestor for your comments. Yes, forward testing does make more sense; however, most system developer depend +4 years of historical back testing using software simulation to have an idea if the strategies has odds of profitability, before moving to forward testing. Forward testing may not produce enough trades for true evaluation of system future performance whereas back testing is more senseable. Do you not back test at all, only forward testing? Thanks you
Your analysis is correct. Unfortunately mt4 has some limitations for testing automated strategies, eg. you can't test with the actual spreads, tick data isn't reliable etc. but if you use open prices and you triple the avg spread it's pretty good.
  • #5
  • Jul 24, 2017 5:31am Jul 24, 2017 5:31am
Quoting goodoboy
{quote} Thank you Madinvestor for your comments. Yes, forward testing does make more sense; however, most system developer depend +4 years of historical back testing using software simulation to have an idea if the strategies has odds of profitability, before moving to forward testing. Forward testing may not produce enough trades for true evaluation of system future performance whereas back testing is more senseable. Do you not back test at all, only forward testing? Thanks you
No I don't really backtest much. If I am trying a new strategy, I manually scan the charts of a few different pairs to try and get an idea of if it makes sense to forward test it. I guess you can sort of call that backtesting, but I don't go through years or data or anything like that. It's an eye test and it is relatively quick and dirty. I then test it going forward under the current market conditions to see if it works and tweak it along the way. Then again, I am a manual trader. I don't use EA's or anything. Just S/R and price action. Backtesting is probably a necessity for system traders.
  • #6
  • Jul 24, 2017 8:16am Jul 24, 2017 8:16am
My whole trading system was built upon backtesting historical data, so anyone who claims it is useless, out of hand, as one poster here claims, clearly doesn't know what they are talking about. Backtesting alone of course will never create a winning trading strategy, but forward testing your backtest results will, and this is achieved through much trial and error. As we all know, technical patterns repeat themselves over and over, not exactly of course, but with variation upon a similar theme. Once we can recognize these patterns as they are forming, we can begin to predict price action, and so backtesting, for me anyway, and I've been trading for ten years, has been and to an extent still is, the building block of everything I have learned about price action.
  • #7
  • Jul 24, 2017 9:27am Jul 24, 2017 9:27am
Quoting cat
Backtesting alone of course will never create a winning trading strategy, but forward testing your backtest results will, and this is achieved through much trial and error.
Thank you cat for your comments and very good response.

Can you please go into detail a bit regarding "forward testing your backtest results"? Does this mean you forward test for about a few weeks, then back test those few weeks and compare the results?

Thanks for input.
  • #8
  • Jul 24, 2017 10:11am Jul 24, 2017 10:11am
Quoting goodoboy
{quote} Thank you cat for your comments and very good response. Can you please go into detail a bit regarding "forward testing your backtest results"? Does this mean you forward test for about a few weeks, then back test those few weeks and compare the results? Thanks for input.
What I meant to say is that forward testing is live trading, whether on demo or not. By forward testing my backtesting results I can begin to bridge the divide between what I see in hindsight, and how I see it jumping in blind as it were. The two are obviously very different, but only because one carries vast uncertainty with it, whilst the other doesn't. Whether one is backtesting or trading live, one's indicators are still there, so your live moving averages for example are more or less in the same position, allowing for the drag factor one sees in historical charts where the smaller moving averages have been dragged into a position a little at adds with live data, but more or less, they show the same S&R and the same positioning as one sees on the historical charts. Once a trader has spent many thousands of hours studying and analyzing historical charts, and the same amount of time forward testing the results, distinct patterns become cognitively recognizable on a live chart despite the absence of any forward data, as the brain has enough retrievable information to be able to process that data to recognize and act upon trading opportunities.

This is why I have always maintained that most people who fail at trading fail not because of psychological frailties, though this undoubtedly plays a part, but because their brains have not processed enough information about the markets to be able to make informed trading decisions, and in my own experience, backtesting greatly facilitates this process. Until it has, a trader is merely gambling and it will be impossible to turn a consistent profit, in other words, traders will just lose money.

Hope that goes some way towards answering your question.
  • #9
  • Jul 24, 2017 3:44pm Jul 24, 2017 3:44pm
Quote
What I meant to say is that forward testing is live trading, whether on demo or not. By forward testing my backtesting results I can begin to bridge the divide between what I see in hindsight, and how I see it jumping in blind as it were. The two are obviously very different, but only because one carries vast uncertainty with it, whilst the other doesn't. Whether one is backtesting or trading live, one's indicators are still there, so your live moving averages for example are more or less in the same position, allowing for the drag factor one sees...

Thank you cat for your response.

What are your views on developing intra-day swing trading (1 to 2 trades per day for big daily moves), scalping (many trades per day for 8 or less ticks), and swing trading (holding trades over night for bigger targets)?

Most of my experience is on the 5 min chart intra-day swings trades for big daily gains.

Thanks
  • #10
  • Last Post: Nov 7, 2017 2:22am Nov 7, 2017 2:22am
But History always repeat itself. But yes time variation may occur. But it must do . So if you back test it with historical data,will empower you more.
0 traders viewing now