With backtesting, who needs forward testing?

Hi All,

Ok, I know this is an insane statement, but I only know it because I have read it a lot, I don't really know why! It seems at first glance that if it you can find a system that has worked consistently for the last 4 years (back tested), then why wouldn't it work moving forward. Even if market conditions change, forward testing isn't going to help, because they could change after you finish the forward testing. What is the difference between forward testing 3 months, or doing nothing for 3 months, then back testing those additional 3 months. And if there is no difference here, why not just pretend you thought of the system 3 months ago, and be sure and back test those last 3 months (as well as more) and that can be your forward test.

Ok, I am getting a little silly, but you get the point. I am sure I am missing something here... what is it?

The problem is actually finding a system, not a curve fit. You would be amazed at the ability of backtest optimizers to curve fit years of data, only to have that same system totally fail on forward data. If it was that easy we would all be rich (which obviously we are not )

Questions you could ask :

1 - Is there any discretion used in the system.

2 - Does the system open trades mid candle, by that I mean if you use a 60m chart are trades opened on the hour, or during the hour.

3 - Does the system rely on any indicators like moving averages, stochs, macd etc.

4 - Is there any discretion in closing trades, taking profits etc.

If you answer yes to any of these MT4 backtesting will be of limited use and not very accurate.

If however the system has a solid clearcut entry and exit based on the close of candles then back testing will be helpful.

One very important reason that backtesting will never give you an accurate picture is that it leaves out market conditions.
With forward testing you will have seen how your system performs under the evolving conditions of market pressure.
Past tick or price data is only a series points in time. This will never show you completely what it took to get to each point in time.

I think a very important reason is psycology... It's very easy to look at the past and see all the nice signals that would be profitable.
But when it comes to trade it realtime I find it difficult to see the signals, like I did in the past.

It's very tempting to think, if I enter here just before the signal is completely true then I will get in the market at a very good price and the profit will be higher, but BOOM then the market goes completely the wrong way.

Hi,
The below is a copy of my posting on the "Fozzies daily - backtesting" if you are interested.........

I have been trading live for about 18mths, not happy with the returns, so I have looked further afield to see what you guys in the Northern half of the world are up to.

Ratrace, I think you have asked a valid question, wherever you post it. I too would like to backtest Fozzies' way of trading. The problem is, having read Fozzies method (ok, ok, I'm only up to page 84), that the original posting Fozzy put out explaining his way of trading has changed so much over the course of the thread that it hardly resembles the original posting and I don't know WHAT to backtest.

As far as backtetsing goes Ratrace, take it from me, take a look at the last 12months of a currency pair that you think may suit the way you want to trade, if it looks ok, go waaaay back to at least the year 2000 and start backtesting up to the current date.

1) Don't forget to take into acct your spreads when calculating stops and profits/losses.

2) DO NOT CHANGE any aspect of your SYSTEM/METHOD as you progress through the years, if you do change even one little part of it, YOU HAVE A DIFFERENT SYSTEM....start again and throw out all the old stuff. This is why real backtesting is a pain in the rear end.

There is a word I have seen used a lot lately, the word is DISCRETION.
If you are going to use discretion in your way of trading, you are not using a system, ie: you are not using something that CAN be backtested. You cannot say "I have backtested my way of trading over the last 10yrs" if you use discretion. A system (or method - whatever you want to call it) is a mechanical way of trading where you take EVERY signal generated and don't deviate from the rules of your system, such as entry method, stop loss, risk, exit rules.
Some people are changing parts of their way of tading to suit the last loss. That is not a true reflection of how that currency pair has performed over the last 10 yrs!

Good luck.

With forward testing, who needs backtesting?

You cannot relive the past, but if you are breathing, you can be fully involved in the present.

Ask Lance Armstrong if he relied heavily on backtesting while training for the Tour de France. Instead, he was always trying in the present to beat his present benchmarks.

I dont' backtest because it ends up leaving me with more questions than it answers.

"cherish the past, respect the future, live in the present"
Mark Olivero

The problem with backtesting only is that it invariably works. This is because the majority of indicators most traders have available to them are based on historical data, not data moving forward. These historical indicators tend to mold themselves to the numbers coming out, at once justifying their existence and providing solid examples of backtesting success. Unfortunately the numbers coming out in the future tend to not support historical data until after the fact so unless you're prescient, you're going to make mistakes.

This is why most systems that rely solely on backtesting for their validity tend to fail.

My advice is to build your own system that you understand and which generates consistent profits for yourself. Do not dwell so much on what has happened but on what is happening. I would not advocate anyone going live with an account until they have seen at least a year of forward testing.

It's a harsh reality but at least 95% of forex traders lose their money. Be careful.

Hi Nathbear,

The theory of fwd testing and not trading live until you have at least 12mths worth of data sounds good, but how many times during the 12mths are you gonna change the system because it's not returning the results you had hoped for? This then means that your 12mths starts all over again because you have changed at least one aspect of your system. You'd never start trading!

When you decide to fwd test, you must have some rules / system / method to start with, where did that come from.......
Historical data I bet!!!!!

You are right, backtesting also has it's limitations, but I disagree with you on fwd testing.

I've done a lot of testing, fwd and back, live trading, lost money, won money and 95% of people lose means that there is more money out there for people who can crack it!!! I'd love to be able to give my kids a profitable system for their 21st.

You must've been happy with the mighty Maroons' come back the other night, the tables will be turned in Sydney!!!!

Good luck.

forward test your system forward for 1 month. then go run a back test on it. if you are within 250% on mt4 i will be surprised (even with 99% modeling)

there are 3 issues

1) do you have the tools to accurately backtest?
2) are you curve fitting, making the best results from the data you have?
3) is 1 month of data a sumation of the market?

if you can get these down, they you probably can back test

yep, as witchazel already said.. there is pretty much only one way to check it. forward test it on a live micro account and then backtest the same period on same data and see if the results match.. they shouldn't differ more than 1-2 pips per trade (due to slippage). note that MT4 does not save ticks and you are backtesting on modeled data. MT4 also doesnt have info on spread widening etc.. and when you test 2004 are you sure that eurusd was 2 pip spread then?

Backtesting is more like a science, hehe.

Aha!
Perhaps we need to consider if we are trading off short term charts (5min to 1hr candles) or longer term. I only trade off daily candles (so I need to consider other options when it comes to different types of data collection on other timeframes), 1 months worth of daily data, be it fwd or backwards, is worth nothing - would you agree?
Where as 1 months worth of 5min or 1hr candles is enough to get some data.
What do you think?
Cheers.....

Cr8 I backtested an EA with 6 years of 1 minute data with amazing results (13,300 pips EURUSD).... I still don't have exteme confidence in it.

Why? When you look at the price year by year, you get a different shape every time. I had 6 "shapes" in a row that did well, but what year will a "bad" shape appear?

Over time, I feel that any EA will show it's weakness. EA trading is gambling, and although I may have an edge most years, there is always that year that can wipe me out.

Also, backtesting, in Metatrader at least, is rarely accurate anyway. A day or two ago, I live tested an EA, and it opened a couple trades that lost money. I backtested that day, and the trades are either gone or completely different. Even if you do not get "Unmatched data errors" while backtesting in MT4, You can never really trust backtesting. At its best, even 90% modeling quality, it is only an approximation. Try demo live testing with various spreads, in various market conditions, etc. and you will have a better picture of EA performance.

Nearly all EAs fail the test of time on a live account. Most of these EAs probably did well on backtesting or they wouldn't have been traded live. I hope this answers your question.

Pete

Talking about backtesting........I've found an EA that seem very good on backtesting.....but i´m only get a 6.66% modeling quality on Daily TimeFrame and I use on M30.
Forward test EA continues giving good results. Two weeks ago I made 80pips on forward test, I made backtest with metatrader and get -60 for same period....so forward test is much better.
This is the backtest that I made from 2003 with default setting, if you can get a better modeling quality backtest on M30 please post it here.

Sorry, I haven't source code.

http://cablebot.webs.com/Performance2003-07.gif