Manual backtest v EA testing

Afternoon everyone

I was wondering if anyone has got any thoughts on the difference between manual backtesting your system or coding an ea to test it?I am in the manual testing camp,not out of choice,just lack of any coding skills.For instance just say i tested the same system using both methods,what issues could there be when i started trading the system live(assuming the system tested possitive).

Depends on the kind of system. I assume your approach is not systematic (quant) and you're interested in testing a system you want to trade manually.

Automated backtesting equals automated trading, at least to some extent. Every trading strategy has a ruleset, but almost all of them require a significant amount of discretion at trader's side, including the most 'mechanical' ones like DIBS, Vegas' or TheRealThing's methods . That discretion is the hardest nut to crack; it's not completely impossible but it's very difficult to quantify and algorithmize all the essences backing a good decision. It's easy to code an EA trading upon indicator data and some price levels. But take price action basics at (real, not artificially calculated) s/r levels and try to achieve a fraction of eyeballs efficiency with an EA. Not so easy! Btw: how many long-term profitable auto-trading EAs running with a healthy risk/reward ratio do you see around? Not many, I guess. And how many threads teaching successful manual systems succeeded to create an EA for that system? Not a single one.

I'm in the manual testing camp, too. Out of choice ..some suggestions: it's good to test the system on data you've never seen before. That's hard to make if you stare at charts all the time since even a subliminal conscience of "what happened here and there" will tend to make you biased. And of course, you MUST NOT see the right side of the chart! If you think you can perserve objectivity, take a look at "Evidence Based Technical Analysis" by David Aronson (a really good book imo).

After comprehensive manual testing, I would try to isolate the few (if any) 100% mechanical elements of that method and try to automate them to provide guidance in manual trading.

But no backtest can beat a forward test on a small real account. Wish you the best of success!

MaryJane

My system is 100% mechanical and that is how i am going to test it,taking every trade.So the end results of a manual test and the ea test would be identical.I see the manual test as being superior to the ea(apart from the many hours carrying out the test).Just one of the things i see as a problem when you actually start trading the system is when you hit a losing streak.In your tests you will have a win/loss % in your final results,but what happends when you hit a streak that is far greater than the average win/loss of your final results?If you only had the results of the ea you might stop trading out of fear that the system has stopped working,well as you might have seen the same sort of streak before,several times(ranging markets etc,etc).

Of course i dont know any of this for sure.I have more questions than answers at the moment.Thanks for the response!

Mechanical systems are prone to huge drawdowns; that's a fact. As I have seen, most successful mechanical systems lose (the really good ones break even) most of the time and make up for their losses with relatively few HUGE winners. First you have to backtest it over a really large sample of data. Trade count is more important than time span, I would say 200 trades are bare minimum, 500 is better, 1000 is really good. Once you gauge drawdown characteristics and win/loss distribution, you can try to find some money management measures (risk adjustment) to improve the system performance during losing and winning streaks. Thorough tests help you to spot them in the future. There's a great advantage to manual tests; you learn the living behavior of the system. Maybe you'll find a key on how to classify 100% mechanically set non-trading periods and make it part of the rules. EA will show you the final curves and numbers but that's just dead flat data.

Manual backtesting can fail due to look ahead bias, where as the only bias EAs have is data mining bias (ie 10 random trades tested 1024 is expected to yield 1 perfect backtest assuming there is not spread)