Formal Trading System Development

I started this thread to present a formal approach to developing trading systems. I am currently a successful trader, but as for the future - who knows. Anyway because of my background I’ve been able to put together an approach to developing trading systems that is serving me very well. I should point out that what I’m presenting here is a method - I’m not about to divulge how I have used the method to build successful trading systems (for now at least anyway).
Every system has two crucial components - the inputs and the outputs.
Inputs are the indicators/price patterns/formulae/price data that you believe have some predictive power. And lets get something straight about the word predictive - it is synonymous with probable. People get hung up on semantics - the word predictive means probable.
If you believe that indicators have predictive power then those will be your inputs. If you are a price action trader then maybe inside bars etc. If you are using inter-market analysis then other instrument prices will be used - and so on. No matter what your approach you have a set of inputs - even if it is something as intangible as intuition.
Outputs are more interesting in many ways. Most people just want a system that has a net pip gain. But other outputs are equally as important. How big is the drawdown? How long does it typically last? How straight is the equity curve (sharp ratios etc)? How big are the stops? - and so on. If you are manually creating a trading system it becomes very difficult to estimate more that a couple of outputs - for that reason I use several software packages to do the donkey work.
Next we come to the mechanics of using historical data. You must split the data into two parts. The first, and largest part is the training data set. So say we have 20 years of daily bars. We might use the first 15 years (1990 - 2004 say) to train the system. No matter whether you build your systems manually or using software the principle is the same. You tweak parameters, modify the patterns you are looking for, change indicators and so on until you find something that works. All trading systems are based on this principle whether you are a price action trader or using neural networks. The problem is that we home in on a set of inputs that work for the data set we are looking at - but does it have general application or is it just a curve fit? Have we tweaked until we’ve made it work just for the training data set?
This is where the last five years of the data comes in handy. The test set, as it is often called (or out-of-sample data) is then used to see if the system built using the training data still works on price bars that have not been seen before. If it does you can be fairly confident that the system is viable. But be warned, many systems built using the training set will fail on the test set. But at least you will not start trading a duff, curve fitted system.
I only trade daily bars simply because I do not want to watch a screen all day. Below is the equity curve for the GBPUSD - and this is one of twenty systems I trade. The last 20 trades (about 10 months) are live trades I have taken - profitably, as the curve shows. The verical line on the equity curve divides the testing data set from the training data set (the largest part).
This is a mechanical approach to trading - I am hopeless at discretionary trading, and so I know that some will find this over-formal - but it works for me and may be of help to others. And just one other thing. Let your imagination soar when it comes to considering inputs and outputs - none of the usual stuff worked for me.

Ok, so unless you just wanted to brag about your 45 degree angle equity curve, how about sharing one of your many profitable systems with the community.

I'm not being mean, I just wanted to know the purpose of this thread with a big 'look at my demo equity curve' screenshot.

Cheers.

Nice thread and I am interested in following this. I am ardently back and forward testing Donchian channels at the moment. I know they work and work really well with the right parameters but I am trying to achieve the best parameters without curve fitting.

You have an interesting approach by using 15 years of data for the initial test and the last 5 years for the final test but are you not concerned about the integrity (read completeness) of the data available for the early years? Also will not your inputs possibly have to change as conditions of the markets you are trading change? I have my own theory on my last question but interested in your view.

that comes close to my conclusion. when developping trading systems is not your logic that works it's the logic of the charts that has to be found which often can be contrary to what you started with.

Well I went to some trouble to explain back and forward testing (in-sample/out-of-sample) because most people in forex don't seem to realise that backtesting is not enough. The equity curve is a bit brash I admit, but it is just to show what that forward testing should bear some resemblance to a back test.
Obviously I'm not going to give away the crown jewels (if indeed they hold water in the long term), but I will be showing some simple pattern based results that traders can use/develop if they wish. The aim being to pass on some experience and hopefully get some (constructive) feedback. What do I get out of it? - some fresh input, and there is nothing better to clarify your thoughts than explaining to others.

In my opinion all trading system development is a form of curve fitting - it's just whether the curves represent some underlying reality, or just happen to be a property of the data sample you are looking at.
I have to admit that I despared of finding parameters for indicators and so my inputs are parameterless - they either work or they don't.
As for changing market conditions - well - you only know they've changed in retrospect and so I developed inputs that have invariant behaviour in strong trends and tight ranges. Time is a much more important input than most folks realise - time based systems offer some interesting angles. The price I pay for this is large stops, and since I only trade daily bars my stops may be as high as ten per cent, but typically six (this usually accommodates any tight ranges).

Ok, good answer. Not sure about your stops bit though, some of the donch channel testing I have done suggests that an initial stop of just 0.5 ATR can be the most profitable on some pairs.

I dream of .5 ATR stops, but tight stops almost invariably go with a high loss rate (doesn't mean the system is not viable) and psychologically I just could not take that. Rashke is on a video somewhere talking about stops (probably youtube) and she make the obvious but unpopular observation that tight stops mean more losers - but of course the extra leverage means the winners make more.
I'm hoping to post some simple pattern based systems over the coming weeks - nothing spectacular, but with positive expectancy and good illustrations of the process I descibed in the first post.
I should add that because my systems typically come in at around 70% winners I happy to average in to a trade - as it goes against me I add. I know it's heresy, but it works if you know most trades are eventually going to come around to your way of thinking.

I'm getting around 8 to 15 losers in a row but with a drawdown of between 10 and 20% and a strike rate around 30 to 40%. Lots of small losers and a few large winners. I think if you're only risking a small % of your account on each trade then a string of losers is less of a worry. Of course a good non correlated portfolio will help too.

I had an EA coded for backtesting purposes and I am awaiting an update to it which adds on after each valid new break. I need to test that yet to see how or if that increases the profitability and then I go live.

It isn't a demo equity curve - I trade this as I've already stated.