One Way to Develop a Winning System

In this thread I'm going to discuss how I approach developing a winning system. This isn't the only way it can be done so it is "One Way", not "The Way", and so those who have differing methods for developing systems are free to chime in, but let's try and keep the topic on this particular way of doing things. My hope is to inspire others to develop their own winning trading system, and to receive valuable feedback from others who have found "a better way". However this thread isn't about developing a particular system, and as such will lack the sex appeal and inflated claims of the many "system" threads found on this site. If you want sex appeal, I ask you to look elsewhere.

Before I start, a little about myself. I've been trading full time for over 10 years, the past 3 in the forex market. My main focus is on developing automated trading systems and implementing these systems in the forex market. I'm not soliciting work, selling a system or otherwise trolling for customers, but if you happen to have a large wad of cash just burning a hole in your pocket then maybe we should talk.

I'll start off by going through a list of steps that may lead one to create a winning system. The first step is probably the easiest as well as the hardest.

The first step to accomplishing any goal is to first determine what you want to accomplish. In this case my goal is to develop (idea into program form) a fully objective trading system that uses high-frequency data to take advantages of the inefficiencies in the forex market.

For me the system must have fully objective rules that can be defined and programmed, tested and automated. I’ve found through experience that while my own discretion can be very good at times, it is at its best when paired with a good fully mechanical system at its base. And I prefer to allow the system to just do its thing, and use my discretion to foster improvements or newer, better system versions, rather than in actual trade implementation.

It needs to have a reason for working. I’m not about just running a bunch of optimizations, tests on random “elements” or things like that. If it is going to work, there must be a reason why it should work. And just so you know, just because something “should” work in theory doesn’t mean that it does or will work in practice. In fact the rule seems to be that most things don’t work in practice. This is a way to test my assumptions and to determine if my beliefs about how a particular market “should” move, is accurate or not.

The system must be robust, meaning that it works across a wide range of variables rather than depending on a magic, though uncommon linkage of events. I want to know that if the market is misbehaving, or not behaving as it usually does, that the system will not toast the account.

The second step is in clarifying what you want. Once you have a broad idea of what you want, it is time to refine the idea down and create an outline and develop an image of what the system might look like.

What markets will this system trade? What timeframes will be used or favored? How many trades will the system make and in what space of time? Will the system follow the trend or trade against the trend? Will the system be a stop and reverse system, a setup, stop, target system, or a grid type system? There are lots of ways to make a profitable trading system.

For me I’m looking to develop trend following systems. So I want to focus on the most trendy trading instruments. This means focusing on the currency pairs that carry traders focus on. The reason for this is that the carry trade creates an underlying fundamental reason for the currency to trend. This is a sort of bias that exists in the direction of the carry trade, and this natural bias is an inefficiency that I can exploit in developing my trading system.

I favor high frequency trading systems and all things being equal, shorter timeframes with more trades. I like high frequency trading systems because I’m more short term oriented, and I like instant feedback, smaller drawdowns, and faster equity increases from “banking” a trade. I prefer a SAR method, where longs are closed out by initiating shorts. I also prefer focusing on times of day when the markets are most active. In the latest system that I’m working on, I tend to focus on a 10 hour block of time that coincides with the most active trading. I realize that in a high-frequency trading system volatility is an ally and lack of activity is an enemy.

Because I’m focusing on high frequency trading systems, it makes no sense for me to focus my time testing longer term data. My testing is best when it is suited to the timeframes that I’m focusing on.

Furthermore, because I’m focusing on trend trading, I’m not keen on most indicators including MACD, RSI, Stochastic, etc., which actually are measures of momentum and not of trending. The moving averages aren’t my favorite either because they lag significantly, which is a bad thing when trying to catch trends using shorter timeframes.

Because I’m a trend trader I’m focusing primarily on breakouts leading to continuation. When the opposite breakout definition is reached, the position reverses. Although I prefer high win rate systems over low win rate systems (due to the smaller drawdowns), my experience has shown that lower win rate systems tend to be more robust, and often more profitable. So I’m happy trading a system that has a 40-60% win rate.

I'll pause here while everyone catches up and to see what comments or questions are out there.

Good stuff, Thank you and look forward to following the thread.

Thanks Rosco, looking forward to some great discussion!

I'm very confused by this thread.

With all your experience, is there some more focused explanation of your systems which you can share?

Appreciate your comments. This thread isn't about any single system or my system's rules. It's about system development, or more specifically my method of system development. I think I have a lot to offer in that area. Feel free to ask me any questions on that topic. If you need any other help, let me know.

Great stuff FXEZ!

I asked a question a while ago, but it's still irking me. You said:

"I prefer to allow the system to just do its thing, and use my discretion to foster improvements or newer, better system versions, rather than in actual trade implementation."

I'm in this stage right now with a system I'm developing and I'm stuck on a couple issues.

1. Over optimization A.K.A. curve fitting.
I'm always concerned that my 'brilliant idea' is simply dealing with the issue(s) I see occurring and that the fix for this one issue may cause further issues later. In other words that I'm being re-active to the market and thereby curve fitting to how the market is behaving rather than actually improving the system. Is this a non-issue?

2. I have a new revision that has less drawdown, but waaaaay less gains.
I know this is a personal question, but is there a somewhat objective answer? If I cut my drawdown in half but my potential gains to 1/10th am I really improving the system?

3. Metatrader backtest SUCKS
I know this is a flawed system. Are there other software apps on the market that I can backtest with to doublecheck my results? I've been forward testing for about 4 months, and the backtest for the same period is very close to identical, but I still don't trust it :-) any suggestions?

Thanks in advance for the answers, and thanks again for the great thread!


Seavo

Like Stunew Me myself also confused??? Could you explain in a way that a bit easier to understand

Very good stuff!

I think everyone at FF can learn a lot from what you are trying to teach here..

I would like to make one point though before you continue on..

It is not until people realize that great systems are not built around the accuracy of a system, but the expectancy, that they themselves are able to design winning systems that work over the long term. This, in my opinion, is one of the most important factors to keep in mind, and I in that respect hope you agree with me.

Thanks for the questions Seavo. The first one is a really good one so I'll just reply to that one in this post and follow up with the other two later.

Optimization can be helpful and can also be a proverbial black hole that is easy to get lost in and hard to get out of! My best use of optimization is in determining if a system has a stable series of "good" results, and to determine if the idea in general shows merit or has a chance to be profitable. I don't believe in using optimization to fine tune a system. I like using it in the broad development, but in the fine tuning of a system too much optimization can cause problems. The main problem it can cause is creating a system that is perfectly curve fit for past data, but which doesn't behave very well with live data. That's one reason why I don't favor choosing the result with the largest profit. I prefer choosing parameters that produce the smallest drawdown, and most importantly parameters in a stable optimization zone - or in other words where lots of nearby parameters "work" well, and not just the best variables.

The other thing about optimization I don't like is that by overusing it, the system you are working with can suffer because development will tend to focus on the optimization of each variable rather than the logic used in the system and how each piece of the system fits together. I believe that it is in fitting the pieces of the puzzle together in a unique, sometimes unorthodox manner that really creates value in a system, rather than the parameter set used. Though I will admit that when testing those unique and unorthodox combinations, it doesn't hurt to run through a large # of parameters to make sure there isn't gold hiding in there somewhere that may be otherwise overlooked!

One thing to keep in mind is that the best system is the one that fits the upcoming data the best. So all systems are attempting to fit to the data to some degree. And markets do tend to have characteristics that are identifiable and that tend to repeat. Take the extreme example a system that only goes long. Place it on an instrument that is in an up trending phase and the system will look brilliant. When the down moves start the system will no longer be as brilliant even though it is the same system. The difference is that the market's behavior changed but the system is locked into the "up trend" mentality. Markets do this all the time but there tends to be a rather long time between MAJOR market changes. If you take this very same up trend system and include some logic to have the system cease trading when certain up trend conditions don't exist, the system becomes even better fit to the market and will tend to do better. But there will always be some moves that don't correspond to the defined 'trend conditions' that will cause the system to draw down. When developing a system the trick is to choose the combination that will minimize the likelihood of future large drawdowns, while accepting the reality that there will always be some degree of drawback or other that is unavoidable, no matter how well written the system. Most of that has to do with the fact that hindsight is typically 20/20 but foresight is often less than 50% accurate due to the uncertainty principle.

So I think it is good that you're both aware of and thinking about these issues. It is true that changing the system in one tiny area can often cause large changes in the overall results of the system, and not always what was intended. So each "tweak" needs to be tested in a thorough manner, and always retain the ability to go back to the previous state of the system so as not to lose the "magic" when you get on a tweaking frenzy. My best approach is to do a series of tests on each component of the system, making note of what eliminations "break" the system and what eliminations help or are system neutral. It's often surprising to discover what combination of elements is actually responsible for producing the profits in your system. But I can't give you a definite answer whether you should or shouldn't be concerned because it really depends based on your system's design.

I'll just add that one thing that took me a long while to become "OK" with is that having different rules for a buy long trade and a sell short trade isn't over curve fitting and is perfectly fine. This is particularly true if the market you are targeting has a natural fundamental bias toward either up or down. And I'll just add this bit that often using different buy and sell conditions will improve the overall system because strangely enough to a purely logical mind, up and down often have very different ways of behaving and that difference tends to persist.

Thanks for asking this most thought provoking question!

Patrick

Hi Seavo,

If you are dealing with a single-lot system then it is easier to answer question #2, and the answer can be virtually objective. Look at the Net Profit / Max Drawdown number. The higher the number, the better the system. If you're working with a multi-lot system figuring the drawdown can be a bit trickier but is still doable. But the difficulty comes in determining which system will actually produce greater gains in real trading. To do this you may need to first determine the max trade size for a given account that you would be willing to trade for each system. And to determine that you need to set a max account drawdown you are willing to accept. Let's say you're willing to accept a max 25% drawdown on any given account. Then it becomes a math problem based on max drawdown for each individual system to determine max trade size.

For example let's say you have a $100,000 account and will accept no more than 25% drawdown (or $25K) on that account, and you have a system that produces a 500 pip max drawdown during its tested "life". For a single lot system, max trade size may be $25K/500 or $50 per pip. If your system trades a max of 10 lots, then you would trade a max of $5 per pip on this system. Systems which trade a variable # of lots can make it impossible to plan in advance, so a good guess at "max lots" must be used.

The answer to #2 is probably not but you can do the math. Let me know if you have questions or need help with running your specific numbers.

Metatrader is a great free program but you're right that the backtesting can give false answers. I don't use Metatrader backtesting. Way back I used TradeStation for system development and testing, but was similarly fed up with its inability to handle bid/ask data. So I ended up creating my own testing environment in visual basic and collecting my own high frequency bid/ask data. Probably the most important ingredient in reliable testing is reliable bid/ask data for the instrument/broker you are going to trade. Even with 1 minute data you have to make assumptions about the bid and ask, and from my testing the difference is huge, particularly when you're looking at data shorter than 1 hour. Forward testing is a great option with metatrader, but if you can collect your own bid/ask data and test it in an environment like Excel using VBA or visual basic, that's the route I'd suggest. Not easy but I'm glad I did. There are probably others who have gone the metatrader testing route (upping modeling quality, eliminating rules that give bad backtest results), and I'm sure there are a few threads that talk about that. I'm not the best resource on that subject because I haven't done a ton of metatrader backtesting once I discovered that some of the tests were severely unreliable.

I'm doing my best. Can you or anyone else who is confused quote sections that seem unclear to you, and I'll do my best to clarify.

Thanks for your comments superdezign! I spent lots of time chasing the win% holy grail. It's just about as thirst quenching as a mirage in the desert. The whole expectancy thing took a long time to sink in for me. And you're right, it is really important to get a grasp on, and the earlier it can be done, the better focused and more clear the system development can become. Understanding that you can have a sub-50% win system that makes a ton of money while you can have an 80% win system that loses money blows some peoples minds. Thanks for that excellent reminder!

Awesome

I've been refreshing the page every hour or so to see your answer (especially to this question since it's driving me nutz! ). This was exactly what I was looking for.

I'm sure I'll have more questions shortly once I fully process what you just gave me.

Much appreciated!


Seavo

Thanks for the feedback Seavo!

I'll continue with the commentary:

The third step is in creating some assumptions and testing those assumptions. The first thing to realize about trading is that there is really nothing new under the sun. The markets are there, the basic ways of approaching the markets are there in the public domain, as are most of the indicators and other tools for trading. Thomas Edison didn’t create the electric light bulb out of magic pixie dust. He tried ten thousand combinations before he had the right combination of components. He didn’t invent the components he tested or the principles behind them, but rather arranged them in a form that increased the value of the whole. This is the idea behind a trading system. We use what is already there and arrange it in such a way as to create a trading system that meets our requirements.

So we take some basic building blocks like trendlines, averages, channels, concepts of trend etc., and use them to build a trading system that will meet our goals or in other words give us what we want. This is where imagination, intuition, innovation, and creativity come in. We need to use our minds to assemble the components that are out there into a whole that becomes our winning trading system.

In this step there needs to be a lot of testing. Each element should be tested individually. My philosophy is that if a given element does not add to profitability, then that element is discarded. The basic idea is to focus in on one element that becomes our foundation for the system. For example, let’s say we focus in on channels as the focus point of our system. We need to define ways of using channels that produce a positive outcome. If a particular way of using or defining channels doesn’t have at least some positive outcome (profitability) then that way of doing things is discarded and another way is experimented with or developed. And so the process goes until we get a way of looking at channels that produces profit. We then go to work on ruling out other possible ways of defining channels that may be better than what we currently have. We also try and figure out why that particular way of using channels “works” and what we can do to improve upon the concept.


More on this subject in the next post...

Step 3 continued...

At step three in my system development I don’t have a trading system but do have a working model that tests with at least some degree of profitability. The idea is to refine the system down from a rough concept into a shiny beautiful diamond of a method that meets all our requirements. So we start adding other things to the core concept and determine if the addition is better or worse than the raw concept. And we do this until we get a combination or two or three additional elements that improves the raw concept and starts to meet our profitability requirements.

Note that the main effort should be put into testing different combinations of concepts, rather than in doing tons of optimization. Optimizing over a wide range of variables can be useful in finding out the range of possibilities and where to focus. However, allowing optimization to drive the creative process rather than the other way around, won’t lead to the desired results.

Unfortunately this is the point at which many people give up the development process because they want to start forward testing and trading the system. But on the bright side, this is also the stage at which the most progress can be made. It’s also the stage at which an honest evaluation must be made. Is there really hope of turning out a system that meets your specifications given the current system design? If not then further clarifying may be needed or a reorganization of priorities may need to take place before a solid system framework can be generated. Then it’s just a matter of putting all the pieces of the puzzle together.

FXEZ

I have developed a breakout strategy that won 87% and for the testing period never had a losing streak of more than 3 in a row. But use a stop 3 x the PT. It does come out profitable over the tested time but due to the 3x larger stop is not a major winner. Am I going of the tracks when using an adapted martingale type MM to up the profitability for this strategy. I am allowing for a straight 6 losers. ie: The total lots to trades 6 straight losers still fall within my risk% for the said account.

More specific:

686 trades = 601 winner + 85 losers

of the 85 losers only had 7 times 3 in a row.

+3430 pips up with adapted martingale type MM or

3005 - 1275 = +1730 taking straight loses.

I have tried smaller stop or larger TP and both decrease profitability.

Am I on the wrong tracks.