Some thoughts on (retail) EAs and automated systems
I don't want to derail thodie's EA thread, and I've included a couple of potentially commercial references, hence I'll post here instead. But I'm not selling anything myself, hence I'm not a commercial member.
I see that Rob Booker -- interviewed in Chapter 3 of Boris & Kathy's book Millionaire Traders -- is now selling EAs.
Just some of my own thoughts, for whatever they're worth, about trading robots. In general, they represent a completely different approach to manual trading, and for some reasons that might not be immediately obvious. Of course their big advantage is that they remove the emotion and guesswork from trading, by consistently and flawlessly executing a predetermined set of rules, while trading 24/7 around the clock, granting the trader free time away from his computer. Providing objectivity and detachment, an EA is a totally disciplined workhorse that never gets ill, tired, stressed, frustrated, discouraged, impatient, complacent, anxious, revengeful, greedy, over-excited, hesitant, etc etc, or influenced by outside noise.
However, there are also some significant caveats and downsides to consider. Firstly, EAs pander to the greed and laziness in human nature. Why would anybody bother spending years learning how to trade profitably, or hours every day marking up price charts, or poring over bank and economic reports, if simply plugging in an EA could guarantee them an effortless, passive, exponentially growing income?
Secondly, people tend to assume that if a human trader can trade profitably, then an EA should be able to perform exactly the same tasks, and generate the same results, just as readily. Not necessarily true. Humans have big advantages over robots: we are not limited to basic TA and MM; for better or worse, we can trade intelligently and adaptively. The human brain can perform economic analysis, nuanced contextual and qualitative analysis, correlation and intermarket analysis, complex pattern recognition, weigh several disparate or conflicting factors simultaneously, and adjust to meet the needs of unexpected, extraordinary or changing conditions. All of these items range from difficult to impossible to reduce to a mathematical formula that's mechanical enough to be coded. For better or worse, EAs can only work to fitted approximations: for example, a human trader might see a 'perfect' shooting star based setup, and make a winning trade, while an EA overlooks it, merely because the SS narrowly missed meeting the coded rule that its upper wick must be at least 60% of the total candle height.
Because the items I listed are too difficult to code, retail EAs tend to use VERY different approaches to those of a discretionary trader. Frequently, entries are based around indicators, due to their being easy to code. Many EAs also seek to gain their edge from MM -- grids, scaling in/out, nedging, loss recovery, martingales etc -- again, because all of this is readily programmable, ignoring the maxim that MM in itself can not provide an edge (myth #2 here). As I see it, this runs the risk that the trader is building his system around programming convenience rather than a proper understanding of price behavior. And some of these MMs -- especially martingale variants -- can deliver continual profit while concealing latent risk -- until one day, the black swan condition suddenly and unexpectedly comes along, and the levee breaks.
Thirdly, there is the danger of curve fitting. Another approach to building EAs is through data mining and backtesting. The assumption is that what was statistically profitable in yesteryear will continue to be so. One should be careful of what I call 'saddle blanket' systems, which comes from the following horse racing analogy: Statistics might demonstrate that a horse with a yellow saddle blanket wins most often on Wednesdays, while a horse with a green saddle blanket wins most often on Fridays. But even if these results were to meet statistically valid standards, the reality is that the color of the saddle blanket has no effect on the race outcome. The same applies to trading. For example, a certain EMA crossover (say 10 vs 20) might generate a profit over past data, but unless moving averages have some influence over the way price moves, or are otherwise somehow grounded in trader psychology/behavior, the researcher is being 'fooled by randomness'. (This post is a 'must read' IMO). And once again, he is trying to impose his desired system upon an unsympathetic market.
Another related danger stems from the finitude of the data over which EAs are backtested. The traders who developed the 'grail EA' explain this well in their fascinating story here. Their conclusion: no matter how thoroughly you test your EA, the market can always serve up a 'black swan' scenario that somehow escaped your testing, highlighting the importance of risk management. As the disclaimer reads, past results don't guarantee future performance. That is one dilemma when backtesting: too little data, and you risk a lack of statistical validity; too much data, and the oldest data might no longer accurately reflect current market conditions. And forward testing over live data is not immune from similar problems to backtesting: after all, at a point in the future when you're seeking profit, today's data will have become the data of yesteryear.
Trend following EAs are less popular because of their low win rate; even if profitable overall, they provide erratic income due to prolonged losing periods while price moves sideways, making them unsuitable for the newbie -- who, perhaps being lured by unsubstantiated claims made by br0kers, and forum members -- is seeking quick, consistent profit. A few years ago my friend mim2005 wrote three such EAs; you can read about their development in detail here. These performed promisingly enough, even on a small live account, but he became distracted by more important projects, and shelved these EAs when MT4 build 600 arrived and the code needed overhauling.
Finally, an EA's inability to adapt to market 'evolution' can give it a shelf life. Markets are ultimately the product of belief systems that can gradually change over time. A good example of a seemingly robust concept is the fact that traders tend to place their stoplosses just outside swing highs/lows, a phenomenon that creates orderflow which can potentially be exploited. But should this practise ever become less prevalent, any related edge will lose its effectiveness accordingly. Two other reasons markets can change character are economic shifts, and advancing technology. For example, the rise of HFT and algos has likely created subtle changes in price behavior, possibly diluting the effectiveness of some systems that were designed to profit during yesteryear's climate. To be robust then, an EA must somehow be grounded in a demonstrably recurrent and overriding aspect of trader psychology/behavior, one that is very likely (given that we can never use the word 'guaranteed') to stand the test of time.
Some good general info about edges by topher here.
Some info from my personal experience, FWIW. Between 2010-2012 I wrote maybe 20 EAs to meet the requested specifications of some of the most highly vouched FF members, handpicked from the dozens of requests I received, all of whom insisted they were profitable. I spent several weeks on some of these, modifying and retesting code as these experts changed their minds and refined their ideas. As far as I know, NONE of these EAs are being run profitably today, making them a total waste of time for everybody involved. I can honestly say that this is not due to faulty coding. Instead, I believe that it's largely because these folk use discretionary nuances that they take for granted, and weren't included in the code. Or perhaps some of them were less than honest about their success, just to get some free programming. As a result I no longer code EAs for other traders, unless they can first demonstrate a third-party verified track record of profit over several years' worth of changing markets. I also attempted to explain some of the difficulties facing programmers here and here.
I'm very skeptical of forum posts claiming that somebody's found a "revolutionary new idea" or a system that's making extraordinary amounts per month. Has it not occurred to these folk that literally millions of people -- including the world's top economists, math and physics PhDs, and veteran institutional traders -- all of whom have been equally determined to beat the market, have never at some point attempted something similar? And do these folk seriously know who they're up against? If there was a plug'n'play system available the retail trader that's guaranteed to consistently beat the market forever, then once again, everybody would be using it to effortlessly exponential gains. But markets are (pseudo)zero sum: you win only because you've outwitted and outmanoeuvred the majority of the smartest heavyweights around. All of which points back to intelligence, an intimate knowledge of underlying market workings, and -- where necessary -- an ability to self-adapt to changing conditions.
Nonetheless, I still write EAs for myself and a couple of like-minded trading buddies. They're not fully automatic, though. For example, one of them ('Ambrosia') makes entries based on user-specified PA patterns that occur in or around S/R zones that I've plotted manually on my chart. (One day I might share some of these EAs publicly). Their flexibility makes them 'tools' that could potentially be used profitably by a knowledgeable enough trader. Their blend of manual/automatic functionality has been designed to meet my own trading philosophy: I like to have a high degree of control over choice of instrument, and setup, while letting the EA manage the exits, which are my personal weakness, overnight (I live in NZ, the UK/US sessions occur during the evening/overnight NZT). While I'm asleep my emotions can't interfere and cause me to take profit too early, LOL.
I should point out that much of the above info applies to the 'retail' EAs you're likely to see being shared on trading forums, or sold by vendors.
For more expert info: There are a few FF members (Craig, mikkom, PipMeUp, FXEZ, algoTraderJo, Copernicus, ...) who have advanced knowledge of (1) statistical math, and also (2) machine learning -- their threads are definitely worth reading, if you can get your head around the highbrow concepts in each case (I can't).
The most knowledgeable creator (and prolific author) of automated systems that I'm aware of is Daniel Fernandez (here is a link to all of his blog entries).
Merlin, one of the founding fathers of FF, wrote a fine essay on developing mechanical systems here.
And this post by topher is definitely worth a read also.
In summary, many retail EAs (especially vended ones) operate to a high win rate/low RR formula, because this appeals to newbies looking for quick, consistent income, and makes them more saleable. I don't see anything wrong with using these types of EA, provided that the trader is aware of the strengths and weaknesses of the system being employed, and is willing to live with whatever limitations and additional risks that they might impose. At the end of the day, it comes down to the individual's own trading preferences and philosophy: account size, desired level and consistency of income, appetite for risk, ability to stomach drawdowns, lifestyle goals and considerations...... and how comfortable you feel about entrusting your hard earned money to a black box. Hence it's not necessarily as simple as saying that a more profitable EA is superior to another (for example, a martingale EA might deliver continuous profit, but also the danger of sudden and unexpected ruin).
Given that arbitrage opportunities are unavailable to the retail trader, there is no guarantee of return without taking some kind of risk, no matter how you trade, anyway. As the successful prop trader Tom Dante says, trading is ultimately always speculation that's based on nothing more than your best guess. Good luck!
I don't want to derail thodie's EA thread, and I've included a couple of potentially commercial references, hence I'll post here instead. But I'm not selling anything myself, hence I'm not a commercial member.
I see that Rob Booker -- interviewed in Chapter 3 of Boris & Kathy's book Millionaire Traders -- is now selling EAs.
Just some of my own thoughts, for whatever they're worth, about trading robots. In general, they represent a completely different approach to manual trading, and for some reasons that might not be immediately obvious. Of course their big advantage is that they remove the emotion and guesswork from trading, by consistently and flawlessly executing a predetermined set of rules, while trading 24/7 around the clock, granting the trader free time away from his computer. Providing objectivity and detachment, an EA is a totally disciplined workhorse that never gets ill, tired, stressed, frustrated, discouraged, impatient, complacent, anxious, revengeful, greedy, over-excited, hesitant, etc etc, or influenced by outside noise.
However, there are also some significant caveats and downsides to consider. Firstly, EAs pander to the greed and laziness in human nature. Why would anybody bother spending years learning how to trade profitably, or hours every day marking up price charts, or poring over bank and economic reports, if simply plugging in an EA could guarantee them an effortless, passive, exponentially growing income?
Secondly, people tend to assume that if a human trader can trade profitably, then an EA should be able to perform exactly the same tasks, and generate the same results, just as readily. Not necessarily true. Humans have big advantages over robots: we are not limited to basic TA and MM; for better or worse, we can trade intelligently and adaptively. The human brain can perform economic analysis, nuanced contextual and qualitative analysis, correlation and intermarket analysis, complex pattern recognition, weigh several disparate or conflicting factors simultaneously, and adjust to meet the needs of unexpected, extraordinary or changing conditions. All of these items range from difficult to impossible to reduce to a mathematical formula that's mechanical enough to be coded. For better or worse, EAs can only work to fitted approximations: for example, a human trader might see a 'perfect' shooting star based setup, and make a winning trade, while an EA overlooks it, merely because the SS narrowly missed meeting the coded rule that its upper wick must be at least 60% of the total candle height.
Because the items I listed are too difficult to code, retail EAs tend to use VERY different approaches to those of a discretionary trader. Frequently, entries are based around indicators, due to their being easy to code. Many EAs also seek to gain their edge from MM -- grids, scaling in/out, nedging, loss recovery, martingales etc -- again, because all of this is readily programmable, ignoring the maxim that MM in itself can not provide an edge (myth #2 here). As I see it, this runs the risk that the trader is building his system around programming convenience rather than a proper understanding of price behavior. And some of these MMs -- especially martingale variants -- can deliver continual profit while concealing latent risk -- until one day, the black swan condition suddenly and unexpectedly comes along, and the levee breaks.
Thirdly, there is the danger of curve fitting. Another approach to building EAs is through data mining and backtesting. The assumption is that what was statistically profitable in yesteryear will continue to be so. One should be careful of what I call 'saddle blanket' systems, which comes from the following horse racing analogy: Statistics might demonstrate that a horse with a yellow saddle blanket wins most often on Wednesdays, while a horse with a green saddle blanket wins most often on Fridays. But even if these results were to meet statistically valid standards, the reality is that the color of the saddle blanket has no effect on the race outcome. The same applies to trading. For example, a certain EMA crossover (say 10 vs 20) might generate a profit over past data, but unless moving averages have some influence over the way price moves, or are otherwise somehow grounded in trader psychology/behavior, the researcher is being 'fooled by randomness'. (This post is a 'must read' IMO). And once again, he is trying to impose his desired system upon an unsympathetic market.
Another related danger stems from the finitude of the data over which EAs are backtested. The traders who developed the 'grail EA' explain this well in their fascinating story here. Their conclusion: no matter how thoroughly you test your EA, the market can always serve up a 'black swan' scenario that somehow escaped your testing, highlighting the importance of risk management. As the disclaimer reads, past results don't guarantee future performance. That is one dilemma when backtesting: too little data, and you risk a lack of statistical validity; too much data, and the oldest data might no longer accurately reflect current market conditions. And forward testing over live data is not immune from similar problems to backtesting: after all, at a point in the future when you're seeking profit, today's data will have become the data of yesteryear.
Trend following EAs are less popular because of their low win rate; even if profitable overall, they provide erratic income due to prolonged losing periods while price moves sideways, making them unsuitable for the newbie -- who, perhaps being lured by unsubstantiated claims made by br0kers, and forum members -- is seeking quick, consistent profit. A few years ago my friend mim2005 wrote three such EAs; you can read about their development in detail here. These performed promisingly enough, even on a small live account, but he became distracted by more important projects, and shelved these EAs when MT4 build 600 arrived and the code needed overhauling.
Finally, an EA's inability to adapt to market 'evolution' can give it a shelf life. Markets are ultimately the product of belief systems that can gradually change over time. A good example of a seemingly robust concept is the fact that traders tend to place their stoplosses just outside swing highs/lows, a phenomenon that creates orderflow which can potentially be exploited. But should this practise ever become less prevalent, any related edge will lose its effectiveness accordingly. Two other reasons markets can change character are economic shifts, and advancing technology. For example, the rise of HFT and algos has likely created subtle changes in price behavior, possibly diluting the effectiveness of some systems that were designed to profit during yesteryear's climate. To be robust then, an EA must somehow be grounded in a demonstrably recurrent and overriding aspect of trader psychology/behavior, one that is very likely (given that we can never use the word 'guaranteed') to stand the test of time.
Some good general info about edges by topher here.
Some info from my personal experience, FWIW. Between 2010-2012 I wrote maybe 20 EAs to meet the requested specifications of some of the most highly vouched FF members, handpicked from the dozens of requests I received, all of whom insisted they were profitable. I spent several weeks on some of these, modifying and retesting code as these experts changed their minds and refined their ideas. As far as I know, NONE of these EAs are being run profitably today, making them a total waste of time for everybody involved. I can honestly say that this is not due to faulty coding. Instead, I believe that it's largely because these folk use discretionary nuances that they take for granted, and weren't included in the code. Or perhaps some of them were less than honest about their success, just to get some free programming. As a result I no longer code EAs for other traders, unless they can first demonstrate a third-party verified track record of profit over several years' worth of changing markets. I also attempted to explain some of the difficulties facing programmers here and here.
I'm very skeptical of forum posts claiming that somebody's found a "revolutionary new idea" or a system that's making extraordinary amounts per month. Has it not occurred to these folk that literally millions of people -- including the world's top economists, math and physics PhDs, and veteran institutional traders -- all of whom have been equally determined to beat the market, have never at some point attempted something similar? And do these folk seriously know who they're up against? If there was a plug'n'play system available the retail trader that's guaranteed to consistently beat the market forever, then once again, everybody would be using it to effortlessly exponential gains. But markets are (pseudo)zero sum: you win only because you've outwitted and outmanoeuvred the majority of the smartest heavyweights around. All of which points back to intelligence, an intimate knowledge of underlying market workings, and -- where necessary -- an ability to self-adapt to changing conditions.
Nonetheless, I still write EAs for myself and a couple of like-minded trading buddies. They're not fully automatic, though. For example, one of them ('Ambrosia') makes entries based on user-specified PA patterns that occur in or around S/R zones that I've plotted manually on my chart. (One day I might share some of these EAs publicly). Their flexibility makes them 'tools' that could potentially be used profitably by a knowledgeable enough trader. Their blend of manual/automatic functionality has been designed to meet my own trading philosophy: I like to have a high degree of control over choice of instrument, and setup, while letting the EA manage the exits, which are my personal weakness, overnight (I live in NZ, the UK/US sessions occur during the evening/overnight NZT). While I'm asleep my emotions can't interfere and cause me to take profit too early, LOL.
I should point out that much of the above info applies to the 'retail' EAs you're likely to see being shared on trading forums, or sold by vendors.
For more expert info: There are a few FF members (Craig, mikkom, PipMeUp, FXEZ, algoTraderJo, Copernicus, ...) who have advanced knowledge of (1) statistical math, and also (2) machine learning -- their threads are definitely worth reading, if you can get your head around the highbrow concepts in each case (I can't).
The most knowledgeable creator (and prolific author) of automated systems that I'm aware of is Daniel Fernandez (here is a link to all of his blog entries).
Merlin, one of the founding fathers of FF, wrote a fine essay on developing mechanical systems here.
And this post by topher is definitely worth a read also.
In summary, many retail EAs (especially vended ones) operate to a high win rate/low RR formula, because this appeals to newbies looking for quick, consistent income, and makes them more saleable. I don't see anything wrong with using these types of EA, provided that the trader is aware of the strengths and weaknesses of the system being employed, and is willing to live with whatever limitations and additional risks that they might impose. At the end of the day, it comes down to the individual's own trading preferences and philosophy: account size, desired level and consistency of income, appetite for risk, ability to stomach drawdowns, lifestyle goals and considerations...... and how comfortable you feel about entrusting your hard earned money to a black box. Hence it's not necessarily as simple as saying that a more profitable EA is superior to another (for example, a martingale EA might deliver continuous profit, but also the danger of sudden and unexpected ruin).
Given that arbitrage opportunities are unavailable to the retail trader, there is no guarantee of return without taking some kind of risk, no matter how you trade, anyway. As the successful prop trader Tom Dante says, trading is ultimately always speculation that's based on nothing more than your best guess. Good luck!
