Why does Automated trading never seem to work?

Given that we (the "retail" traders) trade very small compared to the institutionals ("big bank" FX), wouldn't it make more sense to gear your algorithms, heuristics, or whatever to the games the "big banks" play instead of the little guy?

So, in a sense, before setting up Metatrader for your clients, you somehow model the order flow that you receive from your customers as a whole in regards to PA. Then, model how you lay off the risk (to your "big bank dealer") in a timely fashion, which over time, costs less than the "spread" you make. Maybe, on very short time frames, you can "slow" your quotes sometimes. But if you do this too much, you will lose customer flow, given the current transparency of forex quotes across many retail dealers.

Watching the very short time frames (and losing plenty on them!), I believe it is the "big bank" dealers creating the havoc in pricing that causes many traders and EAs to lose. The "retail dealer" side is just trying to keep a quote up that is close, and that helps them make at least some profit.

Just my 2 cents.

Unfortunately I don't understand the workings of even the most basic EA, so I can't give any credible answer as to how they go about manipulating individual ones sorry. Plus, at the time of our conversations and meetings, I was only just getting vaguely interested in trading myself, so I didn't pay as much attention or understand as much of the lingo as I would today.

I think the basic idea is if they know you are using a robot, they attempt to sabotage the orders in general. It doesn't seem specifically to just hunting stops etc, but can be something as subtle as not processing an order if they know the order was generated automatically. It is very easy to simply blame a malfunction in the script. So you may not always lose money, but can often have missed out on entering a profitable trade simply because they refused to accept your order. By the same token they do cause problems for profitable manual traders at times. It basically came down to their risk exposure. If a trader is firing off multiple orders, particularly at high units, there is a chance the broker gets stuck on the wrong side of those trades and loses capital. This is the reason they need reserves held permanently in their own accounts to cope with these situations. So they do assess the circumstances of manual traders and sometimes throw a spanner in the works for them. But of course robots capable of monitoring multiple currencies and simultaneously setting orders in quick succession pose a greater risk and this is what they seemed to be overly concerned with.



Yes I agree, but I hope you didn't take my opening post to be this way (certainly not paranoia anyway).

Most of the problems people perceive as dishonesty from the brokers is actually perfectly normal as part of trading. Brokers make their money from the customer trading, so they want them to stay trading as long as possible which means they will play it straight as much as possible. For the most part, the broker couldn't care less if you win or lose a trade, because they have already passed the trade and risk off to someone else, so they have no reason to manipulate anything.

However the reason I was able to talk in detail to these guys and get an honest view of how they operate is because at the time the local laws where they were incorporated were changing, and they now needed close to 50 Million (roughly if I remember) held as a reserve for security....without which they would have to close down. And so they were looking for an investor to back them, and thanks to the job I did, I was with the investor and sat in on most of their discussions and negotiations. And when you are trying to get someone to hand over 50 Million, you need to be pretty honest. I was amazed at some of the things they said and were capable of. Their offices can literally see everything every customer does on their platform. They have programs running to monitor who is placing what size trade, on what currency and what liquidity is available at the time. Warnings flash up on screens to indicate they could potentially be exposed to greater than normal risk, and then all sorts of automated processes kick in to prevent the trade from working out the way the retail trader on the other end would hope.

I asked them specifically how they made their money....mainly out of pure curiosity because as I said I was not heavily involved in trading at the time. They said that primarily and for the most part they make money purely from spreads, and 99% of the time what happens to a trade or trader makes no difference to them. But.......they have a huge list of other ways to take money from the trader, most of which are so subtle that we have no idea it is happening. These people are in business to make money, and they make a lot of it....and they'll go to all sorts of lengths to make sure they make a little more wherever they can.

But of everything they said, the issue with robots, and their reasons for delaying introducing MT4 stayed with me. And the post I referenced at the start of this thread got me thinking about it again. I really am not knocking robots. I know a fund manager that uses them with spectacular results. But retail traders on something as basic as MT4 may find they run in to problems, and I believe this is no coincidence.

Am I suggesting an alternative or a solution? I wouldn't have a clue!

I heard an EA shot JFK from the grassy knoll.

Everybody misses the fact that the REAL "ea" was ON TOP of the book depository and was able to hold off the fatal shot until the bullet drifted down to the fifth floor window THEN
wham. Took off and sent that magic bullet.
Or at least that's how they described it on the History Channel

Great post.

The fact that 'robots' or 'EAs' or programmes are often designed to run 24h should tell us something here. Namely that the opportunities most of them are trying to exploit are based on very small moves in the market.

Exploitable opportunities don't come in such high frequency (I am excluding the highly sophisticated algorithms applied in the ECN and equity/futures markets).

Think about price behaviour and FX in particular. Even as an intraday trader myself, I am well aware that opportunities depend on time and liquidity at specific times. What I mean by that is this: a robot as well as a trader needs the price to move beyond the spread. I guess there is not much point in the construction of EA's that only trades trends once a month.

In addition to the fact that MT4 based program's as far as I know have no way of identifying liquidity (unlike algo's that work with ECN's), ECN's are also impartial to whether the program is profitable or not since providers charge per round-trip. The cost for $1m is $50 on average and high freq. algorithms can turn over 10 thousand times/day on amounts far bigger than a million dollars - do the math.

In short, I think that most of these programmes are build with too narrow margins in mind by the creators that leave them open to easy legal manipulation in the RETAIL market. Because most retail brokers (as mentioned) have for a split second to take the other side of the position, it leaves them with precisely that risk - especially when their income depends on the 'spread' only, they rely on the counterpart to honour to deal inside the spread they offer to the clients.

As the OP stated, it makes good business sense to them to try and shave the customers wherever there is a borderline legal opportunity to do so. It seems that many EA's might give them just that opportunity.

regards
daytrading

@ daytrading - thank you for adding such a good post to this thread.

It seems to me that MT4 offers the brokers a license to print even more money, even though it meant quite a bit of extra work to integrate it to their systems in the first place. I may be completely wrong as I have no technical experience in this area, but MT4 appears to be reasonably basic and low tech (in comparison to what the institutional algorithms can do), and therefore the robots which run on it are equally limited in their abilities and easily exploited by brokers.

Hi Flotsom,

Great thread! The information shared here has certainly served to awaken some of us to the reality of retail trading. I am amazed at some of the responses here. The sooner retail traders realise that the the odds are against you the better your trading will become.

Retail brokers exist for one reason and one reason only: TO MAKE MONEY.. They have placed a lot of time, effort and resources into ensuring that they make maximum profits from retail traders. And they will continue to do so at any cost.

fxpilot

To write an EA so a trader can watch TV or play or play golf or go to sleep are also not the origins of thinking behind the inception of automatic trading.

I have been once told that as long as a trader in its human form can execute what he/she is trying to write a program for, it makes more sense to execute manually. I am aware that some will refer to lack of discipline and that the program would work without human emotion - I am for the sake of the discussion presume that the emotional side and the discipline are dealt with.

A little background here: I have been working for as well as with brokers since 1988 - mainly voice broking/trading until 2001/2. I was in the interbank business, traded futures for my own account and for others. As long as we had 'real' contact between brokers and traders (through voice comms.), integrity was given at all times. We never screwed clients and as clients we trusted the broker on the other end of the phone.

This is a dying profession since the turn of the century and these broking houses have become less and many had to join forces in order to survive.
On the other hand, with the electronic execution age, retail brokers appear almost everywhere on a monthly basis - without the actual trading volume increasing that much. There is no need for integrity, which is shown by many of them refusing to operate a real dealing desk and forcing traders to rely purely on visual information.

You might be surprised, but even the trustworthy ECN brokers allow a traders to actually call and place orders per voice.

There is no need for integrity because there is no relationship any more between trader and broker in the retail market.

Back to EA's: the inception of algorithmic, program and HF trading was based on the need to facilitate order execution without negatively affecting the market price. For example, larger volume orders would be broken up by algorithms by way of splitting one large amount into 200 small trades, all timely executed (e.g. one trade every 35sec).

Later came the invention of ultra-low latency, liquidity seeking (dark pool) front-running algorithms that would exploit tiny price movements based on identified inefficiencies for actual profits - which is a difference to the pure 'damage limitation' when having to place huge orders without causing the market to move against you.

Paradoxically, many retail programmers willingly try to compete on such a low level (MT4) that it makes it relatively easy for the brokers to fleece them on a regular basis. There is not much room (if none at all) in terms of an opportunity for retail traders on the 'real' algorithmic front. The prices banks and funds are willing to pay for top programmers to come up with new ways of identifying new minuscule inefficiencies (while other inefficiencies are disappearing) are enormous.

Never give control to a machine if you haven't got the ability to control all the variables - especially when the variables you have no control over can make a significant difference to the profitability of your trading method.

Mechanical (100% rigid and defined rules) are one thing - giving a broker knowledge about the workings of your model by way of an EA is another - especially when the broker might gain an advantage for themselves from that knowledge.

regards
daytrading

What an interesting post, thanks DT.

It serves as a reminder of why I still keep reading FF, the occasional gems make reading all the dross just about bearable!

I'll second that, Pip. Really good summation of the use of programmed trading.

This thread is full of many good points really.

In trading it seems like everything you do, or aspire to do, depends on who you are and what you're trying to achieve. It just doesn't seem to me that a retail trader using a relatively simplistic ON/OFF switch EA with pretty much fixed parameters could work out over time. If your broker doesn't shut you down then the Great Equalizer (naturally varying market conditions) will take care of you. Of course, the dream of sunning on the beach with a drink in hand while your bot generates trades will keep people chasing the end of the rainbow forever.

Crudely: ""Either you're slingin' crack-rock, or you've got a wicked jump-shot. Nobody wants to work for it anymore."

"Things Done Changed".

...or you can rhyme and sell records. Ha! Just having some fun here but, there is this serious point. That if you're driven by a desire for leisure, basically laziness, your prospects of developing a Grail EA or even being a success in trading or anything else may not be good.

I think that one, perhaps unmentioned point is that the reason many traders seek to develop automated trading programs is that - that's their background. They write code. It's only natural that they want to bring it to this arena. It probably gives them no edge at all though as a retail trader with retail resources.

amen to the above post

Well i think there's probably more girls and exotic travel in that game

Yes more than 90% if not 99% of MT4 brokers delay quotes & manipulate prices in live. You may easily notice different price action from demo to live !!

BUT NOW I think there is good ECN Brokers that offre true price feed not manipulated & mainly focus on routing order on market.

Take big care when choosing a brokers. Many arround claiming offering ECN are just a scam.

FX 13

In my opinion, the reason why people think it is not working is they think their profit much more higher than system's expectation. If the system guess the sell price as $169.98 but you want more higher price. In this case, you should have to start manual trading. The automated trading system is perfect for beginner.

I think think this thread is a very good one, looking at the disadvantage of using automated system can leave you in despair;
·High Maintenance – Even though an EA will free up a lot of time, it should be noted that they do require maintenance, and of course we know this might require some monitoring.
Regular Updates – Many people assume that an EA is just something that you can 'set and forget' but this isn't the case. They actually require a lot of regular updates if you want to keep your account in the black.
Expensive – Even though the programming might be quite simply to most, there will be a huge investment even if you can program yourself!
Undermining these facts there are still elements of good in using automated systems, it's all about finding the right balance!

I personally don't feel right to use automated systems, I feel that we are better trading manually, than to invest money in such a system.