Expectancy is always zero

Hi M.A.C.Doug,

Wow, that's not only an excellent example of where expectancy is close to zero, but a great insight into your own trading performance. I wonder how many traders would have thought after the fact to check their total losses against their total spread cost. I'm guessing most would have just thought what you did at first...that they were just awful traders, when in fact they are not.

Many "bad" traders are really just perfectly normal traders that don't have an edge.

And with that one sentence you've defined what much of the rest of Forex Factory is all about.

One of the reasons I brought the issue up is the tendency for a lot of the "new system" threads here to talk about win rates without regard for anything else. I'll see a post say something like "This system has given me a win rate of 90% over the past month." And that may very well be true, but what's the REQUIRED WIN RATE of the system given the risk/reward characteristics. More often than not you'll find that systems with a 90% win rate have Required Win Rates of like 92% or something.

While I don't want to turn this into just another "what's a good system?" thread, I would certainly recommend that traders analyze such systems fully, not only for AWR but for historical RWR as well.

Hi TickerShuffl,

I believe that there is always a balance between the pros and cons of any approach. There are tradeoffs between:

FA vs. TA,
long term vs. short term,
mechanical vs. discretionary,
tight stops vs. wide stops,
etc.

This keeps me from falling into one camp or the other on these and other issues because I always try to see the balance between each side. So since you brought up a great advantage of long term trading, I'm going to restore the balance by bringing up an advantage of short term trading. I just don't want to give the impression that I favor one over the other. I think they both have good and bad points.

While long term trading will certainly decrease the significance of spread cost, it will also defeat the benefits of the law of large numbers on which any successful trading strategy is based.

Let's say I find a system that has a risk/reward ratio of 1:1 and a win rate of 52%. While it's easier to find such a system in long term trading because of lower spread costs, the fact that I have to wait a year to do 100 trades vs. waiting a week to do 100 trades will mean that I have to be VERY confident in the positive expectancy of this system in order to stick with it.

So for me to stick with a long term trading sytstem, the expectancy would have to be pretty high and able to manifest itself in just a few trades. Otherwise, I and most traders will lose confidence in it and abandon it no matter what the historical results are.

I often hear people berate short term traders for being "impatient." I don't think many are. Many short term traders are perfectly willing to wait years for their small gains to compound themselves into wealth, as long as they are confident that their system works. I think the reason they prefer short term trades is that it allows them to use the law of large numbers to get immediate real-world feedback on their trading systems, thus increasing their confidence.

If I'm a scalper, I know in a week if my system works or not because I've got 100 trades to look at. If I'm a long term trader it may take a year or more for me to know if my system works or not.

Hi all

Some thought-provoking posts here, IMHO. Definitely agree with the point that expectancy is zero (ignoring costs), unless one has an edge. Question becomes, how does one define, and measure that edge? (And how great does one's edge have to be - or what margin for error must be allowed - in assuming that past ineffciencies will recur with similar frequency in the future?)

Conversely:
If (and I stress IF) the market was a random walk, then all systems would have zero expectancy, and all profits/losses would be fortuitous.

Hence, on the one hand, a profitable trading system is limited to whatever inefficiencies might exist (which lies outside of its control); on the other hand, it must identify and exploit them (which lies within its control).

This kind of theory fascinates me, and I've already posted a lot on this topic. Not trying to blow my own trumpet (my poor trading results speak for themselves ), just including the links to save me re-typing (or pasting) all of the text.

Interested to know what you guys think.

Thanks
David

Coin-toss entries:
http://www.forexfactory.com/showthread.php?p=1492382

Randomness, relative importance of entries and exits:
http://www.forexfactory.com/showthread.php?p=1441266

Some trading truisms, at least from a mathematical standpoint:
http://www.forexfactory.com/showthread.php?p=1452677
http://www.forexfactory.com/showpost.php?p=198511

Hi Hanover!

I've been reading a lot of your posts around the Factory for a while this morning. I'd love to let you know what I think but I'd be typing for the rest of the week! Seriously, you make a lot of great points and seem to have the same "let's apply some real mathematical reasoning to conventional wisdom and see what happens" mindset that I do.

Because of what I'm struggling with in terms of strategy development at the moment, I found this insight from one of your posts particularly useful:

13. Extrapolating point 12, trading simultaneous (or overlapping, time-wise) uncorrelated positions leverages time to increase return without affecting risk.

I'm still going through some of your other posts.

Hanover, the same post I referenced before also had this point:

12. Where (hypothetically) costs are irrelevant, trading more frequently (i.e. shorter time frame) generates a smoother equity curve. Everything else being equal (and assuming a positive expectancy entry/exit method), this will leave return unaffected while mitigating risk. However, costs being a reality, the shorter the price moves being traded, the greater the edge necessary to overcome the costs.


Now I know this is obvious to most people, but what may not be so obvious is the HUGE extent to which this is true. Mick gave a great numerical example (was it in this thread?) of how the exact same price movement has a much more devestating effect if it goes against you than the beneficial effect if it goes in your favor. So I decided to apply this idea to my (simplistic) model of win rate that I presented in the first post.

To reiterate Mick's point:

Suppose I'm scalping EUR/JPY with a spread of 2 pips and I want to set both my SL and my TP at 4 pips from my actual entry.

If the price is at 165.00 and I buy,
I will get an actual entry price of 165.01.
Thus, my TP will be at 165.05,
and my SL will be at 164.97.

In order to for the bid to hit my TP, the price has to go from its current position of 165.00 to 165.06, a move of 6 pips. But in order for the ask to hit my SL, the price only has to move down to 164.98, a move of only 2 pips.

So using my (simplistic but robust) model that an "edgeless" win rate is equal to:

(distance that price has to move to stop me out) / (total distance from stop to target),

My expected win rate is 2/(2+6) = 25%!
But my REQUIRED win rate for B/E is 4/(4+4) = 50%!
This is a 25% disadvantage right off the bat. I certainly better have some kind of mega edge to overcome this!

In general, calling the spread "S", the SL distance "L" and the TP distance "G" (for gain),

My simplistic expected win rate is (L-S) / (L-S+G+S) = (L-S) / (L+G)
My required win rate is L/(L+G) as usual.

So let's try a few variations. If I use a 50 pip stop and 50 pip TP we get:

Expected (I know, kinda simplistic Win Rate = (50-2)/(50+50) = 48%
Required Win Rate = 50/(50+50) = 50%.
Only a 2% disadvantage due to the spread now! But of course now I'm not really scalping anymore am I? So I have to wait longer for each trade to play itself out.

What about a 95 pip stop and a 5 pip target?

Expected Win Rate = (95-2)/(95+5) = 93%
Required Win Rate = 95/(95+5) = 95%
Still a 2% disadvantage, but at least I'm winning a lot!

Finally, what if I cut my losses short at 5 pips and let my profits run to 95 pips?

Expected (unsophisticated) Win Rate = (5-2)/(95+5) = 3% ugh!
Required Win Rate = 5/(95+5) = 5%
Still a 2% disadvantage, but this method is psychologically nasty.

So anyway, those are my thoughts on the effects of spreads and how to incorporate them into my simplified model.

Sorry my friend, I missed the last sentence of your post...Totally agree and I apologize for missing the most important sentence in the whole post...


Thanks,

Nader

No problem Nader!

I really shouldn't have put the most important sentence at the very end like that. I should have been more clear in that post about my intent.

Thank You, capitalist88!

One of the most useful threads on FF.

Bluefish

Wow, I thought this thread had died out. I happened to be updating my log this morning and noticed it was still active.

I appreciate the compliment bluefish, and Happy Holidays to you and all!

Hi Capitalist, Narafa, Hanover:

I am really impressed, how great minds have found they're place to exchange (and share) some deeper mathematics about trading!

Keep 'em coming. I'm all ear and you definitely have my attention....I am a big fan of mathematical models (although not close to you).

Lately I have been reading a lot about Probability Theories and am quite fascinated with "stuff" like this!

I hope this conversation keeps going.

Wish you guys success!

Peter

Hi petercross and everyone!

Sorry it has taken me so long to respond, but I didn't really have much else to add to this thread at the time. However, recently there has been a bit of a brouhaha (great word! ) on my journal about some research results I had concerning this idea of expectancy. I figured I'd revive this thread as a place to direct any further discussion on the issue.

Ok, here's the situation. In my first post on this thread I presented an argument for why in the absence of an edge, expectancy should be about zero. That argument was purely analytical; it was based only on a logical model in my head, not on any empirical evidence from research.

Over the past few weeks I've been working on a three part research project on price action and structure at different scales. The research isn't specifically about expectancy, but I kind of stumbled onto it again in the course of my work.

What I found wasn't pretty. Forget about "natural expectancy" being zero. From what I found so far it looks like it's a LOT less. I'm using the term "natural expectancy" to refer to the expectancy of a system that has no edge; i.e. a random entry point and a defined stop and TP distance over a defined forward trading window.

I won't get into all the mathematical and testing details in this post because I've done that in my own journal and almost worn out my fingers typing. : But I'll give an informal description of what I'm talking about.

Suppose you take a significantly sized sample of consecutive price bars from any pair or any time frame, say 1000 bars for example. Choose a trading period that's a small portion of the sample, say 20 bars. Finally, choose what you think is an appropriate risk management strategy by picking a stop and target distance in pips. Let's say we keep it simple and use a 1:1 risk/reward ratio and pick a 30 pip target and a 30 pip stop.

Now check to see what would happen if you buy at the close of every single bar. Every trade will either be stopped out, hit the TP, or neither during the 20 bar trading period. Chances are, the result will be a loss. No problem, let's do the same thing again, but this time we'll sell at the close every bar and see what happens. Guess what? The result will probably be another loss.

My conjecture is that no matter what values you choose for the trading period, stop and TP amounts, the net return from both the long and short tests will always be negative, even if there is a strong trend. In the case of a trend, the long trades may give a positive result but the short trades will more than cancel it out. And this is not an effect caused by the bid/ask spread. I didn't even include that in what I was doing at the time, and anyway the BAS effect is peanuts compared to the effect I'm talking about here. What I'm seeing is like an order of magnitude greater than any BAS disadvantage.

So what if this is true? First of all, the idea that good risk/money management is going to save us from a mediocre trading system is in trouble. Forget about the "toss a coin, cut losses short and let profits run" business. That's out the window. What I take away from this, as I said in my journal is that it's imperative that a trading system have good signals; i.e. some kind of definite edge. Without that, you're doomed, no matter how good your money and risk management skills are or how well balanced you are psychologically to be a trader.


OK, now before everyone starts leaping up and down on my head here let me say a few things about this.

1. This isn't a "thought experiment" where I'm just saying how I think the market behaves. I'm basing my conclusion on actual empirical evidence. Granted it's not extensive evidence; I was only working with one pair, but I tested this on about 3000 daily bars, 7000 10-minute bars and about 17,000 10-second bars. The results were the same in every TF. So if someone wants to say, "well I don't agree with you because of XYZ and ABC..." that's fine. Now go and convince the market. The best sounding argument in the world means nothing if the market doesn't agree. So the only real way to dispute this is to find a counterexample where it doesn't work. I would love to see one.

2. This may not be a significant finding at all. Maybe what I'm saying is obvious to a lot of traders in which case I'm just a doofus who discovered that a triangle ALWAYS has three sides. I haven't thought about this enough yet to figure out if it's just tautological. However, don't confuse what I'm saying with the "no-arbitrage principle" that's used in financial analysis to create pricing models. If this was just the NAP, the natural expectancy should hover around zero like I said in the first post. I'm seeing values that are much less than that.

3. I am NOT saying prices are random. In fact, what I'm talking about looks like very non-random behavior. There is a trader in one of the Market Wizards books that talks about a theory that the market will do whatever hurts the most traders. This seems a little bit like that. I'm still trying to bend my mind around what it means. Somehow, the market moves in such a way as to make ANY combination of stop, TP and trading period unprofitable given enough time. Hmmm...maybe that's a good way to phrase what I'm talking about.

4. I am NOT saying that it's impossible to be a successful trader. What I'm saying is that in order to be successful, you MUST have some way of deciding when to go long and short (technical, fundamental, gut feel, reading goat entrails or whatever) which is powerful enough to beat this tendency. I think that there are many such methods and edges (I'll be testing out that whole goat entrail thing soon...). But the point is that risk/money management and good psychology by themselves are NOT enough even to break even.

5. Maybe this weird tendency of the market to break any static MM strategy will provide an insight into how prices behave that I can use as an edge. Again, I haven't thought about it enough.

My fingers are tired again so I'm out.

If I may add, this post contains pure truth!
Thank You many times for sharing Your thoughts based on Your research.

Happy Trading

Bluefish

I am all about goat entrails.

but seriously, great post! I believe it.

I have been thinking about predatory trading patterns; Imagine program trading that presenting pricing data that acts like a fisherman's lure, all the classic analysis tools are considered to validate even the most savvy TA gurus tendencies...

I don't know...I got the feeling on EURUSD recently as if I am being baited into going long here at the top. Granted it will likely make new highs this year, but not until there are casualties in the wet well.

Thanks for your comments TIckerShuffl and Bluefish!

I can't think of anything else to add right now. So I guess all any of us can do is go forth and try to find an edge...

That puts us write back where we started from lol. The risk we face though is that due to the changing nature of the markets, an edge today can fail tomorrow and mayby, that over time, the expectancy is still zero

The edge is luck, I guess... lol

No no, don't get that impression from this thread. All I'm saying is that if we don't have an edge, good MM and psychology can't save us. The market seems to move in such a way that any fixed combination of RR and trade length will eventually have negative return.

But a good edge can defeat that obstacle.

I agree with Jack Schwager: There are a million ways to make money in the market but they are all really hard to find. The only thing I would add is that some of these ways will change and stop working, so as traders we have to work to keep up with the changes.

There ARE edges out there. And there are a lot of them, not just one, which is why I can't understand why everyone keeps talking about this weird search for a "moldy pail." I mean, who wants a moldy pail anyway?


An edge doesn't come from luck. It comes from work. We make our own luck. Which means I have to get back to work...

Just my thought , and the result of this can probably be called an edge itself ...
expectancy is zero when the trade is linear ... meaning you take 1 position n wait for it to hit your tp or sl , ur profit or loss is linear to the market movement.
But consider this , you increase your position in a non linear way as the market moves in your direction and decrease it as it moves against you , that way your profit is always in exponential or to some degree depending on the way you increase your positions and your loss would be negligible (1/e) ...
Does this make sense ? only way to have a real edge is to have exponential move in a linear market ...

btw im big fan of fountainhead n athlas shrugged

Hi maheswara,

I'm a fan of scaling in to winning positions as well because I think that's where the biggest gains eventually come. So I like this line of thinking in general. Also, I've gotten away from setting stops and TPs at the beginning of a trade as you said. I use the concepts of a constant stop and TP as research tools to be able to measure probabilities, but I don't trade with them (I always have an ermergency stop of course, but it's not how I enter and exit trades).

That being said, I'm still turning the idea over in my head.

On the one hand, no matter what series of entries and exits we have, we'll always be in a position that has a definite average entry price. No matter how clever we are in scaling in or out, it won't help us unless we get the direction right. Adding as the trade goes our way will hurt us more if it turns around just after we've added, and reducing the position as it goes against us will hurt us if the price turns in our favor just after we've reduced. So when I talk about an edge, I don't mean a better money management technique, I mean a good directional signal.

On the other hand, what you suggest will help a lot. You're right that my whole "negative expectancy" argument is based on having a stop and a TP set at the outset of the trade. A stop tells us to get out if the price goes against us, and a TP tells us to get out if the price goes our way. This is why I don't like TPs. We should be adding to the position as it goes our way, not subtracting from it.

But it's still not enough. And the way I know that is to ask myself a simple question. Would this money management technique win at a roullette wheel? No, because it's a negative expectancy game, and no MM technique will beat it, given a finite bank (not even a martingale).

The only way to win at roullette is to have a definite edge over the wheel. There were actually some guys in the 70's who did this by putting computers in their shoes which solved the physics equations of motion of the wheel. They were able to predict with pretty good accuracy which octant the ball would land in. It's described in a book called The Eudaemonic Pie. Fun stuff.

Anyway, these guys won over time, but not because they varied their bet sizes based on the last win or loss. That won't do it. They won because they had a clear edge which told them where the ball was most likely to stop. And I guess my whole point in this thread is that THAT is the kind of edge that's necessary to be successful in the market. At the end of the day, we've got to have a good idea of WHICH WAY the price will move over some period.

Wow, I'm really running off at the keyboard here. But since you mentioned Atlas Shrugged, do you remember the scene where Francisco buys his first copper mine using winnings from the stock market? His father asks him how he learned to do that, and Francisco tells him "It's not difficult to determine which commercial enterprises will succeed and which will fail." See? He had an edge!!