Expectancy Management

It is worth adding that the above calculations of the Kelly criterion and Kelly expected growth rate are really only appropriate for trading strategies that use no exit other than a fixed stop loss and profit target.

Kelly is about Money Management. With the high risk of repeating myself this is OFF-TOPIC.

In pseudo-maths, "1+4%" is not a correct notation.

If you want to educate people give the full kelly equation, not the simplified version for binary outcome you copied and pasted from Wikipedia.
I know it is much more complicated and requires pseudo-maths, like numerically solving the first positive root of a high degree polynomial of logarithms.
Now I'm fed up feeding the troll.

Good thread.

Im thinking, eventually filters are part of the system. After phases of testing, you definately would accept the best R:R and winnings rate and try to filter out only the reliable, accurate setups.

this is nothing

PipMeUp, you are right to object to having a very good thread hijacked. However as a point of clarification, there is a bit more to the Kelly criterion than just money management (I hope you will permit a small digression). I also hate to feed the trolls but in this respect Ltc.'s answer with regards to expected growth rate vs. expectancy is more complete (with a caveat - see below).

The part of Kelly that most know about is optimal leverage (f*) or the optimal amount to bet to maximize median log wealth over the long run. The Kelly criterion has its mathematical roots in information theory and is closely related to the information rate.

The flip side of the optimal amount to bet, is the expected growth rate. They are really just two sides of the same equation. There is an extremely good exposition of this subject in chapter 17 of "Paul Wilmott - on Quantitative Finance Vol 1-3 2nd Edition"

The expected growth rate is about maximizing expected growth, which is a concept / goal very similar to maximizing expectancy. But given two strategies, the one with the higher expected growth rate will tend to have better log median performance in the long run (but this outcome is not 100% guaranteed at any single point in time). Maximal expected growth assumes that a Kelly sizing strategy is also used (f*). So in this case there is no escaping the link between maximizing the expected growth of a trading system and an optimal betting strategy (MM). The appropriate form of the continuous Kelly expected growth rate for traders is:

Also, if anyone wants more background on the continuous form of Kelly and its practical applications, see this post and follow the link / skim the pdf, to see how applying a continuous MM strategy it can improve upon buy and hold: http://www.forexfactory.com/showthre...81#post6418881

(now for the caveat) While Kelly's expected growth ratio does indicate a way to compare strategies for possible maximal future performance, and higher ratios will likely outperform all other strategies with lower expected growth ratios, the ratio itself doesn't indicate how to achieve this performance!! Exactly which measures should be optimized, added or eliminated to increase this ratio?

From the formula above, it can be seen that either an increase in average returns / trades, or a decrease in the variance of returns will maximize the expected growth rate. While methods that decrease average returns, decrease trades or increase variance of returns will tend to decrease the expected future growth rate. But what is the link between these measures and more practical trading realities such as the timing / price of entry and exit, stops, targets and the like that have to do with practical system development? (not a rhetorical question, and I believe this post is now finally back on topic)

Because I do my homework even, and especially, when I'm told it is useless... I ran two Monte-Carlo simulations of two Bernoulli trials. In these two games either you lose 1 or you win R. Probability of winning is P.

The first one simulates a system with 70% win rate and a R:R of 1. Its expectancy is 0.4.
The second one simulates a system with only 28% win rate but a R:R of 4. Its expectancy is 0.4 as well. Apple to apple!

In the first 2 pictures below, 100 trades are simulated 1000 times each. Because drawing 1000 curves doesn't look good, the pictures below are the lowest and the highest points reached over the 1000 experiments at a given trade index. The middle line is the mid-point (low+high)/2.

As you can see the first one has a much lower drawdown thanks to its high probability of winning. Starting with $100, the final balance after 100 trades is between 106 and 168. The average is 137.

The second system suffers a DD of 35%. Starting with $100, the final balance after 100 trades is between 75 and 210. The average is 142.5. Quite similar but much higher volatility.

What about using this magical Kelly? The percentage to bet using Kelly's formula is E/R. The first system can bet 40% of the equity. This means that after a winner the balance is increased by 40%. That is the so call "expected growth". The second system can only bet 10%. But its R:R being 4 times the risk, after a winner the balance is also increased by 40%.

The results are shown in the last two pictures. I stopped the simulation after only 12 trials so the difference is obvious. The first system, with a win rate of 70% and betting 40% of its equity reaches the finish line with a ridiculous ~$4000 at best. The system with 28% win rate betting 10% reaches a 8 figure balance!

You are more than welcome Patrick to bring this very interesting contribution

Very nice post Ulrich! This confirms a few things that I believe with the introduction of the additional topic of risk of ruin:
1) Always do your own original research rather than rely upon the work/words of others.
2) Higher win rate systems tend to have lower drawdowns. Drawdown is also closely related to risk of ruin (see picture).
3) Comparison of average profit lines of a Monte Carlo analysis is a fairly good measure to use when comparing quality of systems. Because the low win rate system had a higher average line, I concluded before reading to the end that this system would likely outperform.

After studying the table it is clear that there is an easier path to reducing risk of ruin by increasing win rate (probability of success) rather than Return:Risk (payoff ratio). Another method of reducing risk of ruin is diversification. Some say that diversification is the only free lunch available in the markets. As you stated earlier, this can be market diversification, and/or system diversification. Finally, risking a lower percentage of wealth on each trade can help reduce the risk of ruin.

Table taken from Money Management Strategies for Futures Traders by Nauzer J. Balsara. (IMO the tables were the most interesting part of the book) To read the chart, the win% is down the left column and the payoff ratio is across the top. So for instance going down the far left column, a win rate of 50% with a R:R of 1.0 has a 99% chance of ruin. Increasing the win rate to 55% with 1.0 payoff ratio lowers the risk of ruin to 36.8%. The top table is assuming risking 20% of the bankroll on each trade. The bottom table assumes risking 10% on each trade in calculating the risk of ruin.

In the picture you posted, the variance of the low win rate is clearly greater, and the lower blue line of the low win rate system is more in loss territory than the low line for the high win rate system. I think when doing Monte Carlo analysis, this visually can be seen as a larger risk of ruin in the second subgraph.

I completely agree with your conclusion about the risk of ruin. Do you know the formula used to generate these tables? It would be interresting to compute the value of each cell according to its corresponding Kelly bet size.

Perhaps this validates an idea I have for some times: running in parallel a high win rate small RR and a low win rate high RR. Simply because they are both easier to find than the "healthy and rich man" high win rate and high RR. The high win rate supports (finances) the first one. The collected profits aim at compensating the loss of the low win rate. The high win rate will certainly be a short term system. It will certainly have more opportunities and may compensate a loss before it happens. If they are sufficiently different like a daily trend follower and a M15 scalper I can assume they are independent. Their expectancies don't sum because the frequencies of trading are different. What would the risk of ruin become?

The author apparently used a simulation rather than a closed form solution. Start reading at the highlighted sections for a review of the methodology used to create the tables.

As to your question, I think it depends heavily on the makeup of the two systems as to how the system performance is affected. A few of the many questions that this topic opens up: Will you be trading both systems with equal sizing, taking the naive approach that is it unknowable how the systems will interact in the future? Is there an optimal sizing formula that you can use that will improve on the naive approach such as Markowitz? If so, can you realistically assume that correlations (or some other statistical measure) will hold into the future?

Will you allow the systems to keep the profits they accumulate separately and thus sizing will diverge over time? Is that a good/bad thing? Will you instead take the signals from the two+ systems and integrate them into a single system? Will the higher frequency strategy override the lower frequency strategy's signals in most cases?

By the way, I don't have all the answers to these questions. I have tested some of them and the correct conclusion to draw is not always obvious from the results. But my current preference is toward the naive approach.

His MM isn't %X per trade but %X of the starting capital per trade. He is increasing the risk after a loser trade (in % of the account).

- Will you be trading both systems with equal sizing?
Good question. They shall use their own sizing depending of their own stats. Say we use 1/3 of Kelly, each system will have a different bet size. The corrolary question is X% of what money? Because if one system opens a trade, the remaining capital is smaller for the second. Should the second had open the position first his bet size would have been bigger.

- Can you realistically assume that correlations (or some other statistical measure) will hold into the future?
I have absolutely no idea of how to mesure the correlation between two systems. Of course you can extrapolate by using the correlation of their trading history. But... If you take two extremely good systems, with small DD, their trading histories will look exactly the same. Therefore you will conclude they are 100% correlated, which is perhaprs completely wrong. I thought of using a function R->{-1,0,1} over the time where f(t)=-1 if a system is short, +1 if long and 0 if flat. Then compute the correlation of f1(t) and f2(t). I don't know if this is valid.

- Will you allow the systems to keep the profits they accumulate separately and thus sizing will diverge over time?
Actually I thought of it as "the Knight and the farmhand Peon". One works hard to feed the soldier and his horse. So they would form a single system.

http://mechanicalforex.com/2013/04/d...relations.html
The first comment is exactly what I proposed with the Fi(t) functions
At least this covers the case where the two systems have very different trading frequencies. The money allocation might be modulated the same way as I propose for the allocation between assets. This would allow to trade more than 2 systems.

Nice site.

I wanted to add one more thought to this thread on expectancy management.

The idea of consistency is a fairly important topic as it relates to system traders. Consistency can basically be defined as how many profitable periods a strategy has vs. how many losing periods. How does maximizing expectancy fare with regards to consistency? Over on ET there was a thread a while back that had some interesting insights. I suggest reading all of acrary's posts on page 1 as they relate to this post, particularly starting with the 4th post in that thread.

Executive summary: maximizing win % and expectancy had very little positive impact on consistency. Maximizing trade count and profit factor had a positive impact on consistency.

In the post containing daytrade2.txt. Acary says "he still only has about 80% confidence level with double the expectancy on each trade." But opening the file you see that he DOESN'T double the expectancy. He double the bet size.

In the post containing daytrade3.txt. Acary says "How about if he increases the winning percent to 80% and keeps the expectancy at $400, so that each win now becomes $600 and each loss now becomes $400."
But he actually increase the expectancy from ( 0.7*(1+1)-1= ) 0.4 to (0.8*(600/400 + 1)-1=) 1. If I multiply my expectancy by 2.5 for sure I expect to perform better! To really keep the expectancy of 0.4 with 80% win rate, the winners must be $300 for $400 bet (R=((0.4+1)/0.8) -1 =0.75=300/400).

In the post containing daytrade5.txt. Acary contradicts himself "From this we can tell the win % and expectancy are not critical to consistency." followed by "The higher the profit factor, the more liklely we are to achieve consistent profitabilty". He advocates to maximize the PF. The PF is F=p.R/(1-p). Maximizing F is done by either maximizing p as 1/(1-p) goes to +infinity when p nears 1 and/or maximizing R. Maximizing p and/or R is equivalent to maximizing E=p(R+1)-1. Therefore the win % and expectancy are absolutely critical to consistency.

I keep reading this thread as at page 2 he discusses about running systems in parallel. This makes me very eager to read more. Just that now I know I have to check the formulas he gives...

When I first saw your reply, I immediately thought that you had used the wrong equation for PF [F=p.R/(1-p)]. After taking a few minutes with this simplification I can see now what you mean. It isn't every day that acrary gets schooled! I found this post from his thread rather interesting:


It may not be 100% accurate (how can it be, considering all the factors involved in system design and performance measurement?) but as a general heuristic I think it is very interesting to note how large a factor trade count is in achieving consistency (also PF). But this relates to what I was saying over here regarding consistency. When I think of consistency I think about how often something happens. One synonym for consistency is stability. I think this is a useful term for evaluating trading system performance.

If you look at the equity curve of any given strategy and wanted to compare system A vs. system B for consistency, how would you go about it? One way might be by comparing statistics such as win%, R:R, or PF. Another method might be to analyze the size of the drawdown(s) between the two and do a comparison such as with the Sterling ratios of the two systems such as profit/DD. However, the equity curves are much more complex than a single statistic.

So as acrary mentions, one way to think about consistency is in terms of profitability over a given period, such as weekly or monthly profitability. Simply by counting the up periods and comparing the up/total periods between the two systems, a new statistic based on consistency of returns can be calculated. A system that has more up periods than down could be said to be more stable or consistent. Trade frequency might help in improving the number of up periods due to the edge being applied more often. Other factors may also help/hinder.

Another aspect of consistency is magnitude of drawdowns. Many new traders attempt system trading with the "novel" martingale concept. They achieve smooth positive returns over a period, then an abrupt and very deep drawdown that blows their account. So two other stats that may be important to system developers for measuring consistency are the magnitude of the drawdowns, and whether the losses tend to cluster in one area. Higher win% will help with loss clustering, no doubt. But higher win% might not help with larger magnitude drawdowns. This seems to be the area of R:R.

So it seems you could summarize by saying that win%, R:R and trade count are all important factors that relate to consistency of returns. Are there any others?

Keep in mind that when you increase the trading frequency your trades become more and more linked to the market movement of a short period of time: your trades become correlated. But his simulations as well as the ones I've posted are based on Bernouilli trials which assume independence. Of course if you can trade your system more often with the same reliability while keeping the independence... Why do you believe I'm trying to build a set of (potentially) strongly trending but non-correlated pairs

The autocorrelation, like induced by the overlapping of the trades, which can highly turn a system instable. Sure if it become positively unstable I may accept it. That's why I'm still so unsure about the pyramids.

Whatever the index of comparison between two systems a martingale shall always be classified the poorest. Because not only it has a negative expectancy but it also has the biggest destructive power thanks to the exponentially increasing and unbounded risk. Even if the mother of the losses didn't happen yet it shall be classified poor. If one only look at the equity, a flip a coin with martingale system will look exactly like the Holy Grail before the first big dent in the equity. The equity curve must therefore be studied BEFORE money management, in raw pips/points.

Please allow my ignorance if I did it wrong. I tried to replicate your experiment and got the opposite result. I followed your settings that [RR=1, win 70%, bet 40% by kelly] VS [RR=4, win 28%, bet 10% by kelly].



The system with RR=1, your 1st system, tends to outperform the RR=4 system. Of course, through multiple runs, occasionally, RR=4 system may beat RR=1 system. But more likely than not, RR=1 beat RR=4, according to my spreadsheet. I tested with 100 trades for each run, and manually repeated tens of runs (didn't do monto-carlo).

I attach the spreadsheet here and look forward to your examination. I truly appreciate anyone's insights/comments, as your conclusion shakes my fundamental concepts.

I've had a chance to think about this for a while - specifically the point made in the first sentence in the quote above. I suppose you're talking in terms of a strategy that has an entry / stop / target, or simply entry and some sort of exit / flat period. But if you're talking about an always-in strategy then clearly such a strategy is already linked (correlated) to a great degree with market movement. A buy and hold strategy would seem to be 100% correlated to market movement. Any (always in) strategy where trade frequency increases above the buy and hold would lead to decreasing correlation IMO. A system with 1000 trades (500 long/ 500 short) would clearly be less correlated to the market movement than a system with just 1 trade, no? By correlation, I'm talking about the relationship between the underlying price series, and the equity curve produced by trading a system on the underlying price series. Are you defining correlation in some other manner?

So given an always-in strategy traded on a given market, increasing trade frequency would tend to reduce correlation to the underlying price series vs. a buy and hold strategy. Of course it looks like your main point has to do with correlation to a short time period. This is true but the correlation could be +1 or -1 with alternating long/short positions (averaging to 0 correlation given an hypothetical infinite # of trials?) Thoughts?

Thank you for your effort in reproducing the experiment. Your excel file is correct. It is clear that you found a mistake of mine. Now my own fundamental concepts are shaken! And this is certainly a good thing.

I still have a little problem with the RR1: the equity swings. It sometimes goes really really high but usually falls very deep. 95% DD are regularly seen. To such an extend that it may go under RR4 for some time before rising again. This is really counter-intuitive. It means that when you accumulated a huge amount of money, you should stop trading it. That is favouring under-capitalization... Or stated differently the method is so good that you shall not use it with big money!

I recently found another argument in favor of the smaller R:R. This is the fractal dimension of the market. Because the price doesn't move in straight line, doubling the target more than double the time to reach it. This reduces the frequency of trading.