Alternative Reward to Risk Ratio for Trading System Statistics | Forex Factory

Options

Permalinks
Print Thread

Similar Threads

High Risk/Low Reward Ratio Trading System 108 replies

How Reward to Risk Ratio affects your trading results 0 replies

Rationale behind "Risk Per Trade" and "Risk/Reward Ratio"? 8 replies

Good reward/risk ratio experiment 20 replies

Risk Reward Ratio and Account Risk 30 replies

Alternative Reward to Risk Ratio for Trading System Statistics

  • #1
  • First Post: Edited 9:27pm Jun 24, 2014 8:20pm | Edited 9:27pm
I was going to post this in the Trading Discussion forum, however this is a question that is specific to Trading System Analysis.

When evaluating risk vs reward of a trading system (backtest or real results) has anyone ever worked with using:

Rate of return or ROI
divided by
Max DD Percentage

as a measure of system strength?

The well touted method of money management compares losses vs. gains of individual trades and is somewhat suited for risk management before the trade. However, in evaluating historical trading results, I have been considering that using rate of return over the evaluated period divided by max DD percentage might be a more robust indicator of risk evaluation of the system over a time period compared to other systems.

For example:

A system that delivered real results or a backtested result of 60% return with a 20% max DD would yield a calculation of 3.

Another system for example has a result of 30% return with a max DD of 3% would yield a figure of 10.

Thus the system in the second example would be a more robust system, and given a similar amount of trades or a reasonable amount of trades in both directions for both systems during the same time period, system 2 would be preferable.

A base calculation of 1 would be considered neutral (example: 40% return and 40% DD) .

A calculation of less than 1 would be undesirable (example: 70% return and 75% DD).

Of course, a higher percentage return with the lowest possible percentage DD is the best, but other than this simple calculation there is no simple way to compare results of two different systems or similar systems with different risk profiles.

Anyone currently work with this calculation in their trading system analysis? Is there a name for it? I've never seen it used or discussed elsewhere.

I think it gives a better picture of risk/reward and is more reasonable than simply considering just return, or just drawdown when evaluating system results. Of course there are many other statistical calculations available in trading system results analysis, however, again, I've never seen this simple comparison calculation anywhere.

Shall I call the calculation TRMD? *Total Return vs Max Drawdown
  • #2
  • Jun 24, 2014 9:43pm Jun 24, 2014 9:43pm
I think it is an interesting idea to evaluate the returns obtained by comparing it to the risk incurred. In respect to your idea I think it might be a good way to go but the results you get will only be relevant if the 2 back test are done on the same period of length. The reason being that a abs DD will remain relatively constant but the returns will keep rising assuming that you have a real statistical edge that is proven by a big enough sample. For example, let say you test 2 times the same system, first test being done on a period of one year and the second on a period of 2, you can safely expect the second test to give a score twice as high, return doubled because the test ran for twice as long and the DD should have been around the same assuming constant market conditions. The 2 things you wanted to mesure did not change (risk the same, return per unit of time the same) but yet your metric tells you that one system is twice as "robust" as the other.

In the example I just gave it is easy to remedy to the situation by getting the rate of return annualized to bring both test on the same level but it gets more complicated when you want to compare a scalping method risk adjusted return to the one of a swing or longer-term trading method... Some sort of variation of the sharpe ratio might be more appropriate in this case.
It's not about what you make, it is about what you don't lose
  • #3
  • Edited 12:26am Jun 25, 2014 12:12am | Edited 12:26am
Quoting aceventura69
I think it is an interesting idea to evaluate the returns obtained by comparing it to the risk incurred. In respect to your idea I think it might be a good way to go but the results you get will only be relevant if the 2 back test are done on the same period of length.
I think comparing two systems or two similar systems over the same length of time is a must otherwise you would get skewed comparisons.

Definitely compare over the same period of time, when comparing system results or as you say use an annualized return if not possible to test over same length of time.

As far as comparing a scalping method risk adjusted return vs longer term, it would be a non parallel comparison and possibly not effective.

As far as the Sharpe Ratio goes, it's a bit complex and I think it discounts some very profitable systems. I think the simple method I presented takes into account compounded return more fairly. Compounding returns is a result of position sizing vs. equity (size of risk), and the method I presented allows for adjusting it according to one's overall risk appetite. It would be interesting to see the differences in Sharpe Ratio and my TRMDD ratio on the same systems. However, I don't really care for the Sharpe Ratio. I think it's more suited for equities portfolio management. Not sure how effective of a measurement tool it is for highly leveraged FX trading.

Note: Personally, I wouldn't choose between a scalping system and a longer term system. IF both were effective I would trade both in different accounts or even try merging the two into one EA to run on one account.

Note 2: Haven't found or developed a scalping system yet that gets me excited enough to throw money at. I've seen a couple that look interesting though. I think that over time, I solid compounding longer term system will outperform a good scalping system. However, would like a good HFT scalping system to increase equity to put into more longer term systems.
  • #4
  • Jun 25, 2014 3:07am Jun 25, 2014 3:07am
"my" formula to evaluate system strenght (especially during optimizarions) is:

roi * number of trades \ maxDD

the number of trades also is important imo, the more the better!

eventually when doing optimizations, i went for this:

roi * number of trades \ maxDD * roi (the roi is used 2 times, the formula still care about maxdd and trades number, but is somehow slightly more aggressive)
  • #5
  • Jun 25, 2014 4:03am Jun 25, 2014 4:03am
I like your thinking pipologist. A measure of of robustness. Could be quite useful for automated systems.

However this shouldn't be used if one system is better that the other. Notably a system that scores 1 may be more profitable than another that scores 10.

For example:

A system that delivered real results or a backtested result of 60% return with a 20% max DD would yield a calculation of 3.

Another system for example has a result of 30% return with a max DD of 3% would yield a figure of 10.

At the end of the day the more fragile system with a 20% DD is still more profitable which is the point of trading.
  • #6
  • Jun 25, 2014 5:02am Jun 25, 2014 5:02am
I don't have a ready made formula but let's try to build one.

First let's reject RoI/DDmax. Imagine a system which target 1 pip profit with a huge stop. After that it stops trading forever. This system is edgeless yet its winrate is extremely high. It has therefore a good chance to succeed in several backtests. In case of success of this single trade the measure RoI/DDmax is infinite (DDmax=0 you divide by zero). The formula should contain a penalty for not trading. This will force the system to reveal it is edgelessness. For example you can use the amount of money you would have got from the starting balance with a risk free saving account over the same period.

The formula shall contain the expectancy of the system and give a negative number if the system is a loser. The expectancy is E=p(R+1)-1. p=winrate (probability of success) and R=the RRR.

The formula shall contain some measure of the risk of ruin. This risk depends on the winrate. The lower the winrate the higher the probability of a string of N losers for any N. Note that for a given E, the smaller is p the higher is R. Interestingly the Kelly criteria match our needs. K = E/R = (p.(R+1) - 1)/R.

The frequency of trading is also important. If two systems have the same R and p they have the same K. The one trading more often will generate more profit. If F is the frequency of trading we can use F.K

If the trades have a certain duration D which is longer than the trading interval (reciprocal of the frequency) the trades will overlap. If they overlap they are correlated (otherwise you would stop-and-reverse) and then increase the risk because the system overtrades. There should be a penalty for the duration beyond the interval. The formula should contain something like (1/F - D).

All those elements can be associted with a weight.

For any two given systems the best one must have a better score than the other (hard to validate!). The imaginary Holy Grail system shall have the best possible score and no real system should reach this score (unless you discover the HG of course).

This doesn't give a formula but give some hints to find one.
No greed. No fear. Just maths.
  • #7
  • Edited 1:56pm Jun 25, 2014 5:15am | Edited 1:56pm
Pipologoist,

See http://en.wikipedia.org/wiki/Sterling_ratio Sterling Ratio.

Edit: Sorry about that, link updated.
  • #8
  • Jun 25, 2014 5:23am Jun 25, 2014 5:23am
Quoting FXEZ
Pipologoist, See Sterling Ratio
Your link to Sterling Ratio goes to the previous post ? I'll google it.
  • #9
  • Edited 6:24am Jun 25, 2014 5:31am | Edited 6:24am
Quoting PipMeUp
I don't have a ready made formula but let's try to build one. First let's reject RoI/DDmax. Imagine a system which target 1 pip profit with a huge stop. After that it stops trading forever. This system is edgeless yet its winrate is extremely high. It has therefore a good chance to succeed in several backtests.
Sounds like a Martingale to an extent.

A very small pip target with huge stop...let's say that the stop is triggered eventually and the system reveals a 95% DD or more. The system is over. The system can only take 1 losing trade and 2 consecutive losers would bust it....NOT A ROBUST SYSTEM. I would disregard ANY stats from a system that can not live through at least 10 losing positions in a row. Or multiple trades in a row if the 10 positions are actually only one entry.

I understand your theory in building a formula. However, some things are unstated but obvious, those kinds of systems would be disregarded. If a backtest shows 90% or more win rate and can not handle successive losing trades, then it is out. Period.

The problem with those kinds of systems is that in time they reveal multiple potential black swans and you only need one to wipe out.

One measure of "robustness" that I look for is a system that can handle a 95% DD with no forced margin liquidations in the time period of trading and the resiliency to bounce back profitably even after suffering a 95% DD. Or multiple over 50% DDs. Of course this would also mean that the system is highly leveraged. But for a small speculative account it would be acceptable. With a larger starting equity account, the risk on new positions would be dialed down to reduce the max DD periods.

I understand the rejection of RETURN/MAX DD in order to build a better formula. However, a formula that includes a system that generates a one pip profit with a huge stop loss and then stops trading after either event without successively continuing to trade is not really a trading system, but a hypothetical numbers game. In game theory with limited outcomes, like poker or blackjack, the outlying proposition could work, but everyone who has traded for more than a few years understands that there are basically an infinite number of trading outcomes over time in the markets and especially when dealing with automated systems, especially over the counter FX, it's not just a buy or sell, or win or lose proposition. It would be important in using TRMM ratio (total return / max dd) to disregard the obvious outlying scenarios that one would really never trade in real life since those outcomes would never occur, not that they couldn't mathematically, but because no trader would ever really trade that way. However in cards, if that outcome was one of the 52 cards in the deck, that could come up in a game of hold em or blackjack, then that kind of outlying scenario has to be considered.

I think adding statistical theory and/or game theory can be a vital part of trading analysis, but at the end of the day, there are some scenarios that ANY real trader would NEVER DO, even if it were possible so those scenarios have to be discounted automatically. But in roulette or craps, even the 30plus to one odds needs to be considered as a possible outcome and possibly game scenario.

I just don't thnk that "a system which target 1 pip profit with a huge stop. After that it stops trading forever" would ever, ever be traded by anyone if it was deliberately designed that way, so it's not something that even should be considered. However, if for whatever reason, your system produced that result uncontrollably and that event happened completely unexpected, I would have to say that the financial markets are melting down in a very very super black swan way and there are other more important things to worry about.
  • #10
  • Jun 25, 2014 5:35am Jun 25, 2014 5:35am
Quoting Danny Maina
I like your thinking pipologist. A measure of of robustness. Could be quite useful for automated systems. However this shouldn't be used if one system is better that the other. Notably a system that scores 1 may be more profitable than another that scores 10. For example: A system that delivered real results or a backtested result of 60% return with a 20% max DD would yield a calculation of 3. Another system for example has a result of 30% return with a max DD of 3% would yield a figure of 10. At the end of the day the more fragile system with a...
Yeah, I'm strictly talking about applying this to automated systems and evaluations of trading systems, especially those that allow flexibility in the amount of % capital to risk on each trade or position.

I say trade or position, because many of the EAs I develop now put on multiple positions per entry signal with varied or staggered exits.
  • #11
  • Jun 25, 2014 5:40am Jun 25, 2014 5:40am
Quoting Danny Maina
Notably a system that scores 1 may be more profitable than another that scores 10. For example: A system that delivered real results or a backtested result of 60% return with a 20% max DD would yield a calculation of 3. Another system for example has a result of 30% return with a max DD of 3% would yield a figure of 10. At the end of the day the more fragile system with a 20% DD is still more profitable which is the point of trading.
If the systems have the ability to differentiate amount of risk percentage per trade, then the system of may be more profitable, or the system with a score of 10 could be leveraged higher or lower depending on the effect on DD and reward one *wishes* to achieve.

Of course, over any set of time, a system with a score of 1 could be more profitable.

I like the post and the idea put in by Spaceduck that includes the number of trades. I'll have to think about that one a bit more. I personally would not count ROI twice in the formula though since it overweights and gives higher priority to ROI over DD. I think both are important. More trades definitely gains more weight and as an added bonus if your broker gives you some trade rebates it's like extra cash that the system did not plan for. However, some would argue (mostly mathematicians) that more trades unfairly adds to the robustness of a system. I say otherwise, most traders know how difficult it really is to have a system generate a large number of trades over time and still be profitable. It becomes a good indicator of a system's viability and ability to withstand inherent automatic DD in terms of spread and/or commissions.
  • #12
  • Jun 25, 2014 5:51am Jun 25, 2014 5:51am
Quoting PipMeUp
The frequency of trading is also important. If two systems have the same R and p they have the same K. The one trading more often will generate more profit. If F is the frequency of trading we can use F.K If the trades have a certain duration D which is longer than the trading interval (reciprocal of the frequency) the trades will overlap. If they overlap they are correlated (otherwise you would stop-and-reverse) and then increase the risk because the system overtrades. There should be a penalty for the duration beyond the interval. The formula...
I have to disagree on the statement "The one trading more often will generate more profit". I understand it's in context to your initial logic using an already profitable system to start with. However, a system trading more will only be profitable if the win rate including the costs of trading (commissions, spreads, swaps) are consistently high, and unless the overall risk is dialed down, the system trading more has just as high risk of ruin if not more so. (Trust me on that last sentence, I hate to say it's true but experience tells me otherwise.)

I think a good metric would be "risk of ruin" vs. "probability of profitability". Of course mathematically the first one can be extrapolated, the second in real life, many times, comes to the stars aligning right over a period of time. Anyone who says that they KNOW the probability of success or profitability in trading is in the wrong business and should be psychic to celebrities.
  • #13
  • Jun 25, 2014 6:33am Jun 25, 2014 6:33am
OK. You clearly don't understand the explanation. The trivial loser is absolutely NOT included in anything. BTW it's not a martingale either. That was just a trivial loser used as a trival example to trivially break the ROI/DDmax measure.

You also disagree with statements you don't understand. Sorry for disturbing.Unsubscribed.
No greed. No fear. Just maths.
  • #14
  • Jun 25, 2014 6:36am Jun 25, 2014 6:36am
Quoting Pipologist
Anyone who says that they KNOW the probability of success or profitability in trading is in the wrong business and should be psychic to celebrities.
This is correct but I didn't want to add the variance into the formula to keep the thing simple at least for a start.
No greed. No fear. Just maths.
  • #15
  • Jun 25, 2014 7:40am Jun 25, 2014 7:40am
Quoting PipMeUp
OK. You clearly don't understand the explanation. The trivial loser is absolutely NOT included in anything. BTW it's not a martingale either. That was just a trivial loser used as a trival example to trivially break the ROI/DDmax measure. You also disagree with statements you don't understand. Sorry for disturbing.Unsubscribed.
I'm sorry I didn't understand your explanation. It clearly included the trivial loser in your example from the start. I know it's not a marty either. It's just that the trivial example if repeated over nth trades resembles a marty.

You're probably right that I disagree with statements I don't understand. However, It would have been more clear from the beginning to not even include a trivial example in your discussion if you did not intend to have it discussed. You can't say it was not included in anything because clearly your own statements confirm otherwise.

You can not use a trivial example to break someone's logic and then not have that trivial example used against you in your own logic. That's called playing devil's advocate at best and just breaks all rules of logical discussion.

Your post did not disturb anything. There were several interesting and valid points. I just didn't agree with your initial statement that's all. Not a personal attack.

Sorry for being so wordy and repetitive. Respectfully yours.

You are obviously very versed in mathematics and statistics, much more so than me and most users on FF. Forgive me for my lack of knowledge and simplicity.

I recall someone mentioning Sharpe ratio here and your discussion to me somewhat resembles the Sharpe ratio is some regards. I'm just trying to keep this as simple as possible, yet effective as possible.
  • #16
  • Jun 25, 2014 9:00am Jun 25, 2014 9:00am
Quoting Pipologist
I was going to post this in the Trading Discussion forum, however this is a question that is specific to Trading System Analysis. When evaluating risk vs reward of a trading system (backtest or real results) has anyone ever worked with using: Rate of return or ROI divided by Max DD Percentage as a measure of system strength?
It's not bad as a first stab at getting risk adjusted returns, but as PipMeUp has pointed out, you need a significant number of drawdown periods in your sample for any analysis of historical drawdowns to be meaningful, and even then it may not reflect the actual risk.

I use it the other way up (MaxDD/Daily Return) as a measure of the drawdown in days since percentage DD is pretty meaningless on it's own.
  • #17
  • Last Post: Jun 25, 2014 4:44pm Jun 25, 2014 4:44pm
Quoting euclid
{quote} It's not bad as a first stab at getting risk adjusted returns, but as PipMeUp has pointed out, you need a significant number of drawdown periods in your sample for any analysis of historical drawdowns to be meaningful, and even then it may not reflect the actual risk. I use it the other way up (MaxDD/Daily Return) as a measure of the drawdown in days since percentage DD is pretty meaningless on it's own.
you need a significant number of drawdown periods in your sample for any analysis of historical drawdowns to be meaningful

Absolutely.

I use it the other way up (MaxDD/Daily Return) as a measure of the drawdown in days

Can you give an example calculation of what figures you would use and how you would compute your MaxDD/Daily Return metric? Thank you.
0 traders viewing now