How to evaluate consistency? | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Price of consistency 4 replies

What questions should I ask to evaluate a hedge fund manager? 2 replies

Evaluate string expression in MQL4 13 replies

Length of the window to evaluate performance 0 replies

evaluate a newbie's system 3 replies

How to evaluate consistency?

  • Poll

  • How to evaluate consistency of a trading performance?
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  • #1
  • First Post: Edited 10:08pm Jun 27, 2009 7:41pm | Edited 10:08pm
How to evaluate consistency of one's own performance?

Note: This poll is a multiple selection one and votes are public.

Although one can never be sure that a particular performance will extend throughout all our trading career, how could we reasonably consider that a strategy is yielding a stable and steady return and thus has reached a fair level of consistency and can be relied upon?

Another question would be if a maximum floating drawdown is permanently about 10% of NAV with an average of 6/7% floating drawdown is too much of a drawdown, or if it would be acceptable considering that the real risk is always limited to less than 1.67% on any open position and there are never more than 6 open positions at a given time in extreme conditions, more usually averaging 2 or 3 open positions.

Thank you for your help, should you decide to answer the poll.
  • #2
  • Jun 27, 2009 7:48pm Jun 27, 2009 7:48pm
Quoting doblece
How to evaluate consistency of one's own performance?
How do you define consistency:

1. Uniform risk per trade
2. Consistently following plans
3. Do not enter prematurely i.e. consistently wait for right set-up
4. Trade only certain particular situations
5. Not changing trades once entered
6. Uniform $ value of wins and losses, etc. etc.
The Thief of Wall Street
  • #3
  • Jun 27, 2009 7:55pm Jun 27, 2009 7:55pm
It all boils down to account growth, in percent.
  • #4
  • Jun 27, 2009 8:15pm Jun 27, 2009 8:15pm
Based on my Monthly Withdrawal Rate.

It's worked for me for {10's of} 10's of months now. Even in a down Month, if I can afford to take my wages, It's been a good Month.
Didn't Vote in the Poll...
  • #5
  • Jun 27, 2009 8:17pm Jun 27, 2009 8:17pm
Quoting Chicky
How do you define consistency:

1. Uniform risk per trade
2. Consistently following plans
3. Do not enter prematurely i.e. consistently wait for right set-up
4. Trade only certain particular situations
5. Not changing trades once entered
6. Uniform $ value of wins and losses, etc. etc.

In this case it is a "swing & scalping" strategy. I have tested it in a real account and in the most extreme adverse conditions (that would be with the market going totally against the selected trend) the maximum risk is always maintained. Overall profitability has been steady over more than two years, but this strategy in particular has only been at work since January 1st (after a three-month period of testing and tweaking on demo to make sure that the risk levels didn't go above the percentage assigned in the MM). So officially, it's been 94.25% profitable over 1009 trades. The yield is quite small but the idea behind all of it is to have a stable and stress-free performance. It is still risky though because the R/R ratio is 5 times higher on the risk side (5:1) but the greater weight is put on the money management and constant recalculation and watching the drawdown level, taking the ruled actions whenever needed.

I guess that by "uniform value of $" you mean that all trades considered have the same amount of units at a given time? Indeed it is so, but they are periodically compounded (or reduced) to always reflect the same relationship to the NAV.
  • #6
  • Jun 27, 2009 8:26pm Jun 27, 2009 8:26pm
Quoting doblece
In this case it is a "swing & scalping" strategy. I have tested it in a real account and in the most extreme adverse conditions (that would be with the market going totally against the selected trend) the maximum risk is always maintained. Overall profitability has been steady over more than two years, but this strategy in particular has only been at work since January 1st (after a three-month period of testing and tweaking on demo to make sure that the risk levels didn't go above the percentage assigned in the MM). So officially, it's been 94.25%...
I've been testing the probability of scalping for a while now.
Curious to know, what's your average winner in pips?
  • #7
  • Jun 27, 2009 8:33pm Jun 27, 2009 8:33pm
Quoting doblece
In this case it is a "swing & scalping" strategy. I have tested it in a real account and in the most extreme adverse conditions (that would be with the market going totally against the selected trend) the maximum risk is always maintained. Overall profitability has been steady over more than two years, but this strategy in particular has only been at work since January 1st (after a three-month period of testing and tweaking on demo to make sure that the risk levels didn't go above the percentage assigned in the MM). So officially, it's been 94.25%...
I got you. Would like to add that there are two types of traders 1) who win on balance and compound their earnings with consistency as Trading in The Zone describes. 2) Trader keep their losses uniform to catch a big move like Reminscences of A Stock Operator.

If you are the one from 1) above, Yes to all the above options. But if one is from type 2) trader, their equity curve will be like the curve of the market But in my humble opinion, one should primarily be trader #2 but add their margins for bigger plays by following method of trader #1. Consistency give you an edge, but there cannot be a thing which is called consistent returns if one is not from the category of trader #1. These are just my thoughts and my aim is to earn great money with proper risk management.
The Thief of Wall Street
  • #8
  • Jun 27, 2009 8:43pm Jun 27, 2009 8:43pm
Quoting Ace284
It all boils down to account growth, in percent.
Indeed, though here also we can have mixed opinions: for me, the account growth should be measured yearly but others will take into account the monthly growth. Over the last 15 months two have yielded a negative (on compounded balance) return and two ended at break even, but in the first 9 months I was using another strategy so in fact I can only consider the last 1009 trades and 6 months, where one ended at B/E, one with 6,5% loss (a voluntary loss taken irrespecting the rules because I had to travel away for a few days in the month and preferred to assume it rather than spend so many days under stress - in fact should I had respected the rules and be more confident on the strategy, those positions would have effectively reached the target!) and the other 4 in profit.

So, monthly-wise, it seems a very little time as a whole despite the positive results (monthly average 7,03% over two trading accounts proportionally considered). But "trade-wise" I think that 1000+ trades can be a good indicator for its consistency? (still not 100% sure because I don't want to take it for granted as this is a pitfall to avoid). My daily aim is 1%, and have been effectively trading 65 days over the whole period so that gives me roughly 0.65% daily, which coincides with the results that I get on the majority of normal trading days.
  • #9
  • Jun 27, 2009 8:56pm Jun 27, 2009 8:56pm
Quoting Ace284
I've been testing the probability of scalping for a while now.
Curious to know, what's your average winner in pips?
Average winner is around 4 (in a range, 3.4 to 5.5). Maximum allowed loss is about 15-16 pips but in average I try to make it less than 10. Such a small TF is a totally different dimension of time which in fact seems to slow down while the truth is that, in optimum conditions, one is moving a little bit ahead of time...

My aim is "at least 3 points" but sometimes I get some extra pips
I still have things to tweak over time because I have noticed that I am tending to evolve towards a more longer term approach, which indeed yields a much better return when I'm not around "scalping the swing" but as precisely this 3 point scalping is part of the rules, I'll have to re-assess the whole strategy to see how I can adapt it to longer timeframes (and spend less time glued to the screen and clicking... though in good days I usually spend no more than 5-6 hours full focus with pauses).

At the present moment I use 5 sec charts (mostly for the PA) and enter on 5sec/1m agreement, along with 5min and 30 min agreement as to the direction towards the trend for the swing.
  • #10
  • Jun 27, 2009 9:24pm Jun 27, 2009 9:24pm
Quoting Chicky
I got you. Would like to add that there are two types of traders 1) who win on balance and compound their earnings with consistency as Trading in The Zone describes. 2) Trader keep their losses uniform to catch a big move like Reminscences of A Stock Operator.

If you are the one from 1) above, Yes to all the above options. But if one is from type 2) trader, their equity curve will be like the curve of the market But in my humble opinion, one should primarily be trader #2 but add their margins for bigger plays by following method of trader #1....
I have always tried to be on the number 2 group... but it seems it really doesn't fit my style. Since I started trading in 2005 I've always had better results on very small TFs (working hard and often for a little) which allow me to be out fast and have a small resting period between trades. I am now working at developing a more relaxed attitude which hadn't been possible before because of the pressure of being limited by the minimum lot size on another account I work with (MT4). I definitely won't risk more than 1,67% per trade, hopefully evolving to much less (which I have found to be the "ideal" max ratio at least for me) so I'm working with a true leverage per trade of 1.66:1 and a total max true leverage of roughly 10:1, having pushed it on very calculated conditions to a 15:1 but that was a tweak of the rules...

I can't see without cringing (and I've seen it many times when trying to trade on longer TFs) a trade that has been +50 go all the way back and forth and again back several times, to finally close at stop. Neither a +20 one! I prefer re-entering once and again, most often at a better price, and take some crumbs. I don't like Oanda's trailing stops behaviour as they are quite different than MT4's, but if they worked exactly like MT4's I would be using them to keep a little bit longer in my trades and heighten the average winners. And I'm willing to give back a whole day of work every ten days but even now the losses are being managed in a way that they do not interfere, or just very slightly, with the daily profit.

My aim is to be able to trade without stress, work a normal schedule, and sleep soundly and build up enough capital so I can progressively reduce the risks to a bare minimum, while also reducing the number of trades to be taken to reach the needed percentage to live off.
  • #11
  • Jun 27, 2009 10:09pm Jun 27, 2009 10:09pm
Quoting doblece
Average winner is around 4 (in a range, 3.4 to 5.5). Maximum allowed loss is about 15-16 pips but in average I try to make it less than 10.
Hi CC,

Your question is somewhat open to interpretation, but in the last analysis, consistent profits, over time, would seem to me to be all that mattered. Consistent profitability might suggest that one had consistency in her methods, but not necessarily....

For me, as a new trader I am discovering that a consistent approach to trading is a necessity if for the sake of discipline alone. Consistency, as an expression of a disciplined approach to trading, is proving (for me) to be the hardest part of the game. But I'm on it.

I mostly scalp (M1 - M5) & I also look for a 4-pip profit minimum on each trade. In fact, I tend to make multiple entries (2-5) if I'm feeling it, with default tp's of 4-12 pips and a default 10-15 pip sl. Stop loss is typically manually triggered before default but it has taken me some practice to leave the sl's in-place, & not move or remove them. Well-placed entries are allowed to run longer-term but profits are taken always on at least some of the defaults.

Mostly when I have failed big time & lost mucho dinero is when I have deviated from this approach. Now, if I can always remember what Mother said to Little Red Riding Hood, "Don't stray from the path!" I may be alright and the Wolf won't get me.

May good luck and consistent gains be with you.
"If The Fool persists in his Folly he will become wise." - William Blake
  • #12
  • Edited 10:35pm Jun 27, 2009 10:11pm | Edited 10:35pm
Quoting doblece
I have always tried to be on the number 2 group... but it seems it really doesn't fit my style. Since I started trading in 2005 I've always had better results on very small TFs (working hard and often for a little) which allow me to be out fast and have a small resting period between trades. I am now working at developing a more relaxed attitude which hadn't been possible before because of the pressure of being limited by the minimum lot size on another account I work with (MT4). I definitely won't risk more than 1,67% per trade, hopefully...
Doblece, Sorry I was away with my family for a dinner. This is interesting discussion. I do not promise anything but my this idea may give you some food for thought.

1. For last ten days, I tried to accumulate few pips winning on balance, sometimes with small positions and othertimes with huge positions but very tight stoplosses. You may like to see my journal "Chicky's trading journal" to check my thought process during this whole thing. I was able to grow one of my accounts to 40% plus. This was to gain margin for a bigger play.

2. As senior traders say PA does not like supports/resistances. It quickly touches and moves away and that is why there is very little price action on those points. Having said that, I looked for where will be those likely support/resistances on a trend line. My intention was to place a huge huge trade on that point putting at stake whatever profits I earned in last 10 days i.e. (40/140 x100) = 28.6% of my account balance.

3. I have put several trades on those points on various pairs(just to spread the risk). Since I did not want to take a chance of missing those points, placed my trades little earlier and therefore expected some drawdowns before PA bounces in my favor after hitting those points. I deliberately put those orders on Friday when all big dogs are out and their touts had control of the market. I have great drawdown for now. However if things go according to my plan I will be able to multiply my winnings. If not, I will lose 40% winnings plus 10% of my original investment. Kindly note that if PA breaks those supports/Resistances it is likely to hang around for a while near those points providing me an opportunity of getting out either at BE or tolerable loss. But no one knows for sure.

4. If this deal doesn't work out, I will repeat, first by collecting my own money by trying to win on balance, then again start accumulating margin for another major play. This is how things will go on for me.

5. If this trade goes in my favor, all winnings will be transfered to my main account that I only use to win on balance. The whole game for me is to maximize proifts by using high risk trades on smaller account and that also using house money for big play and accummulate margin for my main account that follows safer method of winning on balance.
The Thief of Wall Street
  • #13
  • Jun 27, 2009 10:20pm Jun 27, 2009 10:20pm
I take a rolling window of my last ~100 trades with a particular setup, I do a students t-test on the mean profit not being zero, if the z-score exceeds 90% I trade live, if not I paper trade, if a setup does not make it to live for say 6 months, I drop it. I reevaluate the z-score for each setup after every trade.
The breaking of a wave cannot explain the whole sea.
  • #14
  • Edited 11:53pm Jun 27, 2009 11:31pm | Edited 11:53pm
Quoting Craig
I take a rolling window of my last ~100 trades with a particular setup, I do a students t-test on the mean profit not being zero, if the z-score exceeds 90% I trade live, if not I paper trade, if a setup does not make it to live for say 6 months, I drop it. I reevaluate the z-score for each setup after every trade.
I tried unsuccessfully to calculate this, but am at doubt as to what value to place as x in the z-score formula. What would be in this case the "raw score to be standardized"? I've got the population mean (3.65 pips) and the sample mean (4.20) as well as the population standard deviation (11.78); so would I calculate: (4.20-3.65)/11.78 ?? Seems wrong.

As t the student-t test, the same goes for the "z" value:
In pips would I take the average last 100 trades mean (which is 4.2)? Then it would give me (4.2*10)+50=92 (but 92 "what"?)
In percentage profit that would be equivalent to (0.07*10)+50=67 ?
I am still quite confused.

The calculations I made as to standard deviation so to get the Sharpe ratio gave me an average Std Dev of 11.78 and a Sharpe Ratio of 2.94 calculated over the last 6 months pro-rata (the monthly profit as statistical unit), and based on a 15% risk-free yearly return (which is the standard return on a savings accounts here in my country).
  • #15
  • Edited 12:31am Jun 28, 2009 12:15am | Edited 12:31am
I'm not sure on the Sharpe Ratio stuff, that always seems a little tricky to get right, but the formula I use for the t-score (sorry if I confused you with the z-score stuff, I got the terminology wrong) is...

t-score = (average profit * sqrt(number of trades)) / standard deviation of profit.

This formula comes from http://en.wikipedia.org/wiki/Student%27s_t-test under the 'Independent one sample t-test' header, it is also discussed in the book 'The Encyclopedia of Trading Strategies' by Katz. The type of deviation used makes little difference as it is only a factor of one in the denominator, you can then plug the resultant number into a t-test table with the degrees of freedom being the number of trades (for example http://www.microbiologybytes.com/maths/t.html, one tail .1 column).

In practice one does not need the table as the table entries don't change much after about 50 degrees of freedom (or trades), I would just look for the t-score to exceed about 1.28.
I'm not saying all of this is mathematically rigorous, but it does provide a good rule of thumb, I have done Monte-Carlo simulations with the profit numbers to eliminate the Gaussian assumption in the t-test so I could compare the results and they are surprisingly similar, so I just go with the simpler method.
The breaking of a wave cannot explain the whole sea.
  • #16
  • Jun 28, 2009 2:06am Jun 28, 2009 2:06am
Quoting Craig
In practice one does not need the table as the table entries don't change much after about 50 degrees of freedom (or trades), I would just look for the t-score to exceed about 1.28.
Ah now it looks much simpler, thanks !

With those parameters, I get 3.08 so it's probable good... Thanks for the tip!

The Sharpe Ratio is more about optimizing the profitability in regard to the extra risk taken on the asset, in terms of excess return on the asset compared to a risk-free investment (which in this case would be the savings return): more volatility would require a higher return. A value of more than 1 is considered good, 2 is very good and above 3 excellent. In simpler terms, it would mean if the risk is worth the while or not.

http://www.investopedia.com/articles...o.asp?viewed=1

Thanks again for all the new study materials
  • #17
  • Jun 28, 2009 2:13am Jun 28, 2009 2:13am
No problem, and cheers for the link.

I think where I get confused with the Sharpe Ratio is when people start quoting 'Annualized' Sharpe Ratios, there is a discussion on this in 'Quantitative Trading' by Chan where he explains the corrections which need to be made to a raw Sharpe Ratio in order for it to be annualized.
I'll read the whole thing one day
The breaking of a wave cannot explain the whole sea.
  • #18
  • Jun 28, 2009 2:29am Jun 28, 2009 2:29am
Quoting Craig
'Quantitative Trading' by Chan
I knew I'd finally have to go there, lol.
Maybe I'll discover I'm a quant trader too!
I don't think I'll end up as a pure algorithmic (and "automated-based") trader, although when I'm finally able to gather orderly on paper all the micro-rules that have been emerging along all the tweaking process it would probably be interesting to build some EA to see if the predictability I think the strategy has runs equally independently of any time span... but this is for much later on!
  • #19
  • Jun 28, 2009 2:34am Jun 28, 2009 2:34am
I measure it by "smoothness" of my balance curve. If there are alot
of sideways and drawdown phases you're probably not consistent.
  • #20
  • Jun 28, 2009 12:03pm Jun 28, 2009 12:03pm
Quoting Tjpld
I measure it by "smoothness" of my balance curve. If there are alot
of sideways and drawdown phases you're probably not consistent.
Slow and small but steady. What I have not been able to be consistent at yet, is setting a daily schedule. Some days I just can't trade, some others I just don't feel like it (and if I do, that's where I make mistakes and lose...).

In terms of productivity, would a non-trading day count as a "zero" or B/E day? That's why the monthly StdDev is still so high... taken daily only on effective trading days, it's at 1.02 with a whole range between 5.01%/-0.86%, with a 31 to 1 ratio (losses are usually absorbed in the same trading day).

Trading has become really boring to me since a couple of months ago when the whole system "fell in place"... for me it's a good sign (ahh the clichés) but is also an alert signal as I am extremely wary of falling into self-contentment, which by painful experience is the very moment one's foot is about to step into the void... (the correction fall after the spike).

This is why even if I admire and sometime consider taking a bold attitude as Chicky's (willing to risk all winnings for a much greater reward), which would seem to be the only way to go past undercapitalization at a faster pace, I'm not good at it (as my riskier account can testimony of )
I don't like roller-coasters... I try to focus on capital preservation at all times, still not fully satisfied with the risk levels as I would like to reach a maximum floating drawdown of about 3%, and raise the R/R a least to 1:1... that's why I asked before about what is the usual acceptable floating drawdown level? 10-15% seems quite manageable but is a little bit scary sometimes.
0 traders viewing now