Trying to calculate the probability of an indicator being correct

After reading evidence-based technical analysis, about half is still not clear. imho a pretty bad book, the author repeats obvious nonsense like "a dog has 4 legs, but something with 4 legs doesn't have to be a dog!" atleast 20 times and takes 400-500 pages trying to explain something that can be explained in 100< pages.

If I have an indicator and want to know if the signals it's spitting out are not random, I could assume there's 2 possibilities: false signal and correct signal. So in 64 different instances where the indicator signals we assume SD = sqrt(64*0,5*0,5) = 4. So if we have 44 correct signals, we can assume that there's a 99,7% chance that the indicator is not spitting out random signals.

But how do I calculate the probability of the indicator being, say, right 80% of the time? My assumption is SD = sqrt(64*0,8*0,2) = 3,2. So to have a 99,7% chance of being correct 80% of the time I need 51,2+9,6 = 60,8 or 61 correct signals out of 64. Is this correct?

Hello Pipsgeek,

If i understand your question here´s the answer:
Test your indicador for a period of time.
Then, in that period of time you have for example 42 bad entries and 74 good entries.

42+74=116

x= 42*100/116 ( 36% bad signals )

x=74*100/116 ( 63% good signals )

I hope i have helped.

Happiness for you.

That's just the percentage of the current sample.

My question is how to calculate the standard deviations assuming different signal accuracy. So how much chance I have of the signal being 80% accurate, 70% accurate, 60% accurate, etc. I already know how to calculate the chance of the signal being nonrandom.

Predictive Modeling for Traders 101: The Basics

There is a difference between the Probability, and the Density (Distribution) of the Probability.

You need to invoke Time. 80% accurate over what time interval? 70% accurate over what time interval? 60% accurate over what time interval? You also need make sure you are comparing apples to apples, or pips to pips. That can't be done until you define what a "Good Signal" and "Bad Signal" look like within the predictive model. This means that you have to define Minimum Target Level per trade, and the Maximum Loss Level per trade.

You then need to set the Range of the model equal to 100%, in terms of Total Number of Possible Occurrences. That means, putting a cap on the range, so that you can later ask the question: "Within the next N number of trades, X are projected to be "Good," while Y are projected to be "Bad." The basic definition of probability is as follows:

P(Event) = TFO/TPO

Where, TFO = Total Number of Favorable Outcomes
Where, TPO = Total Number of Possible Outcomes

Note that your probability will have to reside between 0 and 1, or between 0% and 100% (0 <= P <= 1).

You also have to define the outputs you want to track. In trading, most people think about the two standard outputs of Successful Trade, and Unsuccessful Trade. However, Breakeven Trade could also be an outcome worth tracking, but that depends on you. At a minimum, you will have two (2) possible outcome classifications:

- Good
- Bad

Using a model with a range of 100 trades, if your trading outcomes are producing 70 Good Trades and 30 Bad Trades, then you already know how to calculate the Probabilities for both outcomes: P(Good), and P(Bad). But, you are asking a different question.

You are asking about the Distribution of the Probability, then you want to go inside that distribution and figure out the Timing of the next High Density Probability Occurrence. This is not basic statistical analysis, and you can build the model in a number of different ways. There is a lot to explain, but I'm not going to get into all of it. I'll try to cut to the chase - the rest you will have to research on your own.

In the above example, P(Good) = 70% over the previous 100 trades (range). You can approach this a few different ways:

- You can simply calculate the STDEV (standard deviation) between each "Good" outcome as a function of Time, and then subtract the resultant Time value from the current position (assuming the current position is always the "Now" point), to derive the projected arrival Time of the next Good outcome. As Timing is one of the more important functions that traders care about.

- You can go a little deeper by looking at the actual distribution of % values for both Good and Bad outcomes within the model's range and then calculate the Time based STDEV between the Good outcomes, the Time based STDEV between the Bad outcomes, the Magnitude based ABS (absolute value) between the Good outcomes, as well as the Magnitude based ABS (absolute value) between the Bad outcomes.

You then:

Subtract the Time based STDEV of all Good outcomes from the current time, to derive an ETA for the "next Good."

Subtract the Time based STDEV of all Bad outcomes from the current time, to derive an ETA for the "next Bad."

You then create the simple algorithm:

if(TimeGood < TimeBad), Time_Good
if(TimeBad < TimeGood), Time_Bad

You then develop the same for Magnitude:

Calculate the average Magnitude based ABS value from the Good outcomes.
Calculate the average Magnitude based ABS value from the Bad outcomes.

You then create the simple algorithm:

If(AbsMagGood > AbsMagBad), Magnitude_Good
If(AbsMagBad > AbsMagGood), Magnitude_Bad

You now have two predictors (Time and Magnitude) for both Good and Bad outcomes that you can plug into any Triggering function.
-----------------------------------

Now, the difficult part in explaining this is not knowing how precisely you want to track things. What I would suggest, is that you further define trades by the number of pips they generate. For example:

- Trades that generate a minimum of 5 pips
- Trades that generate a minimum of 10 pips
- Trades that generate a minimum of 15 pips
- Trades that generate a minimum of 20 pips
- Trades that generate a minimum of 25 pips
- Trades that generate a minimum of 30 pips
- Trades that generate a minimum of 35 pips
- Trades that generate a minimum of 40 pips
etc...
etc...
etc...

Out to as many pips as the trade signal historically produces. Now, you can do some rather interesting things. Now, you can be a lot more precise with your probability calculations and your density calculation precision goes up dramatically.

What you will most likely find, is that that probability for your trade signal being good for 5 pips, will be the highest, while the probability of that same trade signal being good for 40 pips, will be the lowest.

This allows you to do three (3) very important things as a trader:

A) It allows you to define the sweet spot of your trading system. No more guess work about the health of your trading signals. You will know what each one is capable of producing at any given time.

B) This will give you the ability to create a Range of Probabilities to a Specified Target. You'll have more confidence when the market initially goes against your signal, and you will have more empirical evidence to holding a position when emotion tells you to get out.

C) It helps you to better understand what the parameters of your Money Management Strategy should be. Now, you can fine tune the MM strategy and bring risk closer to the reality of what your trading actually produces.

Hope this helps.

Thanks for the detailed explanation.

However, it seems you explained how to build a system that tries to predict the "next move" by finding a connection between previous moves and time. Furthermore, you tell me to calculate "the standard deviation" while my exact question here was how to calculate that.

So far I haven't yet been into the profitability of certain signals, more on defining momentum and trend so that I'm sure that when I get a trend signal that this is truly indicative of the future trend, rather then being a 50/50 shot.

Example of an hypothesis:
Does SMA crossover indicate a future trend?
Valid Signal:
SMA crosses over in the direction of trend (Fast > Slow from bottom = Uptrend signal)
A new peak (in the case of an uptrend) is formed after a correction. The signal is invalid when only a new peak is formed without a correction, when a correction is formed without a peak following (including horizontal ranging) or when the correction exceeds the fib of .786 (calculated from the previous peak).

Time is not a part of the signal validation, mainly because any sort of ranging or correction without a new peak following is automatically considered invalid. As such, when next valid signal appears is rather unimportant. What's important is wheter I can consider an MA crossover an indication of future trend when it does appear. I can only go back so far in the charts, and if I have 100 crossovers with 20 of them being invalid then I have an 80% success over this sample. Of course this does not mean every MA crossover will be 80% accurate. What I want to know is how to calculate the standard deviation from this so that I can say (for instance): "Based on this sample I can say that the MA crossover has 99.7% (3 SDs from the mean) chance of being indicative of a future trend 60% of the time.".

Interesting article for finding the probability of an indicator: http://articles.mql4.com/628

You did not specify WHICH "standard" deviation you wanted to calculate. Since this site is about trading, I assume you wanted a probability distribution, or something approximating the same.

Simply take the square root of all variances (deltas) from the "mean" in your data set, and you get the "standard" deviation. Excel, makes this task exceedingly simple. However, that "standard" by itself, may or may not be helpful to you.

I simply took things a few steps further for you, so that you could see WHY having such data available might be helpful to "Traders," as opposed to merely being a fancy bit of information for the statistician purest.


Not until you first find and then define a "future trend" that is repeatable and you can't do that successfully, until you first define your profit requirement per trade, and you can't do that successfully, until you first define what your needs are relative to capital growth, and you probably won't be able to accomplish that task until you have clearly defined your real reason for wanting to be a Trader.

Everything in this business always comes back to the WHY question as that will drive every piece of your research. If you don't know why you are a Trader, then you can't possibly know what to study, test, analyze or optimize. You cannot optimize a model that does not yet exist.

This is a lesson that too many simply never get around to figuring out. Why calculating a "standard" deviation is important to you as a Trader, has everything to do with why you are in this business - believe it or not.


You have just defined what is called a "Pattern." That's the core element of all technical trading. You've also defined Indicator logic. The second step to developing a technical indicator. Every component of this logical statement needs to be verified across Time in order for the trader to have empirical justification for relying upon such an Indicator.

Time is the ultimate validation tool for the creation of any Indicator and the empirical evaluation of any Indicator logic. Timing, is the FIRST most important element to all good trading. Remove Timing from the equation, and you yank the core foundation for being able to calculate revenue growth over period - which is the very meaning and intent to trading in the first place.

Trying to separate Timing from any aspect of trading, is like trying to separate hydrogen from oxygen in H2O. I can be done, but you end up with something far different in the end.


This is somewhat of a contradictory statement. Think about it. At what point can the trader extract the meaning of an Indicator, from the expected results of said Indicator? In other words, can the trend continue without the MA cross?

At some point, all MAs must "cross" in the direction of a trend, depending on its duration. Else, all markets would move in one direction only and no market would be efficient as all pricing would become infinite. Long before that, all markets would shut down even before peak liquidity was reached. So, all trends, therefore, must come to an end through reversal. And, reversal must be accompanied by MA cross over.

Therefore, at some point, some MA cross over becomes an essential component to a trend that continues. If the "validity" of the "signal" is defined as continuance of the trend (which is it), then it cannot also be dually defined as unimportant to the trader (assuming that trading is the point of all this).


You have not yet defined what trend continuation means for you! How much does the trend need to continue AFTER an MA cross over? How many pips are defined as a successful trend continuation? What's the maximum amount of time that can elapse AFTER an MA cross over and BEFORE the trend reverses, in order to define it as a successful trend continuation?

You don't have your terms defined yet, and therefore you don't have any framework for creating such a probability statement. The "standard" deviation only applies to those events where an MA cross over resulted in a failure to continue the trend. But, since you have not yet given the trend any structural definition at all, you have no basis for making such a calculation.

If the MA crosses over and yields 45 pips in 2.5 hrs, was that a successful example of trend continuation? Or, if the MA crosses over and yields 97 pips in 41 minutes, would that be considered successful? Neither are considered successful under the following conditions:

1) The maximum allowable time in any position is 60 minutes.
2) The minimum pips required for any position is 115.

Both MA crosses lead to a continuation of the trend. But, neither event would be components of the mean number of "accurate" results. Therefore, they would become components of the "standard" deviation away from that which is defined as "accurate." But, until you define what "accurate" looks like, the deviation cannot be known.

You have to think this through a little bit more. Hope this helps a bit.

"If you don't know why you are a Trader, then you can't possibly know what to study, test, analyze or optimize."

In addition to the intelligent explanation of mathematics..... I found this quote to be quite interesting.

Agreed.

Hello Pipsgeek,

Here is your major problem. How to define the signal is "correct"? Additionally it may have degrees of correctness. Not only two possibilities... You could write your own 500 page book on this topic. Because like iDouble says you cannot evaluate an indicator outside of the context of its use.

I read a book on the "acuracy" of the candlestick patterns. Pattern X has 63% reversal score. Great. Just the author doesn't explain how he performed his benchmark! I'm not gonna advertise this book.

Let me take your example:

Here you are not evaluating the SMA crossover indicator but the use a given trader may make of this tool. Quite different.

You are evaluating his trading system, not the signal. --And you don't take his exits strategies into account (at least one for exiting a losing position and one to TP).--


Side notes:

That's not non-sense but logic (math version of it). Lots of people can't set in their mind that "A implies B" is by no mean equivalent to "B implies A". It only implies "not B implies not A".
"If the trend changes then the MA crossover" does NOT imply "If the MA crossover then the trend changes"!

No. MAs are causal filters. They only react to the facts. They cannot predict anything. When the MA points upwards it means the price has been going up for some time; not even that it is currently still doing so.
A MA is to be seen as a proxy for some higher timeframe. This is exactly the idea of the MA crossover: having the price, some higher TF (the fast MA) and some even higher TF (the slow MA) now altogether going in the same direction.

I don't know for your evaluation but for an entry the timing is the most important factor. Certainly more than the signal itself. Timing is the difference between a 50 pip SL and and 15 pip one.

100 pages to explain the LONG and a 100 page copy-n-paste to explain the mirror SHORT version. Usually poorly explained twice. I see. +1

Best