Choppy market index: any good ideas? | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

VennD - Any Pairs - Any Style - Any Time 741 replies

Any indicator that tells when Choppy/Ranging starts/ends? 25 replies

Anyone got any good concentration tips/ideas/strategies 7 replies

choppy choppy choppy! 21 replies

Any Ideas on good charts, free too expensive 6 replies

Attachments: Choppy market index: any good ideas?
Exit Attachments

Choppy market index: any good ideas?

  • #1
  • First Post: Nov 5, 2014 10:48am Nov 5, 2014 10:48am
Attached Image

Hi all. I'm sure you looked at the picture above. No doubt you immediately made a clear distinction between the left side and the right side. On the left the market is falling like a stone on the right the price is jumping all over the place: choppy.

There were several attempts here in FF and at other places to build a "choppiness index" indicator. I found no successful results. I wonder if someone has good ideas about building an algorithm which is able to measure how choppy the market is. The motivation for not simply using my naked eyes is that I'd like to screen all the major currencies (28 pairs) on 3 TF. That's 84 charts to monitor 24x5.

Choppiness is usually measured with the ADX indicator. The ADX is designed to measure the strenght of the trend. The picture below shows that a trend can also be choppy. The "trendiness" and the "choppiness" are clearly two independent things. ADX is therefore not at all an indicator of the choppiness of the market. All the indicators or formula I could find make the mistake of considering choppy = ranging.
Attached Image


Some people suggested the use of range bars or renko bars to remove the choppiness altogether. No choppiness = no need for an indi. Brilliant idea but look at the range chart below: constant range bar charts, at any range, can also be choppy!
Attached Image


The more the market is volatile the more it's choppy. ATR measures the volatility. Too bad we just saw that range bar chart can also be choppy yet the ATR is constant on them. Conclusion: ATR is no good either.

I tried to scroll a chart bar by bar to see what I would consider a choppy market. I takes me 3-5 bars to notice a change between choppy/non-choppy. Sometimes I just couldn't decide! So I can't expect a crisp YES/NO indicator more a degree of choppiness. I can also expect up to 5 bars of lag (!).

Drawing a line from the close of the bar to the close 5 bars before you get this: choppy markets look like a webcob and steady markets form a thin ribbon. Perhaps there is something to dig in this direction...
Attached Image (click to enlarge)
Click to Enlarge Name: webcob.png Size: 19 KB
No greed. No fear. Just maths.
  • #2
  • Nov 5, 2014 12:35pm Nov 5, 2014 12:35pm
Some possible approaches I've seen, there are more.

- measure the "energy" of the maximum absolute return at different "frequencies". When trending you'll have a lot of energy in the lower frequencies (weekly, daily, ...).

- measure the coastal length with a standard ruler (length based on TF). When trending the costal length will be smaller, due to less retracing (this is related to the Hurst exponent).

- simulate an ensemble of trending versus ranging models over the recent window. When trending, the trending models (like moving average crossover) will be more profitable. When ranging, the ranging models will make more money (like RSI overbought/oversold trading).

- above + also look at a market cross-section, while taking retail positioning, implied volatility and recent "important" highs/lows into account. This is much harder to do and fit correctly. Machine learning helps.

All of the above require fitting over each individual instrument. This is pretty much against what retailers do, which use "universal" indicators, maybe with a few adjustments (like "period").
  • #3
  • Nov 5, 2014 2:36pm Nov 5, 2014 2:36pm
Quoting PipMeUp
{image} Hi all. I'm sure you looked at the picture above. No doubt you immediately made a clear distinction between the left side and the right side. On the left the market is falling like a stone on the right the price is jumping all over the place: choppy. There were several attempts here in FF and at other places to build a "choppiness index" indicator. I found no successful results. I wonder if someone has good ideas about building an algorithm which is able to measure how choppy the market is. The motivation for not simply using my naked eyes...
hi pick me up,

ive been working on deviations ,from a slightly different view point ,what ive found and this is a not what i was actually looking for is when a collective series of return to price remains constant ,depending on the pip range measured.when that particular range deviates away volatility increases and a choppy range brings that series back to the constant,

my interest lies in the group series of the deviation.using a collective group allows one to capitalize on the efficiency of this market and its corrective behavior to remain efficient.
  • #4
  • Nov 5, 2014 8:56pm Nov 5, 2014 8:56pm
Do you want to PREDICT choppiness (i.e. before it begins)?

Or do you want to MEASURE degree of existing choppiness?
I'm not trying to convince anyone. I'm not in the "convincing" business.
  • #5
  • Nov 5, 2014 9:15pm Nov 5, 2014 9:15pm
Why not set a range limit in pips - say 35 pips, and then count the number of bars (say M15 bars) that stay within that range, where the range is measured as the highest high - lowest low looking back from the current bar.

This would give a number that could be used to compare different times in the market.

A bar count of 6 suggets a short term consolidation whereas a number >20 might show stagnation.

This approach could assist when looking for breakouts.
" check out The Traders Outpost "
  • #6
  • Nov 6, 2014 5:38am Nov 6, 2014 5:38am
@skenobi
I want to MEASURE. I don't think it is even possible to predict. Anyway any attempt at predicting will need the ability to measure first to build the predictive model.

@Adal
Thanks for the ideas. I tried the coast line vs straight line distance. The first problem I encountered is that when the market trends strongly the coast length increases a lot simply because the bars get longer. So I get rid of the trend (by differentiation). The second problem was about the unit of price which may be very different between instruments. This is a problem because I compute the Euclidian distance and the X-step is always one unit. I solved this by normalizing the price with the long term median height of the bars. I'm now facing the problem that the straight line takes a shortcut across a turning point while the coast line follows the peak. Of course this creates a false positive. I'll try to split the running window into two parts: before and after a potential turning point. So far the result is an oscillator with no clear cut threshold. I like the idea of the trading models to find an estimation of this threshold.

@diceman555
Sorry I really don't understand.

@RedLineFred
I'm afraid this will measure if the market is ranging or trending not its choppiness. I've seen two variants of this. The first one estimates how much the bars overlap each other and the second one measures the density of the bars inside the range. Both indexes are mainly driven by the trend.
No greed. No fear. Just maths.
  • #7
  • Nov 6, 2014 5:50am Nov 6, 2014 5:50am
Quoting PipMeUp
I tried the coast line vs straight line distance. The first problem I encountered is that when the market trends strongly the coast length increases a lot simply because the bars get longer.
There's your problem right there. Using bars hides the movement from inside the bars, and skews your result.

Quote
The second problem was about the unit of price which may be very different between instruments.
This is the instrument specific fitting I talked about.
  • #8
  • Nov 6, 2014 6:28am Nov 6, 2014 6:28am
Quoting Adal
{quote} There's your problem right there. Using bars hides the movement from inside the bars, and skews your result. {quote} This is the instrument specific fitting I talked about.
If I use evenly spaced time based sampling then using the bars inside the bar (over sampling) is simply the same as running the indicator on the lower TF with a longer window (in # of samples) and the problem is unchanged. Actually I planned to use it on a range bar chart. So it turns into applying the indicator on a chart of a smaller range with a varying length window. But the problem is still unchanged, is it? Unless the varying length can already be a hint of the choppiness... I'll try that.
No greed. No fear. Just maths.
  • #9
  • Nov 6, 2014 7:09am Nov 6, 2014 7:09am
Number of 10 pip range bars inside the daily bars (orange) versus the coastline/straight line (blue)
Attached Image (click to enlarge)
Click to Enlarge Name: rangecount_vs_coastline.png Size: 26 KB
No greed. No fear. Just maths.
  • #10
  • Nov 6, 2014 8:08am Nov 6, 2014 8:08am
The coast line algo trying to take the turning points into account:
Attached Image (click to enlarge)
Click to Enlarge Name: coastline_turning_point.png Size: 22 KB


The same oversampled using H4 data:
Attached Image (click to enlarge)
Click to Enlarge Name: coastline_oversampling.png Size: 24 KB
No greed. No fear. Just maths.
  • #11
  • Nov 6, 2014 10:31am Nov 6, 2014 10:31am
1.PipMeUp - first, tell us your explanation and differences of ’’choppy vs ranging’’
(I can also try when I get the time...but for now approx. I have my thoughts about it in my mind right now, but differences can be very delicate)

2.diceman555, neither do I understood quite well what you tried to explain.
Please be more descriptive.

3.Adal, if you can you can also give us more descriptive info about your ideas.

Thanks...
  • #12
  • Nov 6, 2014 12:00pm Nov 6, 2014 12:00pm
Quoting skenobi
Do you want to PREDICT choppiness (i.e. before it begins)? Or do you want to MEASURE degree of existing choppiness?
I'm thinking it's probably impossible to predict the amount of choppiness in the future of the market. The only known factor is past performance and there are almost as many unknowns about the future as there are traders. As far as measuring it in the past, It seems to me it would be difficult to find anything that would work better than my tired old eyes. What you are calling choppiness others would call a consolidation after a large move. Many would just draw a horizontal line above and below the consolidation area and trade the breakout of that band.

Perhaps the question to ask is, what is it your eyes see that you interpret as "choppiness"?
  • #13
  • Nov 6, 2014 1:58pm Nov 6, 2014 1:58pm
Quoting Legionary
1.PipMeUp - first, tell us your explanation and differences of ’’choppy vs ranging’’ (I can also try when I get the time...but for now approx. I have my thoughts about it in my mind right now, but differences can be very delicate)
Let me try with a picture. Same initial price. Same final price. Same box.
Attached Image


Quoting Graviton
Perhaps the question to ask is, what is it your eyes see that you interpret as "choppiness"?
It is THE question indeed. Long wicks? well yes but not always and non-choppy also have long wicks. Overlapping bars? Perhaps but steady ranges also have overlap...

@Graviton and @RedLineFred
You propose an application of the consolidation: the breakout of the range box (forexhard SSBO comes in mind of course). But you disregard what in inside the box. Would it bring more information about the probability of a fakey? I'd like to know if when the market is extremely choatic the breakout has a higher chance to be only an overshoot and reintegrate.

Actually I didn't expect to use this to trade a breakout but a swing. My idea is that if the market behaves steadily the price should run directly. In this case an immediate entry is required because the price won't wait for you to jump in the wagon. But if the market is erratic it is smarter to estimate the range and set a limit order. Waiting the price to come at a better price. By doing so you enter at the right price and at the right time: smaller SL. It isn't about catching 5 more pips or increase the leverage like crazy because of a tiny SL. It is that if you half your SL you double your RR.
No greed. No fear. Just maths.
  • #14
  • Nov 6, 2014 2:20pm Nov 6, 2014 2:20pm
Quoting PipMeUp
{quote} Let me try with a picture. Same initial price. Same final price. Same box.
Measure the "time" the price "stays" at each level. In the lower half you'll notice that much more time is "spent" in the "middle". Of course, it's hard to see with bars what really it's going on, since with a simple time shift (phase shift) you could make certain time-series look like either the first or the second (think of a high frequency series which touches the upper/lower half at second :30 versus one which touches it at second :00 in each minute, and a M1 chart).

You could also look into delta-hedging stuff, since the lower half of a picture looks like a pinned option.

http://en.wikipedia.org/wiki/Pin_risk_%28options%29
  • #15
  • Nov 6, 2014 3:27pm Nov 6, 2014 3:27pm
Quoting Adal
Measure the "time" the price "stays" at each level. In the lower half you'll notice that much more time is "spent" in the "middle"
hehe I know. The tick by tick distribution of the price (market profile) becomes very close to a gaussian. That's why I'm pretty sure that in this case it is possible to set a limit order at say the 2nd SD with a SL at the 4th in the direction a trading signal. But when the market moves steadily, it shifts from one bell curve seeking the next one, it won't come back to trigger my order.
No greed. No fear. Just maths.
  • #16
  • Nov 6, 2014 4:18pm Nov 6, 2014 4:18pm
Quoting PipMeUp
{quote} hehe I know. The tick by tick distribution of the price (market profile) becomes very close to a gaussian. That's why I'm pretty sure that in this case it is possible to set a limit order at say the 2nd SD with a SL at the 4th in the direction a trading signal. But when the market moves steadily, it shifts from one bell curve seeking the next one, it won't come back to trigger my order.
It seems you are concerned with execution, not prediction.

In that case you might find it more productive to research execution algos, instead of going at this the round-about way. There are tons of papers on optimal order execution in limit order books.
  • #17
  • Nov 10, 2014 5:01am Nov 10, 2014 5:01am
I'm not especially concerned with execution(1). It is one possible use of this. I have other ideas.

Originally I wanted to avoid adding to a position in such a market because the probability is high that the price retraces right after I have a higher position size. Even if eventually it goes in my favor that's not smart.
--Note: I stopped thinking about adding-in as any way I turn the pyramid idea I always get a lower expectancy--

Another idea is about exiting a losing position. For instance you go long and a few bars later you're in negative territory. If the market is steady (non-choppy) the price is going against you and you shall take the loss. If the market is chaotic (choppy) your position is open above the mean. In this situation the market is gaussian. The mean is also the most likely price. The most likely price is in loss. If the price is currently above the mean your are above expectation. Your loss is smaller than expected: take the loss before it gets worse. If the price is below the mean you can average down and close at the mean: the most likely target.

Another idea is that this is an equilibrum price. This indicator could help spotting S/D zones. This would also improve my S/R indicator which is clearly missing this.
Attached Image (click to enlarge)
Click to Enlarge Name: EURUSD-D1-SR.png Size: 23 KB


-----------
(1) but please post the papers you refer to they may be interesting.
No greed. No fear. Just maths.
  • #18
  • Nov 17, 2014 5:33pm Nov 17, 2014 5:33pm
Quoting PipMeUp
Number of 10 pip range bars inside the daily bars (orange) versus the coastline/straight line (blue) {image}

Well with range bars, say 10 pip. The next bar going up or down 10 pips is a %50 chance. So if one limit number of bars to check back, let say 4 bars. So one can multiply probabilities. So 4 bar up in a row would be (0.5 ^4) would be 0.0625 or %6.25. I think choppiness index could be calculated in a similar fashion, based purely on probabilities. Only for range bars ofcourse..
  • #19
  • Nov 18, 2014 2:35am Nov 18, 2014 2:35am
Quoting GoldTheHun
{quote} Well with range bars, say 10 pip. The next bar going up or down 10 pips is a %50 chance. So if one limit number of bars to check back, let say 4 bars. So one can multiply probabilities. So 4 bar up in a row would be (0.5 ^4) would be 0.0625 or %6.25. I think choppiness index could be calculated in a similar fashion, based purely on probabilities. Only for range bars ofcourse..
Can you develop this idea a little bit please? I like very much the idea of using probabilities but I can't figure out how you see this.

BTW the distribution for 4 range bars is a little bit complex with 3 modes. But after 7 bars it becomes Gaussian which helps a lot with the math.
http://www.forexfactory.com/showthre...65#post7432165
No greed. No fear. Just maths.
  • #20
  • Nov 18, 2014 5:13am Nov 18, 2014 5:13am
Quoting PipMeUp
{quote} Can you develop this idea a little bit please? I like very much the idea of using probabilities but I can't figure out how you see this. BTW the distribution for 4 range bars is a little bit complex with 3 modes. But after 7 bars it becomes Gaussian which helps a lot with the math. http://www.forexfactory.com/showthre...65#post7432165

Exactly it becomes Gaussian, because of the %50 - %50 probability of next bar up or down, just like a coin toss. I think the best way to do it is using monte carlo simulation. This way you can see the probabilities of 7th bar in future (or any other bar number in future) hitting certain levels. If the bars stay within 1 standard deviation (or any other threshold) of expected probabilities and swinging within minus and plus standard deviations you can call this choppy market phase. If it is above 1.96 standard deviation the market could be see as trending phase. Just an idea to get started..
0 traders viewing now