Choppy market index: any good ideas? | Page 2 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

VennD - Any Pairs - Any Style - Any Time 741 replies

Any indicator that tells when Choppy/Ranging starts/ends? 25 replies

Anyone got any good concentration tips/ideas/strategies 7 replies

choppy choppy choppy! 21 replies

Any Ideas on good charts, free too expensive 6 replies

Attachments: Choppy market index: any good ideas?
Exit Attachments

Choppy market index: any good ideas?

  • #21
  • Nov 18, 2014 7:22am Nov 18, 2014 7:22am
I would like to clarify that the 50%-50% is not the reason for the convergence in distribution to a Gaussian (Central Limit Theorem is). Even with a 70%-30% biased coin it would converge; except that the mean would not be zero. The time bars (like H1) also have 50-50% probability of next bar up or down yet the distribution of the N bar return never converges to a Gaussian.

Quote
I think the best way to do it is using monte carlo simulation. This way you can see the probabilities of 7th bar in future (or any other bar number in future) hitting certain levels
There is no need for MC simulation with Gaussian you can calculate in closed form. This is what I meant by "[Gaussian] helps a lot with the math"

Quote
If the bars stay within 1 standard deviation (or any other threshold) of expected probabilities and swinging within minus and plus standard deviations you can call this choppy market phase. If it is above 1.96 standard deviation the market could be see as trending phase. Just an idea to get started..
I'm afraid this will only classify between "ranging" and "trending". As I wrote above these are different concepts than "steadiness" vs "choppiness". But it is also true that the more chaotic the more the movement of the price will behave like a brownian motion (aka gaussian random walk). The RW is bad at visiting its space therefore if the price visits a lot of levels it is probably not choppy... Detrending the price and estimating a distribution with 7 samples only is impossible. Adal is right I'll have to sample at the tick to have a chance.
No greed. No fear. Just maths.
  • #22
  • Nov 18, 2014 9:31am Nov 18, 2014 9:31am
Quoting PipMeUp
I would like to clarify that the 50%-50% is not the reason for the convergence in distribution to a Gaussian (Central Limit Theorem is). Even with a 70%-30% biased coin it would converge; except that the mean would not be zero. The time bars (like H1) also have 50-50% probability of next bar up or down yet the distribution of the N bar return never converges to a Gaussian. {quote} There is no need for MC simulation with Gaussian you can calculate in closed form. This is what I meant by "[Gaussian] helps a lot with the math" {quote} I'm afraid this...
Totally agree that %50 - %50 is not the reason for Gaussian distribution, I just quoted that specifically for the range bar situation, bound by 10 pip up down range. For normal time bars, Sqrt(Number of bars) * ATR(some large value) could be used. It gets very close to Gaussian. But I don't know if it would solve the chopiness measurement problem..
  • #23
  • Nov 18, 2014 1:17pm Nov 18, 2014 1:17pm
Maybe there is something to be done with the distributions. The picture below shows an example of white noise and the histogram of the absolute value of the innovations on the left. On the right, a sine wave and the same histogram. The modes are on the opposite sides of the histogram. A Kullback–Leibler distance could be used to classify one or the other. The drawback is that the method, if it works, requires quite a lot of samples. Here 1000 were used.
Attached Image (click to enlarge)
Click to Enlarge Name: return_pdf.png Size: 9 KB
No greed. No fear. Just maths.
  • #24
  • Nov 23, 2014 4:34pm Nov 23, 2014 4:34pm
Arcsine distribution is interesting: https://www.stat.berkeley.edu/~aldou.../arc-sine.html
  • #25
  • Feb 10, 2016 6:29pm Feb 10, 2016 6:29pm
Not sure how to articulate this so have some good will here. Kinda thinking out loud, so to speak. In a steady market, steady up or down, you "tend" to have a series of higher highs and higher lows for longs and lower highs and lower lows for shorts. But, in a "choppy" market, as I'm thinking of it, that pattern is not present. You tend to have one bar with a top higher than the previous top, then the next bar is lower than the previous top and vice versa for the bottoms. It's erratic and not steady. In that vain, I believe one could create a statistical measure of "choppiness" by it's deviation from a steady state as the context. If in a typical trend of say 20 bars you have 15 bars that behave (higher highs, higher lows, lower lows , lower highs, and 5 erratic bars you might have a baseline. The ratio might give an indication of the choppiness of the current market. In 20 bars, for example, how many conform, and how many violate. I'd love to see something useful on this topic too. I know I chimed in late
  • #26
  • Feb 11, 2016 3:43am Feb 11, 2016 3:43am
Thanks for posting PapaMack. The immediate counter-example I think about the HHHL idea is a powerful directional move. It isn't choppy at all and has no alternating highs and lows. I've tried various variants of this idea of comparing the market with a "steady market". I also tried Mercurio's LAVE method (http://edoc.hu-berlin.de/dissertatio...F/Mercurio.pdf) to estimate the volatility. Without any success. The common point they share is that the choppiness is quickly detected but its effect last too long after in the running average. This "excited state" remains after the market is back in a nice looking flow. Not only it lags but it lags with a non constant delay which "leaks" in the future measurement. As a result the market is seen as always choppy. Hurst exponent and comparision of probability distributions need way too much samples to be usable. They implicitely assume ergodicity. Unfortunately this never holds true: even in the smallest valid window there are several episodes alternating choppiness and steady state.

Note that I'd be happy with a method that classifies even long after the facts as long as they do so with high probability of correctly classifying the periods of choppiness. We could try to have a neural network trained on it.

The only thing I noticed is that the volatility and the momentum are linked. The market is not correctly modelled by "a straight line + independant noise". The market can't be at the same time strongly moving and very choppy. It is as if some "energy" was distributed to either the momentum or the volatility. I suppose it is because the volume of trading is limited.
No greed. No fear. Just maths.
  • #27
  • Edited 6:11am Feb 11, 2016 5:58am | Edited 6:11am
I like the below 4 market states idea.But the samples are too small and I dont know how to determine the percentage of each leg of zigzag ( HHHL ,LLLH etc.. ).What would be the ideal percent value of each zigzag leg? I am not sure if it is codeable with zig zag with one of the pattern recognation algorithms(HMM etc..)
Attached Image
  • #28
  • Feb 11, 2016 6:56am Feb 11, 2016 6:56am
This is exactly the kind of classification I'd love to get. I would also add a fifth "crappy jerk price action" state where no zig-zag is apparent.
No greed. No fear. Just maths.
  • #29
  • Feb 13, 2016 10:10pm Feb 13, 2016 10:10pm
Quoting PipMeUp
This is exactly the kind of classification I'd love to get. I would also add a fifth "crappy jerk price action" state where no zig-zag is apparent.
Have you looked at the rate at which new range bars are printed e.g. the distribution of time duration of each range bar?
Also have you considered the joint PDF of the quote/tick volume and the time of the the range bar.

I put forward these ideas under the untested hypothesis that they differ during ranging and trending periods.
  • #30
  • Feb 15, 2016 5:02am Feb 15, 2016 5:02am
I couldn't compare the distributions when the market is ranging/trending/choppy because I really lack a good estimator for these states. What I can tell is that overall the duration (in sec) of a range bar is distributed like a Possion and the number of ticks in a bar is more or less log-normal. Both unfortunately consistent with random noise. I also tried to find a relation between the duration of a bar knowing the duration of the previous one. Same for the ticks or duration vs ticks... I found nothing. Except of course that the bars print more slowly in Asian sessions.
No greed. No fear. Just maths.
  • #31
  • Edited 3:58pm Feb 15, 2016 3:15pm | Edited 3:58pm
Quoting PipMeUp
This is exactly the kind of classification I'd love to get. I would also add a fifth "crappy jerk price action" state where no zig-zag is apparent.
good for you, keep trying.
  • #32
  • Feb 16, 2016 7:16am Feb 16, 2016 7:16am
I focused on the estimation of the variance because this is the usual definition of volatility (or standard deviation to get the same unit as the price). But I realize that's not enough. Here are two series I hand made. On purpose the blue one looks like a text book range. The red series is made of the same returns but randomly shuffled. They look very different but their returns are the same. Per construction their empirical distributions are the same therefore their volatilities are the same. The red one is choppier than the blue despite they are drawn from the same distribution. The distribution is not enough to describe their movement. Hurst exponent comes in mind again but so many samples are required to estimate it...
Attached Image (click to enlarge)
Click to Enlarge Name: same_variance.png Size: 16 KB
No greed. No fear. Just maths.
  • #33
  • Feb 16, 2016 8:18am Feb 16, 2016 8:18am
Maybe you could set up a fibonacci type grid, or Gann/Murrey type grid so that you could measure the moves over a given period of time. The grid would have to be based on the volatility of the pair over a given amount of time.

If market theory is to be believed, the stronger the trend, the less choppy the market. So maybe you could set a percentage value for a move, and then normalize it. It would be kind of like detrending a market when you are trying to measure cycles.
If you are good at something, never do it for free--Joker
  • #34
  • Feb 17, 2016 6:41am Feb 17, 2016 6:41am
Quoting PipMeUp
....
The red one is choppier than the blue despite they are drawn from the same distribution. The distribution is not enough to describe their movement. Hurst exponent comes in mind again but so many samples are required to estimate it... {image}
half life speed of O-U process?
Nec spe nec metu
  • #35
  • Feb 18, 2016 4:38am Feb 18, 2016 4:38am
I discarded Ornstein–Uhlenbeck process fitting for two reasons. The first is that no mean reversion was found in high frequency data by academics, quant oriented people in this forum and myself. Second I think the market choppiness periods change too quickly to have enough samples in the case it would indeed be mean reverting.
No greed. No fear. Just maths.
  • #36
  • Mar 3, 2016 11:49am Mar 3, 2016 11:49am
I dont know if it helps but for me a good way to see consolidation is to use point and figure and to set the reversal on 1. Thats it, sometimes you will see a consolidation last longer ( not according to timeframe but the number of reversal bars) and sometimes it doesnt last long( the shortest consolidation is 3 bars reversal in a row ).
Have a look at this chart
Attached Image (click to enlarge)
Click to Enlarge Name: pnf2.png Size: 142 KB
So all you have to do is count the reversal blocks. 2= trend reversal. 3 = shortest possible consolidation etc..
of course it is just for a visual look and not to predict.
  • #37
  • Oct 10, 2017 12:57pm Oct 10, 2017 12:57pm
Quoting PipMeUp
I discarded Ornstein–Uhlenbeck process fitting for two reasons. The first is that no mean reversion was found in high frequency data by academics, quant oriented people in this forum and myself. Second I think the market choppiness periods change too quickly to have enough samples in the case it would indeed be mean reverting.
Are you still interested in this problem? Or you already found solution?

Some of the tools (DFG) which I use in my trading are very good for detecting ranging markets. And clearly show the difference between range and trend.

You can use MA clusters to detect if there is a stable range.
Use simple MA with Fibonacci ratios. (21,34,55,89,144,233)
Measure the gap between the highest ma and the lowest ma.

For example this is my definition for range on M1,M5,M15... chart:

If ( highest_ma < lowest_ma + 10 pips && lowest_ma > highest_ma - 10 pips) range=true;

You can use additional measurements to determine if the market is active (moving) or not.
I use tick volume and ATR.

Here you can see some examples. The first 3 pics show choppy range, the last shows trend.
Attached Image (click to enlarge)
Click to Enlarge Name: R1.png Size: 74 KB

Attached Image (click to enlarge)
Click to Enlarge Name: R2.png Size: 87 KB

Attached Image (click to enlarge)
Click to Enlarge Name: R3.png Size: 85 KB

Attached Image (click to enlarge)
Click to Enlarge Name: T1.png Size: 95 KB
  • #38
  • Oct 10, 2017 2:05pm Oct 10, 2017 2:05pm
Yes I'm still interested in a quantitative estimator of choppiness. Perhaps also in finding a better word for this because people always associate choppy and ranging market as being the same thing. I make a clear distinction between these two facets of the PA. A ranging market can be steady while a trend can be choppy.

Admittedly the examples are far from perfect but they may give an idea.
Attached Image (click to enlarge)
Click to Enlarge Name: cases.png Size: 242 KB
No greed. No fear. Just maths.
  • #39
  • Oct 10, 2017 2:23pm Oct 10, 2017 2:23pm
Quoting PipMeUp
Yes I'm still interested in a quantitative estimator of choppiness. Perhaps also in finding a better word for this because people always associate choppy and ranging market as being the same thing. I make a clear distinction between these two facets of the PA. A ranging market can be steady while a trend can be choppy. Admittedly the examples are far from perfect but they may give an idea. {image}
I'm thinking perhaps a ZigZag based indicator that plots the percentage of swing overlap based on n swings (eg 100% is perfectly sideways and choppy) and the linear regression slope using the swing points as inputs.
  • #40
  • Oct 10, 2017 2:38pm Oct 10, 2017 2:38pm
Quoting PipMeUp
Yes I'm still interested in a quantitative estimator of choppiness. Perhaps also in finding a better word for this because people always associate choppy and ranging market as being the same thing. I make a clear distinction between these two facets of the PA. A ranging market can be steady while a trend can be choppy. Admittedly the examples are far from perfect but they may give an idea. {image}
Maybe we can get some useful ideas from the concept of Surface roughness. There are international standards for the calculation. I'm' pretty sure we can apply the formulas directly to market prices.

You should also check the works of Benoit Mandelbrot. As you probably know he is the "father" of fractals. He's done extensive studies on market prices to develop some of his theories.
0 traders viewing now