Cycle measurement: the easy way | Page 2 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Scalping with quick SL measurement to calculate lot size? 27 replies

Pip measurement indicator 2 replies

Backtest interval measurement 0 replies

Discussions on risk measurement 8 replies

Fibonacci Measurement / Spikes 0 replies

Attachments: Cycle measurement: the easy way
Exit Attachments

Cycle measurement: the easy way

  • #21
  • Apr 1, 2014 11:05am Apr 1, 2014 11:05am
[quote=Proximus;7372210]Here is a more illustrated look about what i`m talking. ]


yes, liquidity can change cycle parameters, I found, too. I found cycle sometime keep going, sometime, change fast, breakdown, not reliable.

I bought the commercial version of the gortzal browser. I also got my modified gortzal browser. I use them on M5 chart, only on eur-usd. I found other pairs the gortzal browser seem not work. Don't know why.

I found MESA is quite good. I use it to give signal. But, all these cycle tools fail in sometime, when market is in chaotic. I attach a chart.
  • #22
  • Apr 1, 2014 11:14am Apr 1, 2014 11:14am
Attached Image (click to enlarge)
Click to Enlarge Name: Gortzel_Browsers_Compared-1-4-2014.jpg Size: 138 KB
  • #23
  • Apr 1, 2014 1:50pm Apr 1, 2014 1:50pm
I really like the result you get Jack. --and it is better than the commercial version under it-- Your output is much smoother than what I get. Yet the lag is kept under control.
Would you mind elaborating on the algorithm you use? With Goertzel alone I didn't get this result.

This is the output of my bandpass for the same period E/U M5. As you can see too much high frequencies go through.
Attached Image (click to enlarge)
Click to Enlarge Name: EURUSD-M5-bandpass-output.png Size: 19 KB
No greed. No fear. Just maths.
  • #24
  • Apr 1, 2014 9:29pm Apr 1, 2014 9:29pm
OK so i looked for the files not sure why i got some corrupted files so i could save an archive of a local stock i tryed it once.

1st it is based on wave analysis, the basics are that a stock in a structure of trend, a cycle, and white noise (which we won't look into because is "useless") so we need to get our data. we need the following info. (Date & Price at least volume sometimes help to give the cycle more meaning).

So column 1 is date, from the latest to the oldest, then the next column you need is just counting down (latest date is 1 then 2 then 3... so on until your last data) the next column is price (price action is not good for the cycle calculation because it has the tendency so we use the return instead because you get the cycles there).

Once you have this, you are set to do the math you need to use Fourier Algorithm / transform (trick here the data you select has to be 2^N) so you cant use a data series with 899 dates, you need 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024 and so on, the Fourier analysis will give you complex numbers which again cant really be used but you need them, finally to get how important the cycle is you need to do another trick (use IM.ABS(X) formula) and finally this number column G on the excel datasheet is the importance of the cycle, filter it high to low and look in the cycle column what is the value that is the full cycle so you need to divide it by 2 to get the usable number in the SMA.

Hope this helps i'm not responsible for the math, i did my best to remember everything if you have more questions i just started a Master in Finance today and will have some classes with the same guy that taught this so i can ask him for more info when i meet him, if i wasn't clear on the explanation just ask me i'll try to help.
Attached File(s)
File Type: xlsx SMA SQM.xlsx   141 KB | 1,027 downloads
  • #25
  • Apr 2, 2014 3:32am Apr 2, 2014 3:32am
Thanks for posting this excel.

Quote
price action is not good for the cycle calculation because it has the tendency so we use the return instead because you get the cycles there
This is the reason why I use a bandpass filter: it removes the trend and the mean price at the same time. I prefer this method because the differentiation (taking the return is a first difference) trasforms the trend (α.t+μ) into a time varying offset (α) and also increases the noise, d(sin(ωt)/dt =ω.cos(ωt : the noise is amplified by the (high) value of its (high) frequency.

Quote
the Fourier analysis will give you complex numbers which again cant really be used but you need them, finally to get how important the cycle is you need to do another trick (use IM.ABS(X) formula)
The complex numbers you get aren't useless. You have a value in the form z=x+iy (column F). You can rewrite them in the form ϱ.exp(iφ). The excel function im.abs returns the modulus which is ϱ=√(x²+y²). This is the magnitude of the cycle. The highest value is the most important cycle. But you can also get the argument φ (phi) --using the arctan(y/x)-- this one tells you the phase. This is where you are in the cycle.

Quote
the basics are that a stock in a structure of trend, a cycle, and white noise
I found a little flaw with the model:
The trend is modelled as α.t.
The cycle is modelled as A.sin(ωt+φ)
The price is then modelled as α.t + A.sin(ωt+φ) + μ. μ is the intercept (a constant to match the prices)

The first order Taylor development of sin(x) is x for x small. Because the sin function is periodic each time the cycle ends (or, up to the sign, is at π), we can write sin(x)~x.

Look: sin(x) ~ x
Multiply both side by a constant K: K.sin(x) ~ K.x
Add a constant Y : K.sin(x) + Y ~ K.x + Y

You can interpret K as a blend of α and A because Y gives a degree of freedom. You can't separate the trend and the cycle uniquely since you have an infinity of possibilities for K=α+A.
=> This model has too many degrees of freedom.
=> In traders terms: you can't make the difference between a trend and the first swing of a cycle.

This is why I decided (by visual inspection of lots of charts) that MY trend was the cycles longer then 40 bars. There is no THE trend only YOUR trend.
No greed. No fear. Just maths.
  • #26
  • Apr 2, 2014 9:17am Apr 2, 2014 9:17am
[quote=PipMeUp;7376754]Would you mind elaborating on the algorithm you use? With Goertzel alone I didn't get this result.

I do filter data (pre-process of price (close), then, feed the data to goertzel. I tried with sma, lwma, momentum, macd, Ethler's super smoother filter, etc.
  • #27
  • Apr 2, 2014 12:37pm Apr 2, 2014 12:37pm
[quote=Jack168;7378651]
Quoting PipMeUp
Would you mind elaborating on the algorithm you use? With Goertzel alone I didn't get this result. I do filter data (pre-process of price (close), then, feed the data to goertzel. I tried with sma, lwma, momentum, macd, Ethler's super smoother filter, etc.
Do you use a bank of Goertzel filters, one for each frequency you're interested in? How do you merge their outputs to get the final results?
No greed. No fear. Just maths.
  • #28
  • Apr 2, 2014 1:48pm Apr 2, 2014 1:48pm
No. only one goertzel.
  • #29
  • Edited 11:28am Apr 3, 2014 11:09am | Edited 11:28am
pip me up,

Price cycle forecast is often hard to do for intraday living trading. But, I heard volatility cycle forecast in intraday may be ok. So, you can follow volatility cycle enter or exit. I haven't do any thing on this topic yet.

In short-term chart, price cycle often change too fast for Gortzerl to keep the cycle steady. But, on daily, weekly, Gortzerl work good. A recent case is cad-jpy. But, other technical tool is used with gortzerl, then, more sucessful, I believe.
Attached Image(s) (click to enlarge)
Click to Enlarge Name: Cad_jpy_daily_bounceUp_29-1-2014.jpg Size: 182 KB
Click to Enlarge Name: Cad_jpy_daily_bounceUp_3-4-2014.jpg Size: 163 KB
  • #30
  • Apr 4, 2014 5:15am Apr 4, 2014 5:15am
I can't see cycles in the volatility except for the very long term variation. In the picture below (CAD/JPY Daily) I show the variance of the price (close) in black and the variance of the return in blue. The variance of the return changes in plateau. It is more or less piecewise constant.
(The second indi is a Hurst exponent estimation using the variation index by Dubovikov)
Attached Image (click to enlarge)
Click to Enlarge Name: CADJPY-D1-variance.png Size: 33 KB
No greed. No fear. Just maths.
  • #31
  • Apr 5, 2014 12:01pm Apr 5, 2014 12:01pm
This is a heatmap made of the output of a Goertzel filter bank. The lighter the stronger. The longer periods are at the top. The bottom is the 2 period cycle (Nyquist). The red line tracks the strongest output. The blue line is the result of my estimator. My method is clearly too reactive to the noise.
I'd like to try and fit a polynomial on the bandpass output before integrating/derivating... I also wonder if a bayesian estimator could be used here.
Attached Image (click to enlarge)
Click to Enlarge Name: AUDUSD-H4-goertzel-heatmap.png Size: 104 KB
No greed. No fear. Just maths.
  • #32
  • Apr 6, 2014 2:59am Apr 6, 2014 2:59am
This John, [email protected], love bandpass, ... etc.

I haven't seen mt4 code of Bayesian estimator. When I got time, I will check it.
  • #33
  • Edited 1:35pm Jun 24, 2014 12:40pm | Edited 1:35pm
Quoting PipMeUp
Thanks for posting this excel. {quote} This is the reason why I use a bandpass filter: it removes the trend and the mean price at the same time. I prefer this method because the differentiation (taking the return is a first difference) trasforms the trend (α.t+μ) into a time varying offset (α) and also increases the noise, d(sin(ωt)/dt =ω.cos(ωt : the noise is amplified by the (high) value of its (high) frequency. {quote} The complex numbers you get aren't useless. You have a value in the form z=x+iy (column F). You...
Hey Pipmeup, sorry for the bump but i had classes with the cycles guy, i told him the issues but he said he had a PhD and the math was complex but that his way of doing it works, that i should trust him (i dont have the tools or knowdledge to proove him wrong ).

Anyway so i started doing the cycle thing, but i want to be able to make a graph of the complex numbers so i can see the waves (kinda for the fun of it i just want to know if i can see something there), do you know of any way to make a graph with complex numbers? excel can do the trick from what i've tryed.

Also do you have a manual or can you teach me how to do the decomposition of cycle / trend with your filters? after that you use fourier again? is there a better thing than fourier?

Another question i have is that if we are making the asumption that the structure is a square wave or not, so far i've seen 3 types of waves in the literature adn how can i "see" if the waves or frequency or amplitud i'm calculating are actually of a square or triangular or a sawtooth wave, because that would probably affect my analisis of the price movement.
  • #34
  • Jun 24, 2014 2:11pm Jun 24, 2014 2:11pm
Quote
Anyway so i started doing the cycle thing, but i want to be able to make a graph of the complex numbers so i can see the waves
Did you try to graph the real part and imaginary part or the magnitude and the argument separately? This is what I do in the second picture of the first post where the blue line is the period (the theta angle of the complex) and the green line is the magnitude.

Quote
Also do you have a manual or can you teach me how to do the decomposition of cycle / trend with your filters?
I bandpass filter the signal this removes the trend (and the noise). The idea was to substract the main wave from the original signal to have the trend (+noise). The problem is that I failed at making it work because of the lag.

Quote
after that you use fourier again? is there a better thing than fourier?
The idea was to not use the fourier transform and all the problems that come with it.

Quote
Another question i have is that if we are making the asumption that the structure is a square wave or not, so far i've seen 3 types of waves in the literature adn how can i "see" if the waves or frequency or amplitud i'm calculating are actually of a square or triangular or a sawtooth wave, because that would probably affect my analisis of the price movement.
I don't understand the question. The waves I'm looking at are sine wave (sinusoids).
No greed. No fear. Just maths.
  • #35
  • Jul 19, 2014 7:21am Jul 19, 2014 7:21am
Quoting PipMeUp
I bandpass filter the signal this removes the trend (and the noise). The idea was to substract the main wave from the original signal to have the trend (+noise). The problem is that I failed at making it work because of the lag.
PipMeUp,

Did you find a solution to this? If not, here are a couple of things that you might want to check out.

 

  1. Instantaneous Trendline, by John Ehlers. He has worked out a solution to removing the dominant cycle (and higher frequencies) from price data, leaving the trend. Google it for the articles and presentation slides written by Ehlers. There's an implementation of the Instantaneous Trendline in AllAverages_v3.1.mq4 by Igorad.
  2. Empirical Mode Decomposition (EMD). This is a really interesting alternative to Fourier methods for extracting cyclic signals that are not stationary, i.e. period and amplitude may vary over the analysis interval. After a complete EMD is done, the residual is the trend with all cyclic content removed (lag-free). EMD is noncausal (in trader-speak, that means it repaints), but there are still numerous uses for something like this. Check out "Hilbert-Huang Transform" on Wikipedia. There's an article on EMD at mql5.com, complete with code.

Sure I have a trading plan. Buy low. Sell high. What's yours?
  • #36
  • Jul 19, 2014 2:15pm Jul 19, 2014 2:15pm
I went this -unsuccessful- path because Fourier, MESA, EMD, Wavelets, Hilbert transform, Goertzel... all fail because of the lag.

FYI Ehlers stopped considering the Instantaneous Trendline and all the related stuff based on his homodyne filter years ago (2007 I think) because it fails.
In post #7 I wrote that my filter lags 4 bars whereas Ehlers' lags 21 bars. Yet it is already too much.

EMD and wavelet fail because they need data in the future. Several papers discuss about some "solutions" but all of them introduce a bias.
No greed. No fear. Just maths.
  • #37
  • Jul 20, 2014 1:18pm Jul 20, 2014 1:18pm
Have you read the latest book from Ehlers?

Got some stuff on this, but need to re-read the book as I dont remember completely
  • #38
  • Jul 20, 2014 3:24pm Jul 20, 2014 3:24pm
Ehlers is not an authority on signal processing.There are some very advanced methods definitely worth investigating: like particle filters , Particle Markov chain Monte Carlo methods etc.. http://www.math.u-bordeaux1.fr/~pdelmora/simulinks.html this is the software https://alea.bordeaux.inria.fr/biips/doku.php
  • #39
  • Jul 20, 2014 3:44pm Jul 20, 2014 3:44pm
'' 1) RenTec is not predicting the price – wouldn’t be prudent at this juncture. These days, they do not even predict the prediction of the price. No, taking the idea to its obvious limit, they predict the prediction of prediction of prediction… (take transfinite limit of that if you dare. Go to the first unreachable ordinal, even) of the price.
2) But the key is to do it efficiently. So they conduct so called “interviews”. Oh, they conduct them continuously, year after year, interviewing tens of thousands of people… Only they never hire anybody. Has anybody asked Himself “wherefore”? The answer is inescapable, for, during the interview (after dispensing with obvious nonsense, like, “why do you want to work for RenTec? Is it all about money for you?”) they ask only one question: “What do you think is RenTec’s secret”?
3) Oh, the innocent trading tools of Wall Street! They turn off all their filters, analogous and digital, they disregard Newton’s warning, and begin to feign…
4) It’s impossible for a tiny, fragile human being – no matter how many Ivy League Ph.D.s she holds – to listen to this drivel for long. So here is where the speech recognition algos come handy. But here is the important part of their “”secret””: everybody in the speech recognition community are trying to filter out some meaning out of a speech. Not RenTecs. They are after a dual object, a co-meaning, what Bohr (who, with his concept of complementarity, anticipated the algo) used to call “crazy enough idea to be true (spoiler: it wasn’t)”. In layman’s terms, RenTec is trying to extract complete and utter nonsense out of a massive amount of speculation on their “”secret””.
Have You ever tried to produce complete and utter nonsense (I mean, besides the original post)? Have You noticed how hard it is? But how could it be if all there is, as classical thermodynamics would tell you, the state of the highest entropy, to which your system would run without any effort on your part, anyway? The relationship between meaning and co-meaning are subtle; they are quantum, their hidden Markovian models anti-commute. Co-meaning produced by a large crowd is subject to spin-statistics (I am no expert, but I think it’s called Bozo Stats). The Red Queen got it right: perfect nonsense is rare. The best way to get it would be to listen to a whisper of a Hawking radiation… alas, RenTec doesn’t have it’s own black hole (yet; why do you think they keep pouring money on all those colliders?) Still, RenTec has found the next best way to mine for it.

5) So they filter the co-meaning out, see math details below. I suggest nobody in RenTec could listen to the final result, except Simons. Yes, that’s the terrible price he has to pay for all his wealth… They make him listen, and then they interview him! If he says something like: “I didn’t quite get what was that guy saying about the Laglands Program, but I think the idea to consider zeta functions on an algebraic field with minus one element sounds interesting. Did he say it might be used to model the pricing of the swaps, if only the price continuum was compact?” – then the result is referred to as “low-grade yellowcake” and returned back to the algos for further refining.
But, if he says: “All I got was: Pots of money, pots, pots!” – then it’s perfect. Prediction-to-the-omega-power has been achieved.

5 (again?)) The two main problems with the spectral DSP are:
(a) Modern methods try to “exctract” “signal”. We need to “extract” nonsense.
(b) Speech recognition completely ignores the dance often performed by a speaker alongside with his presentation. To the very least, speaker’s gesticulation and the amount of spitting should somehow enter the model.
(c) (but you said two problems? Oh well. I can get used to that) The nonsense extraction can not be performed in real time, mostly, because it is a delicate process, but trades create too much quantum interference by talking between themselves.

(6) For solutions of (5b): RenTec hired von Frisch, who, as everybody knows, got his Nobel for translating honeybee dance into speech. The rest follows.

(7) For the solution of (5a), first, they realized, that they must replace classical Fourier analysis with harmonics living on tori and such by its non-commutative version. They experimented for a while with locally compact groups, until in the late nineties they had an insight and realized that they should use Hopf algebras (that is, quantum groups) instead. I mean, in retrospect it looks so obvious, and, basically, all stems from duality – you are not trying to “capture” meaning, you are trying to “repel” it, so, in essence, you are studying the inverse scattering problem. For a layman, to extract the harmonics all that they must do, essentially, is to solve a huge Yang-Baxter equation. But I digressed.

(8) Fore the solution “of” (5c), they quickly realized that the computational difficulties are insurmountable, no matter how big a computational cluster you build. But, much like with the Coleman-Mandula theorem, there was a way around it. Turing machines are so yesterday! So they build the best quantum computer in the industry. And not just any quantum computer, but one with an Oracle! From the cold hearts of Wall Street traders, and powerful brains of academic researchers (smooth like the sound of music) they build it, they did. Cold obviously helps with quantum coherence (why do you think they do not ever hire, do not even let Wall Street people on their campus? Disentanglement, man!), and the brains, as Penrose has shown in the early nineties already, perform quantum computations in their teeny, tiny microtubulai.. And the Oracle, you ask? Do you even have to ask? See (5), I mean, the first (5).

(9) Since the markets gone totally random (Fama, rejoice!), all this has become nearly useless without a special trick not described in books… but the margins of this forum are too narrow to contain it. Sapienti sat.'' http://www.nuclearphynance.com/Show%...1&PageIndex=34
  • #40
  • Jul 21, 2014 6:46am Jul 21, 2014 6:46am
Quoting PipMeUp
I went this -unsuccessful- path because Fourier, MESA, EMD, Wavelets, Hilbert transform, Goertzel... all fail because of the lag. FYI Ehlers stopped considering the Instantaneous Trendline and all the related stuff based on his homodyne filter years ago (2007 I think) because it fails. In post #7 I wrote that my filter lags 4 bars whereas Ehlers' lags 21 bars. Yet it is already too much. EMD and wavelet fail because they need data in the future. Several papers discuss about some "solutions" but all of them introduce a bias.
When you say "lag", do you mean phase shift? Or do you mean that the result is not instantaneous? As you pointed out in post #25, you can't tell the difference between the onset of a new trend and the first leg of a cycle. Thus a certain amount of delay in discriminating the two is inescapable.

In your first post you mentioned that your goal was to null out the cycle and be left with the trend. So when you say a lag of 4 bars is too much, it sounds like you are talking about a systematic phase shift. If the approach is to generate a sinewave equivalent to the cycle and subtract it from the data, that can readily be done without any phase shift. But you would have to wait for enough of the wave to appear to be sure it was a cycle. Additionally, your period estimate would have to be extremely accurate, or the inaccuracy of the period will swamp the effects of phase shift and lag relatively quickly.

I use a lag-free LPF to detrend data as a preparatory step to cycle detection. But data points at the far right edge of the chart are necessarily only approximations. In my work, the error this introduces is insignificant. The cycles extracted in this way are free of phase shift. But you are not going to see instant results, it takes a few turns of the cycle at least to get reasonable estimates of period, amplitude and phase.

Notch filtering is an alternate approach to removing the dominant cycle, but of course it is also going to incur a lag, for the reasons already discussed.

Another point is that the SNR of market data is low at the best of times. If your method cannot stand up to 4 bars of lag, it sounds like it would also be overly sensitive to noise in the data. Yes, EMD, Fourier, MESA, Wavelets, etc. are not perfect. I think the real quest is not to find the perfect analysis technique, but to work out ways of using the existing techniques on imperfect, noisy, real-world data in a robust way that gives you a positive expectancy when used for trading.
Sure I have a trading plan. Buy low. Sell high. What's yours?
0 traders viewing now