Optimized Trend Trading

correct sign 52.58
most valuable input CLOSE and lag(1)

correct sign 62%
inputs chosen equally 1,4,5,6 so EMA, HP, HP 2 pole and Butter

correct sign 100%
most valuable inputs are momentum and lag values

Conclusions

Correct sign varied between 51.6% and 62%, its very low. Variation depend from number of bars and its clear impact of Law of Large Number ==> Predictive power of this setup will be very low because even for in sample period was not possible to get high correct sign value. It was confirmed by out of sample period test.

Valuable inputs were chosen sometimes filters and sometimes as Lag/Momentum. There is no clear advantage of using
Filters. For reference signal sin + cos filters were not chosen at all !!!

Fisherized price was never chosen !!!

Does it mean than DSP way is not a "way" but more price action way is a "way" ??

Comments are welcome

Krzysztof

I am sorry for the delayed response. I have not been monitoring this thread for some time. I don't recall having Igorads nlma in AFL, but here is the afl code for the ULLMA as well as almost every filter known to man. (You must change the extension to afl. it is not a zip file, but FF requires certain extensions). Just apply ULLMA1(price) to one of the HA functions. If you have questions, PM me. This is not an afl forum.


MadCow

A lot of good work, but it will take a while to understand what it means.

I have wondered the same thing myself, but is price action really different than DSP, or are we just using it in a different way? What the heck is price action anyway... something like TRO's stuff?
I hope not
MadCow

thnks

...even with all the drama and speculation, this thread generates and stimulates the thought process. There is something here for all of us, the weak and the strong, the dim and the not so'. No, I am not CB.

Hi, I just joined and read this entire thread, great stuff. I was wondering if a long-term member would be so kind as to post a brief summary re which approaches look most promising so far in this realm (application of DSP to forex), and which approaches have been discredited. I've done a fair amount of work with John Ehlers and other DSP indicators on markets other than forex (equity, bond), and have a few profitable EA's for those, but now looking to diversify into forex, which I know very little about. Just want to get myself oriented so that I can hopefully contribute some research...

I've followed most of the thread in a sitting (admittedly I didn't finish all yet) with some interest. After being dazzled by the opening, then cruising forward a bit, it appears that the majority of whizbang conclusions have not revealed too many new ideas under the sun.

I'm hoping some old timers can elaborate and/or confirm the following
1) The ultimate system using goertzel or latest is no better/worse than fft?
2) Since most familiar with fft know that it relies on periodicity (which markets aren't), the assumption is that the some windowed (and detrended?) time period fft will have approximate spectral tone characteristics that have a continuous nature for a small period out of sample? for example, if the dominant tone is 14,5,2 days, I can with high reliability expect that behavior to continue for a very short time frame and bet accordingly.
3) There is a consensus that all truly smoothed methods are non-causal (i.e. rely on future unseen data for smoothness).
4) Since we can't practically see into the future, the goal is to sort of take the true (up to right edge) data and extrapolate estimates based on ~ifft/dft of some arbitrarily optimal sampled window using fft to determine significant dominant tones?
5) The most optimal (and complex) filter derived on the entire thread can be reduced to a simple fir weighted filter (including jurik) only difference being that the weights are chosen differently than typical TA mA filters.
5a) Given that the best possible filter from this thread reduces to a simple FIR with only the coefficients being unknown, the coefficients will vary over time in some optimal adaptive method?
6) I have a pretty broad knowledge of filters, but what exactly is a UULMA filter and HA? Are those standard filters or custom named (I know DSP, but don't recall finding those terms)?

I know I'm likely dumbing down the superior complexity of the thread, but for the IQ 80s in the crowd, can someone confirm...

Would the above assumptions be correct?
Thanks.

ULLMA = ultra low lag MA. In post 699, asm8086 posts his ULLMA which appears to be simlar if not better than CodeBreakers??? And he references posts #526 & #527 re the original design and info. And in post 705 he provides the complete source code.

In addition to jamjamjam's questions, I was also hoping someone could comment on the effectiveness of other well-documented DSP algos (such as John Ehlers') compared to the findings in this thread. Ehlers for example has explored and dismissed Goertzel and FFT, see various papers and powerpoints on his website.

And finally, an assessment of techniques discussed elsewhere on FF 9e.g. trend following with low lag MAs), compared to the DSP approach?

I believe HA = Heiken-Ashi, see here for example

Ah, thank you Charley. It looks like the UULMA, is a combination of ifft spectral weighted inf series of sines and cosines, with some interpolation? I wonder if this was home brewed, or there is a paper on the exact algorithm.

I had thought that the coefficients of these (thread) optimized filters are simply linear real numbers, no? For example, on the reverse engineered example (guess) of the Jurik system, he showed a set of linearly summed real coefficients only.
Can anyone confirm if he reverse engineered Jurik as linear weighted FIR only? Or was it some form of ifft transformation as shown on the UULMA algo charley linked.

I also looked over the jurik page, and from their description it sounds more like a kalman type filter (as they mention it came from radar tracking applications). Not quite the same as linear weighted FIR, as it has AR arguments, for one.

Any more confirmation is appreciated.

CB's ULLMA filter is a sinc filter with some other preprocessing applied. He suggests it uses some HA filtering first. But no one really knows.

asm8086's ullma filter uses HA with a sinc filter and gives and extremely clean signal. I consider the ullma a solved problem. How to use it on the other hand....

Hi Flyboy,

So is it safe to say that it is (or can be replicated by) a linear weighted moving average filter with only real number coefficients (and does not use forward looking data)? The reason I ask is because there might have been a more interesting approach to finding the optimal coefficients.

Not so much to find the best smoothed fit, but perhaps the most profitable.

Sounds like the ultimate jury was a thumbs down, however.

BTW, I don't know how many people follow climatology, but if you want to see some really astute debates on this subject, I suggest looking around those forums.

I don't follow climatology at all, but my ears instantly perk up when you mention it in this forum.

We're dealing with almost exactly the same problems. Predicting the (short term) future of a non-linear dynamic system. No doubt they have some techniques that would be useful.

please post a nice and choppy range to really make us drool...or not?

In truth, not so much. As a background, climate models are not the same as weather models. The first concern timescales of decades to centuries and concentrate on overall physics, the latter concentrate on, of course weather, which has predictability timescales (inverse Lyapunov exponent in some sense) of a couple of weeks. Principal differences are that what weather models take as observed inputs (e.g. surface sea temperatures) are the object of study in climate---for climate you really have to respect boundary conditions and energy conservation laws, for weather you need geographically and temporally accurate immediate forecasts.

Anyway, it's been studied for quite a while----any purely statistical approach for forecasting weather (in modern jargon, 'data mining') is completely inadequate. All weather and climate models now rely on implementing physical laws of nature, i.e. the right parameterizations and approximations of fluid mechanics and thermodynamics.

In finance, it's not clear any such fundamental laws exist.

And back to nonlinear dynamics---the interesting methods rely on low-dimensionality of chaotic dynamics vs high-dimensionality of stochastic processes. Capital markets don't show low dimensionality.

anyways,

I just discovered this thread, the first one I've seen with some serious science discussion here---fantastic! It's what I've been missing.

My background: I worked as a research physicist specializing in nonlinear dynamics, in particular analysis of observed time series. After years of academic life, my day job is now more machine learning/software development for a significant predictive analytics company in commercial banking (not capital markets).

I'm also not so hot on some kinds of DSP-like methods for financial prediction. Often those things are designed to detect or track sinusoidalish signals or some kind of frequency lines in contaminated backgrounds. That's because the underlying problems for which they are developed, e.g. radar and sonar, have physics which make that a good idea.

Anything band-limited tends to have, in effect, exponential tail decay in impulse response. I think financial time series are robustly multifractal, in that a wide variety of timescales must always be taken into account.

Anyway, I offer the attached indicators, developed in isolation.
One is mostly conventional, a Hull EMA with period varying based on a local signal-to-noise ratio. The other is raw material but based on power law impulse responses.

I'd love to see if any of these can be developed into something insightful or profitable in collaboration with the combined membership here.

(corrected typo in original HullAMA.mq4 attachment)

have you got any mail to contact you?can you send me your mail in a private message,i want to prupose something...

mbkennel,

I tend to agree with a lot of your assessments, and find your background very interesting. Regarding Climatology, I was particularly referring to forums like the following, whereby some professor got promptly ripped on for publishing some 'holy grail' of smoothing http://climateaudit.org/2009/07/03/t...-linear-trend/ The debate wasn't centered too much around physical/thermodynamics input based models, but a math based smoothing model, much like the ssa type modeling mentioned somewhere back on the earlier part of this thread.

The debate centered around the endpoints, which as you know, have created a lot of heated debates over interpretation; and much like financial time series, that is the part that is most difficult and simultaneously, most important to model.

Nevertheless, I'll be the first to admit I'm probably the only poster that doesn't have metaquotes, so hopefully I'll be able to add something useful regarding your code. As limited as my understanding is, I looked over your code briefly, and as I interpret it, you are essentially conditioning some of the HULL parameters adaptively as a function of a local SNR value. Would that be correct? I think that's a good potential idea, if so. I think I get the SNR derivation, but am thinking about it a bit more.

Is it correct to say your def is signal = finalvalue(nf)-initvalue(no)
over noise =cumsum(abs(changes(nf,no))).
Seems like an odd way to measure SNR. Not sure whether it is good or bad per se, but just seems unusual, IMO. I would think noise should be expressed as rmse value of the residual (sans signal) changes, no? Signal is usually extracted as the signal with the dominant power component of the spectrum. That is the common signal processing approach at least. Your approach, If I'm understanding, sounds a bit like fractal type interpretation (like hurst, D, or R/S range type measurement). I don't know, there are a few ways to express this, but let me know if I'm on the right page first.

Did you come up with these adaptive equations on your own, or is there some reference to where they are shown elsewhere (for instance, using hyperbolic tangent as part of the scaling param).


Also, could you show a long period plot (with say the SNR period =9) of what this might look like, superimposed over a non SNR adaptive hull? And if you are conditioning on SNR, then how does that account for the fact that SNR can only be derived over a lookback period? Do you expect strong autocorrelation in SNR?

Also, I like the idea about multifractals requiring multiple time frames to express, as it was mentioned earlier in this thread, but I'm not quite sure how you incorporate that concept here.
Thanks,
Jam

P.S. Regarding the power weighted kernel function, could you write out a simple explanation of what that is doing and why it's useful? thanks.

P.S.S. As of yet, no one has expressed whether or not any of these functions can be reduced or expressed as a simple linear weighted coefficient FIR.