Statistical mean of the market [quant corner]

Found a better filter: (FILTER 2.8)

1. 745.71 PROXI SCORE (+90.01)
2. 2.31 RFE (+0.66)
3. 322 TRADES (-75)
4. 17.7% WINRATE (-4.9)
5. 1.53 PROFIT FACTOR (+0.04)

A little bit better ,although i dont like the fact that we had 75 less trades, and also dont like the fact that the winrate is less, meaning that we took longer trades but more accurate ones, i guess 75*spread saved is still a good bargain, although the end profit was a little bit less, but the risk adjusted expectancy was better, thus the PROXI SCORE is higher, even with this handicaps.

But it's far from being good, my research is now at a 1.49142% progress, very slow.

On a price chart it looks like this:



Just ignore the first 500-600 bars, it needs those to stabilize itself, and that part, precisely the first 1000 bars are not counted in the optimization, because that is reserved for the filter to stabilize itself, usually 300 bars are enough, but in this case this one needs a bit more training.

So again, the first 1000 bars are not counted in the optimization, while after that, it can be seen that it signals the reversals very nicely. Although there are still researches to be done, it is still a very primitive moving average, yet way superior to any retail MA that come with retail platforms.

Yeah, thats why im talking to you. You dont buy the bullshit just cause som Ahole blurted it out. I know there are lots of neat math i would like to try out on my charts, but im not a programmer. I AM interested in linguistics, but from a scientific point of view.

I cant tell the bars apart on that picture on account of too far zoom. It would seem the first 1300 bars or so are used to calibrate your purple line there. Im just looking at the purple line and it could be interesting.



I dont really need to be told what to do with it, i have an imagination of my own.

Actually, the first area is March 2008, second is April 2009 and third area is November 2012
The exact days i cant really say.

You know, if we do something but we havent all the knowledge and the skill to explain all the variables, and yet we do it and it sort of works, thats what can be called a form of "art". The artist cant explain exactly why a painting makes a certain impression on the viewer. But if the painting still seem to do something to the viewer, it "works". Take for instance Picassos "The child". Viewable in moderna museet (museum of modern art) in Stockholm in 2010 (dont know if its still there). To me its a very annoying piece of art. Its just an egg made of plastic with an angle cut into it. Very boring. Very upsetting to me that it has got any attention at all. I could make a thing like that in an afternoon if i had some tools. But i still think about it every now and then. I still think about how annoying it is. So maybe in a way Picasso did something that "works". He didnt know about me and he didnt know i would find it annoying even years after i saw it.

Trading is sort of the same, if not done with huge computers and hundreds of PHDs and millions or even billions of $ for research. Maybe we can do that art. Maybe we can. Maybe we can do something that "works", maybe we cant.

I know sometimes i do stuff that works. Sometimes i do stuff that dont work. Maybe we dont have to know WHY it works all the time. Maybe sometimes its enough to know that it works. But do know that we need data for such a hypothesis. When my account goes +, to me all it did was prove a hypothesis or at least add to its credibility. When my account goes -, it disproves or subtracts from my hypothesis. In practice it matters little if i loose money or win money. All i need to do is find a hypothesis that is good enough. Every time i see price reacting my S/R that i put there myself it proves my hypothesis to be sound. The money will come flying automatically after that. Or if i run out of money, its too late and i go do something else.



Maybe eventually my chart will look like so:



So i mean, the money in my account is simply a tool, a variable needed to prove or disprove the hypothesis that i am currently operating. Money in itself dont mean squat. I could have 100.000.000 $, so what if i dont use it for anything? I have a clear idea what to use it for: To build a safe, free world. Maybe it will work. But i know i already started too and i will never surrender. Maybe its not trading that is the way to do it. Or maybe it is. I havent REALLY proved or disproved it as a hypothesis.

Spotting S/R is an artform in itself. I dont have all the variables, i dont know about all the longs and all the shorts and how it all ties into the other pairs, how it ties to gold, to oil, to IMF, to ECB, to the FED, the wars, Syria, Iran, Iraq, Afghanistan, the russians, the chinese and on and on. But i CAN sort of spot S/R: The end result of a EXTREMELY complicated equation going back to the Kennedy assassination, 1945 and Hiroshima, going back to 1932 and the coupe, going back and back and back to stuff i have no idea about.



Just like natural languages (Spanish, English, Swedish, Portugeese for example) can be interpreted this way or that way, numbers are a language that can be interpreted this way or that way. I guess what we are trying to do is to try to understand what they smart people are interpreting in the numbers that they are seeing.

There are much better moving averages out there. But they are written by academics, which you dislike

http://fwd.pics/f/quant_ema_1.png

http://fwd.pics/f/quant_ema_2.png

http://fwd.pics/f/quant_ema_3.png

http://fwd.pics/f/quant_ema_4.png

http://fwd.pics/f/quant_ema_5.png

Did you try those filters? Do they have a recursive formula? How do they compare with a Chebyshev or a Butterworth filter in term of cut off frequencies and phase linearity?

I tried them, and use them when appropriate. The recursive formula is shown in the 4th picture. I didn't compare them with classical DSP filters, since I don't find those useful for trading, but you can easily load the formulas in a math software like Mathematica and do those comparisons yourself.

Also, I don't quite understand how the phase behavior is relevant to trading, since the output of the filter is not used per se as you would with a filtered audio signal for example, and where phase linearity is important since the ear can be sensitive to this.

The 4th picture only shows the convolution formula not a recursive one. Anyway I found the paper by Zumbach. I attach it here for whoever is interested. I didn't finish reading it yet.

FYI1 phase linearity in trading = all the frequencies lag the same number of samples. This is critically important (at least to me).
FYI2 if the filter admits a recursive formula it is rewrittable as a ratio of two polynomials in lag and will be therefore a "classical DSP filter" like any other.

An interesting thing to note is that the paper was written in 2000. And it is about tick-based "indicators" and non-linear time-frames.

14 years later, retail forex still uses bar-based indicators and wall-clock timeframes. I bet that in 2030 retail forex will still work with M5 and H4 candles, fibos and SMAs

So much for retail traders being innovative and thinking outside-the-box (as opposed to academic researchers)

Looks nice, i will try this one out.

But I can tell that my current filter is way superior to any basic filter like ARIMA or AFIRMA models. Mine uses noise calibration and has an automatic noise adjusting feature which will very well approximate the current noise and cut if off.

I really doubt that that particular MA is better than my current model, because it was written in 2000 while my source of "inspiration" is from 2008-2010 noise filtering algorithms academic papers.

I dont dislike academic research, I like it, I just dislike the retail MA's which are the most primitive MA's out there.
Also in my previous comments I criticized the CDO,CDS engineer type PHD's , not the academia, so it's a difference there, I really like academic research.

Precision scales with timeframe, the research I do on H4, can work on tick data aswell, it just needs a parameter recalibration. I think the structure of the timeseries is the same, if you just ignore the element of time and just focus it as a normal series, the inverval of update or the time it takes to get a new element is not important.

What is important, is that by using H4 you actually cut off 4 hours of data inbetween and use only the last tick as a "summary". But for this reason i use MEDIAN price not close price, to account better for the missing ticks.

The priority and the result of each filter is scalable and in the same order as in any TF, it is just that its more accurate in lower timeframe. But by using higher TF i save myself a lot of computing time.

After I found a good filter I`ll just rescale it to the lower TF and it will be even better.

Research shows otherwise. There is very very useful information in the INTERVAL between consecutive ticks.

It contains a lot of information, for example by analyzing this data you can measure the level of informed trading occuring, which can help you decide if you want to follow or to fade the "noise'.

One simple model of the interval:
http://en.wikipedia.org/wiki/Autoreg...ional_duration

Ok but i dont trade at the tick level, this i suppose would be good for HFT hedgefunds. I use the higher candles for filtration purposes, a H4 candle by itself filters out a lot of noise by itself, for example the long consolidations between 5 am and 7 pm or the calm market after 18 pm until the asian session begins.

By incorporation these into the model would distort it's accuracy by introducing useless noise, that 2 hours of noise contains absolute no information for the following period, its only after the move happens you can consider the direction. This is more like following the trend rather than forecasting, but my goal is to spot the trend as early as possible and exit as late as possible while the trend lasts.

The EMA^N operator is simply the EMA of the EMA of ... of the price (applying N times the EMA). In terms of the lag operator the EMA formula is:

This being a time series we can apply the formula again and again on the result. So the recursive formula for EMA^N is simply:

From this it is trivial to expand the powers to get a ratio of polynomials in L. This EMA^N is therefore just a classical DSP filter like any other
Let's compare the EMA^N filter with a SMA of same lag. The plots below compare the EMA^5(alpha=0.5) in blue and the SMA(10) in red:

The frequency responses show that for the approximately same lag
- the lag of the SMA is constant in the passband. This isn't the case for the EMA^N. => filtered signal is distorted with EMA^N
- the cut off frequency is the same as the one of the SMA => same filtering power
- the roll-off is better for the SMA => not so good at rejecting the noise in the transition band
- the rejection in the stop band is much better for the EMA^N => much less residuals oscillations

I also tried the average of the K first EMA^K. The results are very similar. When applied on a chart this average filter is similar to, but smoother than, a SMA of same lag.

When I said classical DSP filter I meant classical in the sense of a filter used in practical application, like Butterworth or Linkwitz–Riley.

I think you are making a mistake in looking in an analytical way at filters, instead of measuring their predictive power. Frequency response and group delay is analyzed for audio filters because they are relevant there. Those measure are less relevant in image processing for example, thus they are not as studied there.

In my mind a DSP classical filter is any filter you can describe in term of the lagged values on its input and output. These filters are used in more fields than just audio. EMA^N is such a filter.

The frequency response is important in image processing too because it determines the kind of noise you're trying to remove. If the group delay is irrelevant in image processing that's only because the kernel of the filter is centered on the pixels. For trading we can't use the values in the future (except for statistical study).

No filter has any predictive power. They just process their input. Some may try to forecast the time series and may update based on the error. They still have no predictive power (and often they overshoot). Only the underlying model they use may have some predictive power. This model could be used with any filter used as a post-processor. I know no deterministic model for the evolution of the market (and I'm open for suggestions). I use a kinematic model simply because I have nothing better. I know this model is wrong since it is trivial to see there is no inertia in the price evolution.

You keep being obtuse and you focus exactly on what I've said instead of what I meant.

You know perfectly well what I meant when I said "measuring their predictive power".

I'm tired of discussing semantics.

As they say, when you only have a hammer (DSP), everything starts looking like a nail.

I don't know why I waste my time with MT4 bar-traders. I should know better.

Perhaps I'm. I just try to be objective. I'm not living inside your head so I cannot know what you mean. I can only focus on what you write.
Just explain yourself better. Write more clearly if what you write isn't what you mean. We only have access to what you write.
That's the point. No I don't. I thought I did. Please explain what you mean when you write "measuring their predictive power". In my understanding they just have none.
Here I understand what you write but I don't get what you mean since I'm not exclusively using DSP. For instance I also use Bayesian inference. To me everything is more probably a nail than a screw but might be both.
I don't use MT4. I'm unable to code in MQL and I'm unwilling to be. I sample the market price with constant range bars which aren't supported by MT4 anyway. So technically I'm also using unevenly distributed time series; just not at the tick level. Simply I only consider the sequentially increasing index of the bars not the timestamp of when they appear. I agree I may be missing something important but the time elapsed between two range bars varies over five orders of magnitude and is nothing near smooth/continuous. I don't see how I can use this information. I only checked that a very short period between two bars is not at all a sign of a sustained price move. Nor is a long period the announcement of a big explosion.

I was speaking in general about this forum, not about you.

Yup, not really good.

I`ve tested all orders of the EMA, from 1 to 6 ( ARIMA(0,6,1) )and the 4,5,6 capped at the same result. (GAMMA & EPSILON & ZETA coefficients both under 0.01 and identical in value suggesting that you dont need more than 3)
Only the ALPHA, BETA & DELTA coefficients were variables, the other 3 was a constant more or less (with slight variations).

So increasing the order count wont make it more accurate in terms of signal giving.

The funny thing is that i can cap the basic EMA's results with only using 1 order, so more than 1 order is really useless, and just wastes CPU power for nothing in gain.




This is how it looks like on the chart (i removed the blue line that is the median price and it hides candlesticks sometimes)



So you can see my VERSION 2.7 of my filter is already better than it with 655 PROXI SCORE, while my current filter 2.8 with 745 PROXI SCORE is way superior to any EMA variation

Of course i work on H4 timeframe, so in other timeframes you could get other results, but i dont think it will differ much as the coefficients could be just identically adjusted if you recalibrate their value for lesser TF's.