Kalman Scalping

Hello All,

I'm still alive and just reporting in from the Bunker on my latest thoughts that may interest some of you....

Attached are a couple of charts showing a Kalman filter I have just managed to get working. I started this quite a while ago but could never get it working properly.

As my reader(s) will know, I have an aerospace background and have designed several guided missile autopilots in my time using Kalman filters. But for some strange reason, I could never get them to work satisfactorily for the markets.

But then I happenned to be reading the EURUSD thread in the Interactive forum and noticed a few posts by Hindustani (for example http://www.forexfactory.com/showthre...09#post6190809).

(Why is Old Dog reading the Interactive threads you may ask -- he is a life-long swing and position trader on the Daily and Weekly charts! Well.... I have a confession to make. Now I am retired and have far too much time on my hands, I am in serious danger of getting sucked in to the scalping world all the dashing young blades on this forum are so fond of. )

Hindustani's "CCI" indicator looks remarkably like my Kalman and I suspect that there is a lot more to his indicator than just CCI. In fact, he says in one post (http://www.forexfactory.com/showthre...29#post6190929) that he is using an external Matlab setup to do the calculations.

So this got me thinking, and I reckon I might have found the bug in my approach!

This thing is a scalpers dream -- the attached charts from today are quite typical in all respects.

Probably the most important thing to realise about the Kalman filter is that it is NOT A FILTER! It is an estimator for signals buried in lots of noise. Not a bad description of the market on low time frames!! (See my thread here http://www.forexfactory.com/showthread.php?t=336715 on this).

There is absolutely no question of "repainting" or any of those silly issues that plague lots of MT4 indicators and the price-position estimates are quite statistically robust (p < 0.05). I am taking a very simple approach basing the current estimate on the current price pivot, and the uncertainty (noise) on the range (hi - lo) of the bar. But simple seems to work just fine...

I made some very good pips today and am in danger of becoming a day-trading addict if I don't exercise some strict self-discipline!

Like Hindustani, I can't post the indicator as it is too complex. I have a rather convoluted system on my terminal that exports tick data from MT4 via the DDE server into Matlab. Matlab then does all the serious crunching that would be far too difficult in MQ4 language, and then returns the results to MetaTrader.

I would hate for any serious programmer to see my code - I would be far too embarrassed!

But some very fine programmers on the Platform Tech forum have made very similar stuff available if anyone wants to have a go.

I am just offering this out of interest.

Kind regards and good trading to all,

Old Dog

.

So what's the point ? Is this an advertising or what? Trade the way you feel and if you would like to share something with the community - do so. Otherwise, i just don't see a good reason for your posting in this section of the forum. If you wanna sell it, there are other ways.

Offering what?

Old_Dog,

I always like your posts:you are always into something new.Unlike you are(aerospace engineer) I am a porter at the local
dockside carrying rice sacks .Long ago I found kalman and similar filters in Russian forums.Some of these guys are math and coding wizards.
Now I m thinking to put these indicators as input/ouput for echo state network.Just be carefull with kalman when the markets are in very strong trend mood,other then you hit pinpoint entry exit with kalman stuff.All the best.

http://i45.tinypic.com/1zeytjs.png

Hi,

I knew it! Kalman is rocket science!

More seriously I'd like to see your code if you mind posting. I barely understand the Kalman filter and don't know matlab so I hope for comments in the source code. I'd like to understand the model you use (the state vector and the transition matrix).

I also have a stupid question about Kalman in general. Because the model is so important but usually unknown, is it possible to have the transition matrix itself estimated by another Kalman filter ran in parallel of the first one using the output of the 1st filter as a feedback? (one can dream!)

I'm asking because I'd like to apply a Kalman filter on my wavelet decomposition (to get a Wavelet-Kalman filter) and I have no idea of the model to use

Thank you

I was thinking about my trading a lot over the past month, and my most recent setbacks and realized that the last four weeks the market has been mostly ranging (I tend to stick towards yen pairs, hard to not make money there from Nov-early Jan).

I decided what I needed was some way to smooth the data with a Kalman filter perhaps (Electrical engineering here, btw). Find it ironic that you have recently been thinking along those same lines - when I was thinking about reversion strategy I found your posts as well.

I am a fairly accomplished Matlab person, in fact all of my strategy modeling is done in matlab but I have never done the DDE data passing from Metatrader to Matlab....

Do you have some quick insight while I do my own research? The farthest I went has been to write some parameters to a text file and then use a program I wrote in C to call Matlab from a command line.... I was considering piping the data to python and using pandas because I already have an interface, but all of the Kalman-related heavy lifting has already been done in matlab....

Casey is this useful? If not, please ignore.

TheMaxx,

Funny I was browsing that site last night I was mostly interested in his Naive Bayesian Classifier, which is sort of another thing I would like to incorporate, and that is a filter of some sort that can identify a trending vs ranging market.

My idea goes something like this, I have this idea of market "velocity" or price velocity, I call dP/dt, and would look at a couple of different lags maybe for the dt term. I figured though, that in order to do this, some sort of smoothing would need to be done because excess noise could give many false signals.

Like Old Dog says, Kalman filters help to identify the the true signals among noise...

Anyway, just barely in the "incubator" at the moment, I'm going to crank out some code this weekend to see what I can come up with.

Casey

Kalman requires a good model of the underlaying process. What model do you have in mind?
I searched if it is was possible to estimate the model itself at the same time as the filter does its job. I found you can, that's called Dual Extended Kalman Filters but they seem to have a poor convergence.

Also I'd like to know how good is a Kalman filter with non-Gaussian noise because the market noise looks much more like a Cauchy distribution where the variance is infinite. That doesn't help for the covariance matrix...

If you come up with anything special don't be afraid to share

OLD DOG,
What is the POINT with this thread????
just to show 2 charts?????????

Best regards

Dan