Kalman filter and currencies strength | Page 2 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

EAs: Spread filter VS News filter 3 replies

Kalman filter 12 replies

Kalman Scalping 10 replies

Extended Kalman Filters 0 replies

Kalman Filter - New Moving Average 0 replies

Attachments: Kalman filter and currencies strength
Exit Attachments

Kalman filter and currencies strength

  • #21
  • Jun 18, 2018 10:49am Jun 18, 2018 10:49am
Quoting PipMeUp
The market return is not Normally distributed but somewhere between Cauchy and Student distributed. Kalman filter performances degrade quickly for non-Normal innovations. That's why I use another Kalman filter which estimates the error of the first one and I feedback this information into the first model as a modulation of the allowed variance of the state. A kind of dynamic cursor "lag <-> smoothness". This first model is a second order polynomial local estimation. I chose a 2nd order polynomial because it can approximate (Taylor) any smooth function...

Are you talking about ARMA processes which generate the price movements? I tried before to get a time series estimation for daily data on GBPUSD, partial autocorrelation showed significant coefficient only for Y(t-1) (partial autocorrelation of Y and Y(t-1)), so the price movement can be basically described as non-stationary AR(1) process with error e , or in other words random walk with drift.
  • #22
  • Jun 18, 2018 1:03pm Jun 18, 2018 1:03pm
Quoting PipMeUp
Now I have an estimate of the most likely trend. It is important to keep in mind that it is a likelihood not a certainty! I can use this information to define if EUR/USD is most probably up or down. OK I hear you, I can also look at the chart and see if it is raising or falling :-). You are right. But the idea I had is to repeat this with each of the majors, not only to know if they are up or down but how much they are up or down. If GBP/USD is going down it means USD is stronger than Pound. If EUR/USD is down as well EUR is weaker than USD too....
"For this reason I use the percentage of variation. By dividing a price difference by a price you get a unit-less value. This way I get rid of the unit/scale and I can compare EUR/NZD and USD/JPY directly."

Nice idea. That alone should improve any currency strength indicators one would think. Correlation indicators as well. Maybe even moreso? no?
  • #23
  • Jun 19, 2018 2:05am Jun 19, 2018 2:05am
Quoting synicz
Nice thread! Have you tried toying around with other Machine Learning algorithms? I have noticed the widespread use of xgboost and gbm within the machine learning community. Edit: I have also read elsewhere on using the outputs from ARIMA as a feature as well. Yet to have tried it thou so Im not sure bout the effectiveness of that.
I coded a few ML algo myself mainly to learn and understand what they do and how they do it. My finding is that ML is to be used when you already know what you're looking for not the other way around. I mean that data mining won't work for market data which sample size is ludicrious small for things like neuron network. Not even talking about deep nets. It can only overfit fluke relationships.

I don't use packaged ML software because they are usually not open and pluggable. You can't see what is inside and even less modify their internal state. If I didn't code the kalman filter my self I couldn't have added my little trick to take colored noise into account without modelling this noise in the input matrices.
No greed. No fear. Just maths.
  • #24
  • Jun 19, 2018 2:16am Jun 19, 2018 2:16am
Quoting available
Are you talking about ARMA processes which generate the price movements?
I'm talking about ARMA processes which describe the price movements. Like a MA follows the trend but doesn't generate it.

Quoting available
I tried before to get a time series estimation for daily data on GBPUSD, partial autocorrelation showed significant coefficient only for Y(t-1) (partial autocorrelation of Y and Y(t-1)), so the price movement can be basically described as non-stationary AR(1) process with error e , or in other words random walk with drift.
Raw price series are indeed very autocorrelated while their return are (almost?) not. Anyway any time series can be modelled as a RW with drift. Estimating this drift from market data is very hard.
No greed. No fear. Just maths.
  • #25
  • Jun 19, 2018 2:38am Jun 19, 2018 2:38am
Quoting PipMeUp
{quote} I coded a few ML algo myself mainly to learn and understand what they do and how they do it. My finding is that ML is to be used when you already know what you're looking for not the other way around. I mean that data mining won't work for market data which sample size is ludicrious small for things like neuron network. Not even talking about deep nets. It can only overfit fluke relationships. I don't use packaged ML software because they are usually not open and pluggable. You can't see what is inside and even less modify their internal...
Fair point about the inherent 'black box' characteristic of ML algorithms.
So you are saying that you mostly do manual adjustment of parameters using techniques such as kalman filter? Hows it coming along so far?

I've been toying around with ML algorithms had some small but statistically insignificant results. Which seems to be a huge problem as nearly all positive results are always statistically insignificant!
I look for value wherever it can be found
  • #26
  • Jun 19, 2018 4:51am Jun 19, 2018 4:51am
Quoting synicz
So you are saying that you mostly do manual adjustment of parameters using techniques such as kalman filter? Hows it coming along so far?
No. I mean that KF is meant for iid white noise with known noise matrices. What if you need estimating these matrices from the data while filtering? The blackbox software usually don't allow that easily.
Market data is corrupted with colored noise. Because of the autocorrelation of the noise a trend filter oscillates because it follows the swings. The usual way around this is to model the noise in the matrices (GPS data processing). All the modelling I found is based on ARMA. But ARMA can't fit the residulas well... What can you do? With blackbox software you can't easily insert a correction between the predict and the update step that uses the current state. That's what I do. It stabilizes the filter (a bit).
No greed. No fear. Just maths.
  • #27
  • Jun 19, 2018 5:25am Jun 19, 2018 5:25am
Quoting PipMeUp
{quote} No. I mean that KF is meant for iid white noise with known noise matrices. What if you need estimating these matrices from the data while filtering? The blackbox software usually don't allow that easily. Market data is corrupted with colored noise. Because of the autocorrelation of the noise a trend filter oscillates because it follows the swings. The usual way around this is to model the noise in the matrices (GPS data processing). All the modelling I found is based on ARMA. But ARMA can't fit the residulas well... What can you do? With...
Apologies im kind of lost in all that terminology.

What i do is i create lags of the data and fees that as a feature into the machine learning algorithm. Along with any other features which i feel may be important.
Similar to an earlier point you made, these algorithms are like black boxes and it would be near impossible learn about the relationship amongst variables...

Well, guess its back to the drawing board!
I look for value wherever it can be found
  • #28
  • Jun 19, 2018 9:29am Jun 19, 2018 9:29am
That's what people usually do. You throw a bunch of features that you expect may contain some useful information and you hope the NN will find it. You add some loss function, often a few steps ahead prediction, and here we go. After few minutes/hours the algo spits out a model. But given the dataset is very small, highly correlated and very noisy, what you get is a neural net that learnt by heart its input (if deep enough), or one that found a set of ad hoc rules only working for this dataset, or a model that says the price is 1.0000 +/- a lot of noise. This depends on the amount of regularization you've set. The more features you add the worse it gets because of the curse of dimensionality.
No greed. No fear. Just maths.
  • #29
  • Jun 19, 2018 10:19am Jun 19, 2018 10:19am
Quoting PipMeUp
{quote} No. I mean that KF is meant for iid white noise with known noise matrices. What if you need estimating these matrices from the data while filtering?
What are sequential monte carlo smoothing techniques (rao blackwellized etc..) like compare to kalman filters for noise filtering, do you know?
  • #30
  • Jun 20, 2018 7:57am Jun 20, 2018 7:57am
Do you mean particle filters? With them I can see two challenges. First the MCMC takes a lot of iterations to converge. The real-time processing is compromised. Second they often generate multimodal distributions: you would see a MA forking and merging again. Can be fun to code a strategy based on that.
No greed. No fear. Just maths.
  • #31
  • Oct 24, 2023 11:45am Oct 24, 2023 11:45am
Quoting PipMeUp
Open question: if there is a cycle in the graph, does it makes sense to basket trade the envolved currencies in a ring?
Hey Pip, amazing work

I currently stumble upon this research of you and tbh i quite interesting in making the same system. Can you please tell me if this idea still work till this day ?. Because i check your other post and you said that you don't use this anymore 2015 to be correct in which this discussion is in 2013
  • #32
  • Oct 25, 2023 2:22am Oct 25, 2023 2:22am
There are a few problems that I found.

Determining if a currency is stronger than another is the same question as knowing if their cross is trending up or down. The measurement will be some kind of moving average. It lags. Choosing the strongest vs weakest pair is essentially equivalent to a MA crossover strategy with all the known problems.

The more you find that a currency is strong the longer it has been so and therefore is about to stop being strong... This means you don't want to trade the strongest currency but the one which is strengthening the most. This is the curvature of the moving average. Not only it lags but it is noisier.

If you succeed in finding a way to overcome these two difficulties you are not yet done! Say two currencies A and B are strong and two are weak (X and Y). You get 4 possible pairs. A/X, A/Y, B/X and B/Y. You rank them and determine that A/X is the one to buy. The next bar the result will be A/Y. Then A/X again, then B/X, then A/X and so on. The first ranked keeps changing all the time. Your strategy will miss signals.
No greed. No fear. Just maths.
  • #33
  • Last Post: Nov 6, 2023 9:11am Nov 6, 2023 9:11am
Quoting PipMeUp
There are a few problems that I found. Determining if a currency is stronger than another is the same question as knowing if their cross is trending up or down. The measurement will be some kind of moving average. It lags. Choosing the strongest vs weakest pair is essentially equivalent to a MA crossover strategy with all the known problems. The more you find that a currency is strong the longer it has been so and therefore is about to stop being strong... This means you don't want to trade the strongest currency but the one which is strengthening...
Thanks for replying.
Have you look into this thread. i think he use the same method that u said in your post:
https://www.forexfactory.com/thread/...erage-distance

Btw, have you tried look into the Premium Discount Array of Currency Pair for it. Just to estimate the strength of the currency pair and it's distance with the EQ ?
0 traders viewing now