Synthetic hedges, cointegration, mean reversion and similar stuff | Page 8 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Simple Mean Reversion 621 replies

Pairs Trading: Reversion to the Mean 22 replies

Forward test of my new mean reversion strategy 11 replies

Question for synthetic and non-synthetic currency pairs 0 replies

Cointegration, Synthetic hedges, mean reversion in R, Tech Thread 0 replies

Attachments: Synthetic hedges, cointegration, mean reversion and similar stuff
Exit Attachments

Synthetic hedges, cointegration, mean reversion and similar stuff

  • #141
  • Nov 19, 2010 1:28pm Nov 19, 2010 1:28pm
Quoting 7bit
No, it should work as usual. Are you getting any errors? (Don't post the MT4 crash log! (If anything then post the DebugView output on level 2 and only after you have read (and followed) the instructions that are always written at the top of all my mq4 and mqh files). Please post only legitimate errors, errors in my code, errors that do not go away after following the installation instructions. I don't write the detailed instructions into the files only to repeat them again and again and again in the...
Yup, I had already worked through all of this with your price forecasting indicator, so knew what to change.

I added the EA to another chart then MT4 crashed with no error messages, just closed. When I opened MT4 again, it simply closed with no message, again and again. I deleted trendomat.mq4 & .ex4 file to stop the crashing, then opened MT4 again, removed all EA's from charts, put trend-o-mat on then arb-o-mat and all worked fine.

JC
  • #142
  • Nov 19, 2010 1:38pm Nov 19, 2010 1:38pm
Quoting 7bit
Any (un)successful trades or interesting insights, anybody today?
If you're using micro lot account, you need to change code

if (AUDUSD) append("AUDUSDm");
if (EURUSD) append("EURUSDm");
if (GBPUSD) append("GBPUSDm");
if (NZDUSD) append("NZDUSDm");
if (USDCAD) append("USDCADm");
if (USDCHF) append("USDCHFm");
if (USDJPY) append("USDJPYm");

Not very insightful, but worth mensioning.

JC
  • #143
  • Nov 19, 2010 1:49pm Nov 19, 2010 1:49pm
Quoting Waka
If you're using micro lot account, you need to change code

if (AUDUSD) append("AUDUSDm");
if (EURUSD) append("EURUSDm");
if (GBPUSD) append("GBPUSDm");
if (NZDUSD) append("NZDUSDm");
if (USDCAD) append("USDCADm");
if (USDCHF) append("USDCHFm");
if (USDJPY) append("USDJPYm");
Yes, I developed it mainly to work with my br0ker and did not care about all possible eventualities. It's still only a prototype for experimenting and the user must know what is going on and how to adapt it if needed. Its not considered anywhere near release quality.
  • #144
  • Nov 19, 2010 3:10pm Nov 19, 2010 3:10pm
Quoting 7bit
Yes, I developed it mainly to work with my br0ker and did not care about all possible eventualities. It's still only a prototype for experimenting and the user must know what is going on and how to adapt it if needed. Its not considered anywhere near release quality.
Sorry, wasn't saying this as a criticism. You have given so much for us to chew on, I would never imply any quality issues. Just adding more info for those who will inevitably come across this.

JC
  • #145
  • Edited 3:45pm Nov 19, 2010 3:35pm | Edited 3:45pm
Can any coder here be kind enough to modify my ratio indicator to do the following and share it with us?:

Please see attached pic on the basic rule of mean reversion.
i.e. When the ratio indicator touches upper or lower boundary of bollinger bands and reverts to the mean 20MA of bbands, let this be ONE count.

I want the indicator to include a count number(C1&C2) over user input N1&N2 no. of days(total range) in any T,attached time frame chart.

Then do a plot of the ratio C1/C2

e.g, Let user input,N1=last 5days, T=5mins, C1=count value
Let user input,N2=last 25days, T=5mins, C2=count value

If C1/C2 is rising, synthetic CoIntegration is increasing...
Attached Image
Attached File(s)
File Type: mq4 instruments_ratio.mq4   2 KB | 837 downloads
  • #146
  • Nov 20, 2010 3:11am Nov 20, 2010 3:11am
7bit, do you normalize price data?
  • #147
  • Edited 9:47am Nov 20, 2010 9:20am | Edited 9:47am
Quoting vitali_yv
7bit, do you normalize price data?
A quick look into the code tells more than 1000 words, I tried to make it as simple and straightforward as possible. (back is the numbers of bars to look back and now is either 0 or some number of bars to exclude from the regression to allow easier visual backtesting):

Inserted Code
   [color=#2f9956]// fill the matrix of regressors[/color]
   [color=#2f9956]// and then copy it over to R[/color]
   [color=#7f0055][b]for[/b][/color] [color=#000000]([/color]i[color=#000000]=[/color][color=#000000]0[/color][color=#000000];[/color] i[color=#000000]<[/color]back[color=#000000];[/color] i[color=#000000]++){[/color]
      [color=#7f0055][b]for[/b][/color] [color=#000000]([/color]j[color=#000000]=[/color][color=#000000]0[/color][color=#000000];[/color] j[color=#000000]<[/color]pairs[color=#000000];[/color] j[color=#000000]++){[/color]
         ishift [color=#000000]=[/color] [color=#000000]iBarShift[/color][color=#000000]([/color]symb[color=#000000][[/color]j[color=#000000]],[/color] [color=#000000]0[/color][color=#000000],[/color] Time[color=#000000][[/color]i[color=#000000]]);[/color]
         regressors[color=#000000][[/color]i [color=#000000]*[/color] pairs [color=#000000]+[/color] j[color=#000000]] =[/color] [color=#000000]iClose[/color][color=#000000]([/color]symb[color=#000000][[/color]j[color=#000000]],[/color] [color=#000000]0[/color][color=#000000],[/color] ishift[color=#000000]);[/color]
      [color=#000000]}[/color]
   [color=#000000]}[/color]
   [color=#000000]Rm[/color][color=#000000]([/color][color=#0000ff]"regressors"[/color][color=#000000],[/color] regressors[color=#000000],[/color] back[color=#000000],[/color] pairs[color=#000000]);[/color]

   [color=#2f9956]// do the regression[/color]
   [color=#2f9956]// first we need a regressand[/color]
   [color=#2f9956]// we simply use a straight line that will be our ideal trend[/color]
   [color=#000000]Rd[/color][color=#000000]([/color][color=#0000ff]"trendslope"[/color][color=#000000],[/color] [color=#000000]0.01[/color][color=#000000]);[/color]
   [color=#7f0055][b]if[/b][/color] [color=#000000]([/color]use_diff[color=#000000]){[/color]
      [color=#000000]Rx[/color][color=#000000]([/color][color=#0000ff]"y <- rep(-trendslope, back-now-1)"[/color][color=#000000]);[/color]                           
      [color=#000000]Rx[/color][color=#000000]([/color][color=#0000ff]"x <- diff(regressors)[seq(now+1,back-1) ,]"[/color][color=#000000]);[/color]
   [color=#000000]}[/color][color=#7f0055][b]else[/b][/color][color=#000000]{[/color]
      [color=#000000]Rx[/color][color=#000000]([/color][color=#0000ff]"y <- trendslope - trendslope * seq(now+1, back) / back"[/color][color=#000000]);[/color] 
      [color=#000000]Rx[/color][color=#000000]([/color][color=#0000ff]"x <- regressors[seq(now+1, back) ,]"[/color][color=#000000]);[/color]                        
   [color=#000000]}[/color]
   [color=#000000]Rx[/color][color=#000000]([/color][color=#0000ff]"model <- lm(y ~ x + 0)"[/color][color=#000000]);[/color]                       [color=#2f9956]// fit the model[/color]
The trendomat.mqh that i posted yesterday has two modes: In the default mode (use_diff=true) it will detrend the data by taking the first difference and running the regression on this. It will then apply these coefficients to the absolute price. The other mode will do the regression with absolute price. I will also post a version of arbomat that lets you chose between these modes (arbomat and trendomat will then be two separate EAs). Both methods have some interesting properties:

when doing it with absolute price (and also allowing an intercept term to be estimated) then the regression will *always* be able to fit either a perfect trend or a perfect range during the fitted time window, no matter whether the time series are actually somehow related to each other or pure randomness. This happens by definition! It will be able to fit to any noise if it has enough pairs to play with and the results might be completely meaningless.

Since there seem to be subtle relationships between these forex pairs the results are not always completely meaningless, over longer time windows sometimes you will see very regular patterns and I have already successfully traded a few but I also made a few losses.

The other method uses diff() to take the first difference and runs the regression on that. I will upload this version of arbomat soon. This will not enforce an absolute trend or stationarity anymore and instead tries to make the curve as smooth as possible by trying to "hedge away" all the short-term moves. The result will only *sometimes* be stationary but is much more meaningful, the coefficients don't fluctuate so much anymore, they drift only very slowly and stay relatively constant.

Fitting the absolute price forces the curve to be perfectly horizontal or a perfect trend but very noisy and fitting the differences makes it as smooth as possible but allows the absolute price to drift freely. I believe it is the diff() method that should be considered for further experiments.

I will post a few plots of the coefficients over time and animations of a few forward runs later (numbers are still being crunched and my CPU fan running at full speed at the time of this writing).

Bernd
  • #148
  • Edited 5:39pm Nov 20, 2010 1:42pm | Edited 5:39pm
http://vimeo.com/17029673 Trend-O-Mat, H1, fitting absolute price
http://vimeo.com/17028948 Trend-O-Mat, H1, fitting diff(price)

http://vimeo.com/17036170 Trend-O-Mat, H4 fitting absolute price
http://vimeo.com/17037303 Trend-O-Mat, H4 fitting diff(price)
  • #149
  • Edited 5:41pm Nov 20, 2010 4:47pm | Edited 5:41pm
I have an idea:

what about doing the regression with diff(prices) over a very short interval and plotting the coefficients into the chart and not trying to trade the basket at all but instead try to use the coefficients as some kind of indicator for "conventional" trading?

When the coefficients in the whole market start to shift this might indicate paradigm shifts, maybe it is visible earlier or clearer than it is visible in the noise of the charts itself.
  • #150
  • Nov 20, 2010 6:02pm Nov 20, 2010 6:02pm
Not to crash the party but doesn't the model have an overfitting property? It seems you have to adjust your ratio every period and that should take a significant ammount out of the profits if not, most likely, all. It's been shown by the model to faill comparing future movements to processed ones.

I don't believe you can find long term cointegration since it depends on central bank rates, their reserves, global economy that isn't static etc. I would look at short term relations between currencies trying to find short movements of money between banks, or news that would go from the mean price. Naturally there is a very big chance those are arbt out and that the mean price is shifted to the closer price level directed by the movement .
  • #151
  • Nov 21, 2010 10:03am Nov 21, 2010 10:03am
Quoting Rikers
Not to crash the party but doesn't the model have an overfitting property?
I didn't start partying yet and you are of course right, there is the danger of overfitting. It has the ability to fit everything into a perfect range. The new version that I will upload soon will behave a little different.

And after playing with the trend-o-mat for some time now it seems this is certainly *not* the holy grail, but moving the time window around and observing how the coefficients behave and how well or how bad it fits *might* still give some insights into the market as a whole.
  • #152
  • Nov 21, 2010 10:13am Nov 21, 2010 10:13am
Quoting 7bit
I didn't start partying yet and you are of course right, there is the danger of overfitting.
It would be a good idea to divide the timespace to 2 sections, first would be the in-sample period and second would be out-sample. All optimization would be done strictly on in-sample. That way could visually see how the curve would look in the future [out-sample].
  • #153
  • Nov 21, 2010 1:59pm Nov 21, 2010 1:59pm
Quoting mikkom
It would be a good idea to divide the timespace to 2 sections, first would be the in-sample period and second would be out-sample. All optimization would be done strictly on in-sample. That way could visually see how the curve would look in the future [out-sample].
in trendomat.mq4 this is already implemented. You can draw two vertical lines into the chart and give them the names back and now. back will override the value of back_bars and it will only use bars between back and now for the regression. It will plot a red line into the R plot at the position where the now line is.

You can move the lines around with the mouse and it will update the plot in real time.

After some cleaning up of the arbomat code I will upload a version of arbomat that has the same functionality too (I made trend and range two separate EAs).
  • #154
  • Nov 22, 2010 6:45am Nov 22, 2010 6:45am
excellent work as usual, professor. Thanks for sharing.

Do you think it will be hard for me to change it so that it can be used across differnt instruments such as non usd pair, futures and metals? I would guess that there are some interesting, stable and vaguely stationary inter-market relationships.
  • #155
  • Nov 23, 2010 8:32am Nov 23, 2010 8:32am
I've made around 50 pips with the original range EA off the 1H chart!

7bit, you posted some tests you've made on the original trend EA which showed that using the same coefficients wasn't good since it was drifting away from the mean because it was a "forced" trend over a string of data.

Did you do it with the range EA? This way, we could see if the same thing happens and how keeping the same coefficients pans out.

Also, is there any new version of the EAs?

Thanks!
  • #156
  • Nov 23, 2010 9:50am Nov 23, 2010 9:50am
Quoting pippero
excellent work as usual, professor. Thanks for sharing.

Do you think it will be hard for me to change it so that it can be used across differnt instruments such as non usd pair, futures and metals?
I don't know how difficult it would be for you ;-)

You can basically do the same things in R completely without Metatrader if you have the data in csv format or you could have a look at the quantmod package which has functions to retrieve data for other instruments from yahoo, oanda and others. (I cannot help you much with questions specific to quantmod, I have used it only once and only to chart the gold/silver ratio, but I think it might be useful because it contains some helpful functionality).
  • #157
  • Nov 23, 2010 9:55am Nov 23, 2010 9:55am
Quoting ultfrisbee
.
Did you do it with the range EA? This way, we could see if the same thing happens and how keeping the same coefficients pans out.

Also, is there any new version of the EAs?
I'm still not finished solving some minor implementation details and cleaning up the messy code. I will post it if it is finished (this week, I promise) and then I will also make a few movies of forward runs. It will also have the "now"-line (like trendomat) to define the border between known past and unknown future to interactively play with it.
  • #158
  • Nov 23, 2010 12:19pm Nov 23, 2010 12:19pm
7bit, I`ve noticed you are using USDJPY in your calculations - it has different point value. How do you consider it?
  • #159
  • Edited 1:20pm Nov 23, 2010 12:25pm | Edited 1:20pm
Quoting vitali_yv
7bit, I`ve noticed you are using USDJPY in your calculations - it has different point value. How do you consider it?
Its documented in the code.

Inserted Code
   [color=#2f9956]// convert the coefficients to usable hege ratios by multiplying[/color]
   [color=#2f9956]// usd/xxx pairs with their quote. The results can then be[/color]
   [color=#2f9956]// conveniently interpreted as multiples of needed Lots or Units.[/color]
   [color=#7f0055][b]for[/b][/color] [color=#000000]([/color]i[color=#000000]=[/color][color=#000000]0[/color][color=#000000];[/color] i[color=#000000]<[/color]pairs[color=#000000];[/color] i[color=#000000]++){[/color]
      
      [color=#2f9956]// convert to units[/color]
      [color=#7f0055][b]if[/b][/color] [color=#000000]([/color][color=#000000]StringSubstr[/color][color=#000000]([/color]symb[color=#000000][[/color]i[color=#000000]],[/color] [color=#000000]0[/color][color=#000000],[/color] [color=#000000]3[/color][color=#000000]) ==[/color] [color=#0000ff]"USD"[/color][color=#000000]){[/color] 
         coef[color=#000000][[/color]i[color=#000000]] =[/color] coef[color=#000000][[/color]i[color=#000000]] *[/color] [color=#000000]iClose[/color][color=#000000]([/color]symb[color=#000000][[/color]i[color=#000000]],[/color] [color=#000000]0[/color][color=#000000],[/color] [color=#000000]0[/color][color=#000000]);[/color]
      [color=#000000]}[/color]

   [color=#000000]}[/color]
Edit: There was a typo in the comment (above it is corrected) but the code is correct. It will multiply USD/XXX pairs with the value of their quoted price (the current price of one USD expressed in XXX).

You can verify the correctness of this with the oanda profit calculator:

Extreme example: If a change of 1 USD in EURUSD (from 1.3400 to 2.3400) and a change of 1 YEN in USDJPY (from 83 to 84) would cancel out each other on the chart (b1 * EURUSD - b2 * USDJPY = const, both coefficents from the regression would then be equal¹) then it would currently need 1 unit EURUSD and 83 units USDJPY to have the same effect in trading.

There would be a more general way to do this with MODE_TICKVALUE, MODE_TICKSIZE and MODE_LOTSIZE but unfortunately I cannot access these values in the backtesting mode of MT4 and so I did it this way.

_________
¹) I am cheating: Its only approximating the real world, theoretically I am not allowed to add b1 * EURUSD + b2 * USDJPY, I should multiply them (or do the regression with log(price)). Simply adding differences (and assuming an (almost) constant tick value for USDXXX) would give incorrect results if there were big moves of UJ from 80 to 40 vs. EU from 2 to 1 but the real-world price changes during our trades are so small that it does almost not matter.
  • #160
  • Nov 23, 2010 12:44pm Nov 23, 2010 12:44pm
I just want to say tanks to 7bit for his works, it opens a new door to forex trading for me.
Actually i'm studying R since 2 weeks ago, when i discovered this thread and it's fantastic

I just want to contribute suggesting a couple of reading that i've found interesting
http://quanttrader.info/public/testForCoint.html
and this books
http://www.amazon.com/dp/0470284889
http://www.amazon.com/gp/product/0387759662

gspe
0 traders viewing now