Synthetic hedges, cointegration, mean reversion and similar stuff

What do I do to get that Arb-O-Mat to display.

Cheers
CanuckCT

Download R from here: http://www.r-project.org/
Install R
Save mt4R.dll into experts/libraries folder
Save mt4R.mqh and common_functions.mqh into experts/include folder
Save arbomat_demo.mq4 into experts folder
Open arbomat_demo.mq4 using metaeditor and edit the in Red below to where you installed R:

#define RPATH "C:/Program Files/R/R-2.12.0/bin/i386/Rterm.exe --no-save"
*I'm not sure if this is relevant but take note of the slash "/" shouldn't be backslash

Save and compile arbomat_demo.mq4

Run the EA in a chart and Arb-O-Mat should just come out automatically.

Hope it helps.

Stuff that comes through my mind...

If you look for signals on M15 (large spread in terms of stdev) and confirm it's the same on the 1H, I think you've got yourself a decent trade that should be played out before the end of the week since you almost have 100 ticks per day.

You could even do it on the 1H and 1D and only have to adjust it once a day if it goes against you.

Avoid doing calculations and adjusting in down times (like 6 pm EST).
1 stdev is too small to be traded consistently without cautious monitoring and lots of adjusting.

This is like trading bollinger bands but the cointegration and scaling makes it more stable.

Could anyone that made a succesful trade using this post the following:

Lots of each pair
Spread when you got in AND out (in terms of stdev)
Pips made/lost for each pair

This is a good idea. What we would need is to choose a base currency (to limit calculations) and have the EA calculate the best basket possible (largest spread in terms of stdev). It would calculate every possible basket of 3 or 4 pairs, chosing among the other 5.

For exemple, if you choose EU and 3 pairs, it will calculate 10 spreads, return a value in terms of stdev and show the results for the basket with the greatest value.

Choosing a basic pair makes it calculate 10 spreads (choose 2 among 5) rather than 20 (choose 3 among 6).

Does not work....is there I file I need to open with "R"?

regarding the doubling up that has been asked: I had to resist the temptation to double my lots more than once today myself. I could easily do it because my lot sizes are (still) small but then I would disturb my win/loss statistics, I would then try to enforce a win all the time. And this is the root of all evil[tm].

Maybe it would work, maybe it will reverse and come back more often than it would drift away and maybe also the permanent re-adjusting of the positions would help to enforce this. But this would add the smell of martingale to it and I want to avoid this at all costs. If this has more winers than losers then any type of martingale would not be needed anyways, it would be profitable in itself. Adding martingale to it would only pollute the statistics and *hide* possible problems until it is too late and the black martingale swan comes along.

For this, all you need is enough testing on your parameters

Data (number of ticks because if you add ticks, it changes the factors)

Timeframe

Number of pairs

Value of spread at which you're confident enough to get in (0.5 stdev is NOT enough!)

Stop loss in terms of stdev (get in at 2, get out at 2.5) or in total pips lost

Take profit once it crossed the mean, at the mean or just before it crosses the mean (from riskiest t o safest)

it is not doubling up like the martingale.

if spread drifts away you could see in trend-mode that the units become smaller and smaller. this means the farther away the spread from meanline is, the lesser you have to risk.
and if you are in red and re-adjust your lotsizes to the actuall showed in the graph nothing happens to your equity or balance you only lower your used margin. after re-adjusting you can then stack the same positions you have again and have the same used margin before readjusting.

example today:

initial start with:

GBPUSD -112
EURUSD 80
USDJPY 90 ...after some move against my posies i readjust to...

GBPUSD -81
EURUSD 11
USDJPY 32 ... this lowers my used margin about 50%

then i doubled:

GBPUSD -162
EURUSD 22
USDJPY 64 ... and had the same margin used like the initial trade.

if i would not double, the spread needs to move more distance that your basket becomes breakeven or better.

it is not realy martingale you only fit your lotsizes in conjunction like the StdDev fit with the spread. yes, you are in red, but you will not loose more in the same distance if spread moves farther away.

pictures tell more than thousand words.

This must have been a coincidence (or only seemed this way): It lowered the total position size because previously there was a big offsetting (profit neutral) gbpusd + usdjpy position that probably only existed because the curve fit was slightly better with this at this moment.

What is always constant is the net exposure into the direction of the y axis. It is always like if you were the amount of base_units long and on the y-axis 0.0001 would be 1 pip. This means base_unit = 100 and each 0.01 (100 pips) on the y-axis is 1$.

It does not scale out, it will only adjust positions to fit the trend as good as possible and sometimes (especially when the whole market starts to reverse and all coefficients have to be slowly reversed) there sometimes appear huge but almost profit-neutral offsetting positions that come and go. During these times of huge fluctuations it is really hard to trade and it almost seems it won't ever stop going down on the right side of the chart.

Its better to wait when this occurs and enter only when it seems to have stabilized again.

I personally don't trust the trend fits, something feels wrong with them. I concentrate my efforts on the mean reversion at the moment.

I think you are both right.

When you get so far from the mean, you don't need as many units to make a profit if you assume that it will go back to this exact mean. If you are 10 pips from the mean and have 100 units, you only need 50 units to achieve the same result once you're 20 pips away.

But when using this tool with 2000 ticks, the mean won't move after the first few following ticks and scaling out is the right thing to do. But as ticks come in, the mean shifts and the scaled out position won't be enough to cover the loss.

This is true because if you buy the basket when it is 1 stdev under the mean and then scale out once it got to 2 stdev under the mean, what happens is that the mean is slowly shifting down as the ticks come in and your 50 units will only make 15 pips if it's long enough.

Doubling the position will offset that, but you're in martingale territory at this point.

Don't forget to tick "Allow DLL imports" in EA properrties, common tab.

JC

"Does not work" is a very vague description. Tell us what exactly you did (step by step) and what exactly happened.

You don't have to open anything in R or even start it. You just need to have it installed on your computer and the path in the mq4 file set correctly, so the EA can find it and start it.

If it immediately crashes MT4 then most likely the path to Rterm.exe is not correct (this will be fixed in the next version and an appropriate error message outputted instead)

Also it might be helpful to set RDEBUG in arbomat_demo.mq4 to 2 (it is 1 by default) and have DebugView.exe running to capture all debug and error messages during the (unsuccessful) run of the EA until it is all working.

I want to believe this is a holy grail. Can I?

another question. the arb-o-mat graph is a floating window of data's. the last data's will be cut if new data's arrive at the beginning. if the last data's would not disapear does this mean that the y-axis could be infinite compund return? what will happen if i open at every 2nd down StdDev a basket and would never close them? would this basket in future give infinite yield or do i have to readjust every open basket? this would be an exponential growing in equity for infinite gain?

well done 7bit, excellent work.

Regarding the trending system, I have done similar work in the past, although I used a different way to combine the pairs the objective was moreless similar. My observation was that the behaviour did not hold very well out of sample. If you want to test this you can either fix your coefficients at some point in time and observe the behaviour later, or, with recomputing the coefficients, whenever you change them, also accumulate your profit/loss resulting from position adjustments. Then you will know if it works or not. My guess is it won't but don't let this stop you from testing.

Regarding the flat ranging system, my research has shown that the cointegration we can achieve in FX is not very good, in statistical terms. Doesn't mean that money can not be made this way, just that the result won't be spectacular. Ideally if you could have access to more trading instruments than FX pairs, I believe you would have far better chance to find better behaving synthetic pairs.

All the best

Careful, the more combinations you try, the more likely you are to encounter http://en.wikipedia.org/wiki/Data-snooping_bias

Progress I think....MT4 crash comments...so something is working...crashes as soon as I drag EA to chart.


There has been a critical error
Time : 2010.11.18 09:01
Program : Client Terminal
Version : 4.00 (build: 229, 16 Nov 2010)
OS : Windows XP Professional 5.1 Service Pack 3 (Build 2600)
Processors : 1 x X86 (level 15)
Memory : 1538544/192612 kb
Exception : C0000005
Address : 003408CD
Access Type : write
Access Addr : 0000000C

Registers : EAX=00000097 CS=001b EIP=003408CD EFLGS=00010283
: EBX=0000000C SS=0023 ESP=0306F3B0 EBP=0000000C
: ECX=7C9113F2 DS=0023 ESI=003408C8 FS=003b
: EDX=030A1E80 ES=0023 EDI=00340000 GS=0000

Stack Trace : 00000000 00000000 00000000 00000000
: 00000000 00000000 00000000 00000000
: 00000000 00000000 00000000 00000000
: 00000000 00000000 00000000 00000000

Modules :
1 : 00400000 0056A000 crogram filessmarttradefx ecn metatrader 4.0-3terminal.exe
2 : 01680000 002C5000 c:windowssystem32xpsp2res.dll
3 : 03070000 00039000 crogram filessmarttradefx ecn metatrader 4.0-3expertslibrariesmt4r.dll
4 : 10000000 00020000 crogra~1aliantnetass~1smartb~1sbhook.dll
5 : 10100000 0000F000 crogram fileslogitechsetpointlgscroll.dll
6 : 5AD70000 00038000 c:windowssystem32uxtheme.dll
7 : 62740000 0000A000 c:documents and settingsall usersapplication datarealrealplayerbrowserrecordpluginchromehookrpchromebrowserrecordhelper.dll
8 : 662B0000 00058000 c:windowssystem32hnetcfg.dll
9 : 71A50000 0003F000 c:windowssystem32mswsock.dll
10 : 71A90000 00008000 c:windowssystem32wshtcpip.dll
11 : 71AA0000 00008000 c:windowssystem32ws2help.dll
12 : 71AB0000 00017000 c:windowssystem32ws2_32.dll
13 : 71BF0000 00013000 c:windowssystem32samlib.dll
14 : 72D10000 00008000 c:windowssystem32msacm32.drv
15 : 72D20000 00009000 c:windowssystem32wdmaud.drv
16 : 73DD0000 000F1000 c:windowssystem32mfc42.dll
17 : 74720000 0004C000 c:windowssystem32msctf.dll
18 : 755C0000 0002E000 c:windowssystem32msctfime.ime
19 : 76380000 00005000 c:windowssystem32msimg32.dll
20 : 76390000 0001D000 c:windowssystem32imm32.dll
21 : 763B0000 00049000 c:windowssystem32comdlg32.dll
22 : 769C0000 000B4000 c:windowssystem32userenv.dll
23 : 76B40000 0002D000 c:windowssystem32winmm.dll
24 : 76C30000 0002E000 c:windowssystem32wintrust.dll
25 : 76C90000 00028000 c:windowssystem32imagehlp.dll
26 : 76F60000 0002C000 c:windowssystem32wldap32.dll
27 : 77120000 0008B000 c:windowssystem32oleaut32.dll
28 : 773D0000 00103000 c:windowswinsxsx86_microsoft.windows.common-controls_6595b64144ccf1df_6.0.2600.6028_x-ww_61e65202comctl32.dll
29 : 774E0000 0013E000 c:windowssystem32ole32.dll
30 : 77690000 00021000 c:windowssystem32ntmarta.dll
31 : 77A80000 00095000 c:windowssystem32crypt32.dll
32 : 77B20000 00012000 c:windowssystem32msasn1.dll
33 : 77BD0000 00007000 c:windowssystem32midimap.dll
34 : 77BE0000 00015000 c:windowssystem32msacm32.dll
35 : 77C00000 00008000 c:windowssystem32version.dll
36 : 77C10000 00058000 c:windowssystem32msvcrt.dll
37 : 77DD0000 0009B000 c:windowssystem32advapi32.dll
38 : 77E70000 00093000 c:windowssystem32rpcrt4.dll
39 : 77F10000 00049000 c:windowssystem32gdi32.dll
40 : 77F60000 00076000 c:windowssystem32shlwapi.dll
41 : 77FE0000 00011000 c:windowssystem32secur32.dll
42 : 78130000 0009B000 c:windowswinsxsx86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.4053_x-ww_e6967989msvcr80.dll
43 : 7C340000 00056000 c:windowssystem32msvcr71.dll
44 : 7C3A0000 0007B000 c:windowssystem32msvcp71.dll
45 : 7C420000 00087000 c:windowswinsxsx86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.4053_x-ww_e6967989msvcp80.dll
46 : 7C800000 000F6000 c:windowssystem32kernel32.dll
47 : 7C900000 000B2000 c:windowssystem32ntdll.dll
48 : 7C9C0000 00817000 c:windowssystem32shell32.dll
49 : 7E410000 00091000 c:windowssystem32user32.dll

Call stack :

Okay....after checking my path 3 times....I did have it entered incorrectly. here is what I get now...

Did you even read my previous post with hints what this most likely means and also how to get the debug info?

And the MT4 crash log is useless. It does not contain any useful information. Please don't post it. All information is in DebugView.

Please take the time to read all instructions that have been given. If had read them you had now


[_] installed common_functions.mqh
[_] installed R
[_] installed mt4R.mqh & mt4R.dll
[_] changed RPATH in the EA to point to your Rterm.exe
[_] changed RDEBUG to 2
[_] downloaded DebugView (and have it running it during the loading of the EA)
[_] watch the output of DebugView
[_] watch the output of the experts and journal logs in MT4
[_] never post this mt4 crash log


Please print out the above checklist and make checkmarks on each item as you follow the procedure.

Now try to put it on different charts: GBPUSD and USDCAD are interesting for the ranging (default) mode. Also try longer timeframes. M5 is not enough, M15 only sometimes and above H1 and H4 it starts to show interesting long-term cycles.

Always use it only with xxx/usd and usd/xxx pairs. It will not work with any xxx/yyy crosses, the lot size calculation will fail completely. Every xxx/yyy can (and will) be expressed with a linear combination of xxx/usd and usd/yyy or usd/xxx and yyy/usd, this means you don't need EUR/JPY if you have EUR/USD and USD/JPY in the basket.

draw a vertical line into the chart and name it back and then yoy can drag this line with the mouse to change the number of look-back bars (the parameter back) interactively with the mouse.

Since it can (and will) forcefully fit *anything* into a range, even if it is not related at all, you will see a lot of things that are not tradeable in reality. You can identify these by expanding the number of back bars and watching how completely the coefficients and the whole curve change and there is no stability.