magic shift | Page 2 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Magic numbers question 12 replies

What does GMT shift and GMT shift Sun do in this indicator? 0 replies

magic is back! 11 replies

Magic Numbers for Mutiple EA 9 replies

Attachments: magic shift
Exit Attachments

magic shift

  • #21
  • Nov 25, 2009 11:45am Nov 25, 2009 11:45am
Quoting 7bit
Yes, this 61 shift only started about after last september (and has been extremely stable since then). Older parts of the history have different shifts, you can easily find them (and the times where they changed) by using the optimizer.

You can also experiment by inverting the rules (go long instead of short and vice versa), there are also positive autocorrelations.

i don't understand this, so, when you decided to change the shift value? now with 61 is good but when this will change?

you say that in the past are other shifts that works better than 61 but you know this now and this is not helpfull

how you determined when other shifts works better and when they must change?
  • #22
  • Nov 25, 2009 12:15pm Nov 25, 2009 12:15pm
Quoting Heral
i don't understand this, so, when you decided to change the shift value? now with 61 is good but when this will change?

you say that in the past are other shifts that works better than 61 but you know this now and this is not helpfull

how you determined when other shifts works better and when they must change?
This is an important question and i have no definite answer.

Like every other system involving moving averages, stochs and other indicators with adjustable period this will need to be retuned at some time or it will stop working just like all the other indicator systems here. Only this system has less degrees of freedom (only one), so it is not prone to overfitting.

I monitor the equity curve (floating + realized) in realtime in a H4 candle chart and currently it is moving up in a nice and well shaped channel. Like with any other trends i hope to see signs of reversal once the lag stops working. This is also one of the reasons why i don't recommend this to newbies. This is not the holy grail, just an interesting system!
  • #23
  • Nov 25, 2009 12:16pm Nov 25, 2009 12:16pm
  •  Porky
  • | Joined Jun 2009 | Status: ...as in Porcupine, not the pig. | 34 Posts
Quoting Heral
i don't understand this, so, when you decided to change the shift value? now with 61 is good but when this will change?

you say that in the past are other shifts that works better than 61 but you know this now and this is not helpfull

how you determined when other shifts works better and when they must change?
From my experience with other systems, the shift will change as soon as I put live money on it.

Interesting phenomenon though.
  • #24
  • Nov 25, 2009 1:54pm Nov 25, 2009 1:54pm
Quoting 7bit
I have made something similar, it doesn't use the accelerator, but it does something very similar: http://sites.google.com/site/prof7bit/autocorr

The above is NOT an EA for this system, it just shares its main idea.
So u aren't gonna share it?
  • #25
  • Nov 25, 2009 2:28pm Nov 25, 2009 2:28pm
attached a little EA that I just put together in post #1
  • #26
  • Nov 25, 2009 3:00pm Nov 25, 2009 3:00pm
The 61 has just been derived from back fitting, you have no way of knowing what the magic shift pattern will be in the future until it has been and gone.
  • #27
  • Nov 25, 2009 3:20pm Nov 25, 2009 3:20pm
Quoting scooby-doo
The 61 has just been derived from back fitting, you have no way of knowing what the magic shift pattern will be in the future until it has been and gone.
Yes, at what point would you determine that 61 no longer applies in the current market and re-optimise?

From what little I gathered from Mikkom's thread about his own blackbox, his genetic algos constantly optimises the blackbox by discarding or utilising strategies according to constant testing. I would surmise that the same would need to happen here in some form or other?

(apologies Mikkom if I got this wrong, your thread is a little above my intellect)
  • #28
  • Nov 25, 2009 4:01pm Nov 25, 2009 4:01pm
Quoting scooby-doo
The 61 has just been derived from back fitting, you have no way of knowing what the magic shift pattern will be in the future until it has been and gone.
This may be true, but looking at the history it seems that the same shift can stay intact many months, the current shift has been in place for a year now.

Unlike many other systems with dozens of parameters, leading to zillions of possible combinations (can be perfectly fitted to anything -> overfitting!) this one has not much room to change. There is only one parameter with only a few hundred possible values.

This model is of such extreme low complexity that it can hardly ever be overfitted.
  • #29
  • Nov 25, 2009 4:08pm Nov 25, 2009 4:08pm
Jesus what the market has done over the last year... if this system can survive that... and a normal market the year before... though we cannot see the future... it seems that this system should be able to work in the future... who knows anyway!

Quoting Limstylz
Yes, at what point would you determine that 61 no longer applies in the current market and re-optimise?

From what little I gathered from Mikkom's thread about his own blackbox, his genetic algos constantly optimises the blackbox by discarding or utilising strategies according to constant testing. I would surmise that the same would need to happen here in some form or other?

(apologies Mikkom if I got this wrong, your thread is a little above my intellect)
  • #30
  • Nov 25, 2009 4:33pm Nov 25, 2009 4:33pm
Quoting 7bit
This may be true, but looking at the history it seems that the same shift can stay intact many months, the current shift has been in place for a year now.

Unlike many other systems with dozens of parameters, leading to zillions of possible combinations (can be perfectly fitted to anything -> overfitting!) this one has not much room to change. There is only one parameter with only a few hundred possible values.

This model is of such extreme low complexity that it can hardly ever be overfitted.

7bit,

If correct this is brilliant! I recommend changing your name to 64-bit processor.....

Are we talking about the MT4 accelerator oscillator, or some other accelerator? I would like to see this visually.

Kostas1
  • #31
  • Nov 25, 2009 4:42pm Nov 25, 2009 4:42pm
Quoting kostas1
ccelerator oscillator, or some other accelerator? I would like to see this visually.
Its the built in accelerator oscillator of mt4. Unfortunately it can not be shifted when put on a chart to visualize it. My EA draws a vertical line at the positon of the shift to help you find the correct position.

Maybe someone wants to write a version of the Accelerator that can be shifted on the chart, so it could be better visualized.
  • #32
  • Nov 25, 2009 6:05pm Nov 25, 2009 6:05pm
How about this guys, start in dec 08 and optimize for that month. Then use best setting for Jan. Then optimize for Jan and use this setting for feb. Wash rinse repeat. After each test write down p/l then use that balance for next month. Write down p/l for each month in excel. This should keep u up with optimal setting as best as possible. See how that equity compares to just using same shift for entire year.

Just a thought
  • #33
  • Nov 25, 2009 6:06pm Nov 25, 2009 6:06pm
So if I understand this correctly with the chart below based on the red virtcal lines I would have 8, 4 hour periods before I would probably go short??? Even if the pair is actually falling as of now?
Attached Image (click to enlarge)
Click to Enlarge Name: eurusd.gif Size: 56 KB
  • #34
  • Edited 6:53pm Nov 25, 2009 6:41pm | Edited 6:53pm
Quoting Scratch99
So if I understand this correctly with the chart below based on the red virtcal lines I would have 8, 4 hour periods before I would probably go short??? Even if the pair is actually falling as of now?
I count 11 more bars (44 hours) long:

Attached Image (click to enlarge)
Click to Enlarge Name: long_long.png Size: 41 KB


But if I look at the equity chart (the small chart to the left) then I dont think all of these bars will go up. There is some room for a retracement at the equity chart, but if you would have been long from 1.4946 already then it may still end in profit.

Bringing this to the next level would mean analyzing the resulting equity chart (on a demo account) and deciding from that whether to switch on or off the expert on the live account.
  • #35
  • Nov 25, 2009 6:50pm Nov 25, 2009 6:50pm
Quoting 7bit
This may be true, but looking at the history it seems that the same shift can stay intact many months, the current shift has been in place for a year now.

Unlike many other systems with dozens of parameters, leading to zillions of possible combinations (can be perfectly fitted to anything -> overfitting!) this one has not much room to change. There is only one parameter with only a few hundred possible values.

This model is of such extreme low complexity that it can hardly ever be overfitted.
It's true that you're not overfitting, but that's only one way to over-optimize. The other way is through the "multiple hypothesis problem" and it seems like that's the route you took here. In an ealier post you said that you arrived at the number 61 by just "testing all of them." This is the classic multiple hypothesis pitfall. There's an old article I wrote about this here:

http://market-geeks.com/articles/002...Datamining.php


The idea of a dynamically shifting parameter based on the changing behavior of market participants is an area where I totally agree with your approach however. The key is just to keep track of how well a given domain of parameter values are performing over time relative to each other. I have another old article on that here:

http://market-geeks.com/articles/004...RandomWalk.php
  • #36
  • Edited 6:26am Nov 26, 2009 6:06am | Edited 6:26am
Quoting capitalist88
It's true that you're not overfitting, but that's only one way to over-optimize. The other way is through the "multiple hypothesis problem" and it seems like that's the route you took here.
But isn't that only possible if i can find a hypothesis for almost every possible outcome? In your example you had 10 coin-flips and 1000 coin flippers. I'm testing over hundreds of coin flips and have only 200 coin flippers. I have found some pairs where this system simply wont work, the equity curves look very different, almost always random, no matter what lags I try, the other extreme is EURUSD, I will immediately find multiple strong lags, 61 is only the strongest one.

And couldn't any form of model fitting be called "multiple hypothesis"? If i have a neural net with hundreds of weights, then each possible combination of weights would be a new hypothesis.

And couldn't we on the other hand just say i need only one hypothesis and it is: "there is strong autocorrelation" and then just measure it's lags?

I agree that out-of-sample testing is the only way to tell that the model is really worth anything. A few months ago I've been running a variant of it (a variant with two lags instead of only one) on the demo account for a while in a minimized chart window, almost forgot it was there, until i plotted it's forward performance from the account history and compared it with the performance over the training set and i was really impressed and by what i saw: The slope of the equity curve was almost identical!

Since then (a few weeks) i am running a few variants of them forward, one of them (unpatient as i am) on a live account and i have not been disappointed so far. The equity curve is still showing the same strong uptrend that it shows when testing on the training set, no signs of beginning randomness so far.
  • #37
  • Nov 26, 2009 7:16am Nov 26, 2009 7:16am
I just made an experiment on H1, I didn't cherry pick a certain period, I just used the time interval that happened to be already set in the tester from some other experiment, switched to H1 (i mainly use H4, didn't extensively test H1 yet) and searched some lags. Search period is 2009.8.27 - 2009.11.27.

Below are shown the result of the parameter search and then for comparison the performance of the best find over the training set. Then i expand the test set going backwards in time, starting the same test earlier and always including the training set at the end for comparison (expanding backwards is allowed in this case)

first the parameter search and the test over the training set:

search plot:
Attached Image (click to enlarge)
Click to Enlarge Name: H1_opt_3month_plot.png Size: 44 KB


results table:
Attached Image (click to enlarge)
Click to Enlarge Name: H1_opt_3month_table.png Size: 54 KB


test over above training set (2009.8.27 - 2009.11.27) for comparison reasons only:
Attached Image (click to enlarge)
Click to Enlarge Name: H1_test_on_training_set.png Size: 34 KB


starting 2 month before (including training set):
Attached Image (click to enlarge)
Click to Enlarge Name: H1_test_on_2month_before_training_set.PNG Size: 34 KB


starting 4 months before (including training set):
Attached Image (click to enlarge)
Click to Enlarge Name: H1_test_on_4month_before_training_set.PNG Size: 33 KB


starting 6 months before (including training set). In March-April it wouldn't have worked:
Attached Image (click to enlarge)
Click to Enlarge Name: H1_test_on_6month_before_training_set.PNG Size: 34 KB
  • #38
  • Nov 26, 2009 9:40am Nov 26, 2009 9:40am
So what can be deduced from these charts ?
  • #39
  • Edited 11:49am Nov 26, 2009 9:46am | Edited 11:49am
Quoting Koyco
So what can be deduced from these charts ?
a shift found by fitting it to only 3 months H1 data worked also during 4 other months. This should also work forward.

I just started a new forward test on EUR/USD H1, magic_shift found by fitting it from March until now is 192, positive correlation (long when positive, short when negative). Note: Same system, but not the same H4 parameters from Post 1: different shift and different trade direction!

I will post updates whenever it opens a new trade:
Attached Image (click to enlarge)
Click to Enlarge Name: magic_forward.gif Size: 21 KB
  • #40
  • Nov 26, 2009 10:12am Nov 26, 2009 10:12am
In your first post you recommended that if AC was neg. go long but in the 1h. chart you appear to do the oppersite is this the case?
0 traders viewing now