scooby-doo BOJ JPY pairs trading robot | Page 298 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Trading correlation pairs by using the other pairs 843 replies

Export All pairs or selected pairs to .csv with script 3 replies

Pairs of Currency Pairs 4 replies

Trending Pairs / Ranging Pairs 0 replies

Robot trading 120 replies

Attachments: scooby-doo BOJ JPY pairs trading robot
Exit Attachments

scooby-doo BOJ JPY pairs trading robot

  • #5,941
  • Feb 13, 2010 5:08pm Feb 13, 2010 5:08pm
I think that the scoobs stuff is great, but let's remember something on forward testing.

a) Stop is 100pts, take profit is 25. This means you need 4x as many winners as losers to break even, in the absence of trading costs. I.e. a mininum of 80% win rate---probably higher just to break even. It's short volatility, not long volatility.

b) People's intuitive perception of a random process doesn't mesh with real fluctuations. True independent random results have more streaks than people intuitively think should be present in random---their personal idea of random is actually short-term anticorrelated.

c) the robot will lose on trend shifts. These are likely to happen in multiple currencies simultaneously because they come from related underlying factors. Similarly the winners will occur in streaks as well because (beyond the native 4:1 stop to tp) market behavior between fluctuations to sharp trends/jumps oscillate but with short-term persistence.

Conclusion: it will be take longer than you think to validate long-term profitability. A 75% win rate robot might still look great for a long time in periods and yet could be long term negative. I doubt this one is, but it may be good while before we're really confident in a scientific sense.
  • #5,942
  • Feb 13, 2010 5:15pm Feb 13, 2010 5:15pm
Quoting tazmaan
Steve,
Something strange happening in backtest, figured i'd make you aware of it in case it's a bug that needs fixing for forward testing.
Take a look at dates 9/30, 11/23, 12/18 and 2/9 in the report, you'll see what's happening.
There was indeed a bug, and a ridiculous one. I intended comparing current time with order opening time; by mistake, I used order close time. 'Tis a wonder there was not more chaos in those backtests.

Esential fix in post 1.

  • #5,943
  • Feb 13, 2010 5:20pm Feb 13, 2010 5:20pm
Quoting mbkennel
I think that the scoobs stuff is great, but let's remember something on forward testing.

a) Stop is 100pts, take profit is 25....
If you just run it on JPY pairs and set the TP to 40 then you will be ok. I only originally provided it with all pairs so people could see what maybe possible.

I stated previously that you need to take into consideration the gap between the 2 EMA's, the bigger the gap then the safer you are as a trend reversal is not likely to occur overnight on these pairs. When such a case occurs then it is likely to be down to interest rate changed in Japan which are easy to watch out for.

Finally, there is no EA or other fully automated system currently in existence that will show positive expectency across all pairs long term, if there were then I and all the other commercial traders would be out of a job.

Scoobs.
  • #5,944
  • Edited Feb 14, 2010 11:15am Feb 13, 2010 5:44pm | Edited Feb 14, 2010 11:15am
Of course you mean by time, number of trades? Or perhaps not...trades over time encompassing various economic climates? What do you think is adequate sample size? Over how long an interval? We may trade all available pairs to accumulate enough data. FWIW, I think most guys with enough dinero to pee away on butterfly hunts know what random is. The price movement of curriencies is highly auto-correlated. We should be able to find a way to profit. I percieve that any edge found will be slight, and that substantial drawdowns will have to be endured with any and all winning methods. Cheero...here we go...again
Inserted Video
Vincit qui se vincit.
  • #5,945
  • Feb 13, 2010 5:55pm Feb 13, 2010 5:55pm
Hello Huston. We have a problem!

I suppose that nobody has checked the correlation formulas in the pairs trading robot. I don't know who is the autor (Kang Gun, Scooby-doo or Steve), but the calculated correlation values are false!

On the mataf website you find pair correlation values: http://www.mataf.net/en/tools/correlation.

I take as example the NZDJPY-USDJPY correlation on H1.
- Mataf calculate -9.2 (or -0.092)
- The robot calculate 0.9279
- I calculate - 0.1431

You can see that in such a case, the robot will take a trade (corr > 0.70) for a worst negative correlation!

In general, all signals are false, so Steve needs to give the broker the taken money back...

After few hours of testing, I think that the explanation for this big bug is in the "symboldif" function, because using the "iMA" here is false. You can see as test-example the correct using of the MA in my code.
Inserted Code
double symboldif(string symbol, int shift, int corperiod)
{
   return(iClose(symbol, 1440, shift) - iMA(symbol, 1440, corperiod, 0, MODE_SMA,PRICE_CLOSE, shift));
}
Here is a simple test program to compare the values. If you test different pairs and compare Matafs values with my values, you will find that they are very good.

P.S. - Just a suggestion: at the time are discussed here other systems and ideas as the pair trading, and maybe is better to open a new thread for this EAs. I wanted to contribute a little to the pair trading theory, but most postings discuss only the new EAs (RSI, Stoch, SMA ans so on) and ignore the best possible strategy...!?
.
Attached Image (click to enlarge)
Click to Enlarge Name: Correlation-Mataf-net.PNG Size: 22 KB
Attached Image
Attached File(s)
File Type: mq4 Test Correlation.mq4   7 KB | 469 downloads
  • #5,946
  • Feb 13, 2010 5:57pm Feb 13, 2010 5:57pm
Quoting Pippopotamus
I percieve that any edge found will be slight, and that substantial drawdowns will have to be endured with any and all winning methods. Cheero...here we go http://www.youtube.com/watch?v=LLbpQl1cCl8
This accords with my experience with the KG Range and Original robot. Yep, it is profitable but the dd can be heavy and the baskets can sometimes get stuck for weeks on end. It is still trading profitably after more than a year - an ice-age in forex terms - but a road to rapid wealth it ain't.

I cannot listen to the link right now as my wife has the telly on. I will listen tomorrow. Benedetti was a wonderful pianist and I love listening to recordings of his playing.

  • #5,947
  • Feb 13, 2010 7:43pm Feb 13, 2010 7:43pm
Quoting scooby-doo
I
Finally, there is no EA or other fully automated system currently in existence that will show positive expectency across all pairs long term, if there were then I and all the other commercial traders would be out of a job.

Scoobs.
Well, there could be one, but like the heisenberg uncertainty principle, you have no idea which one it is until it's too late. So I agree.

More metaish, I remember a guy mentioning (what I later figured out to be) Renaissance with a certain awe while I was at a professional conference back in maybe '96, '97. I was pretty intrigued thinking I may have had basic qualifications (physics, nonlinear time series etc), but rumor was that they didn't accept applications, They Chose Thou. I heard then that they were making 20%+ per year ad infininitum, and later it turned out that it was more like 50%-80% on huge size.

Medallion never blew up either. So WTF are they doing?
  • #5,948
  • Edited 8:49pm Feb 13, 2010 7:52pm | Edited 8:49pm
Quoting SwingMan
Hello Huston. We have a problem!

I suppose that nobody has checked the correlation formulas in the pairs trading robot. I don't know who is the autor (Kang Gun, Scooby-doo or Steve), but the calculated correlation values are false!
......

After few hours of testing, I think that the explanation for this big bug is in the "symboldif" function, because using the "iMA" here is false. You can see as test-example the correct using of the MA in my code.
[code]double symboldif(string symbol,...
What do you mean "false"? Contrary to what a conventional computation is, sure, but why is a conventional computation "right" for trading?

The function symboldif() is a highpass filter and therefore the purpose of the test is to look for short term correlation to make sure that the cause of it is underlying JPY movement: the point is to trade yen, not the other stuff.

More mathishly, the standard correlation coefficient (where you subtract off the observed mean) is relevant for statistically stationary processes. Prices alone (i.e. using iClose()) aren't so. High pass filter as scooby did makes it more so---as opposed to subtracting off mean of an observation period so that the answer depends on the length of the observation period in a nonstationary process.

It's more important to be right than to to be consistent. It's not a bug, it's a feature.
  • #5,949
  • Feb 14, 2010 12:56am Feb 14, 2010 12:56am
I know i replied to this sort of late, this was discussed long ago.

however, i have to say, your suggestion seems exactly contrary to what Scooby proposes, and no longer serves the original filter purpose. or is that your typo?

Quoting sidhujag
Problem with that is if you are trading away from 200 ema too far you will get screwed by a retrace... which usually is the case... trading around the ema is safer... try this:

double Gap = iATR(symbol,1440,1,0) ;
if (EMA_TF1_20 > (EMA_TF1_200-Gap) && MathAbs(EMA_TF1_20-EMA_TF1_200)<=Gap) return( buy);
if (EMA_TF1_20 < (EMA_TF1_200+Gap) && MathAbs(EMA_TF1_20-EMA_TF1_200)<=Gap) return(sell);

This seems to avoid drawdown more and you can get out if price retreats under the EMA farther than you like. What do you think?

Jag
  • #5,950
  • Edited 2:43am Feb 14, 2010 2:33am | Edited 2:43am
Quoting mbkennel
What do you mean "false"?...
Thanks mbkennel,

 -highpass filter, stationary, nonstationary processes, wow!, are you mathematician? I’m not…!??..

-Quote: “It's more important to be right than to be consistent”

The calculation is surely consistent, but not right. Maybe have you not analysed the code.
For me and other millions people 2+2=4, but if you trade consistent succesfully with the formula 2+2=5, this is maybe your trading advantage.
If from the Mataf website or with my test program we get a negative correlation of ca. –0.10, and you trade with the robot formula for a correlation value = 0.93, maybe is this a counter trend trade, and here is the explanation for your consistent success(!), I don’t know.

Here a short explanation of the bug for no mathematicians:

-We have an array of prices (1, 2, 3 ,4 ,5) with the array indexes (4, 3, 2, 1, 0).
-and we will calculate the sum ( x(i) – avgX )

-On index 0 we have for the last three values the average
avg0 = (3+4+5) / 3 = 12 / 4 = 4
-We calculate the sum of differencies:
diff0 = (3-4) + (4-4) + (5-4) = -1 + 0 + 1 = 0;

The formula in the robot calculate falsely with iMA(…,i) for every index the average of the last N values for every index(!):
avg0 = (3+4+5) / 3 = 12 / 3 = 4
avg1 = (2+3+4) / 3 = 9 / 3 = 3
avg2 = (1+2+3) / 3 = 6 /3 = 2
-The sum of differencies is with this formula:
diff0 = (3-2) + (4-3) + (5-4) = 1 + 1 + 1 = 3

Now is evident that the calculated values with the robot formula are greather as the mathematical correct values to use in correlations formulas (“relevant for statistically stationary processes”…).

OK, I would like no longer continue the discution, for me still remains 2+2=4.
  • #5,951
  • Feb 14, 2010 3:50am Feb 14, 2010 3:50am
Quoting charvo
I know i replied to this sort of late, this was discussed long ago.

however, i have to say, your suggestion seems exactly contrary to what Scooby proposes, and no longer serves the original filter purpose. or is that your typo?

No it's what I meant. When price is that far away from the EMA > ATR daily range it usually wants to test the 200 ema before continuing down. A GAP may be sufficient for an entry but a second large difference should be used for safe measure against a retrace.... ideally you don't want to take short trades when it already has fallen x number of pips as you can expect it to go up a bit before it dips again if long term is bearish for example.

Jag
  • #5,952
  • Feb 14, 2010 3:53am Feb 14, 2010 3:53am
Quoting sidhujag
No it's what I meant. When price is that far away from the EMA > ATR daily range it usually wants to test the 200 ema before continuing down. A GAP may be sufficient for an entry but a second large difference should be used for safe measure against a retrace.... ideally you don't want to take short trades when it already has fallen x number of pips as you can expect it to go up a bit before it dips again if long term is bearish for example.

Jag
Thats why we only trade it when it is OB or OS in 3 seperate TF's.
  • #5,953
  • Feb 14, 2010 4:46am Feb 14, 2010 4:46am
Quoting mbkennel
I think that the scoobs stuff is great, but let's remember something on forward testing.

a) Stop is 100pts, take profit is 25. This means you need 4x as many winners as losers to break even, in the absence of trading costs. I.e. a mininum of 80% win rate---probably higher just to break even. It's short volatility, not long volatility.

..........
Which is why I am testing using the mptm trade manager which does not cut profitable trades at +25 but part closes positive positions, allowing the rest to run.

IMHO must significantly lower your stated minimum 80% win rate.

Thanks for your interesting post -
  • #5,954
  • Feb 14, 2010 6:07am Feb 14, 2010 6:07am
Quoting SwingMan
Thanks mbkennel,

 -highpass filter, stationary, nonstationary processes, wow!, are you mathematician? I’m not…!??..
physics, actually, but work I do now is more applied math & statistics.

-Quote: “It's more important to be right than to be consistent”

Quote
The calculation is surely consistent, but not right. Maybe have you not analysed the code....
stop right there. The underlying scientific question is "what is the meaning of 'x(i)'".

Scooby is intentionally not using x(i) = price, but x(i) =HighPassFilter[price] and then asserting that avgX = 0, which is reasonable because of the high pass filter. No it's not true as the *sample* average but it's true as the true population average (what you'd get with infinitely large amount of data in some theoretical sense).

Quote
Now is evident that the calculated values with the robot formula are greather as the mathematical correct values to use in correlations formulas (“relevant for statistically stationary processes”…).

OK, I would like no longer continue the discution, for me still remains 2+2=4.
This is statistics, not mathematics. There is judgement and empiricism in statistics---the mathematics which has correct and incorrect answers is called probability + stochastic processes.

You can compute a correlation coefficient of anything you damn want. The stochastic process side tells you how to do it and what it may mean in some circumstance (e.g. linear gaussian processes can be fully defined by covariance matrices). The statistical and scientific insight tells you WHAT you what to compute it on, and the answer here depends on "what are you trying to learn?"
  • #5,955
  • Feb 14, 2010 6:25am Feb 14, 2010 6:25am
Quoting mbkennel
physics, actually, but work I do now is more applied math & statistics....
There is no need for this discussion. I didnt wright the correlation function, I found it on MQL Forum. I have no idea if it is correct or not.

Keep the piece.
  • #5,956
  • Feb 14, 2010 6:36am Feb 14, 2010 6:36am
Quoting scooby-doo
There is no need for this discussion. I didnt wright the correlation function, I found it on MQL Forum. I have no idea if it is correct or not.

Keep the piece.
It does not appear to be bust in actual trading, so I suggest not fixing it.

  • #5,957
  • Feb 14, 2010 7:11am Feb 14, 2010 7:11am
I'm sorry friends that I've posted about something I am sure that is wrong.
I've posted a program to determine whether the values compare with Mataf are correct or not, expecting that someone would also like to test. Pity for the lost time.

@mbkennel, you read that @scooby-doo has taken over the formulas, and he do not own programmed or tested this. With your knowledge you can made some practical tests because this is in interest of all program users.
  • #5,958
  • Feb 14, 2010 7:15am Feb 14, 2010 7:15am
Quoting SwingMan
I'm sorry friends that I've posted about something I am sure that is wrong.
I've posted a program to determine whether the values compare with Mataf are correct or not, expecting that someone would also like to test. Pity for the lost time.

@mbkennel, you read that @scooby-doo has taken over the formulas, and he do not own programmed or tested this. With your knowledge you can made some practical tests because this is in interest of all program users.
Dear Swingman,

I am sorry that some members appear to be treating you with a little hostility.

It is concerning that you believe this to be wrong as I and many others on FF respect your opinion greatly.

Would it be possible for a change to be made in the indicators and to run both side by side? Perhaps this could be the reason why DD is so high at times?

Warm Regards,

fxtrader08
  • #5,959
  • Edited 7:50am Feb 14, 2010 7:38am | Edited 7:50am
At first glance, i'm thinking about different pairs's digits lenghts when we compare each others to calculate correlation.

i.e. there are 2, 3, 4, or 5 digits, depend on which pairs and which broker are used.
Maybe my thinkings are wrong, don't know.
  • #5,960
  • Feb 14, 2010 7:43am Feb 14, 2010 7:43am
Quoting macman
You need to understand Steve's MPTM to use this, so if you are not familiar with it's use, carefully study the manual.

This is probably not the best settings, but gotta start somewhere.

This is on a 5 digit broker.
Thanks. Will give it a go this week, I've looked over the transactions from the past week and it looks like Steve's MPTM improves profitability greatly.
0 traders viewing now