Optimized Trend Trading | Page 4 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Optimized Risk vs Reward Equation 335 replies

Loss Less Optimized Strategy 63 replies

Optimized Parameters in Metatrader Testing? 3 replies

"Optimized Parameter" feature in EA? 0 replies

Strategy with Optimized parameter 1 reply

Attachments: Optimized Trend Trading
Exit Attachments

Optimized Trend Trading

  • #61
  • Mar 23, 2008 4:19pm Mar 23, 2008 4:19pm
Quoting Codebreaker
I am helping you. Particularly, by NOT giving you a "system". You will learn nothing from that. Far better you work to understand the principles and then design, build and "own" your own system. Trust me.

CB
I could not agree more. Anything that is given is easily forgotten and not understood. Simba has always been very respectful of this on TSD with us. He has always helped us to "fish in the right pond". It's up to you after that. I for one appluad CB for helping us continue to learn. That is the only way to become successful.

Just my .02

All the best,

CL
  • #62
  • Mar 23, 2008 5:55pm Mar 23, 2008 5:55pm
Quoting Codebreaker
I am helping you. Particularly, by NOT giving you a "system". You will learn nothing from that. Far better you work to understand the principles and then design, build and "own" your own system. Trust me.

CB

You are absolutely right ,IMHO..please keep sharing your insights with us,and we will try to contribute a few tidbits of knowledge,mainly in the form of questions,so that the thread can be useful for everybody interested in LEARNING first,then applying...spoonfeeding is not a practical way to teach traders

Best Regards
Simba
Move like a shadow
  • #63
  • Edited 4:17am Mar 24, 2008 4:10am | Edited 4:17am
It is very encouraging to see what folk on this thread are already doing with digital filters -- clearly colleagues have the basics under their belt and understand the importatance of frequency response and phase as separate but related properties of (digital) filters. Also, it looks like the whingers and cynics have got bored and moved on to hassle other threads, leaving a hard core of serious folk who between us, I am confident, can design some excellent systems. Although I know a great deal about signal processing I am a newbie to FX and that side is equally important for success, I feel. So, a few more needed ideas...

Order of Integration

We have already talked about financial time series being non-stationary (ie the cumulants of their probability distributions such as mean, variance, etc vary with time). We can measure the degree of non-stationarity through the concept of integration order. In a discrete-sampled system, we can approximate the derivative of a series simply by taking successive differences, eg, d[t] = x[t] - x[t-1]. Try this - generate a simple sampled sine wave s[t] = sin(2.pi.t/P) where P is any period you want, say 20 bars, and take the differences d[t] = s[t] - s[t-1]. If you plot this on the same chart, you will get a near-perfect cosine wave. Why, because d/dt{sin(t)} = cos(t).

Once we have the difference series d[t], we can take the differences again to get d2[t] and so on. We DEFINE a stationary series to have integration order zero, written I(0). If we have a non-stationary time series but when we take the first differences we get a stationary series, the original time series is said to be "integrated to order 1", written I(1). Why? Because if we differentiate an I(1) series once (ie undo the integration) we get an I(0) series. Likewise, an I(2) series would need to be differenced (ie differentiated) twice to give an I(0) series.

Most single financial instruments such as individual shares or FX pairs behave almost as I(1) series. On the other hand, indices such as the Dow etc which are made up a large numbers of individual assets behave as higher order integrated series. (This is why indices are much harder to trade but that is a story for another day...). If you take your favourite pair/timeframe, say GBPJPY 5-min data, and take successive differences and plot them, you will get a noisy looking series centered almost about zero. If you compute the variance in various windows, it is much the same everywhere. Although this is a far from rigourous test for stationarity, it is good enough for present purposes (to be sure you would need to conduct a proper unit-root test and use the T-statistic to accept or reject the presence of a unit root at a given level of statistical significance).

So the FX data that interests us is generally I(1). Without going into too much detail here (the topic is very large) there exist several different kinds of non-stationarity. Some types of I(1) series are non-stationary because they are random -- the "random walk" is a perfect example. It has a fractal dimension of 1.5 (or equivalently a Hurst exponent of 0.5) and is, as it says, random.

[NOTE: We need to cover fractal properties in a later post].

A random walk cannot be predicted and it cannot be successfully traded. Other I(1) series are said to be "trend-stationary", that is, if we can find and remove the trend component, we are left with a stationary series. (Refer to my previous major post on this). Here, I mean a "real" trend, not an "accidental" one. Even random walks have nice looking "trends" in them but they are just statistical accidents and subvert the unwary into trading and losing money!! This is the basis for testing the success of my filter - if I subtract my filter output (which I call my "trend") from the original data, I want whatever is left to be a stationary series.

Stochastic Volatility

The other idea we will outline this time is stochastic volatility. If you use a nice charting tool such as those found in TradeStation etc and in the IG-Index Java trading platform, everyone will have noticed that as you look at the same FX pair at different time-frames, the series looks very noisy at some resolutions and much nicer and smoother at others. The smoothness does not just increase with timeframe -- I have seen the GBPJPY look pretty horrible at M1, M3, M5, M10 then look really nice at M15 but then go back to pretty nasty at M30, H1. This behaviour is due to a phenomenon called "volatility clustering". Folk who recommend trading using charts on multiple time frames are (whether they realise it or not) exploiting this property and it is a good idea. The amount of actual information, of course, increases as the resolution falls -- if you have the M1 data you can easily reconstruct the M3, M5, H1, H4, ..., etc data, but if you only have the H4 data you can't recover the information at finer time scales. However, the "mutual information" is quite different (we touched on this much earlier in this thread). If you compute the Shannon Mutual Information at different timescales, you will see that the curve has dips (actually peaks, but I use an algorithm that computes things upside-down for reasons that don't concern us here). What are we looking for here? We want to find something we can actually trade!! We want a genuine trend we can latch onto and avoid the dangerous shoals of false trends and ranging markets.

Think of this in terms of a communication system. The transmitter can send either a 1 or a 0 and the job of the recevier is to decide whether a 1 or a 0 was sent. (This is an accurate description of any modern digital comms system, eg, mobile phones). The radio channel adds noise and randomly affects the signal, sometimes changing 0's into 1's and vice versa. If the channel is noise free and we always receive a 1 when a 1 was sent (same for 0) then everything is great. You might think the worst case would be a very noisy channel that always changes 0's for 1's (and vv). But this is easy to handle too -- just invert everything at the receiver! The worst case is actually the channel where 1's are changed to 0's randomly so the receiver can no better decide what symbol was sent than by tossing a fair coin! In digital comms systems, we use powerful "error correcting codes" to code the information prior to transmission to combat the effects of the radio channel, whose properties are quite well known in advance.

So in our trading system, if we have our FX data at a given timescale whose mutual information is close to 0.5, we can no better decide whether to buy or sell (go long or short) than by tossing a coin. WE CANNOT TRADE THIS INSTRUMENT AT THIS TIMEFRAME!! However, if we find a timeframe where the mutual information is biased away from 0.5 (either up or down) then we are in with a statistical chance -- we have an edge and that is all we need. We do not need to win every battle so long as we win the war!! You will have seen from the equity curves of my robot that he is by no means always right, but that overall the equity curve is inexorably upwards!

So in summary.... We know the mathematical behaviour of our time series in terms of its integration order and hence can design filters to extract information we want. We also know that we have to pick our timeframe so that the stochastic nature of the volatility clustering works in our favour. I have described my own "sniffer" software earlier in this thread. Because I have a little account at IG-Index, it is essential I analyse their actual numbers since SB data is so biased from the "real" market. I can set my sniffer to any timescale I want and am currently exploring 4min 17 sec data on one instrument because of the reasons outlined above.

There are not too many more pieces of the jigsaw! Folk who are following this thread, Googling the stuff they are unfamiliar with and doing their research will now be almost ready to put together a system not a million miles away from mine (which currently returns around 25% a week on a small account -- see my posts of the trade files and equity curves)

All the best

CB.
  • #64
  • Mar 24, 2008 4:29am Mar 24, 2008 4:29am
This topic is in the calibre of an academic book and it deserves to be sticked on the top.Thanks Codebreaker, excellent reading , 5 star topic .Also good to see Simba & Co in here.
  • #65
  • Mar 24, 2008 7:09am Mar 24, 2008 7:09am
Just on a cautionary note further to my last post.... When I talk about the results from my robot, I mean that "...for the last 6 weeks since the robot went live, I have made >25% per week...".

Do I now "have a system"? Yes and No. I have something that sure "seems" to work but I don't yet trust it. I do not have sufficient trade history yet to be able to decide whether the results I am seeing are because the robot genuinely has an edge in the FX market or whether he has just had a lucky streak (as many gamblers do).

To decide, we must use another statistic, this time, the chi-squared distribution. (Please be careful you use the right statistics for the right things!! For the unit root test we needed the t-statistic. For other things we will need the F-statistic, and so on!!)

When you look at a table of the chi-squared distribution, you need to specify the number of "degrees of freedom" (dof) of whatever you are trying to measure. This is just jargon for the number of things you can change. Don't ever fall into the trap of building a trading system with hundreds of variables you can tweak. Of course it will fit whatever trade history you want! You can ALWAYS put an (N-1)th order polynomial through N points perfectly but you have proved nothing! The question is whether you can put, say, a quadratic though 100 points. If so, you have found a true relationship. Those who advocate the KISS principle in trading are absolutely right for provable mathematical reasons.

My trading system essentially has 2 filter parameters I can change (answering Qu. 1 of CL's post #57). Therefore, if I want to be able to say with some statistical validity - say the 99% confidence level - that my system is achieving the results it is because I am doing something right and NOT by sheer good luck, then I need to look at the chi-squared tables for 2 dof at, say, the 99% level. I can now read off a number that tells me essentially the amount of trades I need to push though the system to be safe in my belief.

I am not there yet. Most statisticians would demand at least 30 decisions for each degree of freedom at the 95% level so you can easily work out the number of months of history needed at X trades per day. Please be sure you test thoroughly before putting the old hard-earned shekels on the markets!!

CB.
  • #66
  • Mar 24, 2008 11:11am Mar 24, 2008 11:11am
Quoting Codebreaker
It is very encouraging to see what folk on this thread are already doing with digital filters -- clearly colleagues have the basics under their belt and understand the importatance of frequency response and phase as separate but related properties of (digital) filters. Also, it looks like the whingers and cynics have got bored and moved on to hassle other threads, leaving a hard core of serious folk who between us, I am confident, can design some excellent systems. Although I know a great deal about signal processing I am a newbie to FX and that side is equally important for success, I feel. So, a few more needed ideas...

Order of Integration

We have already talked about financial time series being non-stationary (ie the cumulants of their probability distributions such as mean, variance, etc vary with time). We can measure the degree of non-stationarity through the concept of integration order. In a discrete-sampled system, we can approximate the derivative of a series simply by taking successive differences, eg, d[t] = x[t] - x[t-1]. Try this - generate a simple sampled sine wave s[t] = sin(2.pi.t/P) where P is any period you want, say 20 bars, and take the differences d[t] = s[t] - s[t-1]. If you plot this on the same chart, you will get a near-perfect cosine wave. Why, because d/dt{sin(t)} = cos(t).

Once we have the difference series d[t], we can take the differences again to get d2[t] and so on. We DEFINE a stationary series to have integration order zero, written I(0). If we have a non-stationary time series but when we take the first differences we get a stationary series, the original time series is said to be "integrated to order 1", written I(1). Why? Because if we differentiate an I(1) series once (ie undo the integration) we get an I(0) series. Likewise, an I(2) series would need to be differenced (ie differentiated) twice to give an I(0) series.

Most single financial instruments such as individual shares or FX pairs behave almost as I(1) series. On the other hand, indices such as the Dow etc which are made up a large numbers of individual assets behave as higher order integrated series. (This is why indices are much harder to trade but that is a story for another day...). If you take your favourite pair/timeframe, say GBPJPY 5-min data, and take successive differences and plot them, you will get a noisy looking series centered almost about zero. If you compute the variance in various windows, it is much the same everywhere. Although this is a far from rigourous test for stationarity, it is good enough for present purposes (to be sure you would need to conduct a proper unit-root test and use the T-statistic to accept or reject the presence of a unit root at a given level of statistical significance).

So the FX data that interests us is generally I(1). Without going into too much detail here (the topic is very large) there exist several different kinds of non-stationarity. Some types of I(1) series are non-stationary because they are random -- the "random walk" is a perfect example. It has a fractal dimension of 1.5 (or equivalently a Hurst exponent of 0.5) and is, as it says, random.

[NOTE: We need to cover fractal properties in a later post].

A random walk cannot be predicted and it cannot be successfully traded. Other I(1) series are said to be "trend-stationary", that is, if we can find and remove the trend component, we are left with a stationary series. (Refer to my previous major post on this). Here, I mean a "real" trend, not an "accidental" one. Even random walks have nice looking "trends" in them but they are just statistical accidents and subvert the unwary into trading and losing money!! This is the basis for testing the success of my filter - if I subtract my filter output (which I call my "trend") from the original data, I want whatever is left to be a stationary series.

Stochastic Volatility

The other idea we will outline this time is stochastic volatility. If you use a nice charting tool such as those found in TradeStation etc and in the IG-Index Java trading platform, everyone will have noticed that as you look at the same FX pair at different time-frames, the series looks very noisy at some resolutions and much nicer and smoother at others. The smoothness does not just increase with timeframe -- I have seen the GBPJPY look pretty horrible at M1, M3, M5, M10 then look really nice at M15 but then go back to pretty nasty at M30, H1. This behaviour is due to a phenomenon called "volatility clustering". Folk who recommend trading using charts on multiple time frames are (whether they realise it or not) exploiting this property and it is a good idea. The amount of actual information, of course, increases as the resolution falls -- if you have the M1 data you can easily reconstruct the M3, M5, H1, H4, ..., etc data, but if you only have the H4 data you can't recover the information at finer time scales. However, the "mutual information" is quite different (we touched on this much earlier in this thread). If you compute the Shannon Mutual Information at different timescales, you will see that the curve has dips (actually peaks, but I use an algorithm that computes things upside-down for reasons that don't concern us here). What are we looking for here? We want to find something we can actually trade!! We want a genuine trend we can latch onto and avoid the dangerous shoals of false trends and ranging markets.

Think of this in terms of a communication system. The transmitter can send either a 1 or a 0 and the job of the recevier is to decide whether a 1 or a 0 was sent. (This is an accurate description of any modern digital comms system, eg, mobile phones). The radio channel adds noise and randomly affects the signal, sometimes changing 0's into 1's and vice versa. If the channel is noise free and we always receive a 1 when a 1 was sent (same for 0) then everything is great. You might think the worst case would be a very noisy channel that always changes 0's for 1's (and vv). But this is easy to handle too -- just invert everything at the receiver! The worst case is actually the channel where 1's are changed to 0's randomly so the receiver can no better decide what symbol was sent than by tossing a fair coin! In digital comms systems, we use powerful "error correcting codes" to code the information prior to transmission to combat the effects of the radio channel, whose properties are quite well known in advance.

So in our trading system, if we have our FX data at a given timescale whose mutual information is close to 0.5, we can no better decide whether to buy or sell (go long or short) than by tossing a coin. WE CANNOT TRADE THIS INSTRUMENT AT THIS TIMEFRAME!! However, if we find a timeframe where the mutual information is biased away from 0.5 (either up or down) then we are in with a statistical chance -- we have an edge and that is all we need. We do not need to win every battle so long as we win the war!! You will have seen from the equity curves of my robot that he is by no means always right, but that overall the equity curve is inexorably upwards!

So in summary.... We know the mathematical behaviour of our time series in terms of its integration order and hence can design filters to extract information we want. We also know that we have to pick our timeframe so that the stochastic nature of the volatility clustering works in our favour. I have described my own "sniffer" software earlier in this thread. Because I have a little account at IG-Index, it is essential I analyse their actual numbers since SB data is so biased from the "real" market. I can set my sniffer to any timescale I want and am currently exploring 4min 17 sec data on one instrument because of the reasons outlined above.

There are not too many more pieces of the jigsaw! Folk who are following this thread, Googling the stuff they are unfamiliar with and doing their research will now be almost ready to put together a system not a million miles away from mine (which currently returns around 25% a week on a small account -- see my posts of the trade files and equity curves)

All the best

CB.
CB,

Thanks for a wonderful post. I want to make sure I am understanding the concept you are describing. Please see an attached screenshot of an Excel Spreadsheet.

On the left hand side you see our FX data ( EURJPY 1Hr). This is our I(1). It's normal, non stationary data. In order to get some usefullness out of it, we have to integrate the data according to d[t]=x[t] - x[t-1]. This result will be our stationary series, or I(0). This is found on the right hand side of the screen capture. The difference between the open, high, low, and close.

If we have a non-stationary series and a stationary series we can find a tradable "trend". I(1) - Some Filter = I(0). Working backwards, our Filter, or tradable "trend" is I(1) - I(0)? I want to make sure i have this correct.

You did say that a stationary series ( I(0) ) plus a non-stationary series ( our original FX data) results in something non-stationary. Since we aren't adding I(0) & I(1) i wanted to make sure i am on the right track.

Great post and information. This thread is very worthwhile as a teaching aid. Please correct me if i have misunderstood your explanations.

Best,

cl
Attached Image (click to enlarge)
Click to Enlarge Name: Stationary.jpg Size: 440 KB
  • #67
  • Edited 12:06pm Mar 24, 2008 12:01pm | Edited 12:06pm
Hi CL

You are right on track....

On the left hand side you see our FX data ( EURJPY 1Hr). This is our I(1). It's normal, non stationary data. In order to get some usefullness out of it, we have to integrate the data according to d[t]=x[t] - x[t-1].

>> Substitute "differentiate" for "integrate" and you are right.

This result will be our stationary series, or I(0). This is found on the right hand side of the screen capture. The difference between the open, high, low, and close.

>> Yes.

If we have a non-stationary series and a stationary series we can find a tradable "trend". I(1) - Some Filter = I(0). Working backwards, our Filter, or tradable "trend" is I(1) - I(0)? I want to make sure i have this correct.
You did say that a stationary series ( I(0) ) plus a non-stationary series ( our original FX data) results in something non-stationary. Since we aren't adding I(0) & I(1) i wanted to make sure i am on the right track.

>> Market data [I(1)] = Wanted trend [I(1)] + Unwanted noise [I(0)] so we build a filter to pull out the I(1) trend. How do we know when we have succeeded? Because what is left is pure I(0). You can test for I(0) using a Dickey-Fuller test (which you can do in Excel -- look on the Web). Here is the link:

http://foquant.com/wp-content/upload...er-example.doc

http://en.wikipedia.org/wiki/Unit_root

CB
  • #68
  • Mar 24, 2008 2:16pm Mar 24, 2008 2:16pm
Quoting Codebreaker
Hi CL

You are right on track....

On the left hand side you see our FX data ( EURJPY 1Hr). This is our I(1). It's normal, non stationary data. In order to get some usefullness out of it, we have to integrate the data according to d[t]=x[t] - x[t-1].

>> Substitute "differentiate" for "integrate" and you are right.

This result will be our stationary series, or I(0). This is found on the right hand side of the screen capture. The difference between the open, high, low, and close.

>> Yes.

If we have a non-stationary series and a stationary series we can find a tradable "trend". I(1) - Some Filter = I(0). Working backwards, our Filter, or tradable "trend" is I(1) - I(0)? I want to make sure i have this correct.
You did say that a stationary series ( I(0) ) plus a non-stationary series ( our original FX data) results in something non-stationary. Since we aren't adding I(0) & I(1) i wanted to make sure i am on the right track.

>> Market data [I(1)] = Wanted trend [I(1)] + Unwanted noise [I(0)] so we build a filter to pull out the I(1) trend. How do we know when we have succeeded? Because what is left is pure I(0). You can test for I(0) using a Dickey-Fuller test (which you can do in Excel -- look on the Web). Here is the link:

http://foquant.com/wp-content/upload...er-example.doc

http://en.wikipedia.org/wiki/Unit_root

CB
CB,

I have followed the word document. I am getting some sort of error. The values of the excel fine are "paste special" values ( so they aren't formulas). Any insight into what i may be doing wrong?

Thanks,

cl
Attached Image(s) (click to enlarge)
Click to Enlarge Name: before.jpg Size: 323 KB
Click to Enlarge Name: after.jpg Size: 314 KB
  • #69
  • Mar 24, 2008 2:31pm Mar 24, 2008 2:31pm
Quoting cl3307
CB,

I have followed the word document. I am getting some sort of error. The values of the excel fine are "paste special" values ( so they aren't formulas). Any insight into what i may be doing wrong?

Thanks,

cl
I may have figured it out. Sorry for the repetitive post. Please see the attached pics.

I feel like it's a couple of years ago and i'm back in college business stats. :-)

So, i get a t statistic ( based on i did the right computations) of -1.37226. I used 100 observations. Since -1.37226 > 2%(-3.51),5%(-3.17) and 10%(-2.58), we can't reject the Null that there is a unit root. So, we now have a Stationary set of data and it's correct? (whew!)

Thanks,

cl
Attached Image(s) (click to enlarge)
Click to Enlarge Name: before2.GIF Size: 88 KB
Click to Enlarge Name: after2.jpg Size: 288 KB
  • #70
  • Edited 8:02am Mar 25, 2008 6:31am | Edited 8:02am
Quoting cl3307
I may have figured it out. Sorry for the repetitive post. Please see the attached pics.

I feel like it's a couple of years ago and i'm back in college business stats. :-)

So, i get a t statistic ( based on i did the right computations) of -1.37226. I used 100 observations. Since -1.37226 > 2%(-3.51),5%(-3.17) and 10%(-2.58), we can't reject the Null that there is a unit root. So, we now have a Stationary set of data and it's correct? (whew!)

Thanks,

cl
Clahn,

I know about this less than you,so take my comments as just trying to help.
From reading the doc..it seems to me that you should have used x= $C$3 to $C$103 with your y=$D$4 to $D$104..since you should regress deltaprice on lagged values of price...IMHO

I have done the regression on deltaprices versus lagged prices for Weekly GBPJPY data,since January 78 until 23 March 2008 ,total 1578 observations,and with a t-stat of 1.2391 for the interceptor and -1.9201 for the original time series,please see attached,I can`t reject the hypothesis that there is a unit root,and consequently,the original time series should be considered as non stationary....Now,I don`t know how to use this knowledge..yet ..and I have a question,better seem stupid once than be ignorant forever,so,I have to ask..

CB:According to my interpretation of the t-statistics..1.2391 for interceptor and -1.9201 for original time series means that BOTH series are non stationary...could you please comment on it?And,if that were the case,we still do not have I(0),so we need to do a difference of the differences?

Regards
Simba
Attached Image (click to enlarge)
Click to Enlarge Name: TSTATS.jpg Size: 119 KB
Move like a shadow
  • #71
  • Mar 25, 2008 7:47am Mar 25, 2008 7:47am
Quoting Simbafx
Clahn,

I know about this less than you,so take my comments as just trying to help.
From reading the doc..it seems to me that you should have used x= $C$3 to $C$103 with your y=$D$4 to $D$104..since you should regress deltaprice on lagged values of price...IMHO

I have done the regression on deltaprices versus lagged prices for Weekly GBPJPY data,since January 78 until 23 March 2008 ,total 1578 observations,and with a t-stat of 1.2391,please see attached,I can`t reject the hypothesis that there is a unit root,and consequently,the original time series should be considered as non stationary....Now,I don`t know how to use this knowledge..yet

Regards
Simba
I agree actually. My problem is when i tried to do the regression against columns ($D:$D) i got an error. When i specified a lag i also got an error. Perhaps i did something wrong. I will retry right now then keep typing... :-)

I get the same error as above. The first entry is blank on the Y ( the change in N). See what you think. I'm guessing it's user error on my end :-)

cl
Attached Image (click to enlarge)
Click to Enlarge Name: mistake.jpg Size: 96 KB
  • #72
  • Mar 25, 2008 7:53am Mar 25, 2008 7:53am
Quoting cl3307
I agree actually. My problem is when i tried to do the regression against columns ($D:$D) i got an error. When i specified a lag i also got an error. Perhaps i did something wrong. I will retry right now then keep typing... :-)

I get the same error as above. The first entry is blank on the Y ( the change in N). See what you think. I'm guessing it's user error on my end :-)

cl
I see what you mean now Simba. Thanks for the help. Attached is the pic of the current output. T is postive now, which means we are still non stationary ( i think)...back to the drawing board :-)

Best,

cl
Attached Image (click to enlarge)
Click to Enlarge Name: woops.jpg Size: 276 KB
  • #73
  • Mar 25, 2008 11:17am Mar 25, 2008 11:17am
Hi Chaps

Simba is absolutely right! There are two lessons here. (1) Rigourous tests for stationarity are notoriously difficult and need lots of data, (2) We are seeing a phenomenon we touched on earlier at these longer time frames (I think CL used 1H and Simba used 1W).

When we write an integration mathematically, we use a symbol like a long curly "S". This is exactly what the symbol is! 400 years ago the "S" in English was always written like this (like you see on signs in Stratford-Upon-Avon eg "Ye Olde Englishe Tea Shoppe", Ugh! to make the place kinda pseuso-Shakespearian). The original integral sign was shorthand for "Sum". This is exactly what integration is in a discrete series. Differentiation is just subtraction and integration is just addition.

Hence, a simple moving average that just adds up the last N samples (and divides by N to normalise) is just integrating the data. This is why the MA has lag. Rules:

- INTEGRATION
(a) Smooths
(b) Has phase LAG
e.g. MA smooths but has lag.

- DIFFERENTIATION
(a) Unsmooths
(b) Has phase lead
e.g. MOMENTUM leads but is noisy.

Just the laws of physics in action!

Since you have both established that you cannot reject the hypothesis of a unit root on the differenced data, we have to believe that the order of integration of your original series was greater than I(1). Given the timescales you have, this is likely to be so. Why?

Data at shorter timescales is generally I(1). When we make longer timescales from shorter (integration), we raise the integration order of the series. PROBLEM!! The integration order might be fractional!!! This is by no means an obvious idea, but it might be say I(1.2). So when differenced once you get I(0.2) which is ALMOST stationary but still enough to cause the test to fail.

This is the deep mathematical reason why different timeframes behave differently when trading and why MTF analysis can be very insightful. This is a very deep topic but goes right to the heart of WHY trading an instrument at one time frame "works" and at another "fails".

CB.
  • #74
  • Mar 25, 2008 3:52pm Mar 25, 2008 3:52pm
Quoting Codebreaker
Hi Chaps

Simba is absolutely right! There are two lessons here. (1) Rigourous tests for stationarity are notoriously difficult and need lots of data, (2) We are seeing a phenomenon we touched on earlier at these longer time frames (I think CL used 1H and Simba used 1W).

When we write an integration mathematically, we use a symbol like a long curly "S". This is exactly what the symbol is! 400 years ago the "S" in English was always written like this (like you see on signs in Stratford-Upon-Avon eg "Ye Olde Englishe Tea Shoppe", Ugh! to make the place kinda pseuso-Shakespearian). The original integral sign was shorthand for "Sum". This is exactly what integration is in a discrete series. Differentiation is just subtraction and integration is just addition.

Hence, a simple moving average that just adds up the last N samples (and divides by N to normalise) is just integrating the data. This is why the MA has lag. Rules:

- INTEGRATION
(a) Smooths
(b) Has phase LAG
e.g. MA smooths but has lag.

- DIFFERENTIATION
(a) Unsmooths
(b) Has phase lead
e.g. MOMENTUM leads but is noisy.

Just the laws of physics in action!

Since you have both established that you cannot reject the hypothesis of a unit root on the differenced data, we have to believe that the order of integration of your original series was greater than I(1). Given the timescales you have, this is likely to be so. Why?

Data at shorter timescales is generally I(1). When we make longer timescales from shorter (integration), we raise the integration order of the series. PROBLEM!! The integration order might be fractional!!! This is by no means an obvious idea, but it might be say I(1.2). So when differenced once you get I(0.2) which is ALMOST stationary but still enough to cause the test to fail.

This is the deep mathematical reason why different timeframes behave differently when trading and why MTF analysis can be very insightful. This is a very deep topic but goes right to the heart of WHY trading an instrument at one time frame "works" and at another "fails".

CB.
I have a quick question about taking the difference of the difference. When we do the regression on the first DeltaY, that is very straight forward. If we are doing 1000 observations for example, and Close price is in Column A and Change in Y is in Column B, our input for the regression would look something like this....

Input X Range : A1:A1001
Input Y Range : B2:B1002

Our change in Y is lagged by 1 against price data. When you go to do it a second time, is the lag 2 against the original Y, or would you index Y by 1 so you are always lagging the Change in Y by 1 against price? In other words...


Input X Range : A1:A1001
Input Y Range : B3:B1003

or


Input X Range : A2:A1002
Input Y Range : B3:B1003

Hopefully a constructive question... :-)

Thanks,

cl
  • #75
  • Mar 25, 2008 4:10pm Mar 25, 2008 4:10pm
Attached is the output for a EurJpy 5 min analysis. The first regression is the regression of Change in Y against Close. The Second REgression is the Second Change in Y against Close.

I'm not for sure if the first works ( the T stat approves the 10%, but not the others....maybe it's the I[1.2] you were talking about...).

The Second yields a t stat of -3.25, which is less than the 5%.

Perhaps i am getting ahead of myself in my understanding of this. What i take this to mean is that the second Change in Y is now Stationary. If we can subtract this from the original Price Data ( Non Stationary), we will have our filter?

All the best,

cl
Attached Image(s) (click to enlarge)
Click to Enlarge Name: First Regression.GIF Size: 21 KB
Click to Enlarge Name: Second Regression.jpg Size: 275 KB
  • #76
  • Mar 26, 2008 5:48am Mar 26, 2008 5:48am
Quoting cl3307
I have a quick question about taking the difference of the difference. When we do the regression on the first DeltaY, that is very straight forward. If we are doing 1000 observations for example, and Close price is in Column A and Change in Y is in Column B, our input for the regression would look something like this....

Input X Range : A1:A1001
Input Y Range : B2:B1002

Our change in Y is lagged by 1 against price data. When you go to do it a second time, is the lag 2 against the original Y, or would you index Y by 1 so you are always lagging the Change in Y by 1 against price? In other words...


Input X Range : A1:A1001
Input Y Range : B3:B1003

or


Input X Range : A2:A1002
Input Y Range : B3:B1003

Hopefully a constructive question... :-)

Thanks,

cl
CB,

I have a question,in addition to clahn 2 questions..or X=Deltaprice...Y=Deltadeltaprice?

Thanks
Simba
Move like a shadow
  • #77
  • Mar 26, 2008 6:40am Mar 26, 2008 6:40am
Quoting Simbafx
CB,

I have a question,in addition to clahn 2 questions..or X=Deltaprice...Y=Deltadeltaprice?

Thanks
Simba
I was wondering the same thing to....i figured the t stat for the intercept should be negative....

cl
  • #78
  • Mar 27, 2008 2:36pm Mar 27, 2008 2:36pm
Anyone around ? :-)

This is too good of a thread to let it die....

cl
  • #79
  • Mar 27, 2008 2:53pm Mar 27, 2008 2:53pm
Quoting cl3307
Anyone around ? :-)

This is too good of a thread to let it die....

cl
I agree but it is also a challenging one (at least for me).
Looking at post 63, I have done the calculation from the word doc but then I don't know if I have a set of data that is stationary or not. I also don't know what's next with the stochastic volatility and the mutual information
Any idea...??
Thanks

Guiom
  • #80
  • Edited 5:25pm Mar 27, 2008 5:19pm | Edited 5:25pm
Quoting cl3307
Anyone around ? :-)

This is too good of a thread to let it die....

cl
I wanted to post a thought i had...

In rereading this post, CB you described your process as taking the OHLC stream, using the ULLMA on it, then doing a HA Calc, and finally smoothing it again with the ULLMA.

The OHLC you refer to i'm guessing is the "matched filter" OHLC that we create from the original OHLC ( pulling out the noise)?

Also, i may be doing something wrong....but on a 1hour time frame, i did the d{t} about 5 times, and was getting no closer to a valid t state....when you take the difference of a difference of a difference etc.....the # starts to get very big...please see my attachment..( i think i just used 5 min for the example...but in doing it with 1 hour data the dickey fuller was failing)...


cl

Thanks,

cl
0 traders viewing now