Menu

Options

Bookmark Thread

First Page First Unread Last Page Last Post

Print Thread

Similar Threads

Optimized Risk vs Reward Equation 335 replies

Loss Less Optimized Strategy 63 replies

Optimized Parameters in Metatrader Testing? 3 replies

"Optimized Parameter" feature in EA? 0 replies

Strategy with Optimized parameter 1 reply

Attachments: Optimized Trend Trading
Exit Attachments

Optimized Trend Trading

Quoting Codebreaker
Hi Guys


Chirps etc --

Heping Pan -- I liked his work on intermarket influences. I reproduced his NN for getting an 83% correct prediction of the following day's close on the AORD. It works fine due to the orthogonal time zone with the US. It might be interesting to look for intermarket influences on FX.

CB
I haven't tried using the "quote" feature yet so please excuse the note if it appears a bit jumbled.

Thanks for your description of how you use chirp signals.

Re: Heping Pan --

That result he got forecasting the direction of the SPASX or AORD on a daily basis intrigued me as well. It's a very concrete result. What I'm most curious about though is whether or not the return can be captured. Here's the problem. Because of the time difference -- as you correctly point out -- if the US & European markets are up strong, we can definitely expect the Australian (and probably Japanese as well) markets to rally on the day.

However, won't they just gap up at the open in that scenario?

You will correctly forecast the direction with near certainty but not be able to capture the return.

FYI,
Ngawang
Ahh yes!! But then I trade my favourite trade of all -- the Binary Bet. Just UP or DOWN. What could be simpler than that? Pips for old rope.....

CB
Quoting Codebreaker
One is high frequency trading which is perfectly suited to FX (indeed it is only applicable to FX) and uses the tick-by-tick data to uncover meta-beliefs held by market makers. That is, by looking at trade prices and MM reactions to them (up/down/same) and the time taken to revise their quote, we can learn a lot about what THEY think WE think that THEY think. More on this in future.
CB.
You certainly got me interested here. Trying to hack how MM thinks (or how their automated price making algorithms runs) is definitely a most interesting subject.

If you have anything to add to this I for sure would like to hear it (or if you have e.g. any links to relevant articles regarding this).
Quoting Codebreaker
Ahh yes!! But then I trade my favourite trade of all -- the Binary Bet. Just UP or DOWN. What could be simpler than that? Pips for old rope.....

CB
If you're using binary bet , there's no system inside ,right ?
{Promotion Removed}
Hi Codebreaker

When time allow would you check your PM.

Thanks
Quoting BlackMage
You certainly got me interested here. Trying to hack how MM thinks (or how their automated price making algorithms runs) is definitely a most interesting subject.

If you have anything to add to this I for sure would like to hear it (or if you have e.g. any links to relevant articles regarding this).
Hi Guys,

As I posted earlier in this thread, this is worth a read...

http://home.uchicago.edu/~lhansen/survey.pdf

CB
Hi Guys

I am getting PM'd to death with requests about filter design and optimisation. So here is a little math that will enable you to design great trading filters like mine.

Remember that we are taking about FIR (Finite Impulse Response) filters here, like Simple Moving Averages (SMA). We are NOT using IIR (Infinite Impulse Response) filters like Exponential Moving Averages (EMA). This is because, while IIRs are very easy to implement and require almost no data history, they have horrible phase response. This is why SMA's are almost univerally preferred to EMA's.

An FIR filter simply CONVOLVES the input time series (price data) with its IMPULSE RESPONSE. The impulse response is just a set of weights (or "coefficients") that multiply each data point. Then you just add up all the products and divide by the sum of the weights and that is it.

e.g. For a 10-bar SMA you just add up 10 bars of price data (each multiplied by 1) and divide by 10. For a weighted-MA you add up the product of the price data with triangular-number weights and divide by the total weight.

In my filters, I try to design the weights to have MAXIMUM possible smoothing and MINIMUM possible lag compatible with as-flat-as-possible phase response.

So given the weights, how do we find the phase response etc? We just go to the Z-domain where Z is a "delay" operator of 1 bar. So if p is price, p*z^(-1) is the previous price, p*z^(-2) is the price before that and so on.

Then we use the (admittedly rather complex) relationship between the Z-operator and the s-operator of the Laplace Transform and then we are firmly in the world of differential equations that we can solve. The formulae are given below, together with the frequency, phase and lag responses of some filters. Here, zeta is the coefficient (weight) and s is the length of the filter.

Note that the important thing for trading is lag. In the last graph, even for the blue line, which has a smoothing equivalent to a 39-bar SMA, the lag is NEVER GREATER THAN 2 BARS ANYWHERE whereas a 39-bar SMA has a lag of 19 bars!!

This is the secret of successful trend-finding!

CB

Below...

1. TOP - Frequency response
2. MIDDLE - Phase response
3. BOTTOM - Lag response

4. FORMULAE
Attached Image (click to enlarge)
Click to Enlarge Name: Filter_Response.JPG Size: 69 KB
Attached Image
Wowww... Nice pic!

Quoting Codebreaker
Hi Guys

Thanks for all the PMs on port sniffing. Attached are two files for the EURUSD which I sniffed from the Lightstreamer feed provided by IGIndex for their PureDeal trading platform. The zipped one is Tick Data just as it arrived in BID/OFFER/TIME format starting on 20 October and running for a day or two. The other is DATE/TIME/OHLC M3 data for the same pair starting 23 Oct for a week or so. Please be aware that these are RAW files. The Lightstreamer engine occasionally sends the wrong quote to the wrong port, so you get the odd outlier from another pair that you need to filter. (If you plot the data is it obvious where the errors are so you can just chop them out). This is VERY easy...
Hi CB,

I have been following your thread for a while appreciating this great knowledge and the high level of sophistication only seldom present on forums.

Unfortunately, I just understand pieces of what you're trying to convey due to my poor higher math knowledge.

Could someone help us to implement some of his ideas in a human readable format . I'd be more than happy to help with the programming part in MQL4 if it is possible.

best regards,

Daniel.
http://www.forexfactory.com/showthread.php?t=87379
thank you!
Hi Guys

A number of folk have been asking me about the use of spectral analysis in trading, particularly the Singular Spectral Analysis (SSA) add-on from Noxa that is causing a bit of a stir on the forums at the moment.

The important thing to remember about about SSA is that it is a non-causal analysis. It is capable of putting a fantastic, lag-free trend through a price series (much like the Hodrick-Prescott filter we discussed earlier) but ONLY WITH SOME HINDSIGHT.

If you try to build an indicator using SSA you will find that the trendline near the last bar keeps changing with new data. That is, you can get a buy/sell crossover but then the indicator will "change its mind" as new data arrives!

It is also hard to write for MT or similar packages with limited programming capabilities. Just about the only feasible way is to write a DLL. The problem is finding the eigensystem of the circulant (Hankel) matrix. This is a SERIOUS piece of code usually written in C++ or Fortran.

A nice tool but be careful!

CB
Hi Codebreaker,

You've been very generous with your time and your knowledge in this community... I for one have enjoyed reading the papers you've pointed out, and experimenting with some of the techniques you've discussed.

In a recent post, you shed some light on your filter design methods:

Quoting Codebreaker

Then we use the (admittedly rather complex) relationship between the Z-operator and the s-operator of the Laplace Transform and then we are firmly in the world of differential equations that we can solve. The formulae are given below, together with the frequency, phase and lag responses of some filters. Here, zeta is the coefficient (weight) and s is the length of the filter.

Note that the important thing for trading is lag. In the last graph, even for the blue line, which has a smoothing equivalent to a 39-bar SMA, the lag is NEVER GREATER THAN 2 BARS ANYWHERE whereas a 39-bar SMA has a lag of 19 bars!!
I followed most of it, but got lost at the fourth equation. From what I understand, the "Lag Response" of a discrete-time filter is its Group Delay. And the Group Delay is computed as the negative of the derivative of the filter's Phase Response. Am I correct so far?

1) So my first question is: does your "Lag Response" equation compute the negative derivative of the phase of an FIR transfer function?

2) My second question is: what is represented by the delta variable?

Now wandering a bit into design philosophy... Most of the literature on FIR filter design focuses on linear phase systems, (Type I, II, etc.) because if phase is not an issue, then an IIR filter is usually more appropriate. For example, the human ear is insensitive to phase (but very sensitive to delay), so IIR filters are often used in audio to get the group delay down at the cost of phase distortion.

But your words make it clear that phase response is an important design factor for you. Can you briefly explain why? My knee-jerk response was to think that as long as the phase isn't too horrible in the low-frequency band, the filter should be fine. Am I missing something?

Cheers...
Dear BarHopper

I see you are a connoisseur indeed! In response to your questions...

(1) Lag response for an FIR filter is always the Centre of Gravity (CG) of the coefficients. So for an N-period SMA, the lag is (N-1)/2. For any other coefficient set, this is always true. This is the simplest (and correct) way to look at lag.

(2) Delta is simply the period in bars.

The distinction between phase and group delay is important but mainly only in waveguide theory. For trading, we want the correct trend extraction component that may well be cyclic. You will appreciate that one can have exactly the right frequency but if the phase is delayed, then we will get late signals which are useless for trading.

So while the frequency response is important, the phase response is even more critical for TIMING trades.

At least, IMHO....

CB
  • Post #155
  • Quote
  • Edited Nov 11, 2008 12:33am Nov 10, 2008 10:11pm | Edited Nov 11, 2008 12:33am
Yes CB, thanks for the clarification... Knowing the meaning of delta makes it much more clear.

So you are using Phase Delay instead of Group Delay in your designs... very interesting. I wonder why the signal processing texts I've come across tend to focus on group delay. Phase delay seems more intuitive.

Now by way of explanation, for those who may be interested... (and CB please correct me where I'm wrong) the fourth equation (phase delay) can be derived from the third (phase response) with a change of variables (1/delta = f) and a rescaling by the period of the input signal:
1/(2pi radians per cycle) takes us to cycles of phase shift, and multiplying by delta bars/cycle takes us to bars of phase shift
It's an exercise in dimensional analysis. Of course, you can go straight to the definitions if you prefer.

My physical interpretation of your plots... a signal with a 9 bar period using your blue filter will experience a phase shift of about 80 degrees (at f= 1/9 = 0.1111 in the second plot), which is equivalent to a time shift of (80/360) * 9 = 2 bars (in the third plot). Note that I used 1/360 instead of 1/2pi because your phase response plot is in degrees instead of radians.

Now, do you mind if I split hairs with you for a minute If this interpretation is correct, does that mean the leftmost point of your third plot has a period of delta=1 bar, equivalent to a frequency of 1, which is double the Nyquist rate? If so, should your third plot begin at delta=2, equivalent to f=0.5? Of course, this doesn't change the analysis at all, it just means delta=1 is a repeat of the delta=2 point. I just want to know if I've interpreted everything correctly.

Shifting gears, I really like your idea of using the CG as an estimate of the lag introduced by a FIR filter... it makes great intuitive sense... I haven't done any math to try to derive a relationship between the CG of the coefficients and the phase delay (lag response)... are you saying if I write out an expression for the centroid of a FIR filter, and do a little creative manipulation, I'll eventually come up with your fourth equation? If so, I'll be really impressed (once again) with the beauty of DSP. But I have to admit, it seems a little too good to be true.

And going back to the group delay, do you believe it has any usefulness in evaluating the lag properties of a digital filter four our purposes?

Cheers...
Hi BarHopper

Many thanks for your excellent analysis and interpretation...

Quoting BarHopper
Yes CB, thanks for the clarification... Knowing the meaning of delta makes it much more clear.

So you are using Phase Delay instead of Group Delay in your designs... very interesting. I wonder why the signal processing texts I've come across tend to focus on group delay. Phase delay seems more intuitive.
SigProc texts focus on group delay because that is often more important in DSP applications using man-made signals that are nearly always stationary. But "natural" signals (like the markets) are not stationary and so a lot of the conventional analysis does not apply. All DSP books presuppose that the signals they are dealing with are wide-sense-stationary (WSS).

Quoting BarHopper
Now by way of explanation, for those who may be interested... (and CB please correct me where I'm wrong) the fourth equation (phase delay) can be derived from the third (phase response) with a change of variables (1/delta = f) and a rescaling by the period of the input signal:
1/(2pi radians per cycle) takes us to cycles of phase shift, and multiplying by delta bars/cycle takes us to bars of phase shift
It's an exercise in dimensional analysis. Of course, you can go straight to the definitions if you prefer.
Absolutely 100% right. Just substitute Period (delta) = 1 / frequency and you have it.

Quoting BarHopper
My physical interpretation of your plots... a signal with a 9 bar period using your blue filter will experience a phase shift of about 80 degrees (at f= 1/9 = 0.1111 in the second plot), which is equivalent to a time shift of (80/360) * 9 = 2 bars (in the third plot). Note that I used 1/360 instead of 1/2pi because your phase response plot is in degrees instead of radians.
100% correct in your interpretation!

Quoting BarHopper
Now, do you mind if I split hairs with you for a minute If this interpretation is correct, does that mean the leftmost point of your third plot has a period of delta=1 bar, equivalent to a frequency of 1, which is double the Nyquist rate? If so, should your third plot begin at delta=2, equivalent to f=0.5? Of course, this doesn't change the analysis at all, it just means delta=1 is a repeat of the delta=2 point. I just want to know if I've interpreted everything correctly.
Again, you are absolutely right. The delta = 1 point is redundant as the highest Nyquist frequency is f = 0.5 corresponding to delta = 2. But in my plotting software, if you select a LOG scale you cannot truncate the axis except at 1, 10, 100, etc. :-((

Quoting BarHopper
Shifting gears, I really like your idea of using the CG as an estimate of the lag introduced by a FIR filter... it makes great intuitive sense... I haven't done any math to try to derive a relationship between the CG of the coefficients and the phase delay (lag response)... are you saying if I write out an expression for the centroid of a FIR filter, and do a little creative manipulation, I'll eventually come up with your fourth equation? If so, I'll be really impressed (once again) with the beauty of DSP. But I have to admit, it seems a little...
This is little known, but a beautiful fact. If you just imagine the coefficients as weights, equally spaced, on a see-saw (teeter-totter in the US?) the point of balance of the see-saw is the lag. e.g. For a 4-bar SMA, there would be 4 weights equal to 1/4. The middle of these is at distance 1.5 (and NOT 2) because there are 3 gaps. The fulcrum would be directly under the middle of the centre gap with 2 weights either side. 1.5 bars is the correct lag of a 4-bar SMA.

Likewise, if you take a 4-bar weighted (WMA) with weights 4/10, 3/10, 2/10 and 1/10 then the fulcrum would be under the 3/10 weight. This weight then has no "moment". On one side we have a weight of 0.4 at a distance of 1 whose product is 0.4. On the other side we have (0.2 x 1) + (0.1 x 2) = 0.4 which balances the other side. So the fulcrun is in the correct place. And we see that the lag is 1 bar, which is correct for a 4-bar linear WMA.

This technique ALWAYS works no matter how complex the FIR filter. And, if you use REALLY fancy filters like mine with NEGATIVE weights in some places, these are like helium balloons that pull the see-saw up at that place. Just simple mechanics of moments is all that is needed!

Quoting BarHopper
And going back to the group delay, do you believe it has any usefulness in evaluating the lag properties of a digital filter four our purposes?

Cheers...
Not as useful as phase delay IMHO.

CB
Quoting Codebreaker
Hi Guys

A number of folk have been asking me about the use of spectral analysis in trading, particularly the Singular Spectral Analysis (SSA) add-on from Noxa that is causing a bit of a stir on the forums at the moment.

The important thing to remember about about SSA is that it is a non-causal analysis. It is capable of putting a fantastic, lag-free trend through a price series (much like the Hodrick-Prescott filter we discussed earlier) but ONLY WITH SOME HINDSIGHT.

If you try to build an indicator using SSA you will find that the trendline...
This is true that SSA (Singular Spectrum Analysis) cannot be used to generate signals as it is not causal. A SSA package for NeuroShell Trader is free to download from our website www.noxapredict.com and we made sure that users are aware that these indicators cannot be used for trading (see attached printscreens). SSA has potentially a lot of use but requires to be confined to static training ranges; beside this use and eventually learning from market states, SSA is not appropriate to signal generation and cannot be used for trading!!!!!

CSSA however is a causal version of SSA as the C letter stands for and can be used for trading indeed; it does not use hindsight or any forward-reference of any kind and is static given that the training range is properly set. In short, the signals generated by system involving CSSA indicators or by CSSA itself are proper for trading; they don’t change after the fact and they also don't lag but this is beside the point.



You can have a look at this blog where we posted example charts at different time steps to show that signals from CSSA indicators do not change after the fact http://noxapredict.blogspot.com/. To be even more concise, we slightly untuned one of the charts (see "Too good to be true?") to prove with no question that signals are not nice looking as new bar arrive.

One last clarification about CSSA and SSA in general:
 CSSA does not perform a conventional Spectrum Analysis! Instead, it takes all the variability in price and breaks it into a few oscillation patterns (or eigenvectors). The eigenvectors can be seen as elementary patterns that maximize the variance between lagged market snapshots. The eigenvectors are then ordered according to the variance they account for to form the singular spectrum. Cycles and cycle composites are then reconstructed from the eigenvectors in the singular spectrum to best represent the modes of behavior in price with no lag. In that, CSSA view cycles in terms of the volatility they account for in price. This is a radical departure from traditional spectral analysis for which future data consist of a periodic repetition of past data in the form of pure sinusoids or wavelets. CSSA however is not forced to conform to these unsubstantiated mathematical representations; as such, CSSA does not make any assumtion on the way to proceed so that it retains a remarkable resemblance to the original series.


Hope this helps.

Noxa
Attached Image (click to enlarge)
Click to Enlarge Name: do not use for trading 3.gif Size: 41 KB
Attached Image
Quoting Codebreaker
It is also hard to write for MT or similar packages with limited programming capabilities. Just about the only feasible way is to write a DLL. The problem is finding the eigensystem of the circulant (Hankel) matrix. This is a SERIOUS piece of code usually written in C++ or Fortran.

I believe this one is based on SSA, although I have not looked at the code (and my russian is at bit rusty):

http://www.forex-tsd.com/199317-post328.html
http://codebase.mql4.com/ru/1413
Hi Guys

Thanks to Noxa for posting. As you Noxa guys will see, this is one of the more serious threads around with some seriously acid-head math types on board.

So Yowza!! If the Causal-SSA does what it says on the tin, this has certainly piqued my interest!

Anyone tried this and care to comment in depth?

CB
Hello folks from Noxa,

This is the first I've heard of your product/approach, but allow me to ask a question? In your post, you say:

Quoting Noxa
CSSA does not perform a conventional Spectrum Analysis! Instead, it takes all the variability in price and breaks it into a few oscillation patterns (or eigenvectors). The eigenvectors can be seen as elementary patterns that maximize the variance between lagged market snapshots. The eigenvectors are then ordered according to the variance they account for to form the singular spectrum.
It sounds like you're talking about a principle components analysis along some set of feature vectors... probably implemented via SVD? Am I reading this right? If so, what's the general idea behind your feature vectors? You've said they're not simple sinusoids... but what are they?

If I've totally misinterpreted your post, please forgive me.

Cheers...
0 traders viewing now