Can statistical methods be used to create a trading strategy?

Whilst at school, I covered 2 units of statistics. These included things like:
- Normal, Poisson, Chi-squared, t, and binomial distributions
- Confidence intervals
- Hypothesis testing using a given significance level to determine whether a critical value lies within a critical region
- Correlation and regression analysis
- All the basics regarding numerical methods and such

Does anyone implement any of these principals in their approach to trading? If you don't, but are familiar with these concepts, can you see any way in which they would be useful to a trader?

Take a look at some ideas:

Rescaled Range Analysis (nonperiodic cycle detection and persistence measurement)
Box-Jenkins ARMA family of models (immense subject - linear prediction)
Autoregressive Spectral Estimation (the freq domain use of AR models)
LMS adaptive algorithm
Kalman Filter
ANNs

The main thing to remember is that classical statistics are based on the normal distribution which does NOT describe markets. Markets are not random. There are newer statistical methods for applying the Fractal Market Hypothesis. Check out the books by Edgar Peters for that. Makridakis, Wheelwright, & McGee have an excellent textbook "Forecasting - Methods and Applications" that is a classic you should check out.

Stick to price action--it is the one and only approach that works all the time,in all time frames, in all commonly traded pairs,in both trending or neutral (sideways ) markets. It never fails to consistently deliver profits whether your a scalper. a swing trader, a trend follower, or a breakout trader. If you understand and execute a price action strategy you will not ever require any of the analytical tools you mention in your post----the only time it fails is during knee jerk reactions to economic/political news events or immediately prior to major releases. The problem is it takes years of trial and error to develop a workable strategy. You can also dispense with any of the indicators or chart artifacts you may be using with the exception of Bollenger Bands. Devote your time and study to price action ,its worth the trip.

Thank you very much for your response, that sounds like some very interesting stuff. I know relatively little about statistics - as stated, I've only covered relatively basic concepts that are very much based around normally distributed data. I also know relatively little about trading or financial markets; it currently just a hobby, though I hope it one day it may become more than that. However, and based on my lack of knowledge regarding both subjects I may be about to say something stupid, I believe that if markets are not random then statistical approaches must be the most logical way to identify any cycles or patterns that exist.

I can understand why you have understanding of such an area, given your username. Do any of the concepts mentioned in your post make an appearance in your trading strategy?

Anyway, I'll certainly look at the things you've mentioned in detail and I'll probably check out the books you've mentioned. School's nearly over now, so I'll have plenty of time for reading (and investigating any ideas I may have, first hand).

Just to clarify, this was the textbook you were referring to, right?
http://www.amazon.co.uk/Forecasting-...pr_product_top

Firstly, would you mind explaining to me exactly what price action refers to? When I first began reading up on the different approaches to trading, I took to thinking that price action referred the use of certain types of candlestick (doji, marubozo, etc) and candlestick formations. I'm now suspecting that it includes more than just that. Does it?

I can honestly that most of the price action concepts I'm aware of make some logical sense to me. The fact that price action tells a story about the sentiment of market participants - and how that can be used to help identify trading opportunities. However, I don't feel the application of techniques such as price action, other forms of TA, or statistical analysis have to be mutually exclusive. Do you? Maybe one could devise a system involving statistical analysis to identify an opportunity for a trade, and then price action could be used to pick a more precise entry point?

Anyway, thanks for the input.

My definition of Price Action : The fluctuation in successive closing prices within specific time frames and the price ranges that envelope those closing prices. Both elements ,closing price and range,are equally critical, as well as their relative relationship to their adjacent bars as depicted in a classic OHLC bar chart. Use at least 2 independent chart sources to confirm agreement as to range and price, charts can and do vary widely (I use Oanda and Netdania) .Aside from all other criteria I only trade when charts agree. I would rather not go into further detail or describe how I put these diverse inputs together to form an algorithm, but ,perhaps I have initiated a direction for you that may be fruitful for you

My thoughts exactly.
I have been studying these things, off and on, for some 25 years now. I am self taught and did not learn the requisite math needed to be fully versatile with such things. But I do understand the underlying concepts and much of the math that is needed to actually implement such techniques. I do not have a fully-functional technique yet, but I continually work on new technical ideas and am making progress. Recently, I have been working with lowpass filters and Fibonacci-related cycles and have found a treasure trove of seemingly predictive phenomena that I simply can't ignore, so my mind has been occupied in this arena. But I expect that soon I will get back into some of the techniques I mentioned.

The biggest problem with these techniques is that there are tradeoffs in everything, and unless you really understand natural market behavior first, you really can't know what kind of technique makes sense to investigate. You can easily spend a lot of time and effort in making an AR linear predictor and still have it not work. There isn't any off-the-shelf solution; everything needs more work and more thought. It is also quite possible that any technical solution developed will still require wise human interpretation to use correctly. But if we can assemble enough uncorrelated predictive techniques and combine these into our subjective synthesis of the situation, I believe that this will yield excellent results.

So if you really want to figure this out, expect that this will take a lot of time and work. Trading is the hardest way to make an easy buck. You will have to read and understand a lot of technical things, and then figure out the ways in which these techniques are not properly suited for market analysis. Then, you can work on modifications and reconfigurations to the standard forms to make them work in the markets. That's my theory. If you aren't able to approach this like an independent scientist looking for answers that nobody else knows yet, you won't find anything useful. The people who figure this out disappear into lives of quiet obscurity and security; they don't write books or make video training courses.

I recommend a dual approach to your trading education. Work on the crazy tech stuff that I mentioned. But also forget all that and just look at the market itself and try to come to some understanding of why the prices move as they do. Try to trade (demo or microlot) by using just a price chart. You have to develop a feel for what looks right and what looks wrong. There is no substitute for being able to read a chart. I can do this to a minor extent now, and am working on improving this skill. The only way to do this is experience. Your mind has to be exposed to situation after situation, chart after chart, for a long time. Start off on just a single currency pair and trade that only. That forces you to analyze what is going on even when there are no obvious signs. Don't run off to another pair and cherry-pick trades. Stick to one place, one scenario, and do your best with just that. My pair of choice is USDCHF because I believe that is affected far more by changes in the USD than in CHF. I believe (rightly or wrongly) that CHF is a 'neutral' currency that lacks the same political and economic intrigue and complexity of other more turbulent nations. So this reduces the degrees of freedom in the analysis by making it almost all about USD. I'm not a fundy so that could be wrong, but that's my best guess.

The plan is to develop hi-tech objective tools and then use them subjectively, but only after gaining the experience to know how to do this properly. Others rely entirely upon objective means. But I have much experience doing this and it has not worked out that well. Most such objective systems are too inflexible or otherwise don't adapt to changing market conditions. They can work great for a while and then lose it all. And then when it is all over, you have learned nothing because you weren't the one driving.

The book you mentioned looks correct. It is the 3rd edition and I have the 2nd. You will be amazed at what statistics can do when you get into the stuff beyond the normal college fare.

Very nice post. The part I quoted mirrors my own thoughts and experience.

You have my gratitude; yours was one of the most insightful/constructive posts I have read on this forum so far. It's nice to have found someone who seems to share relatively similar beliefs to myself.

Compared to your 25 years of experience, my year and a bit of slowly learning about trading when I have the time seems rather insignificant - so apologies in advance if I'm about to make myself sound like a bit of a simpleton, but there are a couple of things I must ask.
Firstly, what exactly do you mean when you say you've been working with low pass filters? Does this mean selectively ignoring price cycles that conform to certain frequencies?
Secondly, why have you chosen to use Fibonacci-related cysles? I usually aim to only implement something in a strategy if I can explain the logic behind it to myself; I can think of no reason why price should conform to Fibonacci principals, besides through self-fulfilment. It's something I'd really like to be able to get my head around, as Fibonacci based tools seem to make an appearance in so many trading strategies and I feel like I could be missing out on something potentially very useful.

Again, we seem to be in agreement here. Understanding why something works seems to be just as important as understanding what works in order to become profitable in the long term (you can't effectively modify a strategy if you don't understand why it works). As such, it makes sense to try to understand why certain patterns and cycles seem be present in price charts. Do you have any thoughts regarding why patterns exist? Because of economic cycles, maybe?

I'm sure that's true. I should be starting a 4 year mathematics degree this year, which offers the opportunity to specialise in certain areas further into the course. Perhaps the statistics I'd learn there would be more applicable; less linear and constrained.

Algorithm? As in, your price action based strategy is automated?

Obviously stats based methods apply to trading - success of so many quant funds prove that. Which methods do you use is totally upto you.. its like use to stats in any other field - like medical research etc.

Glad that we agree. I hope you take my comments on the other thread as merely a criticism of the specific ideas you mentioned, not any slam on you. I am aware that I am in the minority. Its just that I don't believe that consensus means much of anything. The more people whose opinions you include in any assessment or vote, the more stupid the final outcome.

Glad to help.

Lowpass Filters
OK you probably have used Moving Averages by now. Lowpass filters are a better way of achieving the same function as MAs.
MAs are used for smoothing. But what exactly is 'smoothing'? Simply put, smoothing is a means of doing 2 things: Removing noise and integrating a longer period of time into an instantaneous data point. The ideal smoother should remove all 'noise' while keeping all 'signal' intact. It should create a data point that represents NOW but contains information accumulated from a longer prior period of time so that NOW represents a larger block of time, increasing its significance. As you can see, I just used a bunch of terms that, themselves, need defining.

What is signal and what is noise? Signal is the energy component of the time series which contains the phenomena we wish to observe. Noise is any energy other than the energy of interest in the time series. We remove noise so as to be able to analyze the signal more accurately. Noise obscures a lot.

So this means that anything you don't care about is noise and should be removed. If you are using a high-frequency trading method where the trades last just an hour or two, you really do not care about the overall trend that has been present for the past several months. It is irrelevant, to you, not to somebody else trading a longer-term strategy. If you are that long-term trader, then the trend is signal and short term fluctuations (anything intraday) is just noise. If your trades take 2-6 months to complete, why would you care about cycles whose periods are just a few minutes or hours? So picking a time frame and using the right data for that time frame is very important.

That is one way to look at noise. Another involves a bit more engineering. If you understand the Nyquist-Shannon Sampling Theorem you should get this easy. If not, go look it up and learn it inside and out. It is a basic needed to do anything. The basic idea is that for any continuous process which is recorded by equally-spaced samples, your sample frequency must be at least double the frequency of the highest-frequency wave component you wish to observe within that data. This is Digital Audio 101. Market data is the same thing as audio data recorded on a CD; a series of points in time which represent a continuous waveform. On a CD, the sample rate is 44100 Hz, and the idea is that this can capture frequencies up to 22050 Hz, which is just above the upper limit of human hearing. So for market data, if you are using daily data, the fastest frequency you can observe within it has a period of 2 days. M15 data? 30 minute cycle is the fastest observable.

That is the theoretical limit. In actual practice, you really need a higher sample rate (oversampling) to be able to see those fastest frequencies with any clarity. It gets a bit complicated, but the idea is that 2 samples per wave is often insufficient to properly determine the phase of that frequency. Often, you need 4, 8, or even 16 samples per wave to make it clear. So keep that in mind.

If you understand how a CD player works, you will get this next part. A CD player is given just a sequence of numbers which represent the voltage of an analog audio waveform. That is what is on the CD. It plays that by creating a sequence of voltages that change in time with the sample rate. These voltages are stepped, not continuous. So the output audio waveform from the CD Digital-to-Analog Converter is actually a mess, a stepped waveform that, because of all the sharp angles at each 'step point', includes a whole host of extremely high frequencies that were not included in the original analog waveform that was recorded. These frequencies are artifacts created by the DAC and the whole digital process. This phenomenon is called aliasing and this alias noise must be removed before we can do anything.

The way this is done is by passing the output of the DAC into a lowpass filter. The alias noise is all in frequencies above the Nyquist limit (half the sample rate), so these are easily removed by a lowpass filter. The filter is set to around 22000 Hz, and the output then is almost all pure audio signal and little noise. Without this final step of using the filter, every CD would sound like complete crap.

Remember that I said that market data is the same as digital audio samples? This reveals a hidden truth about market data: it is all 50% alias NOISE! The samples alone have half signal and half noise. You MUST remove the digital alias noise by passing these raw samples thru a lowpass filter. The output of THAT is the actual market data, not the data you see on your chart. Until it is passed thru a 2-bar filter (or longer), you cannot do clear analyses. [This paragraph applies to any kind of math analysis you wish to do on a chart like any kind of indicator - for pure charting techniques, the raw data on the chart is the REAL data because we are not doing any numerical analysis on the data and merely wish to see the extent of its range].

In my experience, I have found that it is best to use oversampling at least at the 4x level (filter set to 8-bars). Sometimes, I oversample way more than this, often at 1000x or more. As an example, say you are trying to observe a 1-week cycle. You could use daily data and set your tuning to 5 bars. But why? Why not use H1 data and set your tuning to 120 bars? You can even do this with M1 data (I do that).

OK. MAs vs lowpass filters. MAs are crap. They leak. That is, in the stopband (noise area) they leak certain frequencies back into the output. They have excessive lag (time skew) making them often too late t o be of much use. The don't exhibit any resonance at the cutoff frequency like LPFs do, making them suboptimal for any kind of cycle analysis. MAs have finite memory, so again they really cannot adapt well to any cycles that are present. I use IIR LPFs which have an infinite memory.

I am including a program you can use to try this out, which I made and call MOVING AWESOME. Enjoy. More discussion on this topic next time.

Fibonacci
There IS a solid theoretical basis to use Fibonacci ratios and periods. In nature, there are many natural systems and organisms that use fib ratios. Look at a sunflower and observe the seed area in the center of the flower. They are arranged in a kind of double spiral which is based on fib ratios. Fibonacci is simply nature's way of compacting as much stuff into as small an area as possible - it is an efficiency technique.

How does this apply to the markets? Well I imagine almost all wave action as merely a reverberation of prior major energy impulses. The most efficient way to dissipate that original energy impulse turns out to be by using Fibonacci-related frequencies to rule the dominant frequencies of the various cycles whose energy drains the system of its original energy. By using fib freqs, the various cycles don't interfere with each other, as in they don't have periods which are multiples/factors of each other, so they efficiently dissipate the energy. Read up on fibs in nature and you will see what I mean. Then try them in the markets and see what I mean.

The cycles exist because humans are merely pawns in a grand system that is the market. Humans do things, en masse, in cyclical ways. The market reflects human behavior. Even the fundamental cycles are merely systems of human opinion and belief. Don't think of the market as some mystical beast. It is the accumulated opinions of a mass of humanity. Psychology is the engine.

We can talk more next time...


If you have any trouble with this program, let me know. BTW this is for MT4, which I hope you use...

Algorithms are merely a set of steps designed to solve a problem,as I'm sure you know. They may or may not be automated depending on complexity and size. Because of simplicity this algorithm, which I suppose could be automated ,does not require it on either count.

This part is quite strange. All the MAs are lowpass filters. Any causal lowpass will always let some undesired frequencies go through. Any causal lowpass will always have group delay (lag). Also the EMA is a IIR lowpass filter (the simplest one). IIR filters almost never have linear phase response which generate distorsions in the output like with the EMA. That's why EMA shall not be used for dynamic S/R. For a good tradeoff lag/smoothing the linearly weighted MA is pretty good despite a not linear phase response.

I like the explanation about the Fibo. I don't know if it is true or not but it makes sense.

In regard to the basic question posed in the title of this thread, I do not believe that any analysis of statistical data alone can predict price movement and direction in the forex market unless it incorporates price action as an integral part of the determining mechanism.There are several factors ,aside from my own trading experience, that lead me to that conclusion. Firstly, markets in general and forex in particular,are dynamic instruments.The ever changing nature of market temperament, the constant ebb and flow of traders, both large and small,entering and exiting the market, the confluence of price points , resistance and support levels, trend lines, pivot points, ect. ,ect. as interpreted by a myriad of traders ,each ascribing their own brand of analysis defy using historical data to determine future market movement, unless that data is bound together by their only true commonalities of price and range. The zero sum nature of the market only tends to magnify the affect of price and range as determinants. Markets have no memory beyond an hour or so and most traders ,in their heart of hearts, are scalpers and act that way on entry and exit if not in the body of their trading.
A lot, if not all, of the proceeding is contrary to established wisdom and accepted belief but is based on my experience trading forex intensively for well over 10 years , 40 or more hours per week, and a lot of back testing and research on weekends. Markets are not random but have causality due to price and range movement and fluctuation.

Strange, my research shows that the EMA, LWMA , SSMA is very ineficcient at predicting price. From the classical moving averages, the simple moving average is the most efficient, although not efficient at all, probably your frequency filters are waaay better, but compared to the others its way more efficient.

The problem with EMA is that it puts exponentially more emphasis on certain bars, which implies that after a downtrend to say, an uptrend has exponentially increasingly more and more chance of happening, so that the individual outcomes are dependent of eachother.

While this is partially true, but as you said the price is not normally distributed, so long tail events (known as trends) can last very long time, so an exponential MA is actually a pretty dumb idea.

Simple moving average, although treats every bar equally, is also flawed, but its still better than to put emphasis on the wrong bars.

While the linear weighted average, and the smoother simple average, using Gordon Gekko's words is:
"The same dog but with different fleas "

If you throw away high frequency data you are losing valuable information. There are techniques for working with non-equally spaced high frequency tick data. For example, the volatility computed this way is much more accurate than when you compute it starting from smoothed data.

The timing between the ticks is also important (see ACD model and it's derivatives). This kind of data is again thrown away if you aggregate ticks into candles, since candles are regularly spaced and lose the timing information in the original data.

Also of notice EMAs created specially for working on unevenly-spaced time series (tick data). They are better smoothers than plain EMA which work only on bars.

All MAs are lowpass filters. Correct. They are extremely poor lowpass filters because they leak and their stopband attenuation is poor. If you look at the frequency response of a moving average and a 2-pole Butterworth IIR lowpass filter, you can see that the Butterworth shows increasing attenuation in the stopband as frequency increases. No part of that curve shows any frequencies in the stopband that 'leak'. The MAs have frequencies where the attenuation curve goes sharply upwards (weaker) allowing leakage.



The effect of these leaks is that a lot more high frequency energy exits from a MA than from a Butterworth, when set to the same cutoff frequency.

Also, the slope at which the attenuation is applied at the cutoff frequency is steeper on the Butterworth than the MA. That makes Butterworth more selective, making it more 'tunable'.

The maximum stopband attenuation is limited in MA filters. It is INFINITE in Butterworth filters.

Any causal filter will have group delay. True. MAs have more than Butterworth, when tuned to the same cutoff frequency, especially when tuned to frequencies farther from the Nyquist limit. The group delay on MAs is a function of the cutoff frequency and grows linearly with it. Butterworth GD grows per cutoff frequency, but at a rate far less than linear. Advantage Butterworth again.

Phase response. The worst part of the Butterworth. But in reality, it isn't as bad as you think because the Butterworth has a very smooth and continuous phase response curve that isn't really very far off. Plus that phase distortion can be eliminated in 2 different ways. You can do the backwards/forwards processing that eliminates 100% of it, or you can do forwards only processing and just apply an estimated phase correction based on the known Butterworth response. That can provide near-total phase correction.