Quantitative and Algorithmic Trading

Quantitative and Algorithmic Trading



This thread is dedicated to Quantitative and Algorithmic Trading.

The first page should be viewed as a focal point regarding above mentioned topics.


This first page is under construction and, if interested, visit it from time to time to see, if new material/links have arrived.



"There is a difference between saying that there is predictability and the ability to predict"
"Although there is always more profit in long term forecasting, from a mathematical point of view, there is more reliability in short term forecasting."
“Make everything as simple as possible.” (A. Einstein) But not simpler.
"Trading's not a game – It's an IQ test" {algodude}


Software For Business Intelligence Analytics http://www.johncon.com/ndustrix/



First, a few things to consider

Behavioural Finance http://www.behaviouralfinance.net/
Gambler's fallacy http://en.wikipedia.org/wiki/Gambler's_fallacy
Illusion of Control http://illusion-of-control.behaviouralfinance.net/
Nonlinear Dynamics http://www.nonlineardynamics.org/
Harmonic Pattern Success Rates http://www.trade-forex-harmonic-patterns...ccess.html




How to Learn Algorithmic Trading http://quantivity.wordpress.com/2010...thmic-trading/

How to Learn Algorithmic Trading: Part 2 http://quantivity.wordpress.com/2010...rading-part-2/

How to Learn Algorithmic Trading: Part 3 http://quantivity.wordpress.com/2010...rading-part-3/


Statistics knowledge

Mean reversion (finance) http://en.wikipedia.org/wiki/Mean_reversion_(finance)
A Mean-Reversion Theory of Stock-Market Crashes Pairs trade http://en.wikipedia.org/wiki/Pairs_trade
Regression toward the mean http://en.wikipedia.org/wiki/Regression_toward_the_mean
Ornstein–Uhlenbeck process http://en.wikipedia.org/wiki/Ornstein–Uhlenbeck_process
Pairs trade Pairs trade http://en.wikipedia.org/wiki/Pairs_trade

Probability distribution http://en.wikipedia.org/wiki/Probability_distribution
List of probability distributions http://en.wikipedia.org/wiki/Probability_distribution
Log-normal distribution http://en.wikipedia.org/wiki/Log-normal_distribution
Fat-tailed distribution http://en.wikipedia.org/wiki/Fat_tail

in finance, fat tails are considered undesirable because of the additional risk they imply. For example, an investment strategy may have an expected return, after one year, that is five times its standard deviation. Assuming a normal distribution, the likelihood of its failure (negative return) is less than one in a million; in practice, it may be higher. Normal distributions that emerge in finance generally do so because the factors influencing an asset's value or price are mathematically "well-behaved", and the central limit theorem provides for such a distribution. However, traumatic "real-world" events (such as an oil shock, a large corporate bankruptcy, or an abrupt change in a political situation) are usually not mathematically well-behaved.

Find The Right Fit With Probability Distributions http://www.investopedia.com/articles...stribution.asp

Probability density function http://en.wikipedia.org/wiki/Probabi...nsity_function


PIMCO - Understanding tail risk http://investments.pimco.com/Marketi...Risk_PU001.pdf

Investopedia explains 'Tail Risk'
When a portfolio of investments is put together, it is assumed that the distribution of returns will follow a normal pattern. Under this assumption, the probability that returns will move between the mean and three standard deviations, either positive or negative, is 99.97%. This means that the probability of returns moving more than three standard deviations beyond the mean is 0.03%, or virtually nil. However, the concept of tail risk suggests that the distribution is not normal, but skewed, and has fatter tails. The fatter tails increase the probability that an investment will move beyond three standard deviations.

Distributions that are characterized by fat tails are often seen when looking at hedge fund returns.
http://www.investopedia.com/terms/t/tailrisk.asp

Are the Skewness and Kurtosis Useful Statistics? http://www.spcforexcel.com/are-skewn...ful-statistics




Books

Quantitative trading http://k-512.googlecode.com/files/AlgoTra.pdf
This is a good book to start reading with…..
Introduction To OpenQuant Strategy Development http://www.smartquant.com/introducti...t_strategy.pdf



Software

Matlab http://www.mathworks.de/products/matlab/
GNU Octave http://www.gnu.org/software/octave/
O-Matrix http://www.omatrix.com/
Scilab http://www.scilab.org/
R http://www.r-project.org/

RIZM http://equametrics.com/
ARB-Maker http://arb-maker.com/



Blogs

Software Trading http://softwaretrading.co.uk/
Quantivity http://quantivity.wordpress.com/
Algodude http://www.algodude.com/
Quantitative Trading http://epchan.blogspot.co.uk/
The Psy-Fi Blog http://www.psyfitec.com/
Quantitative Research and Trading http://jonathankinlay.com/
Quantitative Systematic Market Analysis http://qusma.com/
Quantified Strategies http://www.quantifiedstrategies.com/
marketsci blog http://marketsci.wordpress.com/



Quandl - New search engine for financial, economic and social datasets http://www.quandl.com/



Investopedia - Global Professional Exams http://www.investopedia.com/professi...alarchive.aspx

RE: Quantitative and Algorithmic Trading
http://epchan.blogspot.co.uk/2013/03/what-can-quant-traders-learn-from.html

What Can Quant Traders Learn from Taleb's "Antifragile"?

Here are a few snip-sets I found particularly interesting:


1) Momentum strategies are more antifragile than mean-reversion strategies.

Taleb didn't say that, but that's the first thought that came to my mind. As I argued in many places, mean reverting strategies have natural profit caps (exit when price has reverted to mean) but no natural stop losses (we should buy more of something if it gets cheaper), so it is very much subject to left tail risk, but cannot take advantage of the unexpected good fortune of the right tail. Very fragile indeed! On the contrary, momentum strategies have natural stop losses (exit when momentum reverses) and no natural profit caps (keep same position as long as momentum persists). Generally, very antifragile! Except: what if during a trading halt (due to the daily overnight gap, or circuit breakers), we can't exit a momentum position in time? Well, you can always buy an option to simulate a stop loss. Taleb would certainly approve of that.

2) High frequency strategies are more antifragile than low frequency strategies.

Taleb also didn't say that, and it has nothing to do with whether it is easier to predict short-term vs. long-term returns. Since HF strategies allow us to accumulate profits much faster than low frequency ones, we need not apply any leverage. So even when we are unlucky enough to be holding a position of the wrong sign when a Black Swan hits, the damage will be small compared to the cumulative profits. So while HF strategies do not exactly benefit from right tail risk, they are at least robust with respect to left tail risk.


5) Correlations are impossible to estimate/predict. The only thing we can do is to short at +1 and buy at -1.

Taleb hates Markowitz portfolio optimization, and one of the reasons is that it relies on estimates of covariances of asset returns. As he said, a pair of assets that may have -0.2 correlation over a long period can have +0.8 correlation over another long period. This is especially true in times of financial stress. I quite agree on this point: I believe that manually assigning correlations with values of +/-0.75, +/-0.5, +/-0.25, 0 to entries of the correlation matrix based on "intuition" (fundamental knowledge) can generate as good out-of-sample performance as any meticulously estimated numbers.The more fascinating question is whether there is indeed mean-reversion of correlations. And if so, what instruments can we use to profit from it? Perhaps this article will help:
http://web-docs.stern.nyu.edu/salomon/docs/derivatives/GSAM%20-%20NYU%20conference%20042106%20-%20Correlation%20trading.pdf


6) Backtest can only be used to reject a strategy, not to predict its success.

This echoes the point made by commenter Michael Harris in a previous article. Since historical data will never be long enough to capture all the possible Black Swan events that can occur in the future, we can never know if a strategy will fail miserably. However, if a strategy already failed in a backtest, we can be pretty sure that it will fail again in the future.

http://epchan.blogspot.co.uk/2012/10/order-flow-as-predictor-of-return.html

Order flow as a predictor of return
Order flow is signed transaction volume: if an order is executed at the ask price, the incremental order flow is +(order size); if executed at the bid price, it is -(order size). In certain markets where traders can only buy and sell from market makers but not from each other, a positive order flow means that traders are net buyers of a security. But even in markets where everyone can place and fill orders on a common order book, a positive order flow indicates that informed traders (those willing to aggressively get into a position) are eagerly acquiring a security.

The neat thing about order flow is that it has proven to be a good momentum indicator. That is to say, a positive flow predicts a positive future return. This might seem trivially obvious, but you have remember that generally speaking, a positive past return by no means predicts a positive future return. That FX order flow possesses this predictive power was shown by Evans and Lyons in a series of papers, but this indicator is useful in many other markets, and at many different time scales. For example, in a paper by Coval and Stafford, it was shown that if you can tease out the order flow of a stock due to mutual funds' trading alone, you can also predict its future return up to, say, a quarter. This paper not only shows that order flow is predictive, but that sometimes a specific kind of order flow (in this case, that of mutual funds only) is sometimes more predictive than general order flow. In many cases, traders find that by counting only order flow due to institutional traders, or order flow due to large orders, they can better predict future returns. (No wonder institutional traders are trying their darnedest to break up their orders into small chunks, or to trade in dark pools!) I recently also heard that order flow into sector ETFs can be predictive of that sector's return. If any reader has read papers or has experience with this type of sector rotation model, please leave a comment!

Despite the proven usefulness of order flow, not too many retail traders utilize it. The reason is simple: it can be hard to measure. In FX in particular, many markets do not report trade information, or they report with a sufficient delay such that the information has no predictive utility. Even for markets that report instantaneous trade information, you would need a good piece of software to capture every bid, ask, trade, and trade size, and store them in an array, in order to compute order flow, an operation that most retail trading software cannot accomplish. However, this barrier to entry may just mean that there are still decent alpha to be extracted from this indicator.

Momentum strategies:

http://epchan.blogspot.co.uk/2012/06/momentum-strategies-book-review.html



definitively worth the time to read

Does Averaging-In Work?

http://epchan.blogspot.co.uk/2010/01/does-averaging-in-work.html


read the comment section, too

http://www.snb.ch/n/mmr/reference/working_paper_2011_04/source

Intraday patterns in FX returns and order flow
Francis Breedon and Angelo Ranaldo

SNB Working Paper

This blog about Psychology and Finance has many many good articles

http://www.psyfitec.com/search?updated-m...results=10

Invert, Always Invert http://www.psyfitec.com/2012/12/inve...ys-invert.html

Experience, Rare Events and Risky Choice http://www.psyfitec.com/2012/12/expe...ky-choice.html

Predicting the Information Markets http://www.psyfitec.com/2012/11/pred...n-markets.html


Bayesian Unreason in the Modern World http://www.psyfitec.com/2012/11/baye...ern-world.html

The Emergent Rationality of Markets - Frogs and Philosophers http://www.psyfitec.com/2012/11/the-...f-markets.html

On the Invariant Nature of Investor Ineptitude - Behavior is Forever, Not Just For Life http://www.psyfitec.com/2012/11/on-i...-investor.html

HONTI Home Page http://www.psyfitec.com/p/honti-how-not-to-invest.html

HONTI is a series of articles about how not to invest, covering the wide range of weird and wonderful ways which people find to lose money in markets. The principle here is simple: avoid losing money, exhibit sufficient patience and you'll do all right eventually. The skill is to understand when we're being fooled by our own minds into doing the wrong thing, and then do the opposite!

Mindful Investing http://www.psyfitec.com/p/mindful-investing.html

Mindfulness in investing is one of best defences against behavioural bias. This page is dedicated to ideas and resources for the mindful investor.

The Big List of Behavioral Biases http://www.psyfitec.com/p/the-big-li...al-biases.html


Fantastic list - must read

Combining Mean Reversion and Momentum Trading Strategies in Foreign Exchange Markets http://www.be.wvu.edu/div/econ/work/pdf_files/09-14.pdf


Abstract
The literature on equity markets documents the existence of mean reversion and
momentum phenomena. Researchers in foreign exchange markets find that foreign
exchange rates also display behaviors akin to momentum and mean reversion. This paper
implements a trading strategy combining mean reversion and momentum in foreign
exchange markets. The strategy was originally designed for equity markets, but it also
generates abnormal returns when applied to uncovered interest parity deviations for ten
countries. I find that the pattern for the positions thus created in the foreign exchange
markets is qualitatively similar to that found in the equity markets. Quantitatively, this
strategy performs better in foreign exchange markets than in equity markets. Also, it
outperforms traditional foreign exchange trading strategies, such as carry trades and
moving average rules.


read first page 18ff. ("Conclusion) to get appetite :-)

Reversion to mean and momentum



Whitepaper Lyxor - Trendfiltering Methods for Momentum Strategies http://www.lyxor.com/fileadmin/_file...ublication.pdf

http://www.schulich.yorku.ca/SSB-Extra/NorthernFinance.nsf/Lookup/Kevin%20Wang7/%24file/Kevin%20Wang7.pdf


Regression to the Mean: Of Nazis and Investment Analysis http://www.psyfitec.com/2009/04/of-n...-analysis.html


Combining Mean Reversion and Momentum Trading Strategies in Foreign Exchange Markets http://www.be.wvu.edu/div/econ/work/pdf_files/09-14.pdf


Momentum and mean reversion across national equity markets http://andromeda.rutgers.edu/~yangruwu/MOM_MR_JEF.pdf



Momentum Trading, Mean Reversal and Overreaction in Chinese Stock Market http://www.hkimr.org/uploads/seminar...aper031231.pdf


http://www.istfin.eco.usi.ch/a_sbuelz.pdf Momentum and Mean-Reversion in Strategic Asset Allocation

Are the Skewness and Kurtosis Useful Statistics?


http://www.spcforexcel.com/are-skewn...ful-statistics

That is a really interesting one, because it is sooooo simple.

Comes to mind: simplify your life :-)

Closing Price in Relation to the Day’s Range, and Equity Index Mean Reversion http://qusma.com/2012/11/06/closing-...ean-reversion/


But keep in mind:

the examples have only been tested at relatively strong and continuously trending market and only a "long" strategy was applied. But interesting at all.

Consecutive Up or Down Days, NASDAQ 100 Edition http://qusma.com/2012/11/27/consecut...q-100-edition/



.....Just a short post today. Jack Damn has been tweeting about consecutive up/down days lately, which inspired me to go looking for a potential edge. The NASDAQ 100 has posted 6 consecutive up days as of yesterday’s close. Is this a signal of over-extension? Unfortunately there are very few instances of such a high number of consecutive up days, so it’s impossible to speak with certainty about any of the numbers. Let’s take a look at QQQ returns (including dividends) after X consecutive up or down days:........

Keep Trading As Simple As Possible http://www.quantifiedstrategies.com/...e-as-possible/

Random Walk Hypotheses and Profitability of Momentum Based Trading Rules


http://www.e-m-h.org/Venk03.pdf

BIS Working Papers No 366 Currency Momentum Strategies http://www.bis.org/publ/work366.pdf

Abstract
We provide a broad empirical investigation of momentum strategies in the foreign exchange
market. We find a significant cross-sectional spread in excess returns of up to
10% p.a. between past winner and loser currencies. This spread in excess returns is
not explained by traditional risk factors, it is partially explained by transaction costs
and shows behavior consistent with investor under- and over-reaction. Moreover, crosssectional currency momentum has very different properties from the widely studied
carry trade and is not highly correlated with returns of benchmark technical trading
rules. However, there seem to be very effective limits to arbitrage which prevent momentum
returns from being easily exploitable in currency markets.

Very good reading:

Mean Reversion, Forecasting and Market Timing http://www.norstad.org/finance/rtm-and-forecasting.html

John Norstad http://www.norstad.org/finance/index.html



snipset:

Cheating

Consider once again the pure random walk coin tossing game without RTM. We said there was no timing strategy in this case. But now suppose we find a crystal ball before the game starts that tells us what the ending value will be when the game ends. Recall that this actual ending value is likely to be well above or below 0. Draw a straight line on the empty graph from the starting point to the known ending point. Start playing the game. Whenever the graph is above the line, forecast tails and take your money off the table. Whenever the graph is below the line, forecast heads and put your money back on the table. It should be easy to convince yourself that your forecasts will be much more accurate than 50/50, and you will win with your timing strategy ("win" in the sense that you will do much better than someone who does not forecast or time). This is even without RTM!

Similarly, with investing, if we could somehow know what the future average return will be in advance, we could market time even without RTM.

Today, for example, we know that the average return over the last 75 years is about 10% annualized. Get into a time machine and go back to 1930. Invest for the next 75 years. Whenever the cumulative annualized returns since 1930 go above 10%, lighten up on stocks. Whenever the cumulative annualized returns since 1930 go below 10%, put more money back into stocks. By 2005, you will have beaten the market by a very nice margin.

This is called an "in-sample" test. It has an obvious flaw, because investors in 1930 did not have any idea what the average annualized return was going to be over the next 75 years. They only knew what the past average annualized returns were. If you do the test again and only permit investors to use the information available to them at the time (an "out-of-sample" test), the market timing strategy doesn't work.

This is a simple kind of "chartist" timing, based just on past returns. When past returns are high, lighten up on stocks. When past returns are low, put more money into stocks. In a pure random walk without a crystal ball, we know that this kind of timing doesn't work. The reason it doesn't work is because without the crystal ball, we are unable to define the notions of "low" and "high." "Low" means "below the future average value" and "high" means "above the future average value," but we don't know the future average value. We only know the past average value, and that information is of no use in a pure random walk without RTM.

Most forecasting methods and timing strategies based on the forecasts are more sophisticated. They usually use fundamental financial ratios like D/P (dividend-to-price ratio) or P/E (price-to-earning ratio) to make the forecasts. The argument is that these ratios are sometimes high and sometimes low, but it is unreasonable to think that they can possibly grow or shrink without bounds ("wander off to infinity," as the academics often like to say it). It is much more reasonable to think that while they sometimes get very high or very low, they must eventually revert to some kind of more normal level. RTM, in other words. If these ratios have RTM, it is quite sensible to hypothesize that this RTM in the ratios induces a similar RTM effect in returns, and that the ratios can be used to forecast future returns.

Does this kind of fundamental forecasting actually work? While the general idea certainly seems more than plausible, the proof is in the pudding, and the theories need to be tested. It is possible to examine the historical record to see if the various schemes would have worked in the past. Many people have done these kinds of studies, both in the popular financial world and in the academic financial world.

The key point is that when back-testing these kinds of fundamental forecasting methods to see if they would have worked in the past, it is cheating if you use the actual means of the fundamental forecasting variables calculated over the entire period of the test, because that information was not available to investors in the past. You must back-test using only information available at the time. In other words, you must do out-of-sample tests, not in-sample tests. Most of the popular studies which reach the conclusion that returns are predictable are invalid for this reason. Surprisingly, many of the academic studies seem to suffer from the same fatal flaw.

Amit Goyal and Ivo Welch discuss and explore this insight in their paper A Comprehensive Look at The Empirical Performance of Equity Premium Prediction. When they did out-of-sample tests of all of the popular forecasting variables, including D/P and P/E, they found that none of them worked:
Our paper explores the out-of-sample performance of these variables, and finds that not a single one would have helped a real-world investor outpredicting the then-prevailing historical equity premium mean. Most would have outright hurt. Therefore, we find that, for all practical purposes, the equity premium has not been predictable.

This result also surprises a great many people. The common wisdom is that future stock market returns are highly predictable using common valuation measures like D/P and P/E. Goyal and Welch's research indicates that this belief, like so many others, may be just another example of how people are often fooled by randomness and see patterns in random data that aren't really there.

There is still controversy in the academic community about whether or not stock returns are predictable, and to what degree they might be predictable, and what the best forecasting variables might be. Goyal and Welch have cast doubt on this hypothesis, and they have performed the valuable service of demonstrating how important it is use only out-of-sample tests, but research and debate continues. In any case, predictability, if it exists at all, is clearly much weaker and more difficult to exploit than most people think.

http://www.norstad.org/finance/ranwalk.pdf


6.7 The Fallacy of Time Diversifcation

Financial analysts, columnists, and other pundits are fond of noticing that as
time increases, the standard deviation of the annualized return decreases. This
is quite true, but it's misleading. They cite this as evidence that the uncertainty
of investing in volatile assets like stocks decreases as the time horizon increases.
We have seen quite clearly, however, that the uncertainty of the ending value
increases rapidly with time. For an investor concerned about the value of his
investment at some time in the future, it is the ending value and its uncertainty
that is important, not the annualized return. The volatility of the annualized
return may decrease with time, but because of compounding, the volatility
of the dollar ending value, which is what is important, increases with time.
This misconception, which is nearly ubiquitous, is called the fallacy of time
diversi cation."5

It's the total return that matters in terms of the bottom line. This determines
how much money we have at the end of t years. The standard deviation of the
average return decreases as t increases, but we don't care about that. We care
about the standard deviation of the total return, which increases as t increases.



6.8 Risk Over the Long Run

The fallacy of time diversifcation is very widespread. Most people seem to be-
lieve that the risk of investing in volatile assets like stocks decreases the longer
your time horizon. Indeed, if you read the Financial press, popular books on
investing, or the literature produced by the mutual fund and other investment
management companies, you hear this argument constantly. This problem de-
serves further exploration.
Zvi Bodie, a Finance professor at Boston University, has shown that with at
least one reasonable definition of the notion of risk," it is possible to prove
that, contrary to popular opinion, the risk of investing in volatile assets like
stocks increases with time.6
Bodie's argument is quite simple. It's reasonable to think of risk" as the
cost of insuring against earning less than the risk-free rate of interest." This
insurance is just a simple put option, and the Black-Scholes equation for option
pricing shows that the put option increases in value with time to expiration.
Here's the argument in detail:
Suppose the risk-free rate is r. Let S be the current value of our portfolio.
Assume our portfolio's continuously compounded yearly return has a standard
deviation of . Let E be the future value of S after t years of earning interest
at rate r. This is how much we'd have after t years if we just put our money in
the bank. Note that E = Sert.
We have two alternatives: We can keep our money in our portfolio, or we can
sell our portfolio and put all of our money in the bank. If we keep our money
in the portfolio, it is reasonable to think of the risk we run being not doing as
well as we would have if we had put the money in the bank instead.