The Finance Book Club

Ok, so the notes don't give us the whole answer but there are a couple of interesting points.

1. Osborne didn't make these conclusions alone but with a collaborator - Victor Niederhoffer -one of the fathers of statistical arbitrage. It might be worth adding his book to the reading list - the Education of a Speculator. However Neiderhoffer is most famous for tanking his Matador fund in 2007.
2. Weatherall refers to an article that he simply names Neiderhoffer and Osborne (1966). I believe it is this one: https://www.tandfonline.com/doi/abs/....1966.10482183

I have downloaded it and will add it to today's reading so you should have an answer (even if it's 'I don't know') soon.

My own opinion is based on Mandelbrot's fractal work. Since all charts are basically snapshots of multifractal amplitude over multifractal time, what is 'a little bit' is relative to the local volatility as a whole. So no matter what your trading horizon is, five minutes or five months, prices are mean-reverting in the short term and trending in the long term (in general). But let's see what Osborne and Neiderhoffer have to say if anything.

Osborne’s methodology

1. Hard problems are too hard to solve all at once
2. Study the data
3. Make simplifying assumptions
4. Derive simple models
5. Check carefully to find the places where assumptions break down
6. Go back to the data
7. Figure out how failures of assumptions produce problems for model predictions

This seems like a good framework for solving many problems.

1. A model isn’t necessarily flawed when it breaks down
   1. It just means you have more work to do

2. If you attach too much importance to a mathematical model in your trading
   1. And it has assumptions that are no longer met by the market
   2. You risk ruin the same way a plane with a car engine will not fly

3. Osborne remained convinced that there is no profitable way to make forecasts
   1. However he noticed the ‘whole number’ levels where traders want to buy or sell offer a threshold where likely the price would go up or down from that number but not much less
   2. His method involved taking trades at ⅛ above (when buying) or ⅛ below (when selling a whole number) and risking only that ⅛.
   3. This was in 1966 (and I don’t believe the effect is still observable, but feel free to check it and report back)

Before we leave Osborne I want to read the paper I referenced earlier and see if I can answer HeyYou satisfactorily, or failing that, at least report something of interest.

the idea of entering at pullbacks has always appealed me. I'm considering doing that on indices, mid-long term to avoid complications

Look at what a well-timed question and a little digging can produce. I am in your debt, HeyYou. It also seems if we want specific trading ideas it might be more fruitful to dig through academic papers then books. However the reward/cost ratio for that activity is likely horrendous.

Special supplement - “MARKET MAKING AND REVERSAL ON THE STOCK EXCHANGE”
a paper by Neiderhoffer and Osborne

1. Stock market prices show ‘striking properties of dependence’ (they aren’t random)
2. “We find for example that after a decline of 1/8 of a point between transactions, an advance on the next transaction is three times as likely as a decline.” There you go, good papers aren’t afraid to put the good stuff right in the abstract! Stock prices are no longer measured in eighths but as Jeff Cooper says in ‘Hit and Run Trading’ that’s close enough to $0.10 to simply use whole tenths of a dollar.
3. This is testable, and I aim to see it for myself. Is this pattern still in effect?
4. “Further examinations disclose that after two price changes in the same direction, the odds in favor of a continuation in that direction are almost twice as great as after two changes in opposite directions.” This would also be worth testing but how do we define a ‘change of direction’?
5. The article also mentions that dealer quotes concentrate at integers, halves, quarters and odd eighths.
6. I think a visual check of modern charts shows that this is no longer the case, computerized (and metric-ized) as everything is now. I won’t be checking this, but feel free to “do your own research”. You won’t kill anyone, unlike doing your own virological research.
7. “Although the probability of an advance in the future can be estimated from the relative frequency of advances in the past, this probability does not change from transaction to transaction.”
8. The price dependence they find is due to “ natural consequences of the market making process.”
9. N&O test “six of the first seven” stocks for 22 days in October 1964.

This chart shows the ‘stickiness of even eighths” where all the ticker transactions with no price changes happened at ‘even eighths’ (2/8, 4/8, 6/8), and the authors mention that most of the price changes from those levels were reversals. Again, I think the computerization/algorithmization of stocks has made this meaningless. Was it even tradeable back then with slippage and the delay of phoning orders to a broker?


This matrix simply tells us how likely a price change was from the previous bar to the current bar. What sticks out to me is the change from -⅛ to +⅛ at nearly 36% but the change from -⅛ to another -⅛ is only about 11%, so a reversal is nearly 3x likely as they say. And the ratios are similar, going short from ⅛ to -⅛. The marginal probabilities show that moving more than ⅛ in a single bar is very unlikely.


Changes in the opposite direction occur 3x as often as changes in the same direction.


After price declines or rises by a certain amount the tendency to reverse is greater than when price is near neutral. You wouldn’t expect this if prices were independent.

 

1. the data suggest that large changes tend to be followed by large changes (not surprising)
2. The authors then go on to check second order changes
   1. “For simplicity in presentation and analysis, we have reduced the price movements to just five classes. We have combined into one class the changes of +2/8 and +3/8, and into another class the changes of -2/8 and -3/8.”
   2. Ever since I learned about Simpson’s paradox, I’ve been suspicious of re-formulating/re-shuffling data in any way, even if it’s for the noble purpose of making it ‘easier’ to analyze. I’m not sure if this is a problem yet.
   3. “A negative change is 1.32 times as likely after two consecutive negative changes as after a positive change followed by a negative change (.337 vs .256). In addition, a positive change is 1.32 times as likely after two positive changes as after a negative change followed by a positive change (.319 vs .241)”
   4. A reversal is more probable after a previous reversal than after a continuation; a continuation is more probable after a previous continuation than after a reversal.
   5. “a continuation is approximately 1 1/2 times as likely after three consecutive continuations as after two continuations. Furthermore, after four continuations, a subsequent continuation is 1.27 times as likely as after three. Unfortunately, a paucity of data (only 65 occurrences of three consecutive continuations) prevents us from pursuing this line of analysis here.” We’ll talk about the overall paucity of data later.

   
   
   
   
   

3. The authors suggest that this pattern of reversals followed by continuations is a natural part of the market making process
   1. When buy and sell market orders are equally numerous it will generate reversals
   2. When one type of order predominates, it will generate continuations
   3. Limit orders act like buffers, soaking up the demand for any particular order type until the limit orders are exhausted and then price moves again
   4. The larger the number of limit orders at a given price level, the longer it will take a sequence of market buy orders to break through it.
   5. “We conclude from this discussion that reversals are more common at even eighths than at odd eighths, and more common at integers than half integers.
   6. These mechanics are not peculiar to any one market

   
   
   
   
   

This doesn’t look very overwhelming though - does it?
“It is apparent that the solid line is taller than the dotted line at all positions in Figures 2 and 3. This indicates that continuations are more likely after two changes of the same sign than after two changes in opposite directions. But note that even after two changes in the same direction, reversals are still more probable than continuations at all eighths.”

1. The authors find a “slight preponderance” of rises in consecutive transactions over falls when looking at figs. 2 and 3, which means rising and falling is not exactly symmetric.
2. Limit orders cluster at 8/8 and 4/8
3. Limit orders act as a barrier to continued price movement

Conclusions
(1) There is a general tendency for price reversal between trades.
(2) Reversals are relatively more concentrated at integers where stable slow moving participants offer to buy and sell. There is a concentration of particular types of reversals just above and below these barriers.
(3) Quick moving competitors cognizant of these barriers can take positions at nearby prices, thus "getting the trade" and hoping to make a profit.
(4) After two changes in the same direction, the chances of continuation in that direction are greater than after changes in opposite directions.

It would be interesting to see if these properties of stock market prices hold in other markets.”
Indeed it would. I may pause to do an informal test in some common forex/commodity issues.

A note about the testing data. Is a slice of a single month in a trading year enough to say for sure that this phenomenon will hold up all year long? I would guess not.

Testing N&O's notions of advance/decline

The first thing to note is that it's no longer possible to test the idea exactly as formulated. Stock prices are no longer quoted in 1/8 of a dollar. They are also no longer quoted manually with a human-powered system that would give someone the theoretical ability to react at 'even eighths'. N&O were basically observing reactions between individual 'ticks' and that's just not possible in today's busy market except with sophisticated computer intervention, and using frictionless (fee-free) trading. Retail traders don't have that kind of access.

I don't see any advantage to trying to trade at even dollar amounts. If you see a way, or if you trade that way I'd be interested to know your thoughts.

So what remains is to see if there is a general 'dependence' of price on previous reactions. Can we create a method using patterns of advance/decline as N&O state. Specifically I'm interested in:

1. Does price generally reverse? But if so, how to measure that? Between fixed price levels, or between bars?
2. Does two instances of 'continuation' in the same direction increase the chances of further continuation in that direction? Again there are many considerations to take. How do we define each instance of continuation and reversal?

Not all Charts are Created Equally
I have to pause to explain a bit of my own theory so that you'll know why I've picked the following testing formats.

An ordinary chart is essentially an ostensibly useful abstraction of the overall tick data that composes the market. It tries to show us what's happening in a long time stream but compresses it to fit on a page or a screen. In order to make useful component 'parts' of a chart most of the time we fit price into time-based bundles, drawing a new unit at the start and end of each time unit. So if you're looking at H1 every hour a new bar is drawn, and we attach special importance to the opening and closing prices as well as to the high and low of each hourly bar. This is generally agreed to be useful because so many other traders are using the same basic assumptions. That's a logical assumption, but it's not the only way to think about price.

For many traders there is a distinction between 'useful/signal' and 'useless/noise'. Some price movements are 'real' and some are 'distractions'. Sometimes they claim the real movement is 'smart money' and the rest of it is 'dumb'. However you explain it there is a need to avoid reacting to every bit of information contained in a typical chart. Even if they haven't formally accepted the principle that we're exploring here, they may believe on some level that price reverts in the short term, and trends in the longer term, so they want a way to filter out (or define the boundaries of) the reversion. Almost every indicator does this on some level, but we can also 'quantize' the chart itself, requiring price movement of a certain amount before we accept that a bundle of movements represents a 'true' data point.
There are two common ways to do this:

1. point and figure charts
2. Renko charts

Point and figure charts divorce themselves from time to give you a picture of only rises versus declines. Renko charts 'block' rises and declines into packets of a given size that are uniform over time. I'm going to try both and in different ways, but first let's start with a simple standard bar chart analysis.

To make a truer comparison between my results and N&O's I should really be testing this on stocks, maybe even some of the same ones they used, if they still exist. However I thought I'd start with the trendiest instrument I can find because if dependence exists it should exist in a heavily trending instrument, no?

Here's a test using daily bars. I'm testing the 2-move continuation thesis. I use the standard range indicator but modify it so that it only finds sequential moves. Two up days in a row should spawn a third up day and if it doesn't that's a failure. Otherwise it's a success and I'll count follow-up bars in a trend. I mark successes with a checkmark arrow and failures with a red X.

I start at the new year, as always, but decide to start with 2020, just to get a little down action as well as rising trend action. After two years I get a picture that looks like this.


It looks random because it is random. About a 53% success rate. Was this unexpected? What if we only count the first sequence and leave off the remaining sequence in a trend? By definition trends always end with a failure so what if we only trade the first two-bar sequence and trade a new sequence only if there's an interruption by an opposite direction bar?

Attached Image (click to enlarge)

Does it look similar? That's because it is. A bit worse in fact 47%. I would want at least a 9% edge in any method before I'll put effort into it, and this (sticking with a daily bar trend) is close, so there may be something worth pursuing in this, but it's far from clear. What is certain is this is definitely not the 3x advantage that N&O found.

Maybe daily bars are too broad a quantity to find dependence? What about trading lower timeframes?

The same test using H1 bars.

Attached Image (click to enlarge)

After a month and a bit I think the picture is clearly similar, which is to say, random.

Let's take it down to the very smallest practical timeframe, M1, this time using a real stock chart - AAPL again? I'm looking at yesterday's trading data over 300+ minute bars.

Attached Image (click to enlarge)

Again, the fractal nature of price doesn't disappoint and the general conclusions we drew from other timeframes are valid on this one. However we get the best ratio we've seen yet - 58% successful. Maybe the problem is we need to evaluate this at a tick level, just as N&O did. However profiting from an edge at this level is going to be a technical challenge that I'm not keen on tackling at this stage of my trading journey.

Also worthy of note is that these results are paradoxical as I expected price to revert on lower timeframes more than continue and yet so far we've seen just the opposite on trendy instruments.

Is it worth examining a reversals strategy using bar charts now that we suspect the continuation patterns are fairly balanced around 50%? I guess not.

What isn't surprising is that finding profitable patterns using bar charts is far from a certain enterprise. If it was easy to find profitable bar sequences nearly every beginner trader would be profitable and the market itself wouldn't work.

What to try next? A different instrument? An even lower timeframe? If this were an academic exercise we would try everything just to be sure, but over the years I've learned that a pragmatic trader has to trust his experience and reject what appears to be a fruitless rabbit hole. We don't have the luxury of infinite time, and infinite computing power. However you should not let my reluctance stop you from doing your own tests if you think there's something here. If I can be of assistance, just let me know.

Let's move on to Renko and P&F chart tests next. Since those take a little more effort to set up I'm going to continue reviewing the rest of the book in tandem with the tests which I'll release as I finish them.

Chapter 3 - from Coastlines to Cotton Prices
Now we are getting to the good stuff. Well, for me at least.

1. Benoit Mandelbrot
   1. His uncle, Szolem was already a prominent mathematician and member of the Bourbaki club
   2. He was glad that Mandelbrot followed him into maths but dismissed Mandelbrot’s interests in geometrical solutions as frivolous
   3. While discussing his crazy dissertation ideas, Szolem reached into his wastebasket and pulled out a paper by George Zipf. “This is for you,” he said dismissively. “That’s the kind of silly stuff you like.”
   4. Of course, this doesn’t dissuade Mandelbrot but launches him on a productive path

   
   

2. George Kingsley Zipf
   1. Famously eccentric
   2. Spent his life arguing for a universal law of physical, social and linguistic phenomena
   3. “Zipf’s law said that if you constructed a list of all the things in some natural category, say, all of the cities in France, or all of the libraries in the world, and ranked them according to their size — you might rank cities by population; libraries, by collection size — you would always find that the size of each thing on the list was related to its rank on the list. In particular, the second thing on each list would always be about half the size of the first thing, the third thing on the list would be about a third the size of the first thing, and so on.”
   4. The article Mandelbrot read pointed out how the most common words in a text would be about twice as numerous as the second most common word, while the third most numerous would be about a third of the first, and so forth.
   5. Zipf’s Law is not rigorous on its own being a mix of numerology and guesstimation but it hints at something deeper since it actually does work in many contexts

   
   

3. Mandelbrot
   1. Wrote his dissertation on Zipf’s law without an advisor
   2. Had to push his thesis through bureaucratic channels on his own - highly irregular
   3. Often on the boundaries of respectability as he didn’t bow to academic pressures

   
   

4. Fractal Geometry
   1. The study of the geometry of roughness - think mountains, coastline, broken glass
   2. Mandelbrot realized that the varieties of randomness in nature are more extreme than the randomness you get from simple probabilistic games like coin tossing
   3. If Mandelbrot’s theories are correct, “almost everything” about traditional economics is “fundamentally flawed” (well, tell us something we don’t know)

   
   

5. Mandelbrot’s family history is quite interesting but long, so I’ll skip most of it except to note that the rigid Szolem fled Warsaw for Paris because studying under Sierpinski (of the famous fractal triangles) was too formal. There is always a bigger fish, or a worse boss.
   
   Next - the enimga of fractality - or why so many of our simple experiments just don't work.

Renko test
First let me say what a pain in the ass it is to get a renko indicator to work on mt4 without offline charts. I don't want to use offline charts as they are also a pain in the ass and tend to crash.

I tried about a dozen indicators before I found one that I liked (Tipu Renko) as it draws blocks directly on the existing price chart so we can see how renko distorts the actual price bars to create its own measurement 'quanta'. It has all the bells and whistles I need, and it's free not that I mind buying something that works. However it doesn't play nicely with FXBlue's trading simulator, and a renko block indicator is definitely something you want to test in 'real time' even if it's simulated real time as the way the blocks are drawn (and often, redrawn) makes all the difference in your strategy.

Finally I found something that draws blocks on the chart itself, and simulates trading with the actual renko charts in the trading simulator but it will only draw the blocks at fixed levels, which is fine, for what I'm doing. Just be aware that the actual performance of any strategy using renko depends a lot on the way the renko bricks are sized, and(re)drawn, and behave, which varies a lot from indicator to indicator. You can't say 'renko doesn't work' until you've tried a few different interpretations.

Anyway, this is on D1. We define a 'continuation' as a step up in price followed by another step up (or step down followed by another step down) and then we expect a third step where we'll take profit on the other side of the renko block, even if it hasn't been drawn yet. We'll put a SL at the other side of the second 'step' block the start of the 2-bar step. If price trends we'll re-enter on each new 2-block sequence without interruptions, for now. This means, by definition, every trend is going to be bookended by a lost trade, unless it immediately reverses before breaking our stops but that's practically inevitable if you don't specify an intermezzo/recalculation bar or period.


Example of a trade in progress, with a 2-brick upward move. The observant reader will note that this creates a 2:1 RR so this method can never work, right? It should win slightly more than it loses but when it does win, lose twice as much, which is not allowed!

In this case a picture is worth a thousand words.

Attached Image (click to enlarge)

Here are the results up to the current date. The magenta lines are the failed, losing trades. The observant among you will notice there are no magenta lines because in fact every trade taken was a winner. Now, you obviously can't trust this result entirely. It's full of 'future leakage'. I was able to pick the best block size for the period and I used some extra finesse when entering and exiting, but the point is well made, surely, that this is superior to the naive bar chart system. Way way superior. Note that I didn't trade this perfectly though - I missed one trade. Can you find it?

If you've really been paying attention you may recognize that this is something like what you would get if you traded Hearne's system from 'Reminiscences' or to a lesser extent Old Partridge's.

What is it about this that makes it work?
a) fewer trades
b) a wider stop to account for random fluctuations
c) avoidance of 'artificial' cues
d) pure luck
e) quantifying the potential movement
f) some of these
g) all of these

Attempting to answer questions like this are what separates a pro from a dilettante, and a scholar from a mountebank. Note I said 'attempting' because sometimes you discover something interesting even if you don't come up with an answer.

All right, I think I've said enough about this side quest. Back to the book!

How long is Britain’s coastline?

1. Seems easy to answer at first until you realize the surface area of a coastline just keeps increasing the more accurately (minutely) you measure it.
2. This is paradoxical as with most other measurement problems more accuracy brings you closer to the truth, but with fractal geometry, the more you ‘zoom in’ the more different your estimate gets
3. Mandelbrot wrote a paper addressing this paradox in 1967
   1. Mandelbrot didn’t coin the term ‘fractal’ until 1975
   2. Fractals have a property called self-similarity
   3. You can carve out any piece of a fractal structure and zoom in on it and it will look much like its parent shape(s)
   4. And you can carve up smaller and smaller pieces and they still look similar to each other when examined closely
   5. Self-similarity is an ubiquitous feature of nature and you can find it everywhere - tree branches look like little trees themselves, river streams look like larger rivers
   6. This extends to the social world as well as the physical. A battle is made up of smaller skirmishes that make up a whole war.

4. WW2
   1. Mandelbrot’s father has a close call with the Nazis after being captured and sent to a camp. The camp was liberated by freedom fighters but then the escaping prisoners had to flee to Limoges
   2. In a scene that is practically identical to the story told by George Soros about his own father, Mandelbrot’s father realizes that following the herd will be disastrous and being unable to convince the others he heads out alone to a nearby forest. He hears the sound of a German dive-bomber strafing the other prisoners.
   3. V2 Rockets that hit London clustered in random patterns that are typical of true randomness, but observers might fall prey to the same fallacy as the roulette player who believes that a particular number is ‘due’.
   4. Working out the odds of how many bombs would land in a particular area is not that difficult as the randomness of this type of behaviour is relatively mild
   5. The rockets were fired systematically, and aimed roughly so the odds are similar to a roulette ball falling into a particular slot

5. Modern financial theory
   1. Still makes use of this naive understanding of randomness
   2. Recall Bachelier and his use of the normal distribution for prices
   3. Osborne realized prices are log-normally distributed, and returns are normally distributed
   4. Mandelbrot’s major achievement was discovering that randomness has several characters, not just the ‘mild variation’
   5. A lot of natural phenomena match the normal distribution but many things don’t
   6. Weatherall uses the bizarre example of a drunken firing squad (as opposed to a drunken walker in a hotel hallway)
      1. Most of the drunk firing squad’s bullets will hit in the centre
      2. However the riflemen could be facing any way and stumble any way at all
      3. That means some of the bullets will be wildly off the mark, even directly away from the target
      4. This type of distribution, with extra and extreme outlying results is called a Cauchy distribution
      5. Mandelbrot called this ‘wild’ randomness

1. “The world is a place dominated by extremes, where our intuitive ideas about averages and normalcy can only lead us astray.”
2. Mandelbrot had a ‘freakish’ ability to visualize abstract algebra problems geometrically
3. Vilfredo Pareto
   1. Famous namesake of the Pareto principle (80/20 rule)
   2. Studied income distribution
   3. 80% of wealth is controlled by 20% of the population

4. Hendrik Houthakker - economist at Harvard
   1. Mandelbrot goes to work for IBM and studies income distribution as part of their effort to start selling personal computers
   2. Just before a talk he delivers to Harvard’s economics department he meets with Hendrik Houthakker who has a graph on his chalkboard that is identical to the one that Mandelbrot plans to use in his talk
      1. Mandelbrot assumes the graph is of income distributions but Houthakker is confused
      2. Houthakker’s graph is of cotton price returns
      3. They are the same graph because the principles are the same and this is a major breakthrough for Mandelbrot

   3. Houthakker expected that small price changes would dominate cotton markets if Bachelier’s theory was correct
      1. However in practice there were many large price changes

   4. Houthakker is ready to abandon his project and he gives Mandelbrot all his cotton price data
   5. Mandelbrot confirms there is no such thing as an ‘average’ rate of return for cotton
   6. He was familiar with Paul Lévy’s work as well

5. Paul Lévy
   1. Mentioned in Taleb’s works as well in relation to ‘Lévy flights’ which are one theory of how price moves
   2. Lévy’s work on random processes led to a class of probability distributions being called ‘Lévy-stable’ distributions
      1. Normal and Cauchy distributions are Lévy stable
      2. But there is a spectrum of randomness between the two
      3. ‘Wildness’ can be measured by a number called alpha
      4. Normal distributions have alpha 2
      5. Cauchy distributions have alpha 1
      6. The lower the number the more wildly random the process
      7. Distributions with alpha < 1 don’t satisfy the law of large numbers
      8. An average value for a quantity that wild can’t be defined
      9. Distributions between 1-2 have average values but have no well-defined average variability/volatility/variance which makes it hard to calculate averages even if they exist in the data
      10. The cotton price data’s alpha was 1.7

1. Mandelbrot begins collecting data on many different markets to see if this holds true for all of them- it does. They all have alphas < 2.

Next - where does this leave us?

So where does this leave us?

1. At some point econometricians like Cootner had to decide if they were going to follow Osborne’s simpler but flawed theory or Mandelbrot’s more complex but possibly unworkable theory
2. They chose Osborne and this makes pragmatic sense
   1. “when it came time for economists interested in financial markets to try to extend the ideas presented in Cootner’s book, to put the randomness of stock market prices to work by using statistics to predict derivatives prices or to calculate the amount of risk in a portfolio, they had to pick between the simple theory that gave good results the vast majority of the time and the more cumbersome one that better accounted for certain extreme events. It made perfect sense to start with the simpler one and see what happened. If you make good assumptions, if you idealize effectively, you can often solve a problem that otherwise couldn’t be solved — and get a solution that is quite close to correct, even if some of the details are wrong.” (very smart mathematical/engineering types often are unable to make this compromise and they don’t thrive in trading)

3. Weatherall slanders Mandelbrot (maybe)!
   1. In his 1963 papers Mandelbrot stated that Levy-stable distributions have infinite variability
   2. It should be noticed in his later books like ‘Fractals and Scaling in Finance’ he amended his claim to ‘probably infinite’.
   3. “After almost fifty years of research, the consensus is that rates of return are fat-tailed, but they aren’t Lévy-stable. If this is correct, as most economists and physicists working on these topics believe it is, then the standard statistical tools do apply, even though the simplest assumptions of normal and log-normal distributions do not.”
   4. However, what about in 100 years? 200 years? The big black swan is always over the horizon. This makes it unlikely that Mandelbrot was wrong. Of course it doesn’t matter once one is dead.
   5. Weatherall accuses Mandelbrot of making claims that are too aggressive, but again, it all depends on the time scale on which one is thinking. Count no man happy until he is dead, etc.
   6. Despite all this of course Mandelbrot got the later success he deserved

Ok so Weatherall is drawing a procession of intellectual giants much like the Copernicus > Galileo > Newton > Einstein lineage of classical physics.
For finance it’s Bachelier > Osborne > Mandelbrot > can you guess who’s next? It’s someone we’ve talked about before.

It's Claude f*cking Shannon, that's who! The inventor of modern information theory (by way of Ed Thorp, who is also pretty clever).


Chapter 4 - Beating the Dealer

1. Edward Thorp
   1. Successful hedge fund manager
   2. Specialized in the math of quantum physics
   3. Interested in strategy games
   4. Actually traded (gambled), unlike the academic characters we’ve seen so far
   5. A faulty slide rule prevented him from coming in first in the chemistry test to get a scholarship to UCLA so he entered the physics test and got first place and the scholarship
   6. Thorp needs extra cash for graduate school so he considers roulette. If roulette wheels really don’t have any flaws that should make the mechanics of how a ball moves predictable.
   7. He buys a half-size wheel and filmed a ball moving around it (how did he afford the wheel and the camera if he has no money?) but concludes the toy wheel he’s using is useless for experimentation
   8. Thorp finishes his PhD and gets the CLE Moore instructorship at MIT, the job that John Nash (Beautiful Mind) held
   9. At MIT Thorp meets

2. Claude Shannon
   1. Inventor of information theory
   2. Info theory is the mathematical underpinning for computer science, telecommunications, cryptography
   3. Info theory grew out of a secret project at Bell Labs, an encrypted telephone system for use by generals in a war - SIGSALY that worked by adding noise to the line and then subtracting it with the opposite noise pattern on the other end.
   4. Shannon concluded that the value of information is directly related to how unexpected it is by the receiver on the other end.
      1. The easier it is to predict information the less valuable it is
      2. Information is valuable (cherished) because it offers certainty
      3. Shannon discovered a way to assign a number to a message that measures the amount of information it contains, which in turn was the first major step in building a mathematical theory of information.

3. Blackjack
   1. Thorp goes back to Vegas to try his hand at a strategy devised by the US army (weird)
   2. It doesn’t win but he thinks he can improve on it
   3. He combines a card-counting technique with the army strategy and publishes a paper on it
   4. This is his way to get access to Shannon who at that time was an academic rockstar who refused to spend time on projects or people that weren’t interesting to him
   5. Thorp gives a talk about his strategy at the American Mathematical Society and it attracts national media attention - an MIT egghead taking Vegas to the cleaners
   6. Thorp boasts that with funding he could ‘beat the dealers’ and one casino even fronts him free room and board but not his stake
   7. Casinos have minimum bets (think: lots) and Thorp needed more cash to survive an early losing streak (as do traders)

4. Manny Kimmel
   1. A “legitimate businessman” who made his fortune in gambling and booze offers to front Thorp the money if he can prove the system works
   2. Thorp plays against Kimmel and ‘Eddie the Hand’ for about a month before Kimmel fronts him $100,000.
   3. Thorp reduces the amount to $10,000 and they play in Reno as it’s lower profile
   4. 10k becomes 21k - a story told in ‘Beat the Dealer’
      
      Next: Money Management

so those small moves are random ? I have tested few mean reversion strategies without success.
simple breakout strategies performed very well on backtest and poorly on a live acc.. (I can explain why)... so my idea was to combine mean reversion with trend following and trade only in the direction of the big move.. but it can get very complicated.btw I already tested this many times.. for fun (leverage, no diversification)...

From a fractal point of view they are not any more random than the big moves but it's hard to take advantage of small moves with the practical considerations of spreads and trading costs. What appears to be random is that short sequence of bar/candlestick moves. What appears to be reliable is that a trending move follows a sequence of trending moves. Is it 3x as likely as a reversal as N&O expected? I don't know.

Does price mainly reverse for most moves as N&O stated? This would be a good test to do with the renko blocks.
Suppose we came up with a quick set of rules.

1. The signal is the formation of a pair of opposite-coloured blocks.
2. So if there is a long block followed by a short block that's a long signal
3. We put a pending stop entry at the top of the second signal block. If price never breaks that level, we filter out some bad trades (and it's possible that this alone makes the system profitable, not that reversals are more common)
4. We put a stop loss two blocks away from the entry. Two blocks, not one, because the block itself is the measure of the price variance, so we expect price to hit both sides of it often.

Using this very simple method, a quick glance at the daily S&P500 chart shows that price has only actually reversed three times and, we would have won once and avoided (probably) entering two losing trades.

In any case who is mad enough to trade US equities short right now? Old Partridge voice: "It's a bull market you know".

Well, let's say we are mad enough. Here is a snapshot of a few days using the same rules but a smaller block size on M15. Assuming that price reverts more often on lower timeframes seems to bear out, at least for this small window. We can see the win/loss rate is close to 5:1. 10 wins, 2 losses, for a net gain of 6 blocks. I'm sure we can think of some ways to refine this.


If we tested this on a yet lower timeframe or on a more mean-reverting security (like any major forex pair) we will surely get different results. If I don't seem that enthusiastic about testing this more it's because I share your ambivalence for mean-reversion strategies but for a different reason.


My own view is that reversals are simply fractal generators, a 'roughness' pattern-maker based on the initial condition. So a reversal is a mini-trend and if we are getting surprised by the initiator (parent trend) we either need to find a way to cut losses or trade the trend we can capture. This may be as simple as adjusting settings or it may need a whole different approach.

Money Management

1. When you have an edge over the house the next big question is how to much to bet
   1. Bet too much and one unlucky hand can lose everything
   2. Bet too little (or not at all) and you might as well do something better with your time as the edge is un/under-used

2. Shannon introduces Thorp to a paper written by his colleague at Bell Labs,

1. John Kelly Jr.
   1. “Kelly’s work provided the essential connection between information theory and gambling — and ultimately the insights that made Thorp’s investment strategies so successful.”
   2. A ‘wild man’ who flew planes under bridges, shot guns in his house, WW2 ace
   3. Quantum physics, encoding tv signals, computers that synthesize human voices
   4. The Belmont stakes thought experiment
      1. In the first half of the 20th century there was a delay in communicating horse racing results between bookies
      2. In theory if you had fast method of communication you could find out about the results of a horse race before betting closed
      3. If the messenger was reliable (trusted) and the line was clear you’d be wise to bet as much as you could as there is no risk
      4. But what if there’s noise on the line and you can’t make out what you’re being told?
         1. If you can’t make out anything you’d simply pick the favorite to win
         2. If you could hear one letter you’d maybe be able to rule out one possibility and then you might divide your bet between the favorite and the other horse with that letter in their name
         3. This would be good enough to make the partial information valuable enough to decide how to bet

2. “Shannon’s theory tells you how much credence to give a message when there’s a chance that the message is being distorted by noise, or when the level of noise makes it difficult to interpret the message in the first place.”
3. Kelly worked out how to maximize the long-term growth of the money you start with
   1. The Kelly Criterion (this is featured in so many trading books, a couple of which are on the reading list, and go into it in more detail, so I’m going to leave out the specifics for now)
   2. If your advantage is zero or worse, don’t bet
   3. Otherwise bet the fraction advised by the Kelly criterion
   4. “One of the most surprising things in Kelly’s paper, something that feels almost mystical, is a proof of what will happen if you follow his rule in a scenario like the horse-betting story, where you have a stream of (partial) information coming in: if you always use the Kelly criterion, under certain ideal circumstances your wealth will increase at exactly the rate that information comes in along the line. Information is money.”

4. Meanwhile Thorp and Shannon were working on his wearable-computer for roulette
   1. Their computer could not predict exact numbers but it could narrow it down to an octant (⅛ of the wheel) narrowing options down from 38 to 5.
   2. The device was the size of a pack of cigarettes and used foot switches https://www.youtube.com/watch?v=z2Whj_nL-x8
   3. The roulette experiment is supposedly a middling success, due to technical difficulties, but they don’t pursue it

5. The Stock Market
   1. Thorp’s early forays in investing were not successful
   2. He read ‘The Random Character of Stock Prices’
   3. He subscribed to a publication about warrants that treated them like lotto tickets
   4. “A warrant is a bet on how a stock will perform over a fixed period. The price of the warrant, meanwhile, is a reflection of the market’s determination of how likely the buyer is to win the bet.”
   5. Thorp operates according to Osborne as though stock prices are random but lognormally distributed
   6. “Thorp had an ace up his sleeve, something Bachelier and Osborne never imagined. With five years of gambling experience, Thorp realized that calculating a “true” price for a warrant is a lot like calculating the “true” odds on a horse race. In other words, the theoretical relationship that Thorp discovered between stock prices and warrant prices gave him a way to extract information from the market — information that gave him an edge, not in the stock market directly, but in the associated warrant market.” I don’t actually understand what this means. What is the relationship between warrants and prices that he learned from gambling??
   7. After using a computer to make the necessary calculations however Thorp finds out that all the warrants are trading too high. The ‘lottery tickets’ were just that - overpriced and worthless.

6. Short Selling
   1. In the 1960s it was viewed as dangerous, depraved or unpatriotic (things haven’t improved much to be fair)
   2. The Chicago School of economics changed this somewhat but short-selling serves a good by keeping markets efficient
   3. Thorp realizes if he sells warrants short but at the same time buys shares of the underlying stock you can protect yourself against the value of the warrant unexpectedly increasing in value
   4. Profit can then be guaranteed unless the price moves ‘dramatically’
   5. This is called ‘delta hedging’
   6. Thorp consistently makes 20% for 45 years.
   7. In 2008, one of his worst years ever, he made 18 %.
   8. Thorp writes ‘Beat the Market’. That is mostly ignored. Mostly…

7. Jay Regan
   1. Perceives Thorp’s genius and proposes they start a hedge fund
   2. Although ‘hedged funds’ existed prior to this, the idea behind delta hedging has become synonymous with modern hedge funds so much so that the term may as well have been coined for them
   3. Regan handled the business
   4. Thorp handled the research and maths
   5. Convertible Hedge Associates changed its name to Princeton-Newport Partners

8. Ralph Gerard
   1. Dean of UC Irvine is looking for someone to manage his money
   2. He asks his ‘old money manger’ who is moving on to new projects to check out Thorp and co.
   3. Thorp agrees to drive down and play bridge and chat with the departing money manager
   4. The old money manager is leaving the MM business to rebuild a ‘manufacturing and textiles company’
   5. Thorp and the old MM talk about probability theory and nontransitive dice, which of course Thorp knows all about so of course Gerard invests based on the money manager's endorsement
   6. Thorp leaves the meeting convinced the old MM will one day become the richest man in the world. That man was …

9. Warren Buffett
   1. Warren f*cking Buffett!

10. Princeton Newport Partners
    1. Becomes hugely successful but
    2. Explodes when Regan is raided and accused of ‘stock parking’ with junk bond dealer Michael Milken
    3. The scheme was an elaborate tax dodge
    4. Thorp apparently knew nothing about the illegal practice
    5. He moves on to manage his own money, having ‘ushered in’ a switch in MM to “quantitative, computer-driven methods.”

i much prefer marekt wizards

They are excellent books, that's for sure. Which version of the Market Wizards is your favorite, ReeceRispoli or for which version would you be interested in a summary?

Chapter 5 - Physics Hits the Street
This chapter is all about options and the Fischer-Black-Scholes-Merton pricing methods, and CAPM. For the most part I see this as being of interest for investors with long-term portfolios and it's all based on a flawed methodology that has since been supplanted.

I don’t know if I care enough to summarize it. Ummm..
Yeah, I don’t. Basically

1. The Black-Scholes method has fallen into abeyance in favour of more Mandelbrotian approaches
2. The ‘volatility smile’ is a chart pattern that anticipated the 1987 crash which was caused by portfolio insurance which was a derivative of Black-Scholes
3. The O’Connor brothers were able to avoid the crash because they had some understanding of the dangers involved with Black-Scholes

Sorry, if you want the whole story, I recommend picking up the book

Chapter 6 - the Prediction Company

1. James Doyne Farmer & Norman Packard
   1. Farmer was head of Complex Systems at Los Alamos National Laboratory, famous for having been headquarters of the Manhattan Project
   2. Packard had left a tenured position as associate professor of physics at U of Illinois

   
   

2. The Prediction Company
   1. Goal: do the impossible by predicting the behaviour of financial markets
   2. 3 decades of experience in nonlinear forecasting
   3. NLF - identifies the order ‘at the edge of chaos’

   
   

3. Frank Oppenheimer
   1. Brother of the famous Robert J
   2. Accused of Communism during the height of McCarthyism
      1. Worse than that - he was guilty of having joined the party for 3 years during the Depression!
      2. Blacklisted
      3. Forced to sell one of his inherited Van Goghs to live (clutch pearls now)

      
      

   3. By 1959 things had cooled off enough that he could get a job in science education
   4. At the U of Colorado he meets

   
   

4. Tom Ingerson
   1. Physics major at UC Berkeley
   2. Came to work on general relativity in Colorado
   3. Ingerson made the cardinal sin of mentioning Frank Oppenheimer as a reference
   4. This kills his career and he ends up in Silver City New Mexico
   5. He volunteers with a local boy scout troop
   6. One member of which is 12-year old Doyne Farmer who decides he also wants to be a physicist
   7. Ingerson starts Explorer Post 114 - like boy scouts but the focus is on tinkering and building things like ham radios and dirt bikes
   8. One day a boy who is too young to join, 12-year old

   
   

5. Norman Packard
   1. Gives a talk about new radio technology
   2. Packard and Farmer become friends

   
   

6. Edward Lorenz
   1. The Prediction co’s ideas were first developed by him
   2. MIT meteorologist
   3. Thought computers could assist in forecasting, both of which were in neophyte stages
   4. His colleagues were skeptical but one day Lorenz leaves his ‘Royal McBee’ computer on with a wind simulation and gets a totally different set of data than he did the first time, which is impossible unless the computer is broken
   5. What happened is he had rounded up his world state by 3 figures
   6. Lorenz accidentally discovers Chaos Theory though he calls it “sensitive dependence on initial conditions”

   
   

7. James Yorke and Tien-Yien Li
   1. “Period Three Implies Chaos.”
   2. Extremely small differences in the state of the system at a given time would quickly explode into large differences later on
   3. Even simple systems like pendulums and water wheels are sensitive to initial conditions

   
   

8. “Predictability: Does the Flap of a Butterfly’s Wings in Brazil Set Off a Tornado in Texas?”
   1. Lorenz’ paper that produced the notion of the ‘butterfly effect’

   
   

9. Packard and Farmer try their hands at gambling
   1. Packard uses Thorp’s book - Beat the Dealer
      1. He has some success but always loses his progress during a losing streak
      2. Later he realizes the casinos are cheating card counters with simple methods like crooked dealing (so many parallels to forex)

      
      

   2. Farmer memorized AH Morehead’s book ‘Complete Guide to Winning poker’
      1. Despite his awkwardness he cleans up in Missoula as ‘New Mexico Clem” (good name)

      
      

10. After comparing notes Packard and Farmer try to reproduce the roulette computer of Thorp and Shannon
    1. They reinvent the wheel (literally) knowing nothing about how the original was made
    2. They call themselves Eudaemonic Enterprises
    3. They rent a professor’s house and tinker with roulette wheel experiments
    4. The group grows and becomes part of the Yippie movement - Groucho Marxism is what they call their brand of rebellion
    5. The plan is to take ‘the man’s’ money and build a commune
    6. After some initial difficulties like burning wires, the team starts to profit (?? - very suspicious)

    
    

11. Lorenz
    1. Was the first person to recognize that there is a kind of order in chaotic systems:
    2. If you draw pictures of the paths traced by objects obeying differential equations, they tend to settle down into regular patterns.
    3. These patterns are called attractors, because they tend to attract the paths of the objects.
    4. In roulette the attractors are the pockets of the wheel: whatever trajectory the ball takes, in the long run it will settle down into one of these states.
    5. Attractors have a highly intricate fractal structure

    
    

12. Dynamical Systems Collective
    1. An attempt to form a research centre for chaos theory which at that time didn’t yet exist
    2. Interested in prediction by its very nature

    
    
    

13. Emergence
    1. simple large-scale structures can emerge from underlying processes that don’t seem to have that structure.
       1. For example - hurricanes are built up of simple atmospheric particles
       2. Ants form colonies which are complex systems
       3. Humans form terraforming machines that cannot turn themselves off

       
       

14. Santa Fe Institute
    1. A new research center devoted to study of complex systems; interdisciplinary
    2. International Finance as a Complex System conference
       1. The physicists find the banker/economist models childish
       2. The bankers find the physicists to be talking nonsense

       
       

    3. Not until the third conference, where economists are not invited, but just the traders themselves do things start to ‘gel’
       1. “The traders proved much less defensive than the economists, and by the end of the conference each group had gained an appreciation of what the other had to offer. Farmer and Packard, in particular, left with a clearer sense of how practical trading strategies worked.”

    
    

15. Farmer and Packard finally start to build their company
    1. They have an anti-establishment view, planning to ‘eat the rich’
    2. However they have the sense to hire Jim McGill as their CEO

    
    

16. Farmer and Packard didn’t use chaos theory as a meteorologist or a physicist might
17. They didn’t:
    1. attempt to find the fractal geometry underlying markets, or
    2. derive the deterministic laws that govern financial systems

    
    

18. Their experience with chaos theory helped them
    1. See how patterns with real predictive power could be masked by the appearance of randomness.
    2. Apply the right statistical measures to identify truly predictive patterns
    3. Test data against their models of market behavior
    4. Figure out when those models were no longer doing their job
    5. They could easily apply some of Mandelbrot’s ideas for risk management in ways that people with more traditional economics training could not

    
    

19. Statistical arbitrage
    1. Pairs trading
       1. Coke and Pepsi’s stocks trade in tandem most of the time
       2. When they stop doing that, buy one and sell the other until they do start copying each other again

       
       

    2. They didn’t invent pairs trading but brought a new level of rigor to it

    
    

20. Genetic Algorithms
    1. Possible solutions to a problem compete to find a shortcut to the solution
    2. Black box models - scary because you don't’ know exactly how they work

    
    

21. Craig Heimark
    1. A year into self-funded operations they had no investments and no buyers
    2. They were being coy and picky but now they needed an angel
    3. After Farmer gave a talk at a computer conference he met Heimark of O’Connor and Associates (the firm that made its fortune by successfully modifying the Black-Scholes equation to account for fat-tailed distributions)

    
    

22. David Weinberger
    1. Another O’Connor partner
    2. An early quant
    3. And then

    
    

23. Swiss Bank corporation buys O’Connor and the Prediction Co.
24. It becomes UBS, one of the largest banks in the world and one of the largest forex players in the world
25. “The Prediction Company, following the O’Connor tradition as a secretive high-tech firm, never released any metrics of its success publicly — and none of the former principals or board members with whom I spoke were authorized to share any concrete information. This might seem suspicious. (yes)
26. On Wall Street, success breeds imitation, and the more firms there are implementing a strategy, the less profitable it is for anyone.
27. One former board member [Weatherall] pointed out it is still an active subsidiary of UBS, after more than a decade.
28. Another knowledgeable source told me that, over the firm’s first fifteen years, its risk-adjusted return was almost one hundred times larger than the S&P 500 return over the same period.” (I guess I just have to trust you, Weatherall)
29. “By the time they left, they had made their point: a firm grasp of statistics and a little creative re-appropriation of tools from physics were enough to beat the Man.” LOL. yeah, they’re really wresting control away from the military industrial complex. Look at these guys.

REBELS INC! I mean, look, they’re not even wearing ties!

Weatherall, padding out the chapter, goes into a sideline about black box models and how they along with algo trading are not really responsible for the 2008 financial crisis, but since I already agree, and I don’t think algo trading really got more than a D&D-style 'satanic panic' for a few months afterwards; it’s all worth skipping.

What do you think? I think it’s unclear...at best... that these guys actually made their fortunes by prediction instead of simply being absorbed into one of ‘the largest banks in the world’ and enjoying the corporate gravy train. There are all kinds of ways to seem to be predicting something that is as emergently complex as financial markets while actually engaging in academics far removed from practical considerations.
Still, I’m on the outside looking in, with no data to support my skepticism apart from an increasing number of quant firms selling courses on data science instead of actually trading.

Chapter 7 - Tyranny of the Dragon King

Attached Image (click to enlarge)

1. Whereas the other chapters usually involved multiple characters this one is all about
2. Didier Sornette
   1. A French academic who is most famous for his theory about ‘dragon-kings’ which are merely dressed-up Talebian 'black swans' who announce their appearance with a series of smaller fore-running events
   2. The example used in relation to Zipf’s law is the city of Paris. Most of France’s cities follow Zipf’s law reasonably well, but Paris is an outlier in size even for the number one ranked city
      1. But surely even Weatherall will see how this is susceptible to anchoring bias. Maybe Paris is freakishly large for its surroundings but we notice because it’s what we’re looking at. There are probably a lot more medium-sized cities that don’t rank properly based on their size but we don’t care as they’re easy to overlook. This is a problem with Zipf’s law, not proof of existence of a separate class of uber-black swans.

      
      

   3. Sornette claims to have predicted the 1997 ‘crash’ (which curiously happened post-2000) and other crashes as well, and some ‘anti-bubbles’ which are artificially low prices that will expand in the future
      1. However I can’t find a single public prediction that he has made lately, highly suspicious
      2. His wiki article is barely notable enough to avoid deletion

      
      

   
   

3. The most interesting part of this chapter is about self-organization in kevlar (the bullet-proof material)
   1. Kevlar is very strong but in some cases its structure can self-organize to cause catastrophic weaknesses
   2. It can rip apart along these fault lines under special, mysterious conditions

   
   

4. Similarly, earthquakes might happen after a series of smaller quakes anticipate the arrival of the big ones
   1. However I don’t think it’s possible to use this in prediction

   
   

5. There is some notion of the Mungerian ‘Lollapalooza’ in his worldview
   1. Big events/catastrophes are driven by multiple smaller events that foreshadow their arrival
   2. So for example, we didn’t know when Covid would strike but we had lots of earlier infections that anticipated its arrival
   3. We know climate change is going to get worse as we already see its effects
   4. The Euro is a perennial weakness
   5. US-China aggression will worsen

   
   

6. The thing is all these common-sense forecasts are well and good but without a time horizon they are just as useless as technical analysis or saying a big crash is coming. Yes, for sure, but when?? That’s the only part that matters and if Sornette can’t answer that this chapter is all dragon’s breath. Words are wind, Khaleesi.
7. “If you think of Mandelbrot’s work as a revision to the early accounts of random markets, pointing out why they fail and how, then Sornette’s proposal is a second revision. It is a way of saying that, even if markets are wildly random and extreme events occur all the time, at least some extreme events can be anticipated if you know what to look for. These dragon kings can upend the entire world economy — and yet they can be studied and understood. They are the stuff of myths, but not of mystery.” If you know what to look for, but this chapter does not tell us what to look for. We are told about ‘log-periodic patterns’ but those are never explained.
8. However a little extra-curricular research is never a bad idea and this article gives us a clue from Sornette’s book ‘Why Markets Crash’

https://financial-market-commentary.com/2014/02/

First of all, what crash did this augur?? The little October dip? That’s not a crash.
It seems this is an example of a failed prediction using ‘log-periodic’ wobbles.

In interviews Sornette calls the markets ‘dysfunctional’ as a result of Fed-fuelled madness and it’s hard to disagree, but I also think I detect the bitterness of a seer whose all-knowing eyes have begun to play tricks on him.
You’ll note that obviously the market has to correct after such a series of upward moves otherwise it will just keep going up. That’s all it can do, one of those two things (apart from going sideways for an extended period, which seems to be impossible in US equities).The problem is it can do this for as long as it wants. If you think you can use this to buy put options and cash in on a big crash, good luck to you sir!

Next: a new Manhattan Project?!