S/R and TL on Randomized Price Movement

"Lo and MacKinlay find that markets are not completely random after all"

Pretty sure we discovered this a long time ago. Looks like an interesting read though.

Hi everyone,

These charts are really fascinating. However, I don't think anyone has pointed out that rand() in Excel is at best a pseudo-random number generator. It does not produce truly random series of numbers.

So, I would suggest that the "tradable patterns" are really a manifestation of the non-randomness of the Excel function. Just like patterns in the real market are a manifestation of its non-randomness.

You're right, rand() is not the best function but you wouldn't be able to trade its numbers succesfully. It is not so flawed as you may think. If you could somehow measure its pseudo-randomicity you'd see that it doesn't deviate enough to increase your winning odds substantially.
The market is not random at all but exhibits behaviours like the random numbers because it is the property of any series of numbers to behave like that. You either have a right line or you start seeing patterns.

This probably is true. I have been suspected that it somehow create cyclically random numbers. Sadly, even computational package like Matlab has problems in generating random numbers, each time I turn on it, it generates the same random number series. It is ridiculous, isn't it?

anyone know how to measure the randomness of a set of data? i think it's better to state the randomness with a number (like 90% random) than just saying "not completely random".

This is not a question easy to answer, but it is a branch of knowledge in information science:

http://en.wikipedia.org/wiki/Information_entropy

kk007 - sorry I didn't post my excel file. I didn't see your post at the time, but it looks like you were able to generate the OHLC chart anyway.

Mikkom - thanks for posting the link to the paper on the non-random behavior in stock and bond markets. I'd like to see an analysis for forex as well, but it could be that the academics are right for all markets. It would certainly make me feel better about using technical indicators, trendlines, etc.

I think the forex markets have a general direction which is exploitable, determined by carry interest and fundamentals and news. Also, the markets move more during the London and early NY sessions, so this is a good time to look for breakouts out of a tighter range (also early to mid Tokyo session to a lesser degree). Thanks to Mikkom for linking to his posts on highs and lows for EURUSD, this matches my anecdotal observations.

Hanover, your comments on roulette are apropos. I once made a roulette simulator in excel and tried to find an edge using ranges and trends - there isn't one. It could be that the random charts posted here have a real difference from real price charts since some traders feel like they have found an edge. I certainly find inside bars to be a good trade. It could also be that they are benefiting from fundamental drift, carry interest, and increased ranges per Mikkom's chart.

For those are lazy to test the excel file. Here are 10 charts generated. They are not selected intentional. Price action is so real that I am not convinced it is random.

No problem. I have not worked out the method of creating a OHLC before seeing yours.

But Microsoft told me that they no longer use the pseudo-random number generator in their 2007 version.

"It (2007 version) passes the same battery of standard tests. The battery of tests is named Diehard (see note 1). The algorithm that is implemented in Excel 2003 was developed by B.A. Wichman and I.D. Hill (see note 2 and note 3). This random number generator is also used in the RAT-STATS software package that is provided by the Office of the Inspector General, U.S. Department of Health and Human Services. It has been shown by Rotz et al (see note 4) to pass the DIEHARD tests and additional tests developed by the National Institute of Standards and Technology (NIST, formerly National Bureau of Standards)."


http://support.microsoft.com/kb/828795

I know that I REALLY should not bother wading into these theoretical debates. But I honestly think it's important that traders understand these concepts, so I'm going to try to shed some light on this. Lord, I don't know why I do this...

First, you need to understand that this is not a Boolean equation. One must not either determine that "The markets are random" or "The markets are driven by fundamentals" or "The markets are driven by emotions". The answer to all three of those statements is, "Yes!"

The markets are affected (sometimes dramatically) by fundamentals. If you think you've proven that the markets are purely random, stick around during the next GDP or non-farm payrolls announcement and then try to convince me that it's all random. But just because fundamentals drive markets, doesn't mean that fundamentals account for ALL of the activity in the markets.

The markets are affected (sometimes dramatically) by emotions. The most glowing US GDP forecast will not do a dang bit of good if the whole world believes that the US is about to crumble like the Roman Empire.

And yes, the markets are affected by randomness.

To illustrate this, think about two opposing sports teams. Most of us would agree that all sports involve some degree of skill. Generally speaking, we assume that the most skilled team wins the match. But what happens if two teams are absolutely perfectly matched in every possible aspect of the game? What happens if a team could theoretically play itself?

The answer is that the relative skill of each opponent would get canceled out. Even though each opponent is skilled, and even though skill typically plays a huge role in the outcome of sports contests, in this particular case, skill would actually play NO role because the skill factor would be entirely neutralized. In this case - with two absolutely equal teams playing each other - the outcome is determined almost entirely based upon "luck" (e.g., randomness).

So think of the market as a constant contest between the Bull team and the Bear team. Just as the relative strength of sports teams rises/falls across the span of weeks and seasons, the relative strength of the Bulls and Bears rises/falls across the span of minutes/hours/days/weeks/years.

Sometimes, one of the teams becomes very strong. When this happens, it is not randomness that is dragging the price upward/downward. But at other times, the relative strength of the Bulls/Bears evens out. When that happens, randomness can play a much larger role in the price action.

In almost every instance, something identified as "random" can more accurately be identified as "we don't (or can't) properly understand all of the factors that determine the outcome".

Probably the most common example of a "random" event is a coin flip. The classic definition of a coin-flip probability is that each event (heads or tails) has a 50% chance of occurring. But if you really understood - in real time - ALL of the factors that go into the coin flip, you would find that there is nothing random about it at all. The angle and rotational speed with which it's released from the hand, the density of the surrounding air, air currents, the wear on the coin itself - these all play a factor in exactly which event will occur when the coin is flipped.

If you had infinite computing power that could be deployed to measure and calculate all of the factors involved in each individual coin flip, you could predict with absolute accuracy how each flip would end before the coin hit the ground. The fact that we can't do this doesn't mean that the event is truly random, it is only "random" to us. Randomness is relative.

Pi creates an infinite series of numbers and each of those numbers is either odd or even. If we converted every even number into an up-tick and every odd number into a down-tick, we could create something that looks very much like a stock chart. If I didn't tell you how I was generating the stock chart, it would be, according to your frame of reference, "random". But of course, according to my frame of reference, there is nothing random about it. I am generating the chart by following a series of numbers dictated by pi. If you understood how the chart was being developed, you could use pi to accurately predict every single tick in this "random" chart.

So when you generate your random charts you think, "These look just like the REAL charts, so the REAL charts must be random!" But this is flawed logic. Just because something looks random (like the coin flip described above), doesn't necessarily mean that it is so. It is just random TO YOU because you do not fully understand the factors involved in determining the outcome.

yeah and i guess no one ponders why we think a machine "randomly" generating numbers doesnt actually fall in a sequence.. rather than being truely random no do they... machines are based on rules and rules must be followed in linear fashion so how random is computer generated randomness? so maybe the chart produced for non-random computer generation falls in line with rules and these rules drive the generation of numbers and maybe the number are similar to real charts because that slight non-truely random generation is governed by rules and rules govern our chart so maybe computer random charts are in reality maybe close to random stock or fx charts because they both follow rules... AHHHHHHH! oh terraincognita already said this lol.. but still even if it did pass diehard or whatever.. its still an algorythm..

Most humans assume that patterns are the result of a cause/effect relationship. They also assume that the inverse is true - that if there is no cause/effect relationship (in other words, if it is random) then there should be no discernible patterns. Unfortunately, neither of these assumptions are true.

If you ask the average person to predict the results of a random 1/0 generator, they will predict very few streaks/trends. They might predict some minor instances where the 1's or 0's take the lead, but they also assume that the balance restores itself in fairly short order. But as you have seen with your random number experiment, this is not the case. Even random numbers can go on long +/- streaks that, upon initial inspection, appear to be anything-but random.

But although you can find patterns in randomness, it does not follow that all patterns are simply the result of randomness. Patterns can be created by randomness. But patterns can just as readily be driven by underlying fundamentals. One cannot simply look at the pattern and determine whether it is random or not. The best you can do is to calculate the probability that such an event would occur randomly and then make a judgment call based upon those odds.

but can anything actually be random i mean truly random?

if everything is managed by rules how can anything be random.

I mean I guess like in physics like someone cant just be raptured out of panera bread one day.. like mass is not created or destroyed only transfered... so without heat transfer or energy transfer or mass density changing.. someone can not just vanish from panera bread.. or wouldnt that be truly random?

even the monkey in the infinite monkey theorem is real and has mass thus its held by the rules normal things are governed by like gravity.. the monkey is then bashing on a keyboard... so the keys are arranged perfectly and not rotating in sequence of meanings randomly.. and your assuming a monkey can bash on a keyboard for 10K years and not die or eat.. i mean there are just to many requirements inlife for anything to be actually random right?

I understand and agree with your point. As a software engineer by trade, I know full well that few of the computer-generated "random" numbers are TRULY random. The computer still tries to base these "random" numbers on something (typically, it's the system clock).

That being said, you will see in my previous post that it is also false to assume that because we see patterns in the data, that the data itself cannot be random. You refer above to being "truely (sic) random", but even TRULY random numbers will exhibit patterns.

http://mybroadband.co.za/photos/data...de_by_zero.jpg

Seriously, nice contributions Dom & Bodger

hahaha

but how . how can anything be truly random.. even those few.. they have got to think of a number to generate it and must have something to rely on im not computer expert but if it was me trying to be completely random and generate something from nothing id probably go crazy and fry from confusion.

Some people have posted in this thread that you can't profit from a random system. If this were the case, there would be no lottery. Random systems make people money - sometimes, boat loads of money - every single day. The key is that there is a huge difference between saying that "this system is profitable" rather than saying "I am profitable".

When we claim that a system is profitable, it infers that given a sufficient time horizon and a sufficient bankroll, it will eventually turn a profit. Of course, we want the "system" to turn a profit as soon as possible, but it's dangerous to mix up a system's profitability with your own profitability.

The lottery is a horrible "system". Over the long haul, across millions of people, the net return for everyone is very negative. But the net return for a handful of individuals is wildly positive. It would be very easy for those lucky few to think that they have a "system" for winning the lottery, when in fact it was just random luck.

Why is this dangerous? Think of it this way: Did you trade profitably over the last year? If you did, congratulations. But if you did, does that mean that your system is certifiably, undeniably profitable? No. It means that either, A) your system is profitable AND you were lucky, or B) your system is not profitable BUT you were lucky. Notice that either way you look at it, your profitable returns had at least something to do with "luck" (randomness).

You may be getting indignant thinking that your profitable system has nothing to do with luck. But then I would ask you, "Have you, or anyone else who trades this system, EVER had a losing trade?" If the answer is "Yes", then profitable trading with this system implies at least some small degree of luck (randomness).

Imagine that your system always yields +50 pips or -50 pips and you have historically made money on 99% of your trades. This would sound like an awfully good system. But it also means that on every single trade, there is at least a small chance that you will lose money. And as long as there is a small chance to lose money on every single trade, there is also a small chance that you will lose money on successive trades. In other words, no matter how good your system is, randomness dictates that A) a losing streak could be around the corner, and B) the system may not be as good as you thought it was (because your returns up to this point might have been extremely lucky).

I think this is extremely important for people to understand because even seasoned traders have a horrible tendency to deploy a survivorship bias to their strategies. In other words, they say, "This system made me a profit over the last X trades, therefore it must be a good strategy. And the other system lost money over the last X trades, therefore it must be a bad strategy." But profit (or lack thereof), by itself, is not a good enough reason to accept or disqualify a trading strategy.

In a perfect world, you would truly UNDERSTAND the underlying fundamentals of your strategy. You would know why it works and why it fails at certain times. You would know how likely it is - over an extremely looooong period of time - to return a profit. With that "perfect" knowledge in hand, you could then see the randomness for what it is. When you have a bad month, you wouldn't throw out an otherwise strong strategy. And when you have a wildly profitable month you wouldn't quit your day job and start maxing out your leverage.