codification of the market / Genetic Algorithms

EDIT - this post was from two years ago originally, I've added an update at the end of the thread...



I'm working late tonight and have some free time between tasks, so here goes a bunch of rambling. I'm sure my coworker will enjoy this.

As I've begun to learn about trading, a logical train of if/then type statements and assumptions has formed, with what seems to be a natural conclusion I'm sure many others have reached.

(1) If the market is utterly unpredictable, 50/50 in the long run, then not even the best money management scheme will ultimately prevail.

Contrary to what some math wiz gamblers have concluded, I do not believe there is any way to consistently win in a negative expectancy game (broker's cut figured). All you can do IMO is reorganize your wins and losses into various clumping structures.

I assume that the market is not utterly unpredictable, as there do seem to be honest people managing to consistently make a profit. Not the big floor players that don't rely on price as much as strategy, but regular retail investors playing by the charts and news.

(2) If the market has some degree of predictability, then it can be codified.

Humans are remarkable at pattern recognition, and perhaps the ability to consistently recognize winning conditions is well beyond the simplistic trading rules forming the various systems. After all, a human can recognize a face almost instantly in almost any condition while powerful computers need good lighting, good angles, and the latest software to do a decent job. However, difficult is not impossible, and computers have demonstrated an ability for many years to find patterns in data that humans are oblivious to.

I assume that it is not impossible to codify market predictability, if it exists, due to the presumed existence of traders using fairly simple rules to generate consistent earnings.

(3) If human emotion and psychology are the enemies of good money management and system execution, then eliminating them from decision making should improve consistency and long term profitability.

I've been reading posts and articles about the psychology of trading, and both fear and greed appear to be serious obstacles. Not only that, but simple human error can prevent solid execution of otherwise sound trading rules (like a blackjack player making basic strategy mistakes). Cumulatively, a system that may have on paper had a positive expectancy could turn negative due to error and mindset, or money management could empty an account before it had a chance to succeed.

I assume that if the market is predictable and can be codified, mechanical execution of that code would yield the most favorable results. The obvious "what ifs" include the fundamental external events... earthquakes, profit statements, wars, little country strikes oil, etc. In theory these are simply more variables that can be codified, but their infrequency and variability might make that difficult or impossible in many cases. Even so, since there seem to be systematic rules that generate positive returns during times of relatively "news free" trading, a simple solution is to stop using mechanized trading during those times and return to it when the market is stabilized. A more robust solution is to add frequently occurring events to the rule variables, such as profit statements and earnings forcasts, prime interest changes, etc. As these events occurred numerous times in any market data used for backtesting, a robust system should be able to handle these "routine" events.



Now on to the interesting rambling. What is the ideal form of implementation for a trading system? Not the practical ideal, which would include the ability to actually connect to brokers and execute trades, but rather the ideal theoretical platform on which to develop rules? Here I make a few more assumptions. First, you would want the rule to change as the market behavior changed over time. Second, you would want the system to generate all decisions including entry and exit events and position size.

It would seem that Genetic Algorithms are one solution that could fulfill all the requirements. My programming ability is limited to simplistic scripting, and even then it's shakey, but I have pondered what such a GA system's structure would look like. I'll take a stab at it below...




There would be three primary parts: (1) a supervisor program that reads data, accepts user input, executes "organism" subroutines, evaluates organism fitness, modifies the organisms, and executes trade events based on the "winning" organism, (2) a file containing a list of all organism subroutines, and (3) a config file that controls parameters such as mutation rate, types of mutations, organism building blocks, maximum organism complexity, number of children born each generation, chart timeframes the organisms evaluate, maximum risk allowed to be taken, weighting for fitness evaluation, etc.

The supervisor program should be able to function in both real time and on historical data. i.e., it should accept input from the user to include current data in the updating of organisms or only run with the current winner. It should also have the flexibility to use virtual internal money accounts and trades to evaluate fitness when backtesting historical data.

The main difficulty is the fitness criteria. This could be judged in many ways: prediction that the market turned up or down at a specified time, prediction of a swing over X size, prediction of continuance of a current trend, etc. The problem, if I have any understanding at all of trading, is that knowing the market is about to go up or down doesn't guarantee profitability if you can't generate good entry and exit points. There is a simple and natural fitness solution that encompasses entry and exit points: profitability. The organism that is the most fit is the one that generates the most profit, however that might be accomplished. It does raise the issue of what timeframe to judge profitability over... the last day, week, or month for daytrading systems? The past year? The life of the organism? That question could probably be handled by the inclusion of drawdown limits in the fitness criteria. Technically one might want the organism that could produce the most profit by year's end, but practically if it depleted an account in the mean time it is useless.

The subroutine organisms would be interesting. The little reading I've done on GA used for economic prediction in the past has used them to find "optimal" parameters for a fixed function. i.e., fine tune variables in an equation or set of equations already established as working. However, evolution has demonstrated an ability to produce organisms capable of tremendous pattern recognition abilities (human eye, for example), and I see no reason why a GA aimed at economic markets should not have at least a reasonable chance of finding patterns where they might exist.

Since established systems seem to be based on a set of conditional rules (IF a > b, AND IF a > avg(b,c,d)... THEN...), I think it would be wise to retain this feature of market prediction. I don't believe a set of piecewise functions and conditional statements can be summarized in one grand equation that says IF xyz^abc/def-ghi=lmnop THEN trade. The actions an organism could take in a simple market are buy, sell, and position size for each. In bidirectional markets you have short/long scenarios, so functionally you have enter short, enter long, exit, and position size. So the basic structure of an organism subroutine would likely be four sets of conditionals, each having an action when the conditions are met (enter short, enter long, exit short, and exit long), and a routine for calculating position size for each action. The position routine should look to the supervisor program for information on current account size, money already in play, and user limits to use in addition to market data. Money in play might be differentiated into total money in play in the market and money already in play in that specific market (leading to increasing or decreasing position size). This position routine should be allowed to produce different results for short/long and enter/exit actions, so it might as well be part of each conditional statement - a calculation piggybacking on the IF conditions.

The configuration file is fairly self explanatory but implementing it would take some thought. The first hurdle is controlling the types of variables the organisms are fed, the second is determining the types of data manipulations the organisms can conduct, and the third is determining the mutation rate.

There are already a lot of calculations performed on market data as well as "milemarkers" that perhaps the GA organism subroutines should be "force fed," such as previous period highs, lows, opens, closes, averages, or calculations such true average, moving averages, etc. Or you could let the algorithm operate on raw data without forcing it to look at specific times (opens, closes) or functions (maximums, minimums, averages, etc.). I think the best approach for GA diversity is to use a mixture... some organisms only use current market indices, some use raw price and volume data, others use a mixture. Configuration settings could assure that a certain percent of each generation always remain pure each direction, if desired.

Mutation is another complicated problem. Configuration settings that control how many organisms are retained from each generation (top % of performers and/or all within a certain spread from the winner), how many mutated copies are made of each, and the degree of mutation are fairly straightforward to outline. Types of mutation allowed is more involved. If each line of a conditional statement involves an algebraic comparison, the question is what form are the algebraic expressions allowed to take? I think it is obvious that every conditional line should have access to all of the available variables, so the question is limited to the functions that are available for the organisms to use. Standard functions would be addition and multiplication with the ability to invert and negate terms to produce subtraction and division. Sould array building and matrix operations be allowed? Probably so. What about logs, exponents, derivatives, integrals, etc.? Should the organisms be allowed to "discover" calculus (approximating derivatives with finite difference, etc.) or should these math tools be encoded explicitly? If the coding platform can handle it, I think the more operations available the more diversity produced, and ultimately the greater the chance of finding good system rules.

Once these building blocks are in place, you need to define the types of mutations allowed. Cross-breeding of successful programs is a good idea, as is adding brute mutations. But how many levels? First level would be swapping condition sets (enter short, enter long, ect.) both within an organism and between organisms. Second level would be swapping, adding, and/or removing lines individual lines within a conditional set, among conditional sets within an organisms, and between organisms. Third level would be mutating an individual line by swapping expressions from other lines. Fourth would be mutating a line by swapping variables in an expression. Fifth would be mutating a line by swapping, adding, and/or removing functions and variables. Settings would be needed to control how many children at each level are born, and what degree of swapping/adding/removing is allowed at each level (how many new lines can be added to a condition set in one generation... how many new variables or functions added to a line... etc.). Conservative settings might produce only a few children at each level each only containing one change, agressive settings might produce up to a hundred children (or more) at each level covering the spread from one change to only one thing left unchanged. And of course identical copies of all survivors of any generation.

The configuration file would also have to deal with hard limits on organism complexity (can't add functions or lines past a certain size...), limits on risk allowed, and perhaps variables to steer some organisms toward a desired timeframe (from scalping to long term swing trading). You would also need to control fitness weighting. For example, if the goal of a program is to maximize profits and minimize drawdown, then you need to rank the relative priority of each. Further, since each surviving organism grew through success in previous generations, you need to weight the importance of success and failure as a function of time (newer successes being most important, of course, but how much so?... even nearly perfect systems will make bad trades sometimes and you don't want to kill a proven winner). These rules could become quite elaborate, with for example exemptions from elimination for any outright winner of a single generation for the next X generations, exemption from any top 5% placer for the next Y generations, and taking the top half of the remainders based on a performance scale where losses in last generation count A%, last generation B%, two generations ago C% etc. Interestingly, since the organisms adapt to behavioral changes in data it would be most effective to start the backtesting with the earliest data available and let the organisms crank through it to the present day.



What would an organism look like? Well, take one of james16's rules for a crude example, double bar high close lower. In the appropriate condition section (short, right? ) you might see something like:

IF:
"1:00 to 2:00 high" - "12:00 to 1:00 high" =< 2 pips
AND IF:
"1:00 close - 2:00 close" => +XX pips
AND IF:
.............
AND IF:
................
THEN:
enter


You can see even in this psuedocode example plenty of room for simple mutations. Most will fail, a few might do as well or better than your initial rules.

How long it takes to breed a generation depends on the number of organisms, the complexity they are allowed to have, efficiency of code, and of course the computing power at hand. What is really interesting is the possibility that as the GA program receives realtime data and makes trading decisions, it continues to breed generations and improve. That is exactly what it seems technical traders want... system rules that don't go extinct but adapt to market changes and remain profitable. The idea would be that all organisms have access to real time data and "simulate" trades during each generation, with the supervisor program keeping track of how each of them has done cumulatively whether they were trading for real or not. While all organisms think they are making trades, and operate on the same data, only the chosen one would deal for real if the system were live. A decision would have to be made as to whether the rules used to determine survivors (and therefore produces a ranking of fitness) are the same as the ones used to decide which single survivor gets to make the "real" trading decisions (if actually using the system live) for the current generation. Do you want your last outright winner, who has a lineage of at least good performers, or do you want your time averaged and proven money maker at the helm? I think that question could be answered by simulating the grown of the colony over long periods of data with various rules for survival and "trading king."

A lingering issue is how to handle world events. Profit forcasts and earnings reports, interest rates, inflation, etc. could be entered into the config file each day as additional variables the organisms can use. Perhaps a stock of "special situation" organisms could be retained that proved especially well adapted for specific circumstances, like during a crash, a large natural disaster, a war, etc. The level of potential complexity of the colony once established is limited only by the user.

Once in place, the door opens to even more intriguing questions. What about two organisms that are actually in competition with each other? Instead of each simulating that it had full control over the account resources for a given market and the most fit actually having control, what if two or more of the most fit actually had to fight for their cut of the allowed risk? Would they develop strategies to control that market segment that were more harmful than beneficial? Would certain groups of organisms working together in various markets be more profitable cumulatively than the sum of each operating oblivious to the other players? That is the difference between the position size calculation taking into account what other market organisms are doing at that moment or only looking at config settings. Hard max for each transaction, or float it depending on the money already in play?



Enough thoughts for tonight. Computer just spit out results... back to work.

wow, what a post!!!!

took me a few hours to read, but it was worth it

Orii, you seem to be an very smart guy who is very new to system developement. your thought process is very advanced for someone new to trading, and i can tell that if you got into this game you would be one of the successful quants.

your comments regarding the "reality" of trading, where the system should perform but doesnt due to human errors, is spot on. my mantra is to create the "easiest to trade" systems i possibly can. i will sacrifice big performance numbers for easy trading. this is the sign of someone who has already gone through the process of trading mechanical systems in real time and realized that if you cant trade the system perfectly (according to the EXACT rules), its not worth trading at all.

there is already a software out there that does most of what you are talking about in your GA analysis. its called TradeStation. its the platform of choice for quants, and you will find it can support all of your needs.

it takes a trained eye to spot which backtested systems will perform well in the future. this eye can only be developed by going through the process of developing and trading mechanical systems. once you go through the process a few times (this can take a year or two) you start to get a "gut" feel for what works and doesnt. this is a funny irony to me, because most people become quants because they dont want to rely on thier gut. but in the end is seems to always come back to that.

i do believe that systems have a "shelf life". as my now favorite author Fred Gehm likes to say "no system can beat the market, but it can extract profits for a period of time". good luck man, i hope to see you around the forum.

another of my beliefs is that there is NO algorithm that can predict market behavior. i basically think the general movement of the market is random, there is no ryhme or reason for why it moves how it does. what i try to do is focus on the times when human emotion (which is somewhat predictable) is most at work. for instance, there is a lot of emotion and crowd reaction when the NFP report comes out. if you study just NFP report days, then you start to find some interesting things.

i encourage you to read a few books and get into this business. you have the right mind for it, and you will find many like minded authors who will open your eyes to what has already been established. once you know where we have been over the past 30 years with this quant trading game, you will know how to proceed in the future.

OK here are in short some ideas I had thought of in this area for the last half a year.

First of all I think that the goals here should be finding not one strategy but several which would be uncorrelated. Those strategies were only backtested, so I would be afraid of overfitting. I also would like my strategies as robust as possible to the specific values of the different input data. I would also think that decreasing computational load would be swell, since many times even not a bad GA would be very bad from the point of view of the time it takes to run one.

Here are some random thoughts on the theme.

A) First of all to adress the technical computational problems with the algorithm and in some way to deal with the problem of overfitting i offer the following thesis.
- Since a very large amount of different indicators and trading rules already exist, the right strategy already exists - we just need to choose it from a limited set.
- Feeding the different indicators and strategies directly to the different organisms is preferred from the computational point of view.

Hence the organisms should have the following attributes. 1) a vector of which of the different strategies (indicators) to use 2) different mathematical operations to use on them (I'd prefer to use only the basic four). 3) The description of the logical relationships between the different algorithms.

B) Regarding the speed of mutation - I think that it needs to be differential and controlled by two factors. 1) The overall score of the current population (introduces group selection) 2) The score of the specific organism.
This will cause the increase of mutation rate if a favorable solution is not even close.
C) Recombination - In general I am not sure it is very good in this case. But even if so I think that it must be allowed only for two organisms of the same fitness levels. Since otherwise the stronger specie will bring the weaker closer to itself, while what I look for is that the weak organisms find new solutions.
D) When killing off species it would be wise to keep a certain amount of randomly chosen weaklings - once again because I'm afraid of falling onto one maximum only.
E) When choosing the winning strategy it is very important to look at its robustness level - does it change much if I change the parameters a little? I look for WIDE peaks, not only high.

After running several times the GA we will get more than one different strategies - I think that it is imperatife to asses the correlation between the different methods of trading. Only those that are uncorrelated should be kept and must be traded all together in order to decrease risks.

The money management should be kept at a different place than the strategies themselves, since there are already some rules known regarding the best money management techniques.

I think that adding some random noise to the data might be very beneficial - this might prevent overfitting and lack of future data.

OK, I think that this is all I thought of, at least at this time of night.
Tell me what you think about it. Hope this is helpful.

Just a few thoughts:

1. I don't think Tradestation is favorably disposed to serious GA development due to it's (relative to computational developments) both primitive and ancient language construction.

2. I think that GA's are particularly well suited to both intermarket analysis and also to combining both fundamental and technical analysis which the thread author alluded to when mentioning the use of raw price and volume information. I assumed, perhaps wrongly, the context of that (raw) was pointing to the fact of volume information being fundamental in nature rather than the predisposition of GA's to require pre-massasged data input.

3. With respect to generating and deceasing mutations I'd take a good look at cyclic market theory since one of the greatest problems in the market has to do with mutations appearing, apparantly hibernating, and then reappearing. I think GA's are the more disposed technologically to handling these issues and I think cyclic market theory may finally have it's day should GA's be employed effectively enough to identify cyclical components. (assuming of course that cycles exist and also surmising that if they exist they are also sufficiently predictable to profit from.)

4. I think GA's stand the greatest chance of exploring patterns and relationships that occur between various markets. Something you alluded to earlier. I'd take a good look at some books on intermarket analysis in order to see where others have been with their research in identifying market correlations.

FT

[quote=merlin]
Do you know anyone who is using Tradestation to develop genetic algorithms without calling third party software? This I've never heard about.

What? You mean human pattern recognition functioning on a subconscious basis? Works for me.

But isn't that the whole point of GA's -- to learn and adapt -- to approximate human adaptability?

Don't the two statements in the above paragraph contradict each other?

Just for the sake of argument what if one took the position that most of what has been thought for the last 30 years is inherently flawed? To postulate a position contradictory to my question but still capitalize on it... What our friend said about having two competing algorithms would be an interesting way to model the opposing ideas that the markets are random and the markets are not random. Developing competing genetic algorithms just to see which one would win. Suppose the answer turns out to be that the markets are both completely random and absolutely not random. Suppose that the only thing that defines randomity is contextual and that random cannot be defined as an absolute? If that were true then "true random" would be impossible. What if we live in a deterministic universe?

FT

Now that is some heavy shit right there.

you got that right, i wont touch this thread with a ten foot pole.

Humans are remarkable at recognizing patterns that happen to have some similarities to evolutionarily driven environmental clues. We see edges, because seeing them helps prevent us from running into things.

Humans are not so remarkable at recognizing patterns that have no solid ties to evolutionary environmental pressures. We often times have difficulty in spotting numerical patterns because these patterns have only a marginal impact on our ability to survive "in the wild."

As a consequence, our ability to spot patterns in circumstances not strongly related to natural selection relies primarily on our higher cognitive functions. That some people can learn to spot these patterns for Mensa tests or market trading is remarkable, no doubt, but compare that to the ability of our eye to detect edges faster than the most sophisticated super computer in existence without ever having to consult our brains. Now that's remarkable.

The idea of a GA for this type of a problem is to search for patterns that humans are not well adapted to find. And to search for solutions at a pace much faster than our relatively slow high-level cognitive and logical abilities allow.

After years of practice, perhaps rather intelligent persons can learn (conciously or unconciously, as the case may be) to spot relatively reliable patterns in the market, but just how advance is that ability in the grand scheme of things? What do we have to compare even "successful" traders to? How many solutions have they explored? What is the depth of complexity they have explored (as fiji mentioned, what about intermarket relationships... how much data can even a gifted person process on which to base decisions)?




Well... I need to learn programming anyway. Maybe over the next year I'll tackle this as a good project to really dive into the field. I have some more thoughts on how to evaluate fitness and keep the GA's current to the latest data (adapting to new information), but I suspect that computational power will eventually become a bottleneck. Perhaps before an efficient GA can be found. Who knows...

Then allow me to use my 3 meter pole. It is entirely plausible, and possible that markets are both "completely random and absolutely not random". I cannot prove this, but I can conceive of the possibility because of other antinomies that have been observed in creation. (1) The dual nature of subatomic components that are both particles and waves simultaneously until they collapse into one state or the other upon observation. (2) An electron is in two places at the same instant in time during the quantum leap between shells. Or, the closest parallel, (3) God is sovereign over the minutest detail of the universe, yet man is a responsible free agent. The parallel here, is God would represent the "absolutely not random" markets and man would represent the "completely random market". (I did say it was the closest parallel, but not the most perfect )

So, do I believe the markets are random or do I believe they are deterministic? The answer is......yes, I do.

tsillaiDDiallist

[quote=Orii] So what about the proverbial "glass ceiling". Clearly recognizing it's metaphorical existance is integral to survival and yet it is called such because it is so easily not seen. (just having some fun here)

Just a minute while I quickly calculate the number of lions that are hunting me. Bummer I think I've just become dinner due to another math mistake. They always warned me that, being human, I may fail to recognize a pattern of seven lions hidden in 33.758 thickets at the base of 85 trees, 92 bushes and more saplings than I can estimate.


Rolling off the edge of a counter when only a few months old can ruin your pattern recognition capabilities if you fall on your head.

On a more rational note I think the strength of GA's is more in learning and solving than just spotting patterns. The two things that computers offer are accuracy and selective speed. A computer does not have the innate ability to shift focus when various stimulae intercept the consciousness. It does not matter if other computers have better problems to solve or if it is cloudy outside the computer is completely focussed.

However I wonder if you underestimate the pattern recognition capabilities of the human creature. Art and music do not play any direct part in survival and, when not present in the wild, survival is still possible. However musical patterns and artistic patterns such as seeing lions in the clouds are highly developed pattern recognition abilities.

Even something like perfect pitch is a rather amazing ability considering that many children appear to be born with. In fact it was once thought that perfect pitch could only be genetically acquired until someone figured out how to teach it.

Having an accomplished background in acoustic engineering research and development I have the ability to listen to electronically reinforced music and break the audible characteristics down into the technical architecture that is reproducing it. Since this has never been a primary business interest... and has only ever been a personal passion that I've excelled at due only to my artistic fascination... it would be hard to relegate such a highly developed pattern recognition ability to survival.

I'm not sure the point of many of your questions in the preceding paragraph but I can say that intermarket analysis is a relatively untouched area of market analysis. I'm beginning to think that analysing individual markets is a lot like forecasting the temperature by studying temperature charts. Imagine if all we used to forecast tomorrows temperature range were charts of temperature whether they by update by the minute, 5 minute, 10 minute, daily, etc. We could analyse decades of temperature charts but the degree of accuracy we'd achieve would be limited. However we analyse barometric pressure, wind speed, cloud cover, cloud type, we study pressure charts, temperature, time of year, we look at satellite photos, and perform quite sophisticated models to look at micro and macro weather pictures and we are able to forecast temperature highs and lows the next day with much greater accuracy than we could with just a set of temperature charts no matter how detailed. I think this metaphor applies also to intermarket relationships and I think if we want to achieve any kind of predictive methodology in trading that we should be looking more at related markets.

FT

But see, this is where you're wrong. If the market's 50/50, with a money management system in place you could make a lot of money. You'd just have to cut your losers short and let your winners run. Let's run through a very simplified example for the sake of argument.

Assumptions:
1. The market is truly random, as you said. Thus we make sub-assumptions:
-Your win/loss ratio is 50/50.
-The market moves are also 50/50, so let's set an arbitrary movement amount of 10%.

Now, if you applied a money management system of, say, cutting your losing trades after 3% and letting your winners run to 6%, you'd come out ahead under our rules.

-I know people can poke all sorts of holes in this, but to be utterly rigorous about it I'd have to type a post longer than the original in the thread, and I'm simply not willing to do that.

Oh, and I agree with fijitrader. Computers are best at sifting through and presenting reams of data processed in various ways, and performing a set of programmed tasks over and over again. They don't have decision making capabilities at this point that even approach a human's.

The human brain is the most complex aggregate of matter in the known universe, after all.

You see the problem is that with a totally random market you wouldn't have a win/loss ratio of 50/50 if you decided to break losing after 3% and winning after 6%. You would have about 33.3% winning trades and 66.7% losing trades. So in the end you still have the same money - broker's fee.
This is under the assumption of a random market model.

Just some out-loud rambling thoughts, no particular direction or focus.

Recognizing numbers and basic mathematics obviously has roots in evolution, or we wouldn't possess the ability at all. Whether higher mathematics is a side product of higher cognitive functions evolved for other reasons is a more interesting topic, but not particularly relevant.

My main point in that last post, if there was one, is not that humans are particularly bad at pattern recognition or data analysis, but rather that there are areas where we are better or worse at it than computer algorithms. I don't know which is the case for markets... I'm just wondering.

I think the two are at least strongly related. If an algorithm is limited in its ability to spot patterns by the hard coding prepared by a human programmer, then it is no "better" at spotting the pattern, only perhaps faster in some instances. Only when the algorithm can explore solutions that a human has not considered is it possible for it to truly find a "better" pattern recognition algorithm.

Not at all... I'm a lifelong musician myself. There has been much written over many decades on the origin of these abilities in humans, ranging from what evolutionary selection pressures might have created them, to whether they were selected for or side products, to whether they were evolved or granted by an intelligent designer. Obviously I don't have the answers. But I didn't intend to imply that humans aren't good at pattern recognition at all... absolutely the contrary. There are plenty of examples, from the edge detection I gave previously to your music example to the documented ability of military radio "listeners" being able to pick dialog out of a S/N ratio that is utterly unbelievable. My pondering isn't about whether we are good at recognizing patterns, but rather in what cases, and whether a GA could do better. Perhaps so, perhaps not, I certainly have no answer... just questions.

And yet the most crude and simlistic electronic circuit can accomplish the same feat with arguably more reliability and accuracy, which just illustrates that while many of our abilities are quite remarkable in the human context, they are often not so remarkable relative to mechanistic algorithms. There are some things that we just aren't hard wired for, though our flexible architecture makes practically everything within our reach to one degree or another.

An analogy is generalized CPU's vs. specialized processors. A typical computer CPU can do everything that a graphics processor in modern video cards can do, given the right programming, but it is orders of magnitude slower. Of course, being a generalized CPU grants it the ability to tackle vast classes of problems that specialized GPU's cannot, so there are obvious trade-offs in play.

The real question is just where does market analysis fall in the spectrum of problems out there? Going back to your music example, it seems that humans are capable of spotting pleasing structures, both for tones and music itself, than are computer algorithms. And we are still much better at things like seeing than are current algorithms. But except for precious few people, we pretty much suck at things like factoring large numbers or recognizing the algebraic relationships behind a string of numbers. We can do it, but computers do it faster and arguably better.

So what about a specialized case where we teach a computer to play the market? It obviously wouldn't be any good at doing the laundry or writing a book, but how would it be at playing the market? Better or worse than the best humans out there? Who knows. It's an interesting question.

That makes a lot of sense.

See ivenger's post. This is similar to saying that you can make money off of a series of coin tosses with proper money management, and I've seen suspiciously similar statements made here, on other trading forums, and on plenty of gambling forums and blogs. In short, I dont' believe it, and I've never seen mathematical proof. The most I can agree with based on my understanding of mathematics is that you can cluster wins and losses according to your system, but in the long run it is still 50/50 or worse.

Relevant example, given to me by a CPE friend of mine. A few rather intelligent persons working in the computer information systems hardware industry (think of a setting like Cisco) used a genetic algorithm in an experiement to produce a functioning router chip. The only rules (fitness function) was that it had to work like a router, no other constraints imposed. The result was a network of circuitry unlike anything previously used for a router... in fact, I think the description my friend used was "jumbled mess." It just happened to use fewer transistors and function more efficiently than all those otherwise orginzed and sophisticated looking chips.

This may be an urban legend type of anecdote... I haven't confirmed it's authenticity, but the example provides insight into the idea of GA at least. After all, the complex aggregate of matter we call our brains may have been the product of evolution; if so, why would evolutionary principles not be well suited for producing other remarkable solutions?

It does make you go "hmm..."

LOL!! i just laughed so hard i almost busted a kidney...

between you and dialist this forum is becoming quite entertaining

[quote=Orii]
Right. So far humans have outperformed computers... at least in the known sphere of robotic trading. There are always the unknown factors. If you had a robotic system that was doing much better than most people would even imagine would you tell people about it? Would you require non disclosures signed by any market maker you dealt with?

Indeed they are related but my point was specifically toward differentiating the two ideas. Spotting and solving. GA's are good at isolating through solving whereas spotting to me is a simpler concept. However I understand the relationship if spotting is achieved through reprossesing what has been spotted to determine it's usefulness and therefore arrive at a more rarified item.

I didn't take you to be suggesting humans are not good at pattern rec. I was only suggesting that something as sophisticated as musical expression, and its relatively infinite scope of nuance when it comes to pattern recognition, and how it does not seem all relevant to survival. In fact it seems our most sophisticated pattern recognition abilities may actually not be so much about evolutionary survival.

Excellent. Well substantiated conclusions are not found that easily.

Assuming of course that accuracy is the highest point of musical expression. I recall when the first relatively sophisticated computer generated singing voice was designed the biggest problem they had was that it did not sound like pleasant like a real voice. It was too perfect. Then it was noted that humans do not achive a given pitch instantly as they go from note to note. A live singer may adjust the pitch too quickly for the conscious mind to recognise the process of finding the pitch but when the pitch is truly perfect from the moment it is sung it does not sound pleasing.

I think that is what the relationship between humans and computers is about. Combining strengths. I think that may be why the best of the known traders are most always found with discretionary elements in their trading.

Here we are looking at quantitative and qualitative analysis. Accuracy, speed, and focus are three things about computers that will not likely be matched by humans when it comes to quantitative analysis. Clearly the human is unmatched for speed when it comes to processing power. But when it comes to being fast, accurate, and focussed long enough to maintain that speed to process large reams of numerical data humans are not currently functioning anywhere near the level of even a PC.

I think GA's have a better shot at learning and adapting to changing markets and also processing ideas from multiple markets than conventional algorithms applied to market analysis and I guess that is my point in response to your previous musings.

FT

orii, what "math wiz gamblers" are you referring to? any person good enough at math to be considered a wiz will know that you can not make money with a negative expectancy game.

Van Tharpe claims to have designed a system that has a completely random entry system that uses money management to survive and it actually makes a couple percent annual profit and it includes slippage and commissions.

FT

Does it also have random exits?
If so I don't really get how it works.
Also if we continue along these lines. Why do people say that what is most important are the exits from the trade rather than entries. Wouldn't it hold true if my entries were very controlled and the exits random? I don't really see any conceptual difference between an entry and an exit.

I guess the first conceptual difference I see is that an entry exposes ones capital to risk and/or profit and an exit eliminates risk and/or profit. I think that entry and exit are as different conceptually as "beginning" and "end". Entry always begins with an immediate and quantifiable loss. Exit may produce either profit or loss. Entry is deterministic and has a guaranteed instant loss. Entry is where you give control of your account to a force over which you have no control. Exit is where you take resume control of what happens to your account. An entry has only one possible immediate result. Loss. Exit has one of two possible immediate results either loss or profit but you are not guaranteed of one or the other.

I've never looked at the system but I think it would be clear that you could not apply money management to a system that had random exits. Exits are how we control risk, which is one of three main components of money management -- the other two being position sizing and profits. I think the point of VT's system is to show that a system where you have no control over entries can be profitable by controlling position size and exits. Employing both random entries and random exits would not in any way prove or disprove whether markets are random or not anymore than random entries and controlled exits proves that markets are random or not. The question posed was whether a negative sum game with a 50/50 expectancy could succeed by applying good money management. Merlin said that he didn't think so and cited why. I cited something that may contradict that. So the discussion has to revolve around systems that employ controlled exits or we have changed topics midstream.

FT