Walk Forward Analysis - the only logical successor to backtest

.
.
.
ATTENTION: THIS THREAD IS OUTDATED
Please go to http://datfra-expert-advisor.com/ in order to find the final result of all the research!






















Hello,
I'm Darwin and this is my second article in which I will try to explain how a walk-forward-analysis works and what benefits it brings you as an EA trader.


But I do not just want to explain how a walk-forward-analysis works (as others already did this), no, I want to explain why it is the only logical way to analyse EAs.

And as people keep asking why I release free stuff, let me explain my motivations: I am targeting a job in the trading-economy, and this is not easy to get. Beeing known in the community would help me a lot to accomplish this goal, so I decided to release some of my private tools for free and/or open source to make myself a name over the next months. I am not here to sell you stuff, keep that in mind, please!


I know that the title is a bit provocative, but it's often easier to start a discussion with a controversy
Also, I know that the best EA-traders can use normal backtesting and still be profitable - but most can not. And even if you can, a WFA is still better

I WILL RELEASE A WALK-FORWARD-ANALYZER-TOOL FOR FREE WITHIN THE NEXT FEW DAYS THAT DOES ALL OF THIS 100% AUTOMATED; UNTIL THEN, USE THE TIME TO MAKE YOURSELF FAMILIAR WITH THE CONCEPTS.

Nevertheless, none of this article is needed to use the walk-forward-analyzer tool, it will be as easy to use as the metatrader4-backtester.



After reading, you have 2 choices:
Agree with my arguments and once and for all get rid of the flawed backtesting-approach and use WFA in the future.
Disagree with my arguments, but then please try to argue with me.. Do yourself a favour, don't just stick to backtests because you "know" them or something like that.


Btw, a backtest has many more disadvantages compared to a Walk Forward Analysis than the ones I describe here. If you want to learn more on this topic, read my first article:
"Why backtests are worthless, fixed-logic-EAs are flawed and your parameters are bad [DISCUSS!]" (look it up in google or through forum-search)





Initial Situation

First, let's sum up what the essential parts of every trading system are!

1. The system's logic
The most obvious part! And for a lot beginners it's the only part they know, which is dangerous.
This might be a manual trading system or an expert advisor or any other form of fixed trading-logic / trading-system / trading-strategy (btw: all 3 terms name the same thing in this article)

So far, so good. But you all know that every strategy has some kind of variables/parameters (like the periods of moving averages or stop-loss levels etc), that are NOT FIXED(!) but can vary, which brings us to the second part.
(If you just set them to a fixed value because "this should work".. well, it wont, at least not in the long term)


2. The system's parameter-ranges
The ranges of the parameters are an ESSENTIAL part of every trading-system, as they determine the exact behaviour of it (tough the trading-logic always stays the same).

So, for example, a moving-average-period might range from 5-15 to capture short-term price movements.
It is not 6 and not 11 and not 14, it is 5-15, as the markets change, we can not choose a concrete value, ALWAYS a range!


3. The market, the amount of data, the desired characteristics
Every strategy trades on a market, so we want to determine on which. (eg EURUSD / H4)

But thats not enough, we also have to determine how many past-price-data we want to use to evaluate our possible parameter-choices.
Because, as I said, a system always has parameter-ranges, but for live trading we have to choose concrete values!

And we do this by evaluating all parameter-possibilities on the last X years of price-data, and after evaluation,
we end up with a huge list of possible and "independent" trading-systems (each of them with different parameters, but the same main-logic).
And each has it's own characteristics like "profit" or "profit factor" or "relative drawdown".

So, we also have to determine how to pick the "best" parameters.

But it's not as simple as saying "I want much profit", because the characteristics often don't hold in the future!
Instead we want to choose in a way that gives us a high probability of picking parameters that will succeed during live trading.


Simple, isn't it?





An illustrating example

The system's logic:
Let's suppose a very basic trading-system: "If the price moved more than X pips in the last Y days, a course correction will happen"
(this is not a valid strategy, its just thin air for the sake of simplicity).
The parameters would be X and Y in this case.

The system's parameter-ranges:
To make it all simple, I chose X to be 100-200 pips in this example, and Y to be 2-3 days. (also, just thin air!)

Amount of data & prefered characteristics:
Here we choose the last 10 years of data to evaluate the possible parameters on, and "profit" as prefered characteristic.
(tough, as I said, in reality profit is not a very good indicator for parameters that will have a good future performance)

The process:
Ok, now before we can trade that system, we make an optimisation on the last 10 years.
That means we backtest every possible parameter-combination for our system and choose the best in terms of "profit".

For sake of simplicity, here is a cropped example:

"If the price moved more than 100 pips in the last 2 days, a course correction will happen"
=> 1000$ in the last 10 years
"If the price moved more than 150 pips in the last 2 days, a course correction will happen"
=> 1200$ in the last 10 years
"If the price moved more than 200 pips in the last 2 days, a course correction will happen"
=> 1500$ in the last 10 years
"If the price moved more than 100 pips in the last 3 days, a course correction will happen"
=> 900$ in the last 10 years
"If the price moved more than 150 pips in the last 3 days, a course correction will happen"
=> 950$ in the last 10 years
"If the price moved more than 200 pips in the last 3 days, a course correction will happen"
=> 950$ in the last 10 years

Soo, According to our prefered-characteristic (profit), we would choose X = 200pips; Y = 2 days, and then just trade the strategy.
Well, that's the "normal" process of EA-trading, and I claim it does not work this way.





EA-Analysis: How to ask the right questions

Ok, now that I have described the current process and how it is all done, here comes the "new" part.
The goal itself always stays the same, we want to pick the best parameters (based on some kind of evaluation on the past), and then we want to trade live!

Remember: The only thing a backtest can tell is "How good did my system+parameters perform in the past".
But that is NOT what we want to know! Be sure that you really understand this.

Initial Question; What we actually want to prove with analysis.
"Does the way we choose parameters for live trading ('pick the one with best profit over the last 10 years' in the above example) give us a high probability to pick parameters that are profitable during live trading?"

So we are actually interested in the relationship of past-performance&future-performance, not backtest-results!!

If the answer is No, one or more of the 3 things described in "Initial Situation" are wrong. Might be the logic itself, the parameterranges etc..
If the answer is Yes, the performance in the past and the performance in the future are somehow correlated for our EA, and we can trade the system!






The logical evolution; From Backtests to Walk-Forward-Analysis

Step One: Backtesting - in it's worst form

http://s1.directupload.net/images/131125/zqan4tky.png

Pro:

1. We will get parameters that performed well on a wide range of data

Contra:

1. Every single EA trader that used this method and then tried to trade an EA live, based on good backtests, can tell you: It just does not work this way.
2. Overfitting / Curvefitting!! We first optimise the parameters, and then test them, all on the same data.
   That means we have no clue if we have valid parameters or overfitted ones.
   
   Overfitting means, we optimised towards a random behaviour within our data, that just exists in this particular dataset, and will not hold in the future.
   
   That means, we captured a relationship that existed but was not a sound one. Like this:
   http://s1.directupload.net/images/131125/5ccypnb8.jpg
   
   Don't fool yourself in thinking "ah, this wont happen"... Almost all "relationships" within the markets are like this, as most price-movements are random!!
   
   If you do not understand overfitting, google for more information, as it is our archenemy in mechanical trading.
3. No significance for future performance!! Remember that the initial question is not how good our parameters performed on the past, but how high the probability of succes AFTER the optimisation-timespan (so, "in the future", during live trading) will be.
   
   As we did not do any tests with our parameters that take into account the relative future, we did not even try to answer the initial question.
   We just answered the question "How good did our parameters perform in the past", not taking into account anything about the "future" => very bad.
4. Even if you could somehow magically invalidate my above points, because the parameters worked well on a huge amount of data, they are not really the best for the current market - just average good on all market-conditions.

Step Two: Backtesting using unseen/out-of-sample data

http://s1.directupload.net/images/131125/kc76r7ab.png

Notice: The first dataset, we use to optimise our parameters on, is called "in-sample" (is). The second, unseen, dataset is called "out-of-sample" (oos).


Pro:

1. We now have a lower chance to get overfitted parameters, as we use an independent dataset to validate our parameter-choices.

Contra:

1. Due to the infinite amount of senseless/unsound relationships within the markets, we still have a (too high) risk for overfitting, as chances are too high that we just got parameters that are valid (curvefitted) on both datasets, but not valid in the future.
2. If the system did not work in out-of-sample and you then begin to tune your parameters until you get good oos-results, your oos-results are not longer "unseen" and becoming "in-sample", which makes the whole approach using 2 datasets useless!
3. We still use a very larg part of our data (in-sample) to find the best parameters, which also means we use a lot "old" data. That is not a good decision as the behaviour of the markets in the past is not equal to the behaviour of today.
4. Not just our in-sample dataset is too huge, also our out-of-sample dataset is too huge and therefore un-realistic. In the example above it would be a few years, but would you really like to trade a system for years before choosing new, re-adjusted, parameters? I would not!

Step Three: Backtesting using a more realistic data-amount

http://s14.directupload.net/images/131125/dnhpyp8y.png

Pro:

1. We now only use the recent market-behaviour to optimise our parameters, so we capture the market "at the moment", and not "10 years ago".
2. We not test our parameters on a timespan that is more realistic (as it is not years but months!)

Contra:

1. We only used a small part of the available price-data for our tests. This is not very efficient!
2. Ok, remember the initial situation, where we have settled on parameter ranges, amount of data to optimise on, and the "desired characteristic". Our analysis has the purpose to verify these choices, wether they are valid or not.
   
   But in this case, we only made one test with them, so we optimised on one part of the data, then we chose 1 parameter-combination and tested it on 1 "unseen" dataset.
   
   Facing the million/billion possible parameter-combinations an EA can have, and the infinite ways the markets can change to generate new and "unseen" behaviour, do you really think that 1 test, 1 datapoint, 1 past->future relationship, is enough to judge from? Of course not! So why are you still using normal backtests?

Step Four: Walk Forward Analysis

http://s7.directupload.net/images/131125/9g9s5m6r.png

So, as you might see, a Walk Forward Analysis is the same thing like doing a normal back- & out of sample-test, but we do it over and over again, so we end up not just with 1 test-case but with many (100-150 in most cases, up to 1000 if we choose very small test-period).

That way we can verify our system + our optimisation-methodology on many, many independent test-cases, which is THE reason why we want to use WFA instead of every other analysis-method described in here.

Pro:

1. For our final analysis-report, we only take into account the green test-results, as they are the "unseen future" relative to the red optimisation-windows.
   That way, we simulate the same process we would face during live trading: Optimisation on the past, trading on the (relative) future!
   
   That allows us to draw meaningfull answers to the initial question, as we only analyse performance in "the future".
2. We use all data available for our testing
3. We have 100-150 independent "PAST=>FUTURE"-relationship-tests, which gives us a clue about the future performance, not the past performance!
4. We avoid overfitting, as we use different datasets to optimise and verify our parameters
5. If we want to trade live, we simply make "one more step" of the WFA, optimise on the last available data (the "red" dataset would then end at the end of the chart), and then trade "in the future" (the "green" dataset would be our live trading). So we trade the system using the EXACT same methodology we have tested 100-150 times already.
6. Due to the frequent re-optimisation of parameters, the EA is also continuously re-adapted to the markets, which will most likely increase the overall profit.
7. A traditional backtest answers the question "How good was my EA in the past", whereas a Walk Forward Analysis answers the question "How good will my EA be in the future, during live trading".
8. It does not only evaluate an EA, it also evaluates the corresponding trading plan that determines how to pick the best parameters for live trading.

""Contra"":

1. Most EAs will not pass this test. But this is not bad, because lets be honest, almost all EAs in existance are bullshit. So if almost all EAs tested with this approach would give bad results, that would be great.
   
   Even if a lot people do not like to be disillusioned about their "holy grail money printing machines", it's better to face the truth during EA-development and not during live trading.

Contra:

1. There are some limitations regarding this process which will be discussed in a later article, stay tuned! Also, I am currently working on more sophisticated anlysis algorithms, but it will take a few months until I can show you something.

The main advantage is that we get 100-150 independent test-cases, whereas a Backtest+Out-of-sample-test gives us only 1 test-case (or 1 datapoint).







I hope you like this article, and if you want to discuss it live, visit http://webchat.snoonet.org/forex (it is a web-based chat client),
then click on my nick (FX-Darwin or Darwin or Darwin-FX) and on "Nachricht" / "Message" / "Private Message" (or something like that).
Then you can chat with me directrly, and I will post all interesting chats on the forum.

But the best thing would be to discuss it public here on the forum, so please do so if you have anything to say on the topic, thanks!

-Darwin

Are you selling something?

Could you show me just one trade you took (in a chart). I don't believe in text when it come to forex -- texts mean nada

"I WILL RELEASE A WALK-FORWARD-ANALYZER-TOOL FOR FREE WITHIN THE NEXT FEW DAYS THAT DOES ALL OF THIS 100% AUTOMATED; UNTIL THEN, USE THE TIME TO MAKE YOURSELF FAMILIAR WITH THE CONCEPTS."

EDIT: "And as people keep asking why I release free stuff, let me explain my motivations: I am targeting a job in the trading-economy, and this is not easy to get. Beeing known in the community would help me a lot to accomplish this goal, so I decided to release some of my private tools for free and/or open source to make myself a name over the next months. I am not here to sell you stuff, keep that in mind, please!"

Just want to explain the concepts in a transparent way before I release anything - and it will be for free & open source.






Wrong When it comes to forex "one trade" or "a few months of trades" mean nada - they are just statistically irrelevant. Also, I am researching and coding at the moment, and will only trade live when I have finished this, which might still take a few more months. Because as I said, a few days, weeks or months of live-trading won't mean anything, so first I have to finish my algos, then I can test them live. But I can not test them live while I still develop them.

Also, this is not a trading-system, it is a way to analyse trading systems.
So even if I could show you a meaningfull amount of trades (like a few years), it would not prove or disprove the concept of WFA, just show you that the one particular system I traded works

So it is hard to prove what I have written with hard numbers, as you would need to trade it live using MANY EAs for MANY YEARS.

But when Charles Darwin came up with the theory of evolution, he also could not just "prove it", but nevertheless he could come up with a logical argumentation.
So a theory can exist and be logical, even if it is not yet fully proven - and I can not give you anything else than a logical argumentation I fear.

Tough, if you choose not to believe it, at least argue with me

-Darwin

you are too intellectual for my persona. Good luck anyways.
PS. One trade can show almost everything -- your logic, mentality, and money management.

Statistically irrelevant means, well, imagine tossing a coin 1 time, get a Head and then claim "The coin always shows head".. You see? If you have a too small number of results, you can not draw conclusions about the real statistical probabilities of any system. And to show you one trade would have the same problem.
So, perhaps you would have to toss the coin 1000 times to get the real 50:50 probabilities. Same with trading, only MANY trades show you the real characteristics.

Don't wish me good luck on that - I just release a concept, a way to analyse EAs, so it's nothing that can work or not



One trade might show a small part of your logic and your money-management, yes. But the result of the one trade is meaningless to judge if it works - for that you will need years of live trading (to be sure about your results). Tough, a few months of live trading might already be a good hint about the real performance.

Anyways, "one trade" is dependent on a particular system - but you can test any system with WFA, so the "one trade" would just show you the one particular system and nothing about the WFA



-Darwin

Now I understand you are not talking about trading, but about back-testing.

I think you should move to the commercial category where you will get much more responses/interests.

IMHO...

Yes, but it's a way to backtest that closely resembles the way you trade a system live
Thats the purpose of a WFA, replace the flawed backtesting approach with one that is like trading.

Well, as I do not want to sell the tool, the commercial catergory is wrong for me I guess?!
Also, with this article I want to educate traders in a way so they understand the concept and really want to use it because they fully understand the benefits - not just make them use my tool because I have some neat slogans or charts or promises or something

-Darwin

I can only disagree with this statement. Look at the image below. This is a backtest (not a WFA). There are more than 500 trades. What is the logic behind? What is the "mentality" (if ever an EA has a mentality). What is the money management?

Over the 7 year period of backtest the account has been multiplied by 2.5, this is 13.9% per year. The profit factor is 1.19, the expectancy is 15.4 pips per trades.

Is this good or not?



Answers:
- Is this good or not?
Absolutely NOT. This EA is completely edgeless!

- What is the money management?
Fixed size, 1 micro lot. One trade at a time.

- What is the logic?
It flips a coin to go long or short. SL=100 pips TP=500 pips. When the trade closes it opens a new one.

This is simply a cherry picked lucky realization of a random walk! A single trade or 500 of them prove nothing at all.

A great argument! I completely hat off to you

But wait! The description above just shows the logic, mentality, and money management, doesn't it??? (even if it was one trade)
Anyways, let's not get serious, man. It's just my stupidity talking

Questions for you Darwin:

You say that the WFA aims at adapting the EA parameters to the evolution of the market. But you seem to assume, as a given, that the EA already has some edge. When you optimize an EA and you get a result such as the curve above how do you evaluate that it isn't due to pure random chance?
The first question I'd like to get an answer for is: "Is this system is really working?" even before trying to optimize anything.

Second question; on the optimization part. Let's say you optimize a system using a MA. In the first sample block SMA 23 was the best. So you choose it for the next round. At the end of the 2nd period SMA 31 would have worked better. So you use it for the next round. In the 3rd block SMA 7 was the optimal setting.
Clearly each time the next block fails. The value of the parameter changes widely. Strong evidence of overfitting. If SMA 31 is optimal, SMAs 27,28,29,30,32,33,34,35 should work good as well. How do you estimate the dispersion of the results around the parameter vector you get from the optimizer?

BTW SMA 31 is optimal with respect of what? What is the loss function you use to compare the results?

Correct but could you tell me the logic, mentality, and MM from the picture alone before reading the answers?
The WFA is a tool to evaluate the idea behind the EA not to build the EA.

Logic: Guts feeling entry in the market.
Mentality: Refuse to lose, revenge the market to show it who's the boss!
MM: Martingale.

This is the input, in the form of an AE script to the WFA optimizer. The job of the WFA is to tell you: The best multiplication factor for the martingale is ZERO lots.

Right. Tough I would argue that it depends a bit on the way you look at them. If you just look at "the trades", 500 might be really useless - depending on your test, but if you look at them in a way where you analyse the correlation/relationship of past->future, and can conclude that the past and the future performance is somehow correlated for 500 trades, it might be an good indicator for a system with an edge.

Tough, your point stays valid and this is still a dangerous bet, so the algo I am coding right now will be capable to analyse something like 1.000.000-5.000.000 single trades per system (depending on the degree of freedom and the trade frequency of your system), and the first tests look quite promising
Looking forward to discuss it in a new thread soon!

Also, thanks for the post, the coin flipping example is always a very good argument - i should have included it in one of my articles, shame on me.







Well, a WFA does not just analyse a system, it analyses the correlation of the past->future performance.
So, when you optimise your parameters on the past to get good results, and then you analyse the relative future performance and can see that there is a clear correlation between them, it indicates that there is not just "random chance" showing its ugly face

At least that's the idea behind the approach, and this thread is here for discussion, so, what do you think?






I'm a programmer and not much into statistics, could you explain this in .. well, a language I understand?

But I try to answer anyway, it is "optimal" relative to all other SMA-periods. The user defines the prefered characteristic, like profit/drawdown, and then the system takes the optimal parameter, compared to all others, based on the prefered characteristic.

Also, it's up to the user to define the parameter-ranges and therefore, more or less, the degree of freedom.






So, that would mean looking not just at the "best" parameter but at the whole picture. Thats the purpose of my new algo, analyse the whole parameterspace.
But the first WFA-tool, which I will release in a few days (I aim for saturday), will just be able to analyse the one "best" candidate









-Darwin

Also, we have more possibilties than just taking the "best" parameter based on some characteristic.
If you do an optimisation and end up with this huge list of possible systems, you can implement any kind of "mutators" (ranking, filtering, truncating, shuffling etc) to get the one "best" parameterset for trading. I call this a trading-plan. (at least thats the name of the C++ class handling it haha).

So, it would not be very hard to add a mutator that takes away some degree of freedom from the initial strategy, based on what works best, analysed on >1.000.000 single trades.
This might be a good approach regarding this topic.

Also, I am just thinking about a way to add a mutator that does not just reduce degree of freedom but "group" parametersets together with their "nearest neighbours" or something.. interesting

At least the new framework allowes to add any kind of these post-optimisation-mutators, so if you have some cool ideas, tell me, please.
And you might guess that the purpose of it will be to optimise a trading-plan, instead of parameters, based on the whole parameterspace of a system. So it might be a much more robust approach than a WFA, but I do not have enough code running to make any tests yet.

Anyways, thanks for mentioning that


-Darwin

Can you develop a little bit more? Is a high correlation a good thing to you?
Correlation can been seen as a hint of causality: one variable explains the other one. But the correlation can also be looked at as a convolution: how two curves have a similar shape. And what looks the most like a random walk than another random walk? Two random walks often show a high correlation coefficient, whereas per construction this is nothing at all.

I think a picture is the best explanation. The x-axis is your parameter like the period of the MA. The y-axis is the performance you measured like profit/DD. As you can see there are two peaks. The first one gives the best result. It would be selected by your algo. But slightly changing this value makes the result drop significantly. The second one performs much worse but it is also way more stable/robust. This one should be selected instead. The first one is clearly overfitted.


This curve illustrates one period of test. How different will be the curve for the next periods? Same number of peaks? Are they at the same place or do they move a lot?

How important are the parameters? Let's take again the coin flipping example. There are two parameters: SL and TP. How important is the SL? Is it more or less important than the TP? Is the ratio TP/SL more important than the values themselves? Your optimizer will always find a best value. It is its job. But we perfectly know that these two parameters are completely irrelevant. Any value of them will always give a perfectly edgless system (expectancy of 0).

I don't understand. Having one or ten parameters is the same. You simply optimize a vector instead of a scalar value. Unless you're talking Genetic Algorithms? Did you read mikkom's thread Systematic trading? Instead of trading "the very best" he selects a bunch of "very good" and run them altogether.

The only post-optimization I think of is to test the result against the null-hypothesis "this is a random walk".

Damn this is awesome! PipMeUp, you must be a professor, no? Not only you have knowledge, but also you convey them so well!

Hey PipMeUp.


http://s7.directupload.net/images/131127/cxrywck4.png

Red = Profit in Optimisation.
Green = Profit in relative future.

Left = Sound system
Right = Random/Useless system

So, if you use profit || drawdown || profit_factor || whatever to pick a parameterset from your optimisation-results, and this process tends to give you parametersets that perform good in the relative future, this is the correlation I talk about.
I don't know if my point of view is clearer now, if not, please ask again, and I will try better

A WFA can test this approach ~150 times, I am not completly sure how safe this is, but it is at least better than a backtest, where you have 1 of these tests.

But if you test this on like 100.000 independent samples (the new algo, again), and all these samples indicate "A good performance in the past has a high chance of a good performance in the future", this should eliminate random-walks.
Because there is no correlation between past and future profitability in them, at least not if you look at the whole picture
(and I think 100.000 independent test-windows with about 1million single trades is quite good, at least if you dont have too much degree of freedom).









Ah ok
Yes, you are right, the algo would, in this example, select this huge spike with the overfitted parameters.
But it's the purpose of a WFanalyzer to evaluate how vulnerable your system is to the problem you described.

If it is, the WFA will give you bad results, as it would always choose overfitted values and then get bad trades in the testing-timespan, leading to a bad WFA-result in the end.







Well, if the parameters are irrelevant, then also the according backtests. Therefore the way you pick parameters for testing can also just be irrelevant.
And then the correlation, as I tried to explain in the first part of this post, will not hold into the future for many samples, leading to a bad WFA-result in the end.









Well, these "mutators" I talk about just define the way you choose "the very best" or "very good" parameters:
The classical way, and also the only one the WFanalyzer will be capable of, is to just "pick the candidate with most XXX" (XXX can be profit or drawdown or whatever).

But you could also say: "Well, first remove all candidates with profit<0, as they might not work. Then remove all candidates with profit > 5000$, as they might be overfitted. Then sort the list according to the profit_factor and take on of the best 10 randomly".
So each step "mutates" the optimisation-result, they all together are a "trading-plan".

And the new algo (and this is the part I work on right now) will be able to find such a "trading-plan", which is more or less just the config for a WFA, without any human interaction


Back to your quote, selecting a bunch of "very good" parameters is the best choice I think, but I would prefer to switch parameters after every trade instead of trading more parameters altogetther - but it boils down to the same thing.

Nevertheless, if you want to take this road, you also need a way to analyse it. The WFanalyzer will not yet be able to do this, the new algo will. (again.. well, sorry about that xD)
It can first look at the whole picture, for example, analysing the "best" 1000 candidates on all Walk-forward-windows. Then it does some mutations, ending up with a bunch of "very good" candidates and then analyses all of them together. On its own, all automated.

That is also one of the limitations of WFA that I mentioned in my post, it only takes "the very best" to make tests, the new one will just take them all into account




-Darwin

True that, it's a pleasure discussing with him

-Darwin

Thank you very much. I'm a software engineer for a living.

Darwin do you know the genetic alogorithms? I think that's the way you're taking
GAlib: Matthew's Genetic Algorithms Library - MIT
Evolving Objects (EO): Evolutionary Computation Framework

Where do you find 100.000 independent samples? as soon as your training windows overlap their are no more independent.