.
.
ATTENTION: THIS THREAD IS OUTDATED
Please go to http://datfra.com/ in order to find my current research [ Strategy Builder, EA Collection, Papertrading ]!
As promised some times ago, here is my last article about trading system analysis.
This is quite much to read, true, but I promise that you will find the article very usefull and that the concept layed down here is superior to every other algo trading and system analysis approach you have ever heard about
Also, I am currently starting a blog whichs only purpose will be to analyse public MT4 Indicators and free EAs with this method, whichs purpose is to replace the "dafault parameters everybody uses". In the future, we can use the empirically proven best ones
To not miss this, subscribe to me on this forum or keep an eye on my blog!:
http://darwins-fx-research.com/
Btw: Please read my other 3 articles before you continue with this one, you can find them here on the forum.
How Parameterspace Analysis (PSA) works, Part 1 - What is the fundament of all, what data do we work with?
Parameterspace Analysis is no fixed and inflexible method, it is a datamining approach that first collects all important data of a system, and then gives you the tools and interfaces to analyse and dig into this data.
What kind of data do we look at when aanalysing trading systems, what is a "datapoint"
First of all, I will talk about data and datapoints frequently. So what is a datapoint?
Let's first recall how Walk Forward Analysis (and also Parameterspace analysis works):
http://darwins-fx-tools.com/images/art4/4.png
One datapoint can be generated PER PARAMETER-COMBINATION(CANDIDATE) and PER WINDOW.
It consists of:
- The performance in the RED optimisation window
- The performance in the GREEN forward trading window
- The used parameter-combination for this specific test
- (Some statistics about the RED window as a whole. For example, how many candidates were profitable in optimisation, what was the average profit, what the average profit factor etc)
So in WFA, one datapoint per window is generated, as it only forward trades the very best candidate per optimisation. In PSA, one datapoint per candidate is created.
For comparison, a Walk Forward Analysis usually works on ~50-250 datapoints. So it does 50-250 optimisations, takes the "best" result, forward trades them -> 50-250 datapoints.
PSA, in contrast does... well, read on.
What is the actual difference between WFA and PSA?
PSA basically does the same thing that WFA does (look at the image above), but after each red frame (optimisation), it takes all parameter-combinations (==candidates) and forward-tests them, instead of only the "best" one.
Then it saves each of these "optimisation->forward trading" datapoints to a database (this is what I call the parameterspace of a system).
Compared to a WFA this can easily produce 1.000 or 10.000 as many datapoints, that descibe a given trading system.
And there lies the power of PSA, algotrading is about data and information, and it makes sure you get as many data and information about your system as somehow possible, so you can make funded decisions.
What might now just sound like "the same thing" (but with more datapoints) leads to a varity of possibilities that are simply not possible with common WFA.
PSA by example - and some ideas how this data can be used
Taking my old example strategy (from article about Walk Forward Analysis) "If the price moved more than X pips in the last Y days, a course correction will happen", such a parameterspace-database could look like that (its a simplified example of such a parameterspace, where each row is one datapoint):
http://darwins-fx-tools.com/images/art4/db.png
Some ideas on how this data can be used
Make sure you fully understand how this PSA-Database is structured, it is essential!
Forward-trading all candidates instead of the "best one" is just the logical next step from WFA, but this approach gives us enormous advantages:
1. No data is thrown away, it is all saved to make sure our evaluations are highly reliable and robust!
2. No assumptions have to be made. In WFA, you have to decide some things, before you start the analysis. For example, what optimisation fitness value (like "highest profit" or "lowest drawdown" etc) is used to pick the candidate, from optimisation, for live trading. So some very important decisions have to be made without solid data, based on intuition, which is very bad, of course.
3. All the different parameter combinations, over all the different "past->future" windows, are in the database, so we can just see which parameters are best for our system, no educated guesses or trial&error needed. Perhaps a moving-average of 100-200 is best? Or is 50-100 better? Who knows...
4. Correlation tests based on ranks: As we have all the "past->future" datapoints in the PSA, we can see how robust our strategy performs, how high the chance is that we get good live trades out of good backtests/optimisation on past data.
5. We can see what market conditions our strategy is suited for. For example we can spot simple thresholds for it, like "if a strategy makes >XYZ$ during optimisation, it has a very high chance to make a good profit in forward trading." Same thing for overfitting, we could see when our strategies tend to be overfitted, with the same threshold approach.
6. I am currently experimenting with this, but it shows very, very promising results: We can train artificial intelligence (in my case Artificial Neural Networks) on all this data. And the AI can then tell us if we should trade a given strategy at the moment, with what risk we should trade it and how likely it is that it will make profit in the next weeks or days
All these evaluations are not possible with a WFA, as it is not "producing" the data needed for it. And these are just the first ideas, I am sure the community will come up with dozend of ways to analyse this data treasure
How Parameterspace Analysis (PSA) works, Part 2 - Practical examples: How PSA data can be used to analyse a trading system
Now that you understand how PSA works and what data it generates, and what could be done with it from a theoretical point of view, let me show you DATFRA as an example, and what power comes with all this information.
A word about the analysed trading system
All examples were done with the default moving average expert advisor that comes with every mt4 installation. That is because I did not have time to develop strategies yet, as I am still busy coding all this, so do not expect the examples to show any good results!
A word about findings and testing
First of all, as all the data we could need already is in the PSA database, we do never have to re-do any simulations. So if we find that parameter X works best if it is between 75 and 100, we can simply filter our PSA database, so it just contains datapoints with where "parameter X" is [75,100] and then continue our analysis.
Or if we want to look into the characteristics of all candidates that were making very much profit during optimisation, we could filter the database to only contain these candidates.
Example 1: How are different Optimisation Results related
If we want to know if higher profits during optimisation are due to higher trade frequency, or higher profit factor or whatever, we can simply look at the corresponding data.
For example, this strategy achieves higher profits due to higher profit factor
http://darwins-fx-tools.com/images/art4/ex1.png