But there is still sooooo much to explore... What if we used NN? What if we used SVM? What if we used KNN? What if we used random forests ? What if we created a large portfolio using all of these ??? 
Machine Learning with algoTraderJo
- #482
- Edited 4:03pm May 26, 2015 3:47pm | Edited 4:03pm
Linear Regression is already very impressive!
However, the combinations you came up with is cherry picking of a massive exploration. So in order to believe that there is no or little bias, I would need to do the experiment from the beginning (as described the past pages) - lots of computing power. Otherwise I would need to trust that you did
I will experiment further.
It is great to see these results out of ML algorithms! Very nice!
However, the combinations you came up with is cherry picking of a massive exploration. So in order to believe that there is no or little bias, I would need to do the experiment from the beginning (as described the past pages) - lots of computing power. Otherwise I would need to trust that you did
I will experiment further.
It is great to see these results out of ML algorithms! Very nice!
Linear Regression is already very impressive! However, the combinations you came up with is cherry picking of a massive exploration. So in order to believe that there is no or little bias, I would need to do the experiment from the beginning (as described the past pages) - lots of computing power. Otherwise I would need to trust that you did
Is it cherry picking? You will need to measure mining bias to know . Measuring data mining bias is extremely important! I wonder if anyone will try to measure it for the portfolio I have posted
In case anyone wants to perform the data-mining assessment here is the information about the mining process:
The genetic optimization settings for the ast.config (note that F4 uses gaul (open source genetic library) for optimizations):
However it is better to reduce the above space to 5-10K and mine all possible combinations... Genetic optimizations have some problems when doing data-mining assessments, you have to perform several trials per random data set to ensure you get to the global maximum for the statistics you're optimizing for.
The genetic optimization settings for the ast.config (note that F4 uses gaul (open source genetic library) for optimizations):
Inserted Code
optimizationGoal = 0 ; 0 = profit, 1 = Maximum DD, 2 = Maximum DD in Length, 3 = PF, 4 = R2, 5 = Ulcer Index population = 100 maxGenerations = 10000 ; 0 to not limit stopIfConverged = 1 discardAssymetricSets = 1 minTradesAYear = 20 crossoverProbability = 0.5 mutationProbability = 0.5 migrationProbability = 0.0 evolutionaryMode = 0 ;0 = Darwin, 1 = Lamarck Parents, 2 = Lamarck Children, 3 = Lamarck All, 4 = Baldwin Parents, 8 = Baldwin Children 12 = Baldwin All elitismMode = 1 ;0 = Unknown, 1 = Parents survive, 2 = One parent survives, 3 = Parents die, 4 = Rescore Parents mutationMode = 0 ;0 = Single point drift, 1 = Single point randomize, 2 = Multipoint, 3 = All point crossoverMode = 0 ;0 = Single points, 1 = Double points, 3 = Mean, 4 = Mixing, 5 = Allele mixing
However it is better to reduce the above space to 5-10K and mine all possible combinations... Genetic optimizations have some problems when doing data-mining assessments, you have to perform several trials per random data set to ensure you get to the global maximum for the statistics you're optimizing for.
You can also build a portfolio using systems for only one symbol since the mining space contains several candidates that can be combined successfully. Here are 4 USDJPY candidates using linear regression that give decent results (there are many more, these are just examples). Remember that we always retrain the machine learning algos before each trading decision.
These are their simulation results:
This makes perfect sense since a machine learning algorithm might have different results under different market conditions using different parameters. Since changing parameters changes the trading logic like crazy (the ML reaches very different conclusions under some conditions) we get some good complimentary behavior on the same ML approach using different variables. Above you can see that the most common number of returns used to build examples is 6, but a system with 2 also exists with good results. All systems have in common that they trade around the Asian opening time (23-1 GMT +1/+2, makes good sense since we are trading the USDJPY).
The correlation of returns matrix posted also shows that the relationship between the different ML systems with different parameters is limited because this is not like changing the period of a MA on a crossover system, these parameters, in very real terms, lead to very different real systems. The parameters control how a model learns how to trade!
Portfolio results soon...
These are their simulation results:
This makes perfect sense since a machine learning algorithm might have different results under different market conditions using different parameters. Since changing parameters changes the trading logic like crazy (the ML reaches very different conclusions under some conditions) we get some good complimentary behavior on the same ML approach using different variables. Above you can see that the most common number of returns used to build examples is 6, but a system with 2 also exists with good results. All systems have in common that they trade around the Asian opening time (23-1 GMT +1/+2, makes good sense since we are trading the USDJPY).
The correlation of returns matrix posted also shows that the relationship between the different ML systems with different parameters is limited because this is not like changing the period of a MA on a crossover system, these parameters, in very real terms, lead to very different real systems. The parameters control how a model learns how to trade!
Portfolio results soon...
This is the simulation for the portfolio using the 4 systems I described before for the USDJPY (0.25% risk per trade):
The CAGR is 16.8% and the maximum drawdown is 19.7%. The annualized Sharpe ratio is 1.18. The results are inferior to combining linear regression approaches from different symbols but still gives something usable. We can perform something similar with the other symbols and then combine everything into the same portfolio.
Trading 20 or more systems from 3 different symbols would give us a very nice portfolio with a very clean long term equity curve and a high Sharpe and sterling ratio. Of course, all the systems traded must have a low data mining bias (at least 99.97% confidence that they don't come from chance for me) otherwise you are just wasting your time mixing results that are only due to spurious correlations.
The road is clear, construct large portfolios from low data mining bias, high Sharpe ratio machine learning algorithms that constantly retrain
The CAGR is 16.8% and the maximum drawdown is 19.7%. The annualized Sharpe ratio is 1.18. The results are inferior to combining linear regression approaches from different symbols but still gives something usable. We can perform something similar with the other symbols and then combine everything into the same portfolio.
Trading 20 or more systems from 3 different symbols would give us a very nice portfolio with a very clean long term equity curve and a high Sharpe and sterling ratio. Of course, all the systems traded must have a low data mining bias (at least 99.97% confidence that they don't come from chance for me) otherwise you are just wasting your time mixing results that are only due to spurious correlations.
The road is clear, construct large portfolios from low data mining bias, high Sharpe ratio machine learning algorithms that constantly retrain
{quote} Thanks to the cloud computing, almost anybody with programming chops can do these test now.. http://www.bloomberg.com/news/articl...ng-wall-street not sure if you can do that using f4 framework but if you program on your own, you should be able to easily do these tests
.
- #491
- May 27, 2015 8:55am May 27, 2015 8:55am
{quote} I have a cluster for this purpose where I am working at, so I just use F4 and OpenMP/MPI to perform this analysis (along with some python scripts I wrote to manage the whole thing and process the outputs). But you're right, anyone who has the money or means to use cloud resources and program should be able to carry out these tests... I wonder why so few do
- lazy
- no *real* programming abilities (even CS grads from good schools)
- no idea this is important (i.e. not a good understanding of stats and bias)
BTW all institutional quant traders I know do something to estimate mining bias so it is not that uncommon either.
Think : Trade : Live Life
- #492
- May 27, 2015 8:55am May 27, 2015 8:55am
- Joined Oct 2005 | Status: Pip Slappa Extrordinaire | 1,013 Posts
{quote}I wonder why so few do
You are in a maze of twisty little passages, all alike.
{quote} yes. the money hurdle is now quite low but need programming expertise. So few do it because - lazy - no *real* programming abilities (even CS grads from good schools) - no idea this is important (i.e. not a good understanding of stats and bias) BTW all institutional quant traders I know do something to estimate mining bias so it is not that uncommon either.
- #498
- May 28, 2015 7:52am May 28, 2015 7:52am
- Joined Aug 2011 | Status: Trader | 1,327 Posts
No greed. No fear. Just maths.