DislikedSince we have both NN (100 examples, 3 inputs, 500 cycles per training) and SVM (145 examples, 20 inputs) models with decent results (the first a regression model and the second a classification model). It's cool to attempt to build an ensemble of the two models and see if we can improve our results. This means retraining both models on each bar and simply only acknowledging a long signal when both say "go long" and a short signal when both say "go short" (do nothing if they disagree). We build a model ensemble using both classification and regression,...Ignored
Machine Learning with algoTraderJo
{quote} Well i think i sort of explained something that seems interesting here: http://www.forexfactory.com/showthre...82#post7876282 Since May E/U has been going down pretty clearly if you ask me. There has been a bit to suggest differently, but not in a big way. Of course now that we know what occured between Say May 10 until now, just selling where ever and holding was a apparently a good idea. But lets assume we didnt know THAT much. So, in a sell trend like we have had on E/U since beginning of may, sell as the high is...
I do not want to know exactly how you trade or what ideas you have for discretionary trading (sadly I don't have the time to read it with the attention it deserves and come up with an ML approach that simulates it). This is a thread about machine learning (ML), if you have an idea for a clear input/target/model structure that you want me to test or if you have questions related to input/target/model choices, tell me. Explanations about the way in which you trade, not related to ML, are better left inside other threads.If you have ideas for ML approaches in the form of a clearly explained, input/target structure and model selection, do tell me (read the thread from the start for a better explanation of how input/target structures work, what models are available, etc). I sadly don't have any time to invest in attempting to understand discretionary trading approaches.
- #266
- Jan 19, 2015 6:39am Jan 19, 2015 6:39am
- Joined Jul 2010 | Status: Determined | 5,346 Posts
{quote} I probably did not explain myself very clearly before
My signature is: "Classified".
As training is important when building NN, we can also try an ensemble using rNN models with different levels of training, then act only on a majority vote of the models. This means that we are attempting to extract different levels of generalization and we only act when most levels agree about what is going to happen. The following test uses an ensemble of 6 models using optimization cycle values 100, 200, 300, 400, 500 and 600 per training (remember that training of all the models is always carried out on each bar using the past 100 examples, each example with 3 inputs). We take signals when the net prediction adds to at least 3/-3 (models give 1 for long signals, which is a prediction of return above 0 and -1 for short signals which is a prediction of returns below 0).
The results are better than when using a single NN model. Attempting to capture a consensus between different levels of generalization is a very common strategy when using NN, because it is difficult to try to choose an arbitrary optimization cycle level and obtain good enough trading results. This is the tip of the iceberg in the things that you can start doing with NN, they are perhaps the most useful models in the building of ML approaches for trading.
The results are better than when using a single NN model. Attempting to capture a consensus between different levels of generalization is a very common strategy when using NN, because it is difficult to try to choose an arbitrary optimization cycle level and obtain good enough trading results. This is the tip of the iceberg in the things that you can start doing with NN, they are perhaps the most useful models in the building of ML approaches for trading.
Change your hidden layer and output neurons to Hyperbolic Tangents (instead of the FastSigmoid/Linear neurons used on the last post) and you get very different results:
Changing functions means changing everything, so you need to be very very careful when choosing your hidden layer and output neuron functions. These results don't mean that Hyperbolic Tangents are "bad functions" for the problem but simply that they have different training requirements and different characteristics than the functions use before. The neuron types used are perhaps the most critical initial decision when building an NN model.
Changing functions means changing everything, so you need to be very very careful when choosing your hidden layer and output neuron functions. These results don't mean that Hyperbolic Tangents are "bad functions" for the problem but simply that they have different training requirements and different characteristics than the functions use before. The neuron types used are perhaps the most critical initial decision when building an NN model.
{quote} Hi Jo, the question was how to use the information about high/low as inputs. So keeping the existing systems, how about just adding a couple of inputs. In the first case I would try adding one value for each bar (0, 1, 2 or 3). So in the SVM example you are using the last 4 bars as input. If you tried adding the four values (one for each bar) about the high/low information, would that add value to the model? The second example would add 2 values for each bar (one for high and one for low). So it would be an additional 8 inputs. Hope that...
An SVM is robust to the addition of noise, so adding this information only deteriorates results slightly, nonetheless a more sensitive model, like an NN, would change from profitable to losing with the addition of the suggested inputs. The inputs suggested are also quite redundant, using several High/Low comparisons is practically the same as using a single one, because the information is all contained there. For example the High > Previous High, Low < Previous low already contain the information that you place on the 4th input, as if they are both true the model already knows this (they are both 1), you do not need a separate input to contain this information.
The issue why the High/Low information seems irrelevant (even if the inputs had a better design) is related to the fact that we are attempting to predict bar directionality which has little to do with the High/Low information. When attempting to predict bar directions the High/Low do not add an amount of information greater than the Open-to-Open bar direction comparisons. If you're in a down/up trend most likely the low-to-low and high-to-high comparisons follow similar trends as the open-to-open. The high-to-high and low-to-low could help us if we were looking at different targets (where they would be more relevant).
Good try! In machine learning doing experiments is the best way to learn new things about how to create better systems
Thanks for testing it out! However, I might not have explained it right. The idea was 1 additional input per bar. This input can have the value (category) of 0, 1, 2, 3 0 = high and low are weithin previous range 1 = high is higher than previous high 2 = low is lower than previous low 3= high is higher and low is lower So the existing model would have 145 samples (each 20 bars return and 20 bars the category info = 40 inputs) But even that might not make much difference. ----------------------------------------------------------------------- As...
It's great to hear you're building your own framework for testing! Just some small advice. These things have nothing to do with ML but if you fail to implement them things will become painful later on:
- Make sure you can refactor timestamps, make corrections for DST, GMT shifts, have NTP synchronization, etc. You will save yourself a lot of pain if you implement this from the start. Make sure you can correct any back testing data to a set timestamp with a chosen DST and GMT shift, make sure you can do the same with your live broker's data.
- Make sure you can easily move from back testing to live trading! (I made the mistake of ignoring this step when developing in R and things were a huge pain)
- Make sure you can live trade the exact same code you back-test.
I chose the Asirikuy framework because all the above are a pain to develop and debug individually (at least for me they were). I preferred to pay for something that had already been developed and is being actively supported and debugged by a trading community (I chose to spend more time doing my actual ML development for trading than simply framework development). If you have the time to develop all the above using your chosen language then that's absolutely cool, deeply encouraged
- #276
- Jan 20, 2015 4:12am Jan 20, 2015 4:12am
- Joined Aug 2014 | Status: Trading | 2,163 Posts
Trading is the hardest way to make easy money...
To continue moving in the exploration of NN I am now going to show you the effect of making variations in the hidden and output functions as well as the number of optimization cycles per training for the NN using 100 examples to train (last three bar returns as inputs, next bar's return as target), remember that training of the models happens on every bar. The balance chart below shows you a comparison between FastSigmoid (FS), Linear (L) and Hyperbolic Tangent (HT) functions at different optimization cycle numbers.
There is a very large amount of information in this picture. You can see that the FS functions allow for a dramatic increase in overall profits after some increases in optimization cycles, while their performance under recent market conditions tends to deteriorate. We can also see that the HT functions seem to give the best performance under recent market conditions with high cycle numbers, along with the FS that uses less optimization cycles per training. Decreasing cycles for the HT leads to a decrease in the 2003-2008 period that does not appear when we have a higher cycle training regime. Linear functions are the most stable with 500 to 10,000 cycles giving the same curves (not showed as they simply overlap), although results are inferior to other functions.
Optimization cycle number and function choices can therefore have a very relevant effect in the performance of our models when building trading systems. It is difficult to know what model and cycle lengths to use; so a general strategy is to bring the different choices we have into ensembles or portfolios. It is easy to think we might benefit from putting together an ensemble using models that use FS and HT functions with higher training (1500 cycles), which gives the following result (ensembles acts only on agreement of the two models):
The Sharpe for the FS model is 1.25, for the HT model it is 0.999 and for the ensemble it is 0.872. We can actually see a deterioration in results. Why do you think this is the case? Why are sometimes ensembles so beneficial and other times so counter-productive? Can we tell in advance if an ensemble will be better or worse? Penny for your thoughts
There is a very large amount of information in this picture. You can see that the FS functions allow for a dramatic increase in overall profits after some increases in optimization cycles, while their performance under recent market conditions tends to deteriorate. We can also see that the HT functions seem to give the best performance under recent market conditions with high cycle numbers, along with the FS that uses less optimization cycles per training. Decreasing cycles for the HT leads to a decrease in the 2003-2008 period that does not appear when we have a higher cycle training regime. Linear functions are the most stable with 500 to 10,000 cycles giving the same curves (not showed as they simply overlap), although results are inferior to other functions.
Optimization cycle number and function choices can therefore have a very relevant effect in the performance of our models when building trading systems. It is difficult to know what model and cycle lengths to use; so a general strategy is to bring the different choices we have into ensembles or portfolios. It is easy to think we might benefit from putting together an ensemble using models that use FS and HT functions with higher training (1500 cycles), which gives the following result (ensembles acts only on agreement of the two models):
The Sharpe for the FS model is 1.25, for the HT model it is 0.999 and for the ensemble it is 0.872. We can actually see a deterioration in results. Why do you think this is the case? Why are sometimes ensembles so beneficial and other times so counter-productive? Can we tell in advance if an ensemble will be better or worse? Penny for your thoughts
