Loving this thread. Will continue to read...
Another thing we can play with is the prediction target. I previously selected it as 0.5*ATR(20) but you can change it to higher values in order to try to predict a more favorable move in the currency pair. Changing the target to 1.5*ATR(20) means that we are predicting a much larger move, meaning that we are seeking to get a larger edge from our prediction. Predicting a larger target also means that the prediction frontier needs to be moved in order to prevent having many examples where the target is not reached, due to this I also increased the frontier to 35 days. This means that the target is now whether a +/- 1.5*ATR(20) target is reached within the 35 days following the evaluation day. We assign a 1 if the upper target is reached first or a 0 if the lower target is reached first, if no target is reached we simply skip the example and get the next one in the past.
In line with my last post I also use an ensemble of models instead of a single model to make predictions. This can be applied to the K-NN as well as the SVM in order to obtain predictions that are not dependent on the exact number of inputs and number of examples used for training on each bar. I only take a trade if all models point to the same decision. These are the results for the SVM and K-NN ensembles:
With this we now have better results in the 1991-1995 region (reduced the drawdown from 35 to 25%) and we have good performance in recent market conditions. The SVM model beats the K-NN model but they both achieve good results throughout the test. I still would like to have a higher linearity near the end of the test (less volatility) and I would also like the 1991-1995 period to be a net growth period instead of just a flat patch within the test. These are two things I believe we can achieve. How would you improve the results of these ensembles? Post your thoughts
In line with my last post I also use an ensemble of models instead of a single model to make predictions. This can be applied to the K-NN as well as the SVM in order to obtain predictions that are not dependent on the exact number of inputs and number of examples used for training on each bar. I only take a trade if all models point to the same decision. These are the results for the SVM and K-NN ensembles:
With this we now have better results in the 1991-1995 region (reduced the drawdown from 35 to 25%) and we have good performance in recent market conditions. The SVM model beats the K-NN model but they both achieve good results throughout the test. I still would like to have a higher linearity near the end of the test (less volatility) and I would also like the 1991-1995 period to be a net growth period instead of just a flat patch within the test. These are two things I believe we can achieve. How would you improve the results of these ensembles? Post your thoughts
A good question is to wonder about the sensitivity of the target. Are things very different if we use slightly different targets for prediction? Keeping the frontier at 35 and using the same targets with exactly the same SVM ensemble (as mentioned on one of my last posts) you get the results below:
You can see that although changing the target does change your results the change is not very dramatic. Results only start to change significantly when you go above a 2.0 target or below a 1.0 target. Trying to predict +/- 2*ATR(20) seems to be too ambitious for the set of predictors we are using to build the models.
You can see that although changing the target does change your results the change is not very dramatic. Results only start to change significantly when you go above a 2.0 target or below a 1.0 target. Trying to predict +/- 2*ATR(20) seems to be too ambitious for the set of predictors we are using to build the models.
Hi AlgoTraderJo, I am sure you will get to it, but what is the result if you try some of these algorithms on shorter timeframes, e.g. 1 hour?
When using ensembles an interesting thing you can do is to take signals that are favored by a given % of your models. Playing with this threshold can help you adjust trading frequency and avoid not trading whenever you have low disagreement (for example 1 model is against all others). This is the result we get for the SVM ensemble if we take valid signals as those where at least 75% of our results predict the same direction.
We get an improved result under the past few years of market conditions but again we have a significant deterioration of results in 1991-1999. What else do you think we can do to make results more stable? what can make results stable all around?
We get an improved result under the past few years of market conditions but again we have a significant deterioration of results in 1991-1999. What else do you think we can do to make results more stable? what can make results stable all around?
Let us make this thread a little bit more interactive 
Post some ideas you would want me to try to improve the machine learning results I have posted so far (predictions on the daily timeframe on the EURUSD), I will try them out and will post the results here!
Post some ideas you would want me to try to improve the machine learning results I have posted so far (predictions on the daily timeframe on the EURUSD), I will try them out and will post the results here!
Let us make this thread a little bit more interactive
- #154
- Edited 7:45pm Dec 17, 2014 1:23pm | Edited 7:45pm
When you add new information that contributes with predictive power you actually get better results. The graph below shows you our K-NN test using 2 bar directions as input and a second test using 18 bars as inputs. As you can see the addition of 16 new inputs adds enough information to improve results altogether. {image} It is also interesting to see how the distribution of accuracy in predictions has changed significantly and our two systems take opposite directions through several parts of the simulation. This means that we may be able to obtain...
Below is R code used to download Oanda daily data into R for EUR/USD from December of 1998 to 2014 (with Oanda data you can only get the Close price). The last 3 lines of code show how you can call the function getyears to start downloading data. The function getyears is essentially a convenience function for the much smaller function, dailyoanda to put the data together into a zoo object (and so I don't have to type out the full date).
Inserted Code
library(tseries)
dailyoanda = function(symbol="EUR/USD", start="2010-01-01", end="2010-12-31", provider="oanda",
quote = c("Open", "High", "Low", "Close")) {
return(get.hist.quote(instrument=symbol, quote = quote, start=start, end=end, provider=provider))
}
getyears = function(symbol="EUR/USD", startyear=2000, endyear=2013, provider = "oanda",
quote = c("Open", "High", "Low", "Close")) {
year = startyear
hist = NULL
while (year <= endyear) {
sy = paste(year,"-01-01", sep="")
ey = paste(year,"-12-31", sep="")
dat = dailyoanda(symbol, sy, ey, provider, quote)
if (is.null(hist)) {
hist = dat
} else {
hist = c(hist, dat)
}
year = year + 1
}
return(hist)
}
#quote = c("Open", "High", "Low", "Close")
quote = c("Close")
EU = getyears("EUR/USD", 1998, 2014, "oanda", quote) [Edit #2] After messing around with the way the classes are assigned and the trading rules I end up with this
:
[Edit #3] tdazio: Can you tell me why an acf(1) == 10.8% is too high to be true? (I get the same value)
I would like to strongly emphasize the importance of the daily rate structure. Daily candles can be constructed in a wide variety of ways (weekly starting and ending times, GMT shifts, DST or no DST, etc). The way in which the candles are structured is very important for the success of ML strategies that work on the daily TF because the return series will be very different depending on how you construct the daily series. I have constructed a series of tests using a linear regression model, the same I posted about before. The model uses the last 2 bar returns (Open-LastOpen) as inputs and attempts to predict the next bar's return. The model trains on every bar using the last 100 examples and uses a 60% of the ATR20 stop. Look at the results:
You can see that the best results are for GMT DST +2 non-DST +1, as you go away from this GMT shift results start to fall apart because the structure of your returns is now very different and it becomes harder for the machine learning model to make predictions. Results also don't fall randomly apart, they consistently become worse the more you move away from the best shift. The DST setting is also critical because market session times change with DST. Here you have a comparison of results with/without DST off-set:
Data structure is one of the most critical aspects in FX machine learning (this market is especially complicated as you need to correct different broker's data to match a desired structure). It is very important to have the same data structure if you want to be able to reproduce the same results. You will see as we move along the thread that this shows heavily when we develop systems for the lower timeframes (as market volatility cycles depend on market opening times, that change with DST as well).
You can see that the best results are for GMT DST +2 non-DST +1, as you go away from this GMT shift results start to fall apart because the structure of your returns is now very different and it becomes harder for the machine learning model to make predictions. Results also don't fall randomly apart, they consistently become worse the more you move away from the best shift. The DST setting is also critical because market session times change with DST. Here you have a comparison of results with/without DST off-set:
Data structure is one of the most critical aspects in FX machine learning (this market is especially complicated as you need to correct different broker's data to match a desired structure). It is very important to have the same data structure if you want to be able to reproduce the same results. You will see as we move along the thread that this shows heavily when we develop systems for the lower timeframes (as market volatility cycles depend on market opening times, that change with DST as well).
- #157
- Dec 19, 2014 7:49am Dec 19, 2014 7:49am
- Joined Aug 2011 | Status: Trader | 1,327 Posts
No greed. No fear. Just maths.
Hi ATjo I'm still trying to reproduce the linear regression results. My parameters are: Currency of the account: USD Starting balance: $100000 Money Management: 1% Timezone: Europe/Paris,Berlin Price used: open price, (bid+ask)/2 Stop loss: 0.6 x ATR20. ATR uses the daily candle in GMT time and excludes the week-end candles (including the W-E changes nothing). Take profit: None Number of price return per sample: 2 Number of samples in window: 100 Strategy: If no position is open, open in the direction of the forecast. Else if the current position...
If you can please post some balance graphs so that I can take a quick look and see what might be wrong (sadly do not have time to go through and process txt files). Have you compared your predictions with the ones I posted before? Are they in alignment? Differences could be because of a few reasons, the ATR calculation (mine is done with the ATR as implemented in TA-lib) or the linear regression procedure (I use the models as implemented within the Shark library, see here). Another possibility is that our feeds have very significant differences (what is your source?). I obtain very similar results with data from Oanda, data from Alpari UK and data from Dukascopy. Using the exact same libraries for machine learning and indicator calculations should make reproducing the results easier, if you want to follow/reproduce results of the thread more easily I strongly encourage using Shark for model building and TA-lib for indicator calculations. Since our testing frameworks are so different there is not much I can do besides sharing all information about what I'm using
Do ask any questions you may have.
When doing regressions we can also try other models to see if we can improve the results obtained using the linear regression approach. The first thing that comes to mind is to use models that are more complex, that can find relationships that may be very hard for a linear regression approach to accurately describe (non-linear relationships). The next regression model I want to share with you is a Random forest model, this model is able to find non-linear relationships between input variables and can also find relationships between a large number of variables. Another great thing about random forests (RF) is that they are very robust to the addition of unnecessary information, they are quite good at leaving out variables that do not add any predictive power to the mix. Using the exact same setup as in the previous linear regression model I have now repeated the test using a random forest model. This is the result:
The results are similar to the LR approach (nothing surprising here, although behavior is indeed much better in the 2007-2014 period) but they will become more interesting as we increase the complexity in inputs given to the RF model, as we go beyond the limits of the possible relationships that the LR model can find. The use of RF models will be important as we move further along the thread, into higher levels of complexity. Another great thing about using RF is that we can get measurements of variable importance from this model type. What additional inputs do you think an RF model could use?
The results are similar to the LR approach (nothing surprising here, although behavior is indeed much better in the 2007-2014 period) but they will become more interesting as we increase the complexity in inputs given to the RF model, as we go beyond the limits of the possible relationships that the LR model can find. The use of RF models will be important as we move further along the thread, into higher levels of complexity. Another great thing about using RF is that we can get measurements of variable importance from this model type. What additional inputs do you think an RF model could use?