Machine Learning with algoTraderJo | Page 3 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Algorithmic Quant Trading (Machine Learning + Stat-Arb) 25 replies

Forex with Machine Learning Software Project 1 reply

Machine Learning + Retail Forex = Profitable? (Quant) 1 reply

Potential new machine learning style software. 79 replies

My most recent advancements into machine learning 16 replies

Attachments: Machine Learning with algoTraderJo
Exit Attachments

Machine Learning with algoTraderJo

  • #41
  • Dec 9, 2014 9:33am Dec 9, 2014 9:33am
Quoting stt
{quote} I think deep networks will be widely used in all sort of places. the problem acadameics are dealing with (computer vision, spatial awareness) are so much more complex then finding patterns in financial data..
I also do machine learning for a variety of problems (in industry) besides trading and I can tell you that trading is perhaps the worst of them all. In academia/industry most of the problems you tackle are deterministic (like computer vision and spatial awareness) and are often not based on time series, while financial time series tend to be chaotic and their prediction is certainly very complicated. There are certainly people doing complex things in industry but financial time series prediction is one of the worst available problems to study (from a difficulty standpoint).

Quoting stt
{quote}I am experimenting with this in hft space so we have a lot more that certainly help control data mining. the paper olsen linked didnt really use a lot of data so i think that all is doable in under $1000 machine.
I don't doubt that someone can probably do this using GPUs but sadly I don't have the time or desire to construct deep learning libraries on openCL/CUDA to be able to run models using graphic cards. Deep learning certainly may have potential in financial applications but since current machine learning algorithms do the trick for me I still have no desire to go into this level of complexity (I still have tons to study outside of deep learning).
  • #42
  • Dec 9, 2014 9:52am Dec 9, 2014 9:52am
I'm amazed by your results! Each time I try to cluster FX data I get a single cluster that encompass the whole dataset. That's why I tried another approach. Instead of clustering then testing in which cluster the query vector is I select the N nearest neighbors of my query vector (using a VP-tree).

In the picture below I take all the sequences of 40+15 close prices (shift to the 40th close as the origin). The 40 first are supposed to be a pattern and the next 15 are the prediction. At some bar I query for the vector made of the last 40 close prices. I retrieve the 30 nearest neighbors. I average the predictions (red). I also draw their median (blue). The black line is the last close price for reference. The result is very often close to the black line. The variance is always very big.

How can I know if this forecast makes sense or if it is just plain random? If it is valid, how can I measure the predictive power of such an indicator?

I can't backtest since there is no trading rules. If I wanted to create trading rules from this indicator how do infer it from the data? For instance is it better to follow the bullish/bearish bias (red above black I buy) or "mean revert" by fading a price 2-3 sigma too far away from the forecast?

Wouldn't it be fun to feed this to a GP engine and let it discover the rules?

Attached Image (click to enlarge)
Click to Enlarge Name: EURUSD-D1-forecast.png Size: 36 KB
No greed. No fear. Just maths.
  • #43
  • Dec 9, 2014 10:24am Dec 9, 2014 10:24am
Hi Algo,
nice thread !
What do you think about latest Asirikuy NN delevopment called AsirikuyBrain ?
http://mechanicalforex.com/2014/01/n...everal-nn.html
I'm planning to run it in live account but Daniel suggested to run at least 3-5 different instance to have a better performance evaluation in the long run.

Regards,
Skyline
  • #44
  • Dec 9, 2014 11:15am Dec 9, 2014 11:15am
Quoting algoTraderJo
Consider now that we change the model to a still simple yet more powerful classifier (a K-Nearest Neighbor approach) using the same input/target structure as above (two past days to predict next day's directionality). However we now have a stoploss of 70% of the Average True Range (risking 1% per trade) and we train using 70 instead of 200 examples. We still rebuild the model on each daily bar. See how our balance curve changes drastically: {image} We now have something that...
A quick question regarding this just to clarify for myself (and perhaps others).

Are you using last two days returns or just direction (up/down). seems like it is just up/down so for last two days, it is just a combination of 11, 10, 01 and 00. In effect you are just looking at the direction of next day in each of these cases in last 70days, doing an average and predicting based on that (perhaps >= .5 mean 1 and <.5 means 0).
many people might be doing similar prediction just may not realize they are applying knn algo. In this there will be no nearest calculation needed since the new input would exactly match one of the classes.

This would be considered very basic use of knn since the dimensions are low and each dimension is discreet.

Is this correct or i am missing something fundamental.
Think : Trade : Live Life
  • #45
  • Edited 12:31pm Dec 9, 2014 11:22am | Edited 12:31pm
Quoting algoTraderJo
{quote} I also do machine learning for a variety of problems (in industry) besides trading and I can tell you that trading is perhaps the worst of them all. In academia/industry most of the problems you tackle are deterministic (like computer vision and spatial awareness) and are often not based on time series, while financial time series tend to be chaotic and their prediction is certainly very complicated. There are certainly people doing complex things in industry but financial time series prediction is one of the worst available problems to...

Agree. there is a lot that can be done with other ml techniques. so no point to skip past those just because other fields are on DL bandwagon.
Think : Trade : Live Life
  • #46
  • Dec 9, 2014 11:42am Dec 9, 2014 11:42am
I like the thread, but sadly this brute force method will not crack the market. Two reasons for this, firstly markets have nothing to teach that a neural net work can learn. Its a bit like sitting in your car with a blind fold on trying to learn to drive. We don't learn like this and neither should a machine algo. Programmers think that they can create code to forecast the market. I can tell you now that you can do that already with about a 70% accuracy. With no code, its easy, but its not always right, however 70% correct is ok for me.

Secondly, making money in a market is about buying or selling either low or high, you can already see where that has taken place in the past which is about all you have for the future. That's the key to making money in the markets, the other thing is how far is this move going, Hurst had a rough method for that, which predefines a zone where a reversal may take place, feeding that into a artificial machine learning algo might yield results. Hoping to uncover the golden goose by unleashing artificial learning on the market will result in a highly complex piece of code that will not predict jack, simply because its not learning the right things.

More than happy to share insight with like minded folk, if wanted.
  • #47
  • Edited 12:24pm Dec 9, 2014 12:09pm | Edited 12:24pm
I have a simple question for the experts: How does the machine learning differ from curve-fitting? As far as I can see (which may not be far, to be honest), it is the same thing, the only difference being that we have removed the decision (responsibility?) to a piece of software.

Two remarks to a previous conversation: (a) if one can predict the following bar direction with >60% accuracy, one can make money with binary options, regardless of the range of the bar. (b) Also I would expect that setting a learning period over one of the typical cycles of the market could significantly improve performance of the system. So instead of 2 previous days' bars, one should perhaps use 5 bars (one week).

Cheers,
k
  • #48
  • Dec 9, 2014 12:24pm Dec 9, 2014 12:24pm
Quoting kprsa
I have a simple question for the experts: How does the machine learning differ from curve-fitting?
Here is a very good and comprehensive introduction to machine learning.
http://research.microsoft.com/en-us/...cmbishop/PRML/
(http://research.microsoft.com/en-us/...RML-sample.pdf)
No greed. No fear. Just maths.
  • #49
  • Dec 9, 2014 12:26pm Dec 9, 2014 12:26pm
Quoting kprsa
So instead of 2 previous days' bars, one should perhaps use 5 bars (one week)
How many bars is the best is part of the machine learning scope.
No greed. No fear. Just maths.
  • #50
  • Edited 1:02pm Dec 9, 2014 12:32pm | Edited 1:02pm
Quoting PipMeUp
I'm amazed by your results! Each time I try to cluster FX data I get a single cluster that encompass the whole dataset. That's why I tried another approach. Instead of clustering then testing in which cluster the query vector is I select the N nearest neighbors of my query vector (using a VP-tree). In the picture below I take all the sequences of 40+15 close prices (shift to the 40th close as the origin). The 40 first are supposed to be a pattern and the next 15 are the prediction. At some bar I query for the vector made of the last 40 close prices....
In my experience using close prices for predictions is always a poor choice because close prices are strongly auto-correlated. This is a reason why you often get predictions that are just the last close price when using close prices on machine learning algorithms. The model learns that the close price must be a close price in the vicinity of the current one but where in the vicinity becomes noise. Big variance is definitely an indication of randomness, which I would say is likely strongly related with the use of close prices. I would attempt to reduce the variance of predictions first, before attempting to go further. I would suggest using bar returns instead of close prices as this removes the auto-correlation in the series.

Building trading rules using a GP engine would be problematic due to data-mining bias (becoming nearly impossible to measure) I would say that in this sense the ideal target for prediction is not simply a value for the series but a trade outcome (which we will get into later on this thread). Instead of answering the question "is price going to X?" it's better to answer the question "is a trade entered at this point with X/Y targets going to succeed?"This in itself constitutes a guideline for trade entry.
  • #51
  • Dec 9, 2014 12:34pm Dec 9, 2014 12:34pm
Quoting Hurst
I like the thread, but sadly this brute force method will not crack the market. Two reasons for this, firstly markets have nothing to teach that a neural net work can learn. Its a bit like sitting in your car with a blind fold on trying to learn to drive. We don't learn like this and neither should a machine algo. Programmers think that they can create code to forecast the market. I can tell you now that you can do that already with about a 70% accuracy. With no code, its easy, but its not always right, however 70% correct is ok for me. Secondly,...
yes, agree you cant just throw all the past data into a big machine and expect it to just magically become profitable.

Can you please be specific about how you can do that. 70% right is great (depends on how you define "right") to make money. Not every quant is looking for a full proof method.

the whole research is into using techniques of ML to get better insights from the data. Your S&R can be one input if they prove themselves to have any prediction power.
Think : Trade : Live Life
  • #52
  • Dec 9, 2014 12:38pm Dec 9, 2014 12:38pm
Quoting skyline
Hi Algo, nice thread ! What do you think about latest Asirikuy NN delevopment called AsirikuyBrain ? http://mechanicalforex.com/2014/01/n...everal-nn.html I'm planning to run it in live account but Daniel suggested to run at least 3-5 different instance to have a better performance evaluation in the long run. Regards, Skyline
This is quite more complex than what I am doing here. The K-NN and linear map methods I have showed here have the advantage of always giving the same result (they are in this sense very deterministic) the neural networks used in the Asirikuy Brain are not deterministic in a quite real sense, you require to run several instance to obtain a mean result. This is due to the neural networks used and the predictors used by Daniel, which are more complicated than those exposed here (one of the neural networks even uses a chart image for predictions). For these systems, the more you run the better! I would run 10+ instances if possible to obtain a true approach to mean simulation performance.

I think Daniel has also never measured mining bias for the machine learning instances (I personally have no idea if the brain suffers from significant bias problems), I believe he has focused in bias measurements for price action based strategies (which I think is excellent work, you as a member may me familiarized with these systems).
  • #53
  • Dec 9, 2014 12:41pm Dec 9, 2014 12:41pm
Quoting kprsa
I have a simple question for the experts: How does the machine learning differ from curve-fitting? As far as I can see (which may not be far, to be honest), it is the same thing, the only difference being that we have removed the decision (responsibility?) to a piece of software. Two remarks to a previous conversation: (a) if one can predict the following bar direction with >60% accuracy, one can make money with binary options, regardless of the range of the bar. (b) Also I would expect that setting a learning period over one of the typical cycles...
yes traditionally neural networks had this big issue. thats why you cant just blindly throw data to a network and expect it to magically work. but there have been good development in techniques to control the curve fitting (hinton's dropout method is one).

but also machine learning is not a precise term. in some sense linear regresssion is machine learning though most dont consider it as such.
Think : Trade : Live Life
  • #54
  • Dec 9, 2014 12:41pm Dec 9, 2014 12:41pm
Quoting Hurst
I like the thread, but sadly this brute force method will not crack the market. Two reasons for this, firstly markets have nothing to teach that a neural net work can learn. Its a bit like sitting in your car with a blind fold on trying to learn to drive. We don't learn like this and neither should a machine algo. Programmers think that they can create code to forecast the market. I can tell you now that you can do that already with about a 70% accuracy. With no code, its easy, but its not always right, however 70% correct is ok for me. Secondly,...
Agree to disagree. I have been working with machine learning methods for years and I have achieved good success in live trading. You do need to be aware of the statistical biases present and you need to ensure that you do have something that has predictive power. However I assure you that building strategies that succeed in the market using machine learning is possible.
  • #55
  • Dec 9, 2014 12:53pm Dec 9, 2014 12:53pm
Quoting kprsa
I have a simple question for the experts: How does the machine learning differ from curve-fitting? As far as I can see (which may not be far, to be honest), it is the same thing, the only difference being that we have removed the decision (responsibility?) to a piece of software. Two remarks to a previous conversation: (a) if one can predict the following bar direction with >60% accuracy, one can make money with binary options, regardless of the range of the bar. (b) Also I would expect that setting a learning period over one of the typical cycles...
Your questions are quite fundamental !!

If you trained a machine learning model with data from 2000 to 2014 and then executed a simulation of that same period using the model you have trained you would have effectively curve fitted the model to the training period. However in the case of the systems I have showed I am always building the model with recent data on each bar and I am then performing a prediction for the next bar (which the model has never seen). In this sense I am not using the machine learning process to curve-fit to the data. However there are variables that I am indeed choosing based on simulation performance results, like the inputs, the stoploss, etc. The selection of these values introduces curve-fitting bias. The way in which I limit this bias is by performing the simulation across more than twenty years, which is a very broad array of market conditions. Any system you develop is somehow curve-fitted to the historical data you test it on, the only defense you have is to use so much data that the curve-fitting is expected to reflect a more general scenario.

Now about your other questions:

(a) Yes, that is true. But do you need to predict with more than 50% accuracy to make money when trading? Since we make/lose money based on the amount of pips per bar we actually depend more on the distribution of predictions than on the net success of predictions. Perhaps we only predict 40% of bars correctly, but those turn out to be larger bars. When trading binary options your success does depend primarily on the bullish/bearish prediction, when trading it depends on the distribution of that success, meaning it's not the best target for a machine learning approach.

(b) That does not happen every time. However this does bring the question of seasonality into the picture. What if the examples used for training are more specific? (for example I only use outcomes of Tuesdays to predict Tuesdays, etc). This is an interesting topic, it does make a difference.
  • #56
  • Dec 9, 2014 12:54pm Dec 9, 2014 12:54pm
Quoting stt
{quote} A quick question regarding this just to clarify for myself (and perhaps others). Are you using last two days returns or just direction (up/down). seems like it is just up/down so for last two days, it is just a combination of 11, 10, 01 and 00. In effect you are just looking at the direction of next day in each of these cases in last 70days, doing an average and predicting based on that (perhaps >= .5 mean 1 and <.5 means 0). many people might be doing similar prediction just may not realize they are applying knn algo. In this there will...
I am using direction (1) bullish, (0) bearish. You're indeed right that this is an extremely basic K-NN case, which is what makes it a good case for the building of a basic practical example.
  • #57
  • Dec 9, 2014 1:28pm Dec 9, 2014 1:28pm
@PipMeUp: Thanks for the reference, I will obtain and study the book as soon as I am convinced about the path. (Also, I have a feeling that for the understanding/description of the 6-month drop in eurusd one should probably include the data dating back 2 or 3 years.)

@stt: Thanks, I found the paper on arxiv, but in order to understand the language I think I'd better at least read the book PipMeUp recommended.

Quoting algoTraderJo
{quote} Your questions are quite fundamental !!
Thank you. It turns out that, in order to answer a simple childish question "Why is the sky blue?", one needs to go and study physics.

Quoting algoTraderJo
{quote} If you trained a machine learning model with data from 2000 to 2014 and then executed a simulation of that same period using the model you have trained you would have effectively curve fitted the model to the training period. However in the case of the systems I have showed I am always building the model with recent data on each bar and I am then performing a prediction for the next bar (which the model has never seen). In this sense I am not using the machine learning process to curve-fit to the data.
This is all absolutely clear to me. And I think that what the ml algorithm does is curve-fits the past, and extrapolates one step in the future. My question, in other words, is why do you need a machine learning software for this? Does a ml algorithm have any advantage over a person doing the fit manually (i.e. with a script or software that does the fitting procedure). For example, I ( :nerd: ) can do fits to data and extrapolations based on those fits by various deterministic methods or by a Monte Carlo algorithm/simulated annealing. Would the ml algorithm, given the same input data and the same given model reach a different conclusion than me, and if yes, why? If it will not, isn't this just shifting of procedure/decision/responsibility to a black box?

Quoting algoTraderJo
{quote}(a) Yes, that is true. But do you need to predict with more than 50% accuracy to make money when trading? Since we make/lose money based on the amount of pips per bar we actually depend more on the distribution of predictions than on the net success of predictions. Perhaps we only predict 40% of bars correctly, but those turn out to be larger bars.
No, of course you do not require more than 50% accuracy. Thanks for specifying this!

Thanks everyone for their inputs. I am on the verge of starting to play with this...

Cheers,
k
  • #58
  • Dec 9, 2014 1:35pm Dec 9, 2014 1:35pm
Quoting algoTraderJo
{quote} This is quite more complex than what I am doing here. The K-NN and linear map methods I have showed here have the advantage of always giving the same result (they are in this sense very deterministic) the neural networks used in the Asirikuy Brain are not deterministic in a quite real sense, you require to run several instance to obtain a mean result. This is due to the neural networks used and the predictors used by Daniel, which are more complicated than those exposed here (one of the neural networks even uses a chart image for predictions)....
So looking at your result it seems you're a step ahead than Daniel, it's really interesting that your NN could be backtested with the same outcome for different runs!
About mining bias problem , I don't know if it's a concept that could be applied to NN, btw a nice question to ask Daniel

Will follow with interest your work here !

Cheers,
Skyline
  • #59
  • Dec 9, 2014 1:38pm Dec 9, 2014 1:38pm
Quoting algoTraderJo
{quote} In my experience using close prices for predictions is always a poor choice because close prices are strongly auto-correlated. This is a reason why you often get predictions that are just the last close price when using close prices on machine learning algorithms. The model learns that the close price must be a close price in the vicinity of the current one but where in the vicinity becomes noise. Big variance is definitely an indication of randomness, which I would say is likely strongly related with the use of close prices. I would attempt...
Thanks for the input. This is the result using the Euclidian distance of the 39 past bars returns. The "prediction" of the past is pretty bad but the variance seems lower too.
Attached Image (click to enlarge)
Click to Enlarge Name: EURUSD-D1-forecast-diff.png Size: 37 KB


Now if I average both metrics:
Attached Image (click to enlarge)
Click to Enlarge Name: EURUSD-D1-forecast-average.png Size: 32 KB


How important is the choice of the distance (metric) is nearest neighbor? L1, L2, L∞, Mahalanobis.... Do you have a preference?
No greed. No fear. Just maths.
  • #60
  • Edited 2:25pm Dec 9, 2014 1:39pm | Edited 2:25pm
Quoting kprsa
For example, I ( :nerd: ) can do fits to data and extrapolations based on those fits by various deterministic methods or by a Monte Carlo algorithm/simulated annealing. Would the ml algorithm, given the same input data and the same given model reach a different conclusion than me, and if yes, why? If it will not, isn't this just shifting of procedure/decision/responsibility to a black box? {quote} No, of course you do not require more than 50% accuracy. Thanks for specifying this! Thanks everyone for their inputs. I am on the verge of starting to...
Well, machine learning methods find relationships that might not be obvious enough to be found through a more transparent method (like those you suggest). What can be found also depends greatly on the model you use which allows you to control complexity. A neural network using a complex structure can device rules that are extremely complicated while the K-NN example I have given finds a simple relationship which is so simple that it can probably be extrapolated to a simple rule based approach (as stt suggested).

I think that both paths are ways of doing the same thing. However I prefer machine learning because:

  1. You can constantly adapt without having to worry about how to adapt
  2. You can evaluate data-mining bias in a straightforward manner
  3. Many machine learning methods are computationally cheap (although not all but you can do enough with those that are)

Certainly you might choose to predict future market values using something besides machine learning (I am not claiming it to be the best) but at least it the method I have found to be the best for me.

If you want to get into this I would advice you about the following:

  1. Do a basic machine learning course so that you understand basic principles (the Stanford course in coursera is quite good)
  2. Get a good handle of programming (do R and python courses)
  3. I would also advice you to get a framework you can use to easily program/test/live trade your machine learning strategies. I can recommend the programming framework available at Asirikuy for this but I am sure there are others as well.

2 traders viewing now