Price Analysis with Neural Networks | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Any success with neural networks in trading 29 replies

Neural Networks: Myths And Reality 19 replies

Extreme scalping using neural networks - EA/Manual 646 replies

Cognitive indicators and neural-networks 19 replies

Neural Networks in Trading (Java, Algodeal, Technical Analysis) 3 replies

Attachments: Price Analysis with Neural Networks
Exit Attachments

Price Analysis with Neural Networks

  • #1
  • First Post: Edited Feb 15, 2008 9:09am Feb 11, 2008 11:38pm | Edited Feb 15, 2008 9:09am
About This Journal

This will be my personal journal detailing my development of an artificial neural network to attempt to gain a statistical edge on the market on multiple time frames. I will post my progress as I reach milestones, and we will see together whether it is in fact possible to gain a statistically significant edge with a machine learning method.

I will also discuss how I approach money management with respect to whatever statistical edge I might gain with my learner. I will post probabilistic simulations of my learner's performance using various money management techniques.

Please note that I currently trade fairly successfully (on a small, ~4K live account) using simple chart patterns and S/R analysis. I am also a full-time engineering student, however, so it would be highly preferable to have an automated agent either trade in my absence or alert me of high-probability setups. If my development of this indicator/EA is successful in gaining an edge, perhaps I will release it to the community in some form.

I would appreciate your thoughtful input to the thread. Development functions best when there are watching, constructively critical partners - or so the open-source software movement has taught us. So if you have something valuable to add, please by all means post it.
  • #2
  • Feb 11, 2008 11:45pm Feb 11, 2008 11:45pm
What are the inputs?

Purely price? COT? Interest rates? CPI?

Second, what is the criteria for entering/exiting? A "majority wins" based on multiple NNs in multiple timeframes?


I am pretty educated on NNs, and if you introduce noncorrelated (data with no strong linear dependency to price) inputs you will introduce error to your learning. I bet you already knew that.

So the question is, what inputs are strongly correlated? I would bet that forex is extremely efficient, so "new" external data such as COT and CPI are most likely fully realized by the price within a few minutes of the data being introduced, thus VERY small lag and inefficiency.


Well, NNs are not dinner table talk. Either you get what I'm saying or you don't. Hope it helps.
  • #3
  • Feb 12, 2008 1:11am Feb 12, 2008 1:11am
Initial Design Concepts

First of all, an explanation of the theory of neural networks. The neural network structure I'll be using is similar to the one shown in the picture below.

http://www.gamedev.net/reference/pro...nn/figure1.png

Each circle you see in the picture is called a "node". Each of these nodes acts as a sort of mathematical operator. Information moves from the left to the right in the image, from the inputs to the outputs. At each node, the inputs are summed to create the output. Each input has an adjustable "weight", or scaling factor, by which it is multiplied before they are all summed at that node.

Ok... so we've got nodes that add scaled values. How on earth are we going to predict future price action with this thing?? Well, it turns out that if you adjust these weights in an intelligent way, you can essentially "train" this network to recognize patterns on the input layer and produce the outputs you desire according to those patterns.

So here's the design for the network:

Inputs

Time-delayed pip movements. Essentially these are values whose magnitudes correspond to the size of each bar on a chart of a certain timeframe. For example, let's say you have the following chart:

http://www.tradingcurrencyonline.com/images/jap_4.jpg

If your neural network were designed to analyze four-bar patterns, it would have 12 total inputs. That is, for each bar, it would have an input for the lower wick, the bar body, and the upper wick. Both wick values would be either positive or zero, and the bar body value would be either positive or negative. So for the candle chart above, you'd see the following inputs (approximately) in the format upper|body|lower:

0.02|-0.26|0.18|0.06|-0.12|0.00|0.20|-0.03|0.23|0.00|0.30|0.10

As you may have noticed, all of these values are below 1.0; that's because the inputs to the neural network must be a decimal between 1 and -1.

Outputs

So now we have inputs. What do we want for the outputs? Let's choose upside and downside risk indicators for different timeframes. Say... two on each side. In short:
One output for the future high after 1 hour.
One output for the future high after 4 hours.
One output for the future low after 1 hour.
One output for the future low after 4 hours.

To train the network, we look at the highs and lows after the input pattern, calculate the outputs, and then use what's called a "back-propogation algorithm" to intelligently adjust the network weights so that the desired outputs appear when we present that input pattern. We then do this over a large number of input/output combinations, and we will hopefully arrive with a network that, when presented with a new set of inputs, can predict the future highs and lows with some degree of accuracy.

Once we have a network that can predict future highs and lows, we can place trades with TP and SL levels that maximize our likelihood of success. We can also choose to only take trades that have a good risk-reward ratio. For example, we may choose to only buy when the network predicts a 2:1 high to low ratio, and only sell for the inverse. This can help limit market exposure to risk, while maximizing the statistical chance of success.

If you want more information about neural networks, check out the Wikipedia entry on Artifical Neural Networks.

Expect another post soon about the (custom) software and data I'll be using for network setup and training.
  • #4
  • Feb 12, 2008 1:47am Feb 12, 2008 1:47am
Quoting tdion
Well, NNs are not dinner table talk. Either you get what I'm saying or you don't. Hope it helps.
They are at my house . Or at least when I worked back at the Neuroengineering department at UPenn.

The hope is that the the network will be able to learn complex patterns after a large number of examples. I'll be separating the data into sections and doing round-robin training/testing/validation with numerous reincarnations of the neural networks in question.

I will have networks for significant timeframes. Perhaps one for the 15min, one for the 1hr, one for the 4hr, and one for the daily. Once each is appropriately trained, I will have to experiment with different trading decision-making schemas. Most obvious would be a majority-voting or perhaps unanimous decision-making process. These networks could also simply serve as an alert for a high-probability setup, and then discretionary technical and psychological analysis could inform the final trading decision.
  • #5
  • Feb 12, 2008 2:26am Feb 12, 2008 2:26am
Software

I will be writing my own custom C/C++ programs that manage the input data and build training/testing data sets. My programs will also manage the training and testing of the neural networks created.

The neural networks themselves will be implemented using a high-performance software package called Fast Artificial Neural Network (FANN). It has bindings for many languages, but I will be bundling the library directly with a C/C++ program for maximum efficiency.

Eventually I will probably inspect the results visually by programming an MT4 indicator that displays the outputs of the network on the chart.
  • #6
  • Feb 14, 2008 7:47am Feb 14, 2008 7:47am
everything sounds good, it looks like that many people got interested in neural networks applied in forex after great success of EA based on neural networks in EA competition.
  • #7
  • Feb 15, 2008 2:06am Feb 15, 2008 2:06am
Quoting Ahil
everything sounds good, it looks like that many people got interested in neural networks applied in forex after great success of EA based on neural networks in EA competition.
I suppose. My interest was certainly renewed after seeing this in the MetaTrader Automated Trading Championship, but I originally began researching NNs and other machine learning techniques in relation to market analysis in late 2006. In addition, this neural network will be reading price action directly rather than computed indicators, as the recent winner's EA did.

I have some new revisions to my plan that I will post when I am more awake and don't have to finish a VHDL design before 11am tomorrow .
  • #8
  • Feb 15, 2008 9:32am Feb 15, 2008 9:32am
Instead of converting the numerical value of the outputs to pip values, it would likely be preferable to instead output a sort of probability histogram. That is, for each time frame under consideration, have a set of outputs as follows (for example):

Likelihood of future high +100 pips
Likelihood of future high +50 pips
Likelihood of future high +30 pips
Likelihood of future high +20 pips
Likelihood of future high +10 pips
Likelihood of future low -10 pips
Likelihood of future low -20 pips
Likelihood of future low -30 pips
Likelihood of future low -50 pips
Likelihood of future low -100 pips

Thus, each output would predict the likelihood, given the current pattern on the inputs, that the future high or low would be a certain distance away from the current price.

These outputs would likely be preferable for reasons that become clear once you've used NNs for a while. All machine learning tools end up being a sort of statistical analysis, but NNs are especially so, as the function under minimization is the mean-squared-error over the data set of the outputs versus the targets. Translated: the outputs generally tend to the mean of the targets. So if you have 100 examples of a pattern and its future highs vary around some mean, the network will learn to simply output that mean. Unfortunately, the mean doesn't tell us anything about the shape of the distribution. What if 50 out of those 100 cases move only ten pips up, and the other 50 move 100 pips up? The network will tell us that the expected move is mean(10,100) = 55 pips, when this move never even happens!

Using a histogram-based set of outputs, however, we would see that the output strengths of the nodes that correspond to 10 and 100 are both 0.5, and the output strengths of the other nodes are 0. Thus, when building algorithms that actually use this output to trade, we can analyze not just the mean of the expected price change but the distribution.

Note that by dividing the future range into enough price bins and timeframes, we would end up with a set of outputs that resembles an image like the following:
http://www.understandmarket.com/skin...ges/pmpic1.gif

To train, test, and especially trade automatically based off of that image would be extremely time-consuming, however, so we will stick to a smaller set of bins and timeframes for now. All parameters of the network and training/testing system will be configurable, however, so if the network proves to be valuable and trains/tests quickly as is, perhaps we can try to increase the resolution by adding more bins and timeframes.
  • #9
  • Feb 15, 2008 7:04pm Feb 15, 2008 7:04pm
Hi Drummer!


I anticipate that this thread is going to be a lot of fun and a great learning experience! I already have a couple of questions, one on inputs and one on outputs.

1. Will you be varying the number of bars used to generate the input parameters easily? If so, this produces another degree of freedom but it may provide an insight into how the "length" of a predictive pattern affects performance. Are 4-bar NNs more or less successful than 10-bar NNs, that sort of thing.

2. I understand why you switched to a distribution, as I usually look at quartile breakdowns in my own research. However, it's still hard to "trade from a distribution." What about using actual trading decisions as the output? There could be three variables; an action variable for buy, hold or sell with values (-1,0,1), an SL value in pips and a TP value in pips (since your inputs are measured in pips I believe).

When you back-propogate, you would be comparing the actual outputs with the ideal outputs. For example, if over the course of the test period, price reached a low of 37 pips below the close of the last output bar and 19 pips above that close, then the "perfect trade" would have been a sale with a stop of 19 pips and a TP of 37 pips, i.e. an output of (-1,19,37).

I've never worked with NN software and I only know the basics of the theory and structure of nets, so this may actually not make sense. What do you think?
  • #10
  • Feb 15, 2008 8:21pm Feb 15, 2008 8:21pm
Quoting capitalist88
1. Will you be varying the number of bars used to generate the input parameters easily? If so, this produces another degree of freedom but it may provide an insight into how the "length" of a predictive pattern affects performance. Are 4-bar NNs more or less successful than 10-bar NNs, that sort of thing.
Yes

Quoting capitalist88
2. I understand why you switched to a distribution, as I usually look at quartile breakdowns in my own research. However, it's still hard to "trade from a distribution." What about using actual trading decisions as the output? There could be three variables; an action variable for buy, hold or sell with values (-1,0,1), an SL value in pips and a TP value in pips (since your inputs are measured in pips I believe).

When you back-propogate, you would be comparing the actual outputs with the ideal outputs. For example, if over the course of the test period, price reached a low of 37 pips below the close of the last output bar and 19 pips above that close, then the "perfect trade" would have been a sale with a stop of 19 pips and a TP of 37 pips, i.e. an output of (-1,19,37).

I've never worked with NN software and I only know the basics of the theory and structure of nets, so this may actually not make sense. What do you think?
The "perfect trade" data you mentioned is actually what I spoke about in my second post in this journal. What you're talking about is targeting the future high and low for each input pattern. The buy/sell signal is actually redundant, given that you can calculate it by the expression (high>low)*2-1. After training over a large set of these patterns, however, the neural net will learn to output the mean of these values over the set of similar patterns.

Instead, I prefer to have the network learn the distribution. Knowing the past statistical distribution allows you to create strategies based upon the calculated likelihood of hitting different price points. For instance, we are used to the idea of taking trades with a 2:1 or 3:1 TP to SL ratio, but we can't be sure that they will actually increase our winnings, as without a predictive edge, a closer stop is usually more likely to be hit than the farther take profit. If we knew the statistical distribution of price points over the next hour (or day, or week... whatever), we could set stops that, historically, are actually twice less likely to be hit than our take profit level.

Ideally, we would be able to choose trades that are very high-probability and high-reward, where the stop is unlikely to be hit, the take profit is very likely to be hit, and the take profit is significantly farther than the stop. Warren Buffet always attributes his success to making low-risk, high-reward investments. Our goal should be the same - that is, to maximize the probability of winning while minimizing the probability of losing - or in trader's terms: to have an edge on the market.
  • #11
  • Feb 15, 2008 9:07pm Feb 15, 2008 9:07pm
Hi Drummer,

I don't have any real experience with NN but it seems a lot like distributed (parallel) programming, so I think I have an idea what you are looking to with your program and your explanation was of your NN will be is very good. I could be jumping the gun with this question but I have to ask.

Are you going to look at news reports for additional info once you get the raw inputs trained? I think it might be a valid data set if the correct filtering and sources could be found and used. What are your thoughts?

I am currently working on just such programming for news as a indicator for trade signals. It's functional but needs adjusting and teasing before I will be using it for trades.
  • #12
  • Feb 15, 2008 9:39pm Feb 15, 2008 9:39pm
Quoting drummer486
Yes



The "perfect trade" data you mentioned is actually what I spoke about in my second post in this journal. What you're talking about is targeting the future high and low for each input pattern. The buy/sell signal is actually redundant, given that you can calculate it by the expression (high>low)*2-1. After training over a large set of these patterns, however, the neural net will learn to output the mean of these values over the set of similar patterns.
You're right, it's the same thing you started with. That expression threw me for a minute. (h>l) is Boolean, 0 or 1, got it. Anyway, as you said the NN will just spit out the means so you've got to use a distribution.

Quote
Instead, I prefer to have the network learn the distribution. Knowing the past statistical distribution allows you to create strategies based upon the calculated likelihood of hitting different price points. For instance, we are used to the idea of taking trades with a 2:1 or 3:1 TP to SL ratio, but we can't be sure that they will actually increase our winnings, as without a predictive edge, a closer stop is usually more likely to be hit than the farther take profit. If we knew the statistical distribution of price points over the next hour (or day, or week... whatever), we could set stops that, historically, are actually twice less likely to be hit than our take profit level.

Ideally, we would be able to choose trades that are very high-probability and high-reward, where the stop is unlikely to be hit, the take profit is very likely to be hit, and the take profit is significantly farther than the stop. Warren Buffet always attributes his success to making low-risk, high-reward investments. Our goal should be the same - that is, to maximize the probability of winning while minimizing the probability of losing - or in trader's terms: to have an edge on the market.

You're talking my language here. I have a closed thread called "Expectancy is always zero" where this is one of the topics. A TP/SL of 3:1 means that win rate would be about 25% in a strategy without an edge. In general, the required win rate (RWR) for any strategy is (Avg. Loss)/(Avg. Gain + Avg. Loss).

The thing is, I've tried to wrap my head around trading from a distribution in the past and had trouble envisioning how to do it, especially in the case of a separate distribution for highs and lows. Actually, I guess what you'd really have is a distribution over all pairings of highs and lows, meaning over an area with high on the X axis and low on the Y for example. At any given point theoretically you'd have a value although I know you're actually going to bin them.

Ultimately, in order to make a trading tool, there has to be some process for getting from the inputs at one end to the "3 big decisions" (buy/sell, stop, TP) at the other. This may turn out to be a much thornier problem than the construction and training of the net itself.

I'm bringing this up now because it's key to know at this stage what you want the output to look like. It's a pain to get something all built and then realize at the end that you want it to do something different. Hopefully you'll avoid that problem, and maybe you've already got it covered.

I'm out of here for the night, but looking forward to your updates. This thread is going to rule!
  • #13
  • Feb 15, 2008 10:15pm Feb 15, 2008 10:15pm
Drummer,

Awesome thread, man!

You've already upped my understanding on Neural Networks and programming them.

I'm supposed to be doing other things!

...Damn you!!! ...kidding. LOL

I'm going to enjoy this!
Bundy's status today: "Waiting..."
  • #14
  • Feb 17, 2008 12:51pm Feb 17, 2008 12:51pm
Quoting capitalist88
The thing is, I've tried to wrap my head around trading from a distribution in the past and had trouble envisioning how to do it, especially in the case of a separate distribution for highs and lows. Actually, I guess what you'd really have is a distribution over all pairings of highs and lows, meaning over an area with high on the X axis and low on the Y for example. At any given point theoretically you'd have a value although I know you're actually going to bin them.

Ultimately, in order to make a trading tool, there has to be some process for getting from the inputs at one end to the "3 big decisions" (buy/sell, stop, TP) at the other. This may turn out to be a much thornier problem than the construction and training of the net itself.

I'm bringing this up now because it's key to know at this stage what you want the output to look like. It's a pain to get something all built and then realize at the end that you want it to do something different. Hopefully you'll avoid that problem, and maybe you've already got it covered.
I actually have given a little thought to this problem. I know from one of your PMs that you are interested in genetic algorithms. This might be a case where GAs could be quite useful in determining good ways to make use of the outputs of the neural network. On the other hand, there are a number of other machine learning or classification techniques that could be applied to the problem of trading decision-making using the distribution data.

We may also be able to create a trading strategy through simple theory, however. As long as we understand what the distribution means, we can make intelligent use of it. Essentially, the distribution will tell us how likely it is that a certain price point will be reached in a certain timeframe. We know that, to reach a high of 50 pips above the current price, price also (typically) has to pass through the 10 pip, 20 pip, 30 pip, and 40 pip levels. So, knowing the approximate distribution, we can sum the probabilities of more extreme levels to calculate the expected probability of hitting any level during a future timeframe. We can then create trade criteria that select only the highest-probability trades.

We could also break up the take profit and stop loss orders into a number of stratified orders across the distribution according to the probabilities of different levels. This is also a setup that could be calculated in order to facilitate the highest reward with the least risk.

Anyway, this is all conjecture that will have to be reconsidered once the final network is done. I think I will begin developing the network with the structure described, and if we realize later that the probability distribution is less useful than another target, we can change them out fairly easily. The real difficulty of programming is simply setting up and debugging the framework for training, testing, and viewing the output.

It'll be quite a while until I have something useful developed, but I will keep this thread updated as I make progress. Have a good weekend, or whatever's left of it
  • #15
  • Feb 17, 2008 2:20pm Feb 17, 2008 2:20pm
Very good approach! I am doing something similar.

What do you think about incorporating very large scale trend into the equation by normalizing the bars.

What I mean is say 10000 bars ago price was 3000 pips lower. That means that on average we are moving up 3 pips per 10 bars.

Could this be useful if you normalize the OHLC bars to reflect that in your distribution equation?
  • #16
  • Feb 17, 2008 3:29pm Feb 17, 2008 3:29pm
Quoting charlinks
Very good approach! I am doing something similar.

What do you think about incorporating very large scale trend into the equation by normalizing the bars.

What I mean is say 10000 bars ago price was 3000 pips lower. That means that on average we are moving up 3 pips per 10 bars.

Could this be useful if you normalize the OHLC bars to reflect that in your distribution equation?
Interesting idea. Without normalization, the distribution would be skewed towards whatever the long-term trend is. This would be fine if we were still in the same long-term trend, but not fine if the trend had changed recently. I suppose you could do some kind of normlization, but it's also not clear exactly how you should normalize appropriately. It might be better to simply include a set of moving averages over different periods on the inputs. This would help the network to separate the cause of the skew, and be able to replicate the correct amount of distribution skew for the current market.

Thanks, charlinks... great point!
  • #17
  • Feb 17, 2008 9:19pm Feb 17, 2008 9:19pm
Quoting drummer486
I know from one of your PMs that you are interested in genetic algorithms.
Genetic programs (GP) actually, which was a concept developed after GA by a researcher named Koza I think. I'm just nit-picking on a Sunday night. :
  • #18
  • Feb 18, 2008 9:59am Feb 18, 2008 9:59am
Here is an indicator I made that may turn out to be useful.

It is a simple price distribution. Fiddling with the parameters can create quite different results.

cell_width
how many bars to use for the price distribution per column

cell_rows
how many rows to split the price distribution into

cell_cols
how many columns to generate... for some heavy computations you can reduce this number if performance suffers as you most likely need the recent price distribution

cell_step
the higher the number the more blocks are skipped for each step it is still calculating cell_width number of bars. so if cell_step and cell_width are the same you will see a clear picture. if cell_width is significantly higher you will see data "blurred" in.

cut_off
all cells are normalized with a value from 0 to 1 if you draw all of them it will be very resource intensive. so everything smaller than cut_off is just not drawn. (I put 0.8 sometimes to identify strong support/resistance areas)

count_inside_bars
count_support
count_resistance
and here is some magic... instead of counting the whole bar I have split it into
inside bar = ( Open - Close )
support = ( Low - Min(Open, Close )
resistance = ( Max(Open, Close) - High )

That way you can focus your study only on support or only on resistance areas.

Happy Neural Networking!!!
Attached Image (click to enlarge)
Click to Enlarge Name: BHPD.png Size: 49 KB
Attached File(s)
File Type: ex4 BH_PriceDistribution.ex4   5 KB | 1,319 downloads
  • #19
  • Feb 18, 2008 11:09am Feb 18, 2008 11:09am
Quoting drummer486
Interesting idea. Without normalization, the distribution would be skewed towards whatever the long-term trend is. This would be fine if we were still in the same long-term trend, but not fine if the trend had changed recently. I suppose you could do some kind of normlization, but it's also not clear exactly how you should normalize appropriately. It might be better to simply include a set of moving averages over different periods on the inputs. This would help the network to separate the cause of the skew, and be able to replicate the correct amount of distribution skew for the current market.

Thanks, charlinks... great point!
You know... it just hit me...

All you have to do is adjust the price for interest swap rates!!!
In fact some brokers do exactly that... they adjust your entering price to reflect the swap so if you held GJ for 100 days you'd have like a purchase price of 200 pips below your original trade in there...
  • #20
  • Feb 18, 2008 11:21am Feb 18, 2008 11:21am
Quoting capitalist88
Genetic programs (GP) actually, which was a concept developed after GA by a researcher named Koza I think. I'm just nit-picking on a Sunday night. :
Yeah, I guess GP is a practical application of GA theory. The machine learning book I'm most familiar with (Machine Learning by Tom Mitchell) treats every type of learning algorithm as a search method. So GP is the application of the GA search method to programming tasks. Taxonomy aside, applying a genetic algorithm search method to the problem of optimal trade selection could definitely produce some good solutions that we would never think of ourselves.

Quoting charlinks
Here is an indicator I made that may turn out to be useful.

It is a simple price distribution. Fiddling with the parameters can create quite different results.
...

That way you can focus your study only on support or only on resistance areas.
Very cool. I made something like this once that drew a number of s/r lines using different colors according to the past high/low distributions, but this is way more configurable. Well done!
0 traders viewing now