Gathering Empirical Data on Forex Signals

Hey everyone,

I'm starting this thread today so that we could collect actual data on some of the most common and widely used technical signals.

***Update*** // Check the table with the overall signal test stats

For example some of the most common technical signals are those that result from oscillators exiting overbought/oversold areas. If you load an oscillator on the chart, and scroll back through the historical data, you could see that most often than not, when RSI crosses the 70 level downwards, the price goes on to follow it downwards. But how often is that the case?

We all know that, when visually backtesting, we are often biased to disregard instances where our strategies fail. So what I want to do in this thread is to actually backtest these simple and common signals, collect their data, and then maybe we could get to see patterns from all that data, and get better insight on what works, how often does it work, and how we could exploit it in trading.

I've just started my first test on RSI. We want to see how often does price go down when RSI crosses level 50 downwards, and how much price goes up when RSI crosses level 50 upwards. I want to see how often this occurs, what's its accuracy in predicting price movement, and how big is the movement when it actually happens.

Now I have decided to create a system to test this. It's the first draft, but we could refine it further together. Here's how it works:

1. I will test 3 different RSI periods i.e 5, 14, and 21.
2. The tests will be done on the 7 major pairs, on 7 timeframes.
3. I created an EA that opens trades whenever the signal occurs.
4. 5 tests will be done for each RSI period on each one of the 7 timeframes of the 7 pairs:

1. Fixed SL and TP at an ATR distance from the open price (1:1) - gives an unmanaged result of how often price goes our way
2. SL placed at the previous swing hi/lo - TP at a 1:1 distance - same as above with a varied distance of SL and TP
3. Normal Trailing at ATR - Initial SL placed at the previous swing hi/lo - makes sure we detect small movements in our direction
4. Trailing behind previous candle hi/lo - Initial SL placed at the previous swing hi/lo - catches longer movements in price
5. Trailing behind previous candle hi/lo on nearest lesser timeframe - Initial SL placed at the previous swing hi/lo - somewhere between the two tests above

I will then focus on 3 results of the tests:

1. number of trades/number of bars - This gives us the frequency of the signal
2. winning ratio - This gives us the accuracy of the signal
3. average profit pips - This reflects the average movement in pips that the signal predicts, when it gets it right

So I've started running these tests on EURUSD and AUDUSD, and I will move on to the next pairs in the next few days, but here's a quick view of the results I've gotten so far:

You will see the different pairs and timeframe constituting the different rows.
On the left you will see how many bars and trades, and the calculated frequency:


In the middle you'll see the winning ratio of each test, which reflects the accuracy of the signal. It's colored from red to green to reflect low to high accuracy, and then the average accuracy on the right of this image which is an average of all 5 tests. The bottom is the average of each type of tests:


Then to the far right of the table you will see the amount of pips for each move, also with averages of both right and bottom:


While I haven't finished testing yet, we could already get some insight on this signal on RSI 5.
By looking at the first two columns of the winning ratio, you see that the un-managed signals are about 43 to 44 percent accurate, which is less than a coin toss.
But with some management in the next three tests, we were able to raise the accuracy to go above 65 percent. But we could clearly see that the higher the timeframe the more accurate the RSI(5) overbought/oversold exit signal.

Then when we look at the average movement in pips, we could understand 2 things so far:

1. This signal predicts bigger moevements in EURUSD than in AUDUSD.
2. Using this signal to scalp on M1 and M5 gets you no more than 5-6 pips, which could easily disappear with spreads, stop levels and commissions.

As I said, I will continue testing this signal for the next few days, then I will move on to test RSI exiting the overbought/oversold 70/30 areas, then RSI entering the overbought/oversold areas.

Let me know what you think of these tests, if perhaps you have some more data you would think is good to collect from the tests, or maybe if you think the testing system is flawed and should be tweaked.

Would love to hear your thoughts while I continue the tests

very, very interesting i would like to sub and follow your results/testing, good work

I have finished testing on RSI(5), and I have attached the detailed results in a pdf.

Just like in the first two tests on EURUSD and AUDUSD, the rest of the pairs showed little to no improvement in accuracy, leading to an overall 47% accuracy of the signal, with a 20.46% frequency, which means that it occurs about once every 4.89 bars.


However, if we look at the tests and sort them out by timeframe, it's very clear that the higher the timeframe the higher the accuracy, crossing the 50% milestone at H1.


This makes sense, especially since we see that the movements this RSI(5) signal predicts, are very small on lower timeframes. The 6.1 pips and 4.3 pips movements on M5 and M1 respectively will be easily wiped out by spreads and stop levels.


But we do see that the difference in accuracy between pairs is rather significant, reaching up to 5% of difference.


So in conclusion, it looks like if we were to use the signal of RSI(5) exiting the overbought area downwards, and the oversold area upwards, we should only consider trading it on H1 or higher timeframes (maybe not even H1), in order to get an edge, otherwise we might as well flip a coin in order to decide if we buy or sell.

Take a look at the detailed pdf report, you could see the results for each test, and see the exact movements in pips on each pair and timeframe.

Next test is going to be on RSI(14) with the same signal. Let's see what we'll get.

I just finished testing the same signal on RSI(14) and the results are interesting. The accuracy is better on all timeframes and all pairs by at least 2%.

Here are the overall Frequency and Accuracy rates:


Compared to RSI(5), Overbought/Oversold area exit on RSI(14) is barely half as frequent, but it's 4% more accurate overall. Which is a considerable difference considering that we went from below 50% to above it. It's still almost like a coin toss, but it's more accurate on higher timeframes, and it's now even above 50% on M30.


Also the difference in accuracy at the pair level has decreased. The largest gap in accuracy on RSI(14) is 3.34% between GBPUSD and USDCHF, while the biggest gap on RSI(5) was about 6.6%


And what I found interesting, is that even though it's more accurate on RSI(14), the movements the signal predicts are almost the same length as those that RSI(5) predicts. The difference in the number of pips is barely noticeable, apart from the 10 pips on D1.


I have attached the detailed report in pdf format below.

My next test is going to be RSI crossing level 50. In my experience this was a relatively accurate signal, but let's see if I will be proven wrong.

Subscribed. Really interesting analysis thus far. Well done mate!!

As I was about to start the new tests, I realized there was a mix up in my previous tests. The results above were actually for RSI crossing the 50 line, not the ones for it exiting the overbought/oversold areas. Sorry about that, my files got mixed up.

Everything stays the same, same analysis, but it's for the wrong signal. I will re-post the charts and files here with the corrected titles, then I will post the actual test results of the overbought/oversold area exit signal in a couple of hours.

Since the results are those of RSI crossing the 50 level, it makes very good sense that RSI(5) had such bad accuracy, and RSI(14) barely achieved a 50% accuracy. Usually this signal is good for longer periods of RSI.

I have just started the RSI overbought/oversold area exit signal tests, and so far they seem slightly more accurate, and definitely less frequent, but haven't done enough tests to confirm if it's a better signal overall.

Sorry again for the mix up, here are the charts with the corrected titles:

RSI(5) Crossing Level 50:





RSI(14) Crossing Level 50:





Detailed Reports:

I just finished testing the RSI(5) signal from exiting the overbought/oversell areas, and I have to say I didn't expect these results. The signal was much less accurate than I had anticipated.

The signal I tested is when RSI(5) crosses level 70 downwards, and level 30 upwards. And the overall accuracy is 45% which is less than the signal from crossing level 50 on both RSI periods.


Looking at the accuracy at the pair level, we see that it never reached 50% on any pair.


Same with the timeframes. With the previously tested signal, we saw that higher timeframes yielded good results, but with this one we barely hit 50% on D1.


I don't know if the reason is that I'm using a small RSI period, or if the signal all together is not accurate.
I will wait and see what are the results of RSI(14).

The movement in pips seems similar to the previous tests. So no surprises there:


My test is RSI(14) overbought/oversold area exit signal. If you have any suggestions about to test after that, share them with me so I could add them to the list.

I'm thinking maybe we could try Parabolic SAR, or maybe ADX where DI+ and DI- cross.

I have just finished testing the overbought/oversold area exit on RSI(14), and the results are not as uniform as the previous tests. By that I mean that if you look at the detailed report you won't see the same thing that we saw in previous tests where it was obvious that the higher the timeframe the more accurate the signal.
In this tests we have M15 giving better results than H1 on certain pairs. We have 8 points difference in accuracy between D1 and the rest of the timeframes on GBPUSD...

Anyway, take a look for yourself.

As a start, the overall frequency is 2.71% which is far less than any previous tests. That equals about once every 30-40 candles. As for the accuracy it is 46% which is also low, like that of the RSI(5) exiting overbought/oversold areas.


That kind of confirms that RSI crossing level 50 is more accurate as a signal than it exiting the overbought/oversold areas. I suspect when it enters those areas we could have a more accurate signal, but we'll leave that for the next test.

We're back to having a big difference in accuracy between the pairs, even though this signal didn't cross the 50% accuracy on any of the pairs.


We even barely reached 50% on D1, just like the previous test of the same signal on RSI(5).


As for the size of the movement, the results from this test resemble that of the RSI(14) crossing level 50, while the two previous tests on RSI(5) had smaller movements in pips, and they were similar to each other. That makes me think if the period of RSI is more related to the size of the movement than the actual type of signal is.


The detailed report is attached below.

My next tests are going to be on RSI entering overbought/oversold areas i.e how often does price go up when RSI crosses level 70 upwards, and how often does it go down when RSI crosses level 30 downwards.

Can you attach your excel files?So I can see how you calculated different frequency distributions?Thanks.

Sure, here they are. I basically divide the number of occurrences by the total numbers of bars in a test.

Finally! A signal that is at least a bit accurate, or at least one that qualifies as an actual signal.

The following tests were made on RSI(5), to check how often price goes up when RSI crosses level 70 upwards and thus enters the "overbought" area. And subsequently checking how often price goes down when RSI enters the "oversold" territory by crossing level 30 downwards.

If you have been following the previous tests, you'll see that this signal is much better, even though it isn't ultimately great.

Let's start by the overall stats. First of all we could see that the frequency is 11.91% (about once every 8 and a half bars) which is exactly the same frequency we got with RSI(5) entering overbought/oversold. That makes perfect sense because whenever it enters the area, it will exit it. So if you're planning on trading the overbought/oversold area, I couldn't recommend it more to drop the signal from RSI exiting these areas. Because while they do have the same frequency, with an accuracy of 53%, the signal from RSI entering overbought/oversold territory is 8% more accurate.


This edge in the overall accuracy is even reflected on the pair-level, where the signal is at least 52% accurate on all major pairs.


And on the timeframe-level, you could begin getting an edge starting from M15, achieving over 60% accuracy on H4 and D1.


The size of the movement seems also to be bigger on this signal than on our previous tests, which makes it even better to trade.


And here's the detailed report in PDF

I decided to organize the overall results in a table so that we'll always have a reference:
https://www.nelhajj.com/entry-signals-stats/

I will add it to the first post for future reference.
You could sort all the entries by indicator, period, signal, accuracy and frequency. You could also search for a signal and it filters out all other entries.

Do you think I should also make a Pair-based table? or would that be too much?

Because averaging all the tests' accuracies by pair is kind of misleading. Unless there's another way to make a pairs table.

If you have suggestions let me know and I could make the table.

Here are the tests for RSI(14) entering overbought/oversold territory. I can say that the results were just as anticipated.

We have seen that this type of signal was more accurate than the other signals tested on RSI. We also know by now that RSI(14) is more accurate than RSI(5) in general. And this test confirms both expectations: it is the most accurate signal yet.

At an overall accuracy of 57%, this signal on RSI(14) is 4% more accurate than our previously most accurate signal.


And just like the tests of the same signal on RSI(5), it's accurate across all 7 major pairs, with an accuracy of at least 56.52%


This time even the accuracy on M5 crossed the 50% milestone. Then it shoots up to 57% on M15, and almost reaches 70% on D1.


The movement size from this signal seems even 30-40% bigger than on RSI(5).


So finally I regained some faith in RSI. I will probably go on to test RSI(21) on all previous signal for my next tests. Then move on to divergences on RSI. That way we could maybe conclude our tests on this indicator, then move on to the next.

The detailed report is below.

I was still unsure if I should test both classical and hidden divergences together, but then I decided that it's better to test each one by itself to get a better idea of the stats of each one.

So here are the test results of the classical divergence on RSI(5), and again, they weren't good.

The overall accuracy of this signal is 47%, and it's far less frequent than any other signal on RSI, happening about once every 175 bars.
However, you will see that both accuracy and frequency are not consistent across timeframes.


If we look at the different timeframes, we see that the accuracy shoots up by 9% from H4 to D1.
And also in the detailed report you could see that the classical divergence was more frequent on higher timeframes than on lower ones, which I found to be a bit odd.


As for the accuracy according to the pairs, GBPUSD remains the pair with the highest accuracy of RSI signals. I don't know yet if other indicators will also be more accurate on GBPUSD because it usually has higher volatility, or if GBPUSD and RSI go well together. We'll have to find that out later. But for this test, there's a 5% different in accruacy between pairs.


As for the size of the movements, it doesn't seem to be as big as the previous signals on RSI.


Here's the detailed report, and next test will be on RSI(14) classical divergence.

Here's a screenshot of the table I have here, just so that we could still keep up with the results: