Rel's Lab

Update 09 Sept 2014 : Lab is officially closed.
Please refer to http://www.forexfactory.com/showthre...92#post7724492

3 simple rules / guidelines for participation of this thread :

1-No indicators will be released. No amount of asking/begging will get me to release anything. It is really up to me to decide what I would like to release to the public, not anyone else.

2-Be prepared for the unconventional. If you find that these methods are not for you, please do stay out of this thread. Do be warned that often I am forced to write in point form as there are too many things to expound upon.

3-Do consider the rational and context of the research methodology and results before commenting. If you have your own real research that are backed by real numbers, then you deserve a shot at commenting. After all, this is a laboratory, so do not assume all research found here works 'as is'. It is highly recommended that you do your own research as well.

Current Status (09 Sept 2014) :
1-Price Data :
M1

2-Chart :
Relative Bars

3-Wave Tool to map out key price points :
Modified Barros Swings

4-Tool must be able to help identify all market conditions :
Four types of 3 swing states; Trend, Squeeze, Expand Resume/Reverse
To consider 5 swing states. Improved classification method.

5-Gather statistics as per market condition :
Done, can improve further if more swing VS swing relationships are found, especially those beyond the 3 swing range. Improved classification method.

6-Use the statistics to create my own wave theory :
Done, can improve further, dependent on #4 and #5. Improved classification method.

Other important considerations to model and test :
-Reattempt at Sequence Checker for states; 100% completed.
-Multiple Price Frames using MMLC (major move life cycle) v2 methodology : Done.
-GMMA levels confluence with swings (canceled idea due to redirecting focus on next project phase)
-TCD relationships with swings (canceled idea due to foundational differences)

Scientific Rigor is key.


This is really not for everyone.
http://www.forexfactory.com/showthre...54#post7688354

-Reasons why I do not use conventional charts and indicators anymore.

The reasons are purely from a technical standpoint;

1-The real chart is the tick chart.
In reality, there is no such thing as 'Open' or 'Close'. The exceptions can be ;
-market session opens and closes
-holidays
-monday open and friday close (but again, debatable as the market is actually opened on the weekend via dark pools)
-daily high and low (debatable; as back in the day, daily charts are the only charts available and 'tradition stays'. Today we have timeframes lower than D1 easily available; did the market and trader perceptions change?)
Therefore the market can be seen as a continuous process, like a living entity. Orders that already exist in the market currently do have some weightage on future price action e.g. Losers and Winners exiting positions.

2-Conventional time based charts are just compressed price movements.
What if you wanted to trade the movements within say, a D1 chart? Then you would have to use a timeframe lower than D1 to trade the movements e.g. H1 or M15. From a pip/time efficiency perspective, meaning getting the most pips within the shortest amount of time; isn't that what we all seem to want? I've done countless of studies in the past to determine which is the most efficicent time frame to watch and the answers vary on how volalilte the instrument is. e.g. EURJPY is really fast, compared to USDJPY. But most pairs seem to take M15 and H1 very well, but not well enough (see next point).
How to calculate pip/time efficency;
-Select a chart; H1
-Select a number of bars for sample; 100
-Calculate High - Low for all 100 bars
From here either;
-Apply a Frequency Distribution and get the Median and Outier values
OR
-Just average the results
-Take the Outier OR Average and divide it by 60. (for 60 minutes is H1)
-Do the same for all the 'standard' timeframes; M1,M5,M15,M30... up to MN1
-Combine the results and as far as I can remember, more or less, one should see M15,M30,H1 peak

3-The problem with multiple timeframes
There is no end to it when one looks 'up or down'. I am not discouraging anyone from using MTF. It does work, but ...
-I find the common problem of identifying 'which timeframe is currently in control' disconcerting.
-We are also all 'taught' (as far as I know) that higher timeframes control the lower timeframes in a trend + lower timeframes control the higher timeframe in a reversal / trend continuation. Or at least 'do not fight the higher timeframes'. But the question of 'how far to look' and 'how high is consider higher' isn't exactly answered IMO.
-MTF says the market works in fractals. Are there actual statistical studies to prove this? I do think, from observation point of view, markets that are cyclic heavy do show up in fractals. But how about choppy markets prone to manual bank interventions / 'pump and dunp' e.g. EURCHF?
-Why are we trying to map and model the market using fractals, which are of geometric progression, but the order of conventional time based charts (M1,M5,M15,M30..) are not in geometric progression or even fibonacci sequenced (M1,M2,M3,M5,M8,M13,M21,M34...etc)?
-Not saying the works of MTFers do not work; e.g. Ray Barros, iDoubleStoch himself, MasterForex-V, or even later Elliot works have MTF concepts. I respect them, but is there a better and clearer way? (there are a few, which I would mention in time)

4-Most classical indicators skew what we already have
-Bar charts are already the best we got, in absence of tick data. They already show more or less what is happening, other than the problems I've mentioned above. Why should we skew this data further by taking the average of what these charts provided? Aren't we hiding from what is really happening in the 1st place by doing so? Or worse, add further interpretations to what these indicators say.
-One can say, bar charts are already indicators themselves; see point 1 and 2 again. Ray Barros explains this best.

-So what does all this really mean?

It means...
1-Our tools for effective trading are already broken.
We cannot work properly if our tools are not in order to equip and help us work. Yes we can apply money management and psychology, but IMO these should be separated from the understanding of how the market moves. MM can only go so far if one is 'already wrong'.

2-Note who are those who created these 'timeframes'
In reality, companies that provide charting software and services aren't really interested in you making money, but them making money using their software. Yes I am grateful for these software in some sense, and MT4 is free (in general) but its only as far as it goes. But I have to take care of myself. One can counter this by saying that I did not learn how to 'use these timeframes'; I've tried (will post more about this later). They do work but I know it can be much better. MT5 allows custom timeframes, but that’s a minor comfort.

3-Note who are those who created these classical indicators
I have to first say that I respect their effort and legacy, in an effort via evolution (if one can call it!) to help us trade better. They did their part, and it worked for them back in the day, but I say we must do better, not just to survive, but thrive. The market is forever changing. This means we have to create something that works today, but also takes into account of the future market. MasterForex-V explains this best : http://www.masterforex-v.com/mf_book.../chapter1.html

4-Not everyone has the skills to create tools
Like I've said again, it is harsh, but when you realised that no one is there is look out for you in the trading world... this means one must improvise to survive and thrive. Are you ready and responsible for this? Not everyone is! If you have a job and you realised that your employer has not equipped you sufficiently, of course you can always look for another job. But what if you have circumstances beyond your control and you must stay in the line of work? What would you do? Learn, adapt, survive, or die =X

-Why Chose Wave Analysis?

1-Price already moves in waves in the tick chart. This is the purest form, so lets let it stay that way as much as possible.
2-Objectively, we all want to enter at point A and exit at point B and do not really care about what happens in between (aka waiting). Wave Analysis filters out the noise between point A and point B.
3-Wave Analysis is based on the premise price movement in repeatable patterns from point A to point B

Therefore, why not? Sounds simple isn't it?

Unfortunately, most Wave Analysis is theory and are plagued with too many problems. Sound statistical work is a key foundation of making something theoretical into practical. I have not seen statistical work done for WA, with 4 notable exceptions;
-James L. Bickford / Michael Duarne Archer (Getting Started in Currency Trading; GWT)
-MasterForex-V (they have a department of statistical analysis; they seem to use standard ZZs)
-Ray Barros (Nature of Trends book; Barros Swings)
-Bulkowski (http://thepatternsite.com/studies.html; Price Patterns are part of WA)

If anyone has seen more, please do tell me! =)
Wave Analysis / Tools that I am aware of;
-Dow Theory (the mother of WA)
-Elliot Wave (the father of WA)
-Wolfe Waves (single fixed pattern; not flexible for all market conditions)
-Harmonic Patterns (fixed patterns; not flexible for all market conditions)
-Weis Waves (the problem is again, time based chart candles are already broken and use of volume is questionable from a MTF view; so who is correct?)

I like all their work, but again, I encountered the following problems;
1-Quality tick data is hard to get. I am aware of Birt's patch, but I feel its way too much work and do not have the CPU resources to create what I needed using ticks.
2-The standard chart is already broken, so the next best is M1. It turned out pretty good actually.
3-With chart data problem out of the way using M1, the next problem is to objectively map out the price points using a wave tool. No one has a standard way of mapping the waves other than Barros (read his book and you will understand why). I also find...
-the standard ZZs MasterForex-V uses have questionable and complicated formulas
-GWT too rigid as it is only good for trending markets
-Bulkowski's work seemingly too manual and are for stocks

And besides, all 4 of them based their work using their waves on standard charts. So this means I have to reinvent everything from scratch. Therefore, the development route would be :
1-Price Data : M1
2-Wave Tool to map out key price points : Barros Swings (but note that I did my own modifications)
3-Tool must be able to help identify all market conditions : ?
4-Gather statistics as per market condition : ?
5-Use the statistics to create my own wave theory : ?

But there would be a development issue if I take this route. Remember, for point 5 to be sound and be actually usable for trading, development from point 1 onwards to point 4 must be solid. I would explain why I ended up using range bars (somewhere in between 1 and 2).

Subscribed!

Excited to see how this thread unfolds. If you have any development related questions, feel free to ask.
Are you looking to use an external software or develop and app to do the number crunching or use MT4? If you tell me the languages you are comfortable with, I might be able to suggest something.

Sounds like you are looking to build a statistical tool to analyze waves.. Interesting..

Yes I would be posting more later, so you should definitely see something.

I use the following as my primary tools;
MT4
Excel

That's it =) I surprised myself to do so much with so little (you will see later why and how)

If you are talking about relevant external software and languages (not an impressive list IMO);
R (I gave it a good shot but decided that its not worth the effort as all I really need in the end is just frequency distribution studies, which Excel does very well)
SQL (for big data; I thought I needed it, later I figured how to scale down my projects to the point I don't really need it)
VB (good when used with Excel via VBA, yet again, I realised I didn't need it)
Powershell (yes I actually use it to backup my MT4 directories)

Yes, a statistical tool to analyse waves. Its already completed and on live. I am just journaling the process of how I got here plus looking into how I can further advance the current 'product'.

-An objective view of price movement in time based candles

Let’s look at a simplified breakdown of conventional MTF analysis. What do we see in these charts?

http://i61.tinypic.com/25s36v9.png

http://i58.tinypic.com/33adrox.png

1-There are 'so many charts to look at, but we only have two (2) eyes. Yes I am aware we can write indicators to get a single view of things. But that’s not the point...

2-Basic VSA would say the high of the day coincides with high volume candles seen on M15 and H1 charts, signaling a reversal. I say volume is just another time compressed indicator that faces the same problem like MTF. There is no end of how high or how low one should objectively go.

3-There are moments when timeframes take turns to make their moves; in a high volatility period of day, the movement is better seen on lower timeframes at M5 and M15, because the higher timeframes H1 and D1 compressed the view. This is vice versa for low volatility periods where higher timeframes hold their worth and lower timeframes are just 'noise' and distractions. Oh really?

4-There are many small moves shown by the smaller timeframes throughout the day. One should be able to just see that the 'pips distance covered' is definitely higher than the daily range. M15 and H1 generally show these small moves within D1. M5 is too small and uncomfortable to work on a visual level. But 'pips distance covered' is potentially more profitable than just the daily range. More pips, more money right?

5-Likewise, volume records every tick moved. Obviously there’s way more volume than the daily range. Why can't we trade these ticks? Look at the orange line in M5/M15/H1 that marks a swing low to high of the day aka major move of the day. The 1st half of the day is low volume. The 2nd half of the day is high volume; there are missed opportunities in the 2nd half of the day. Why not extract and exploit them?

-What can we do and why?

To solve the problem time-based candles give, the last thing one can do is to use time further via alternatives such as MTF or try to answer a very difficult question many traders had asked themselves; which timeframe should I use? I've tried to answer the question statistically using Pip Time Efficiency, but it falls apart in a market when the 'controlling timeframes' changes and switches to make their moves.

Classical answers to this would be to assume...
1-MTF works and one must learn MTF to correctly pick out the 'controlling timeframe' and trade it. What if one (like me) just can't get it? Or maybe I got it, but I don't like it and do not dare to vouch or bet my life on it because no one provided statistical proof of its effectiveness. Further more, MTF teaches that the lowest timeframe is the start of the reversal. This is true, but how can we objectively see this and pick this up, when there are so many timeframes to sort out from? How low is low, given the situation? (again, unanswered question)

2-Time duration are a reflection of our personality and trading style (patient and slow VS quick and responsive). I think this is partly true. But science is objectively detached in order to ensure the results can be repeatedly produced by another party without bias. This is what Ray Barros tried to solve in his book Nature of Trends, so that no 2 traders using the Barros Swings with MTF can disagree with the construction, use and results. I like his work very much as it is clear and concise, and more importantly, repeatable. I would however, chose not to use MTF because of all the ranting of complaints one can see so far (very sorry =d)



This means one has to eliminate time duration, but not time markers. Time markers are still useful and should be retained (you will see later why).

The following are are known methods eliminating time duration;
1-Price Range based; Range Bars / Point and Figure / Renko
I eventually decided on just my own version of Range Bars as I find the rules of Point & Figure and Renko a little inflexible; Standard Point & Figure and Renko ignore excess values. I want to retain price wise everything possible, so these are out.

This leaves me with Range Bars and here are the standard rules of construction;
A-Each range bar must have a high/low range that equals the specified range.
B-Each range bar must open outside the high/low range of the previous bar.
C-Each range bar must close at either its high or its low.
...which I want to keep things really simple, so I only use A. That’s all.

As for what range number to pick, I chose not to go into the details as it can confuse people. But basically it adjusts every 5 days on a rolling basis.

2-Price Level based; 2-Line Break / 3-Line Break
Good for reversals, but I don't just want map reversals; I want to map out everything.

3-Volume based Bars
From a practical point of view, tick data isn't available so I avoided this method. However I did try my luck to use M1 candles to construct these and it didn't turn out well.

4-Tick based; JChart
This is an FYI, but basically I can't see how this will help lead to WA. Besides to effectively create this, one needs tick data.



Therefore the development route now would be :
1-Price Data : M1
2-Chart : Modified Range Bars
3-Wave Tool to map out key price points : Modified Barros Swings
4-Tool must be able to help identify all market conditions : ?
5-Gather statistics as per market condition : ?
6-Use the statistics to create my own wave theory : ?

-Which chart would you really prefer now?

This?
http://i59.tinypic.com/2lm3xcg.png

Or this?
http://i62.tinypic.com/4qrur.png
http://i61.tinypic.com/14e9yxe.png




Are you able to see where the real opportunities are now?
Are you able to see that not every trading day is not the same much more clearly now?
Are you now able to imagine yourself trading almost every swing, extracting the maximum potential?
Are you now able to learn market movement much better than the average trader?

Do you realized now, no price movement can be hidden from your view; that its much easier to be confident on what you can see clearly, rather than guessing in the dark? Seeing is indeed believing.
And the wonderful thing is that you are only seeing 1 chart. Nothing more, eliminating every single hassle / problem I objectively mentioned in my earlier posts.

I would leave it to you guys to decide before I continue the rest of the parts

Your last chart looks like regular candles with zz legs is that correct? anyway I prefer the last. Please continue.

In case you didn't see the yellow words in the pictures, here you go;
1st ; time based candles M30 with day boxes and time based candles D1
2nd ; range bars with day boxes
3rd ; range bars with day boxes and ZZ legs

Yes the last is best.

Yes, I did see what you wrote but I said, "looks like".

-Market Conditions (part 1)

A recap of where we are now :
1-Price Data : M1
2-Chart : Modified Range Bars
3-Wave Tool to map out key price points : Modified Barros Swings
4-Tool must be able to help identify all market conditions : ?
5-Gather statistics as per market condition : ?
6-Use the statistics to create my own wave theory : ?

We are now at step 4.

There are many traditional ways of identifying market conditions. Most get the basic gist, but I have a problem with them... Its often not accurate enough, because these methods often use averages or are pretty manual.

Lets investigate a popular one; Bollinger Bands. Price breathes in and out; from flat to trend to flat to trend. Or to be more specific from Bollinger Bands POV; Trend -> Squeeze -> Expand -> Trend
http://i61.tinypic.com/10nyopk.png

Questions;
1-Does the market really behave in this manner of cycle all the time?
2-Is there any way to predict when price would transit from state to state?

Boilinger Bandwidth attempts to answer these questions;
-Historical low of bandwidth = squeezing -> impending breakout of trend
-Historical high of bandwidth = slow down of breakout trend -> squeezing
http://tradingsuccess.com/blog/measu...ction-926.html

This was my basis of identifying market conditions until I realized I needed more accuracy and it could be done. If someone can gather better statistics of market states using Bollinger Bands, lets compare notes if you may e.g. I've found that squeeze states in a H1 chart tends to be around 8 to 12 hours.

Now, would this be much clearer, having and using key price points mapped out?
http://i60.tinypic.com/rrlcaa.png

Congratulations my friend!!! . Glad you finally got the ball rolling. Lets hope we can keep things decent now eh? Behind you all the way!

Some thoughts before I nap

The more I think about it the more I feel like correctly developing states is just as hard as correctly developing a good tool to map price. For example your wave tool is created in such a way that each bar or wave, in order to register, has to be "significant"; a term which you ultimate had to define yourself. I think a "correct" wave tool does the following:
-correctly maps price (not too much distortion of missing highs/lows)
-follows mechanical rules (RB/modded BS swings)
-offers the user a way to actually trade from them (ratios)
-offers a pattern (freq. dist./awareness of: number of continuation moves, trouble areas, make or break areas, areas of no return, turning points, etc).


At a glance states run into the same issue of "tailored to fit". I think it's impossible to say that one persons classification of states is better than another's, although their states should match up in some scenarios (strong trends).

Question: What qualities are in a "correct" state? Likely something that answers "yes" to your two questions haha

I'm thinking this is something that one learns from actually creating states and working through them (aka the need from going from 3 states to 6? lol) but I feel like there are a couple of different sets one could use that would in theory work:
1. trend/expand/squeeze (with or without it's extensions)
2. moving/flat (exhale/inhale)
3. strong bullish/bullish->flat/ flat/bearish->flat/strong bear
etc
and of course even if we were to pick 1 category to say was "correct", then the nitty gritty definitions from person to person would differ too.


Lastly perhaps BB isn't the way but I feel like you have to use averages in some fashion. After all, since price is in a state, it must be in a state compared to something else right? How else would you be able to tell the difference between an expand or flat, if it weren't for knowledge of previous sizes of said states?

This thread is very interesting. But I'd like to drop this little post to warn you about the fact that the Bollinger Bands is a defective tool. It leads you to wrong interpretations.

The first issue with the BB is that it uses as many samples to estimate the variance as it uses to compute its moving average. For instance a BB with a SMA(30) will use the last 30 samples to estimate the variance (the square of the width of the bands). This is absolutely wrong! The value you get this way for the variance comes with an error of up to 300% (yes really!). At least 200 samples are required to keep the error below 10% assuming a Normal distribution and the market is not at all normally distributed.

The second issue is that the variance is estimated by taking the average of the squared difference of the current value of the SMA and the current price. This is also incorrect because the SMA lags by the half of its period. One shall measure the difference between the current value of the SMA and the price half the period ago.

The screenshot below shows in red the original BB and in blue the corrected algo. The dotted lines are their respective 2 standard deviations bands. Both use the same SMA30 so the orange line can't be seen (it is hidden by the blue one). Clearly you can see the difference. In the uptrend the BB widens its bands to capture the PA. It keeps giving you a reading of ~2 standard deviations. The interpretation being "an overbought condition with a retracement expected in a near future". At this same time the blue version reads 6 standard deviations. Under mean reversion assumption this event is supposed to appears with a probability of 1 out of 500 millions. The application of Bayes theorem rejects the means reversion assumption with 99.99999999% confidence. In clear text the blue indicator tells "there is 99.9999% chance that we are in a uptrend".

Same in the down trend. In the end of November around 1.3500. The blue version warns quickly of the high probability of a down trend (PA outside) while the
BB stubornly considers a range (PA inside).

Nice post PipMeUp! The corrected algo looks a lot like TMA with the bands? Are they similar?

Thanks. You can certainly change the SMA with another MA of your choice. Here it is just a good old Simple Moving Average with period 30. The bands are build by taking the lag into account and by using 200 samples instead of only 30 to compute the variance.

I had to ask google to know what TMA stands for. I found on this page http://daytrading.about.com/od/indic...Triangular.htm it was a Triangular Moving Average which formula is (SMA1+SMA2+SMA3+...+SMAn)/n. As I was curious I expended the terms of a TMA 7. The coefficients are (60*p7+130*p6+214*p5+319*p4+459*p3+669*p2+1089*p1)/2940. If you plot these coefficents you just recognize an exponential. The TMA is just a over-complicated way to compute a truncated Exponential Moving Average!

so...EMAs and/or SMAs will do just fine then...

pardon my ignorance, but how EXACTLY do you get the bands on the charts using the 200 samples as you say. Is that same as period?
How do you 'take lag into account'?