Z-Score Kit

Hi apipshort, thanks for the last post. I checked the number of bars in the chart and it had 35000, ( thanks for telling me about the hover ), the MA indicator rejected the 7200 in the parameter so Im guessing theres an upper limit for calculating MA bars, ( it reverted back to 14 each time I entered 7200) I realise that my last three posts are probably not Z kit related and I dont want to clutter this thread so am happy if you delete any of my posts. thanks for the advice on version control. I am getting addicted to watching the nodding donkey.

I've always had a interest in Z score for some reason, but wandered off into the world of Harmonic Elliott Wave (which I do like). However, I recently stumbled across this:



I like his approach to Z score. Just FYI

Hello stryker and aminpip, both a very late thanks for your last.

I’m sorry the video went on the blink, but hope it was useful to you.
And, aminpip, it looks like you are getting that Bell Curve. It doesn’t move, but is nice to look at.

apipshort

A Bell Curve Indicator

One can find the very detailed contributions Mr. Brian Twomey has made in earlier posts here.

After the lay-off in this thread, Mr. Twomey has continued to give me even more help and pointers, for which I owe him the greatest of thanks.
Nothing much would have been possible otherwise.

In return he has been inundated with all kinds of experiments and gobs of numbers, making I’m sure, not a thread of sense.
This has been due to my extremely slow uptake; it took months to realize Mr. Twomey works with series of MAs, drawn from the Daily TFs mostly.
I remained totally focused on the MT4 platform, with things moving quickly every which way in 9 TFs according to PA.

Mr. Twomey is a long time poster at http://www.forexstreet.net/profile/BrianTwomey718
He made an impressive run of winning trades there, as an experiment, all announced and set-up.
What was it? 15 or 16 at least, various currencies and durations. That gained him an instant following and he got run ragged for a while …

His latest book: “Using the Z Score to Trade Foreign Exchange and Other Financial Instruments”, can be found on Kindle and Amazon book store.

I did try to “get” his system on MT4, but that has so far eluded me.
What came along are rather building blocks, some fairly solid and geared towards his ways of thinking.

One was to get a Bell Curve up on MT4. This version is for a Standard Normal Distribution.
That is a mainstay of statistics, and will serve as a Standard Deviation grid to show objects set out according to their Z-Score:



The immediate on-line references are:
http://en.wikipedia.org/wiki/File:No...and_scales.gif
http://upload.wikimedia.org/wikipedi...stribution.svg
That last image can be greatly magnified in a browser.

In the Demo some remarks can be found at the top of the code and links and material are to follow.
Please use a .tpl to get a clean look and become acquainted.

Continuation from prior post #24: finding your way around.
Please refer to the Bell image in that post or better yet, put the mg4 up.

The horizontal Baseline of the bell is graduated in units of Standard Deviation (SD).
For purpose of calculation the y values for plotting the bell are generated by using 0.01 step increments.
This gives 100 sub-units per 1 SD, and is useful for displaying Z-scores at the proper location. Like centimeters to meters.
Put the cursor on the Mean and measure a hundred bars to the right; z = +1.0 SD. 125 bars; z = +1.25 etc.

y values on the vertical scale are displayed for z = -PI to +PI. (-3.14 to +3.14)
The total of bars used to partition over is more than 600, one for each y value; this is why one needs to zoom out almost all the way.
All y values also appear in the terminal Data Window.

The mathematical average or Mean of observations is symbolized by the vertical line in the middle.
At this location the z-score = 0.0 and y = 0.4

Above the bell, a probability ruler:
The bottom row holds p values, between 0 and 1.
On top the same p values are expressed as percentages, after multiplication by 100.

Let’s hear it from the man himself, Count Laplace, 1814:

On general principles of probability

“The first of these principles is the definition itself of probability which, as we have seen,
is the ratio between the number of favourable outcomes and that of all possible outcomes.”

The favourable outcomes being the ones considered of interest.

If something always happens exactly half the time, then one would say fifty-fifty or a toss-up.
A quant will think “fifty out of a hundred” and, even in his sleep, will call it p = 0.5
50% “favourable” and 50% not. All is well with the world.

The numbers shown in between on the p-ruler are z-scores.
Both these z-scores and p values are deemed critical p and z values.
The hash marks on the bell and on the ruler all line up with these numbers.

If you lay the cursor over the hash at 95%, a z-score of +1.65 is found and can be placed on the baseline.
So, if you are a winning retail trader, your place is to the right, with the remaining 5% and your z-score is over +1.65. That’s plenty critical!
If not, one needs to work on one’s z-score. Trading is really that simple ;-)

At the very top are some markers for critical Q values. Those are part of a Box Plot that will be, well, plotted in that location.

Honestly, if a student in stats 101 turned in a paper like this, he would get a big fail and an even bigger question mark behind his name.
So, it’s back to class for you lot. I don’t care, retired and all.

All kinds of pics, but nothing beats MT4:
https://www.google.com/search?q=%22z...w=1286&bih=888

How to:
http://www.wikihow.com/Category<img ...and-Statistics
http://www.wikihow.com/Calculate-Var...g_Variance_sub

Wikipedia
http://en.wikipedia.org/wiki/Normal_distribution
CDF: http://en.wikipedia.org/wiki/Cumulat...ution_function
PDF: http://en.wikipedia.org/wiki/Probabi...nsity_function

On Box Plots
http://en.wikipedia.org/wiki/Quartile
http://en.wikipedia.org/wiki/Boxplot
http://upload.wikimedia.org/wikipedi...lot_vs_PDF.svg

Phi, the Golden Ratio: on Phi/phi encountered in PDF and CDF
http://en.wikipedia.org/wiki/Golden_ratio
http://www.maths.surrey.ac.uk/hosted...nacci/phi.html
http://opinionator.blogs.nytimes.com...ype=blogs&_r=0

Statistics, free study material:
Start: http://www.usablestats.com/tutorials.php
Advanced: http://statweb.stanford.edu/~tibs/El...II_print10.pdf
Laplace original (French): https://archive.org/details/thorieanalytiqu01laplgoog

Still don't get it? Ask Josephine: http://web.stcloudstate.edu/jtkulas/josephine.jpg
"A child of five could understand this. Fetch me a child of five!"

Wooohhhh,
Very interesting thread.
Apipshort, thanks very much for these brilliant pages.

Tomcat98

thx :-)


PDF and CDF as an Excel file download.



This is a combination of the 2 images shown at the top of page: http://en.wikipedia.org/wiki/Cumulat...ution_function

Excel does the numbers here, but they are the same the Bell Indicator calculates.
This file can replace many printed tables out there for p values, and also adds the y lookup which is not usual.

Z=0 found around row 400.

The red curve is identical in all respects to the Indicator curve.

This is an outlay only.

5 SMAs derived from ECB Daily quotes, provide data from the chart window, which is overlaid on the bell.
The numbers are dummy, but show how values could display.



If the grid is correct and the numbers were meaningful, quiet a few statistical ducks get lined up.
A lot needs to be reviewed before anything workable comes along ...

This image situates the box plot over the curve.
http://upload.wikimedia.org/wikipedi...lot_vs_PDF.svg

Hi apipshort, can you share the mq4 file to reproduce how the daily data is overlaid on the bell curve and explain how to use the result to trade? What are the main elements to look for in the resulting image of data overlaid on the bell curve? I'm trying to wrap my head around all of this.

Hi apipshort, thank you very much for all the info you shared so far. Amazing stuff. Wow! I'm hoping that I can use these Z-Score ideas to help forecast the candle direction on the Daily time frame, then use that forecast to filter out misleading trade signals from my own trading system. I'm looking forward to any advice from you as to how to import the current chart data to integrate it with the bell curve as in your outlay example above. Then once the data from the previous candle close is integrated, then how can I use that to forecast the current Daily candle direction? I do some mq4 programming myself so once I can see an example of how it's done I should be able to put it all together -- at least I'll try my best with it. I have a gut feeling that this Z-Score thing is the missing link to correct market analysis. Thanks, and I'm looking forward to hearing from you soon.

Hello global, Thx for posts and sorry for late reply.

As to the first; the working code is a big do-all mess, and separate demos are needed just to show how to get data under the bell.
Please wait up on that for simplification and details on the coding.

Regarding your second; actually I just added another track to this thread, with work based on Mr. Twomey's earlier input here.
As he closely follows the Standard Deviation, think of it as an SD chapter; an evolutionary branching where quants wave us a sad yet fond farewell ;-)

So, regarding use as trade support, please make a firm distinction between "simple" Z-Scores and the SD part being added on now.

Initially only the z-score was addressed, as seen in single TFs.
Those indicators seem stable and work with data straight from the terminal.
They are found in the zip at foot of first post, as a collection of Basic Indicators along with documentation.
Treat them as BB thermometers or weather stations. No need to keep a finger in the wind and guess regarding anything to do with MAs, BBs or extremes of market behaviour.

Some tries at an MTF view can be found in the Experimental folder there. More should follow shortly, intriguing but flaky ...
Best kept in a zip for editing or yanking.

I shy away from giving any advice on trading, but it seems the basic "z-score" indicators can support or aid in refining most any traditional MA/BB strategy.
For a coder it is simple to work in the z-score and mostly concentrate on testing various conditions/trade signals.
One or two of those indicators have been specifically written for them to grab some functions and run.

All the above concerns our happy but messy TF views with candles and whatnot.
Not to knock it, but what seems a crystal ball one day turns into yarn the next.

This is where Mr. Twomey caught up with us and he doesn't hold with any of that ...
He's more one to sit down of a quiet Sunday and chart the future of the world's economy and next week's markets on the back of an envelope.

For all explanations regarding trading, his reasoning and expectations, best follow him and consult his publications.
I expect to put up, bit by bit, the mechanics to get the kind of numbers he looks at, but none of the expertise.
No bell at all is actually required, straight layouts of numbers is sufficient.

The bell from the demo is a formal statistical grid. Changing values have to be shoe-horned (normalized) into a fixed outlay.
Added calculations such as skewness and kurtosis are required to interpret this artificially symmetrical view.

A bell using real values would wobble and squish like a blob of jelly on a plate once junior gets hold of it.
A trader with good MM could likely build a strategy predicting today's market from whether he hits floor, ceiling or walls.
I'd lay odds on Mom putting a quick stop to all that though.

The basic statistical concepts useful for interpreting this thread will be found, in most presentations, in a rather fixed format:

Introductions.
Read up on Objects, Variables and Scales.
Typically grouped under Data Reduction: Frequency Distributions, Central Tendency and Variation.
Basic Probability as an introduction to Inference.

After that it's duck and weave with the boffins and if they catch you out, you're toast ... probably.

Thank for your response apipshort. I tried various way of using z-scores to aid my trading but I've found they give no additional help than %B or Bollinger Bands do since they basically measure the same thing. However, I found that the coefficient of variation (CV) which is a normalized value that measures volatility, is more useful as a filter for my trading signals.

Here is the simple code to get the CV as a percentage:


I have the habit of using the DivZero function just in case the denominator may be zero but that shouldn't be necessary in this case so just use:


Nevertheless, I'll continue following this thread to see if it leads to something that can increase my edge. Thanks for all your efforts. Brian Twomey certainly knows his stuff but it may take some time to reach his level of understanding.

https://en.wikipedia.org/wiki/Coefficient_of_variation

Hey global, a big on that.

That also got me looking at the quartile coefficient of dispersion:
https://en.wikipedia.org/wiki/Quarti..._of_dispersion

and more related concepts which I managed to miss out on so far.

Your code works very well. I saw you wrote it out to fit in with the other functions.
It is used in a demo version of At Price.mq4 to generate the %CVs both on Open and Close as a "Comment".



Maybe you would compare those numbers to your own, and advise on where they should be best integrated in the displays?
Right next to Z-score and SD info? Sorry, I don't have a feel how to work it in yet or for the values you filter on.

The ZeroDiv function is also new to me. At least I never bothered since none showed up so far in mega flops.
I wonder what the z-score would be on the occurrence of a division by zero in nature ... infinity? Oops, re-boot big bang? (Y/N).

Please set the period in the mq4 dialog box equal to your MA/BBs.

Since I trade very short term, instead of looking at the open and close %CV values of the current candle I use the open %CV value of the current candle together with the Rate of Change (ROC) of the CV value from the previous candle open as a way of filtering out false signals. I'm sure that there must be other statistical values that can be used to help predict the next candle direction but I really don't know what they are. Whatever values are used, they must be normalized so they can be applied to any time frame or currency pair. I know of one very successful trader who says that his trading system is based on counting probabilities and Brian Twomey uses statistics successfully so there is so much more to know about using statistics for successful trading. Eventually we will get there.

Global, thank you very much for your input and interest. You may expect more here.

A new thread was started recently concerning the z-score.
The intention was to show Rookies some lines that got me going on that subject a few years ago.

At present, one should follow both threads since new information of interest can be found at:
Gramps' Greedy Indicator @ Forex Factory

It is an occasion to use tools from the Z Kit for some real investigation in trade related matters.
Feel free to post over there if not too technical, otherwise PM please.

I expect to keep new developments here and the rookie thread for more basic discussions.
Things should shake out in a while.

The next post will be an mq4 to put Mu on the trade screen; The Average of the 9 TF averages.

Mu.mq4
This set-up is pure MTF and simple enough to be useful. Download from first post, please.

It combines better orientation between TFs, along with a permanent read on price movement.
It is also the starting point for any other statistical MTF work and consists of completely neutral information.

It pairs two statistical concepts; Mu and the Average Deviation of Price in the TFs. I call that deviation muDev for short.
Together ... cherry.

Mu
In statistics MU is an Average of averages.
It's use is only correct when it is the average of an entire known "population".

A Population holds - to our satisfaction - information on all (price) elements that are known, or that we care to know about.
A Sample is only partial. Complete information is not available. Estimates can be made though, using slightly different calculations.

The 9 TFs we use on MT4 suit the requirement for a population. Heads are nodding.
The population consists of Reduced Data; the 9 Simple Moving Averages from each individual TF.
Other types of averages won't do, as they do not represent a Mathematical Average.

Summing the price value of all 9 TF SMAs and dividing by 9 results in the value of Mu.
A Price Value, valid on-tick, through which a horizontal line is drawn by Mu.mq4 and updated.

Note: The only variable that sets the Mu value is the SMA Period. You must manage and check that in Dialogue Box.
This does not mean any BBs or MAs are actually needed on-screen. If showing, those MAs should line up exactly with the markers.
The default Period is 20, like the whole Z kit.

Mu on Gold, Minutes - squished

muDev
MuDev is calculated by analogy; Sum the Price Deviations in each TF and divide by 9 for an average.
We do not obtain a Price Value, but a Price Difference, expressed in Pips.

muDev is a given name in this context, it is also adopted in the code. It's a real concept, so a bit of jargon then.
Its job is do the least, do the most.

Once the mq4 is up and you have identified the Mu line, click through a few TFs.
Maybe squish the screen and also observe the Bid Line.

Now, please lay on the cursor and measure the pips between Mu and the Bid line. Write it down.
Do a few more. All the same ... on every TF.
Congratulations, you are now a confirmed applied statistician. Yes.

You have calculated, by eyeball, the Average TF Price Deviation; muDev.
It can also be summed and averaged, a lot of useless work really.
This because the equation Mu + muDev = Bid holds in all TFs and so muDev = Bid - Mu.

muDev, as a pips off-set from Mu, is congruent with Bid.

A double freebie. Traders can work up strategies with Bid near Mu (SL) or look to TP coming in, etc.
Also get instant orientation. On any Pair, one Mu line. In any TF, the same Mu line.
And always Bid offset with a current pip value, doing the yo-yo.

Double, because as the trader looks at Bid and Mu, statistics looks at muDev.
muDev is an "in". With a muSD and a muZ-score added, one has all the current major market statistics averaged.
One can already tell the sign of such a muZ; Above Mu, z > 0. Below Mu, z < 0. At Mu, z = 0.

The horizontal Mu on screen is the same as the vertical Mu under the SND Bell (posts #24 and up on the Z kit thread).
Experiments in Demo now may serve, as these two concepts will be used exactly the same in different outlays.
There will be a Doc to sort the Dialogue box, but the comments in code will get one going.

The Mu.mq4 and user information can be downloaded together now.

The code for the Basic Indicators from the Z-Score Kit has been updated for 600+

All new code has a Last Edit date of August 8, 2014. The old versions should be over-written.
Downloads and information can be found in a new out-lay at foot of first post.

The aim is to provide a solid code base for investigating the Z-Score, at least in the 9 TFs on mt4.
That should also serve to develop more elaborate projects, as z-scores there mostly show up as variations on the same theme.

Please note that the Mu.mq4 has had a slight update and been re-posted.

Good Luck, apipshort

00sd Average Joe.mq4

Average Joe completes the Mu.mq4 (post #36) by adding a calculated SD and Z-score to form an Averaged TF called the "Mu TF".
Called? Yes, just a name because in this form it is not a statistical distribution about Mu.

We will see and use such an "official" SD as a formal standard. That involves a distribution of the TF SMAs about Mu.
The result though requires a whole new interpretation, as the respective 9 TF z-scores change sign and value.

However, using the "muDev is congruent with Bid" trick as found in the Mu.mq4 description, Bid can be seen as another Average.
Taking the given Price Deviations in the 9 TFs, it is possible to calculate an averaged "muSD" for the 9 TF SMAs about the muDev/Bid tandem.
We can understand this concept trading-wise, as it is analogous to any normal TF and built only from familiar parts.

The behaviour of this "Mu TF" is dynamic and directly related to PA. It is effectively the "wild" version of the formal distribution of TF SMAs about Mu.

When Bid = Mu, both Standard Deviations - the formal SD and the calculated muSD - are found to take on exactly the same value!
This not only ties these distributions together through a numerical equivalence; It also sets up a shared "zero" reference for SD comparison once price moves away.

Average Joe is all about eye-balling. Values are seen moving at scale on the Chart and only concern the new "Mu TF" Data:
Template with added PumpJack.mq4 from the Z-score Kit.

Average Joe along with Mu.mq4 and full documentation can be downloaded from the new SD-Kit Zip in first post

Thanks for the update apipshort. Since you uploaded the Mu I've been playing with it and I find it very useful. Now with this new Average Joe and "Mu TF" stuff there is so much more to experiment with. I'll go through the documentation. Thanks again.