null | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Modifying indicator to account for NULL values - A Little Help, please! 8 replies

Attachments: null
Exit Attachments

null

  • #1
  • First Post: Edited Jun 21, 2018 11:41am Jan 5, 2018 8:42pm | Edited Jun 21, 2018 11:41am
Just gathering some market statistics this Friday night.

Attached Image (click to enlarge)
Click to Enlarge Name: Screen Shot 2018-01-05 at 8.35.39 PM.png Size: 239 KB


I really love frequency distribution...

--- About the Journal ---
Trading is very boring once you figure it out...
Leaves a lot of time to sit, think, and write...
This journal is where I spend a lot of that time.

This is for me to document research/work so I can reference it later.
Some of this is for the LOLs and to share my opinions on matters I find important.
  • #2
  • Edited 9:43am Jan 8, 2018 6:43am | Edited 9:43am
Heres the flip side of that.
Attached Image (click to enlarge)
Click to Enlarge Name: Screen Shot 2018-01-08 at 6.20.07 AM.png Size: 82 KB


Really interesting stuff.

Attached Image
  • #3
  • Jan 8, 2018 9:20am Jan 8, 2018 9:20am
Quoting Sis.yphus
Just gathering some market statistics this Friday night. {image} I really love frequency distribution...
can you explain the above stuff Please
  • #4
  • Jan 8, 2018 9:42am Jan 8, 2018 9:42am
The Kelly Criterion

Is there one betting amount that is statistically better than another? Yes.

Have you ever wondered why Casinos have table limits? Because the answer to the first question is yes.

If there was a statistically significant amount to bet at any given time, how would you know? The Kelly Criterion

In probability theory and intertemporal portfolio choice, the Kelly criterion, Kelly strategy, Kelly formula, or Kelly bet is a formula used to determine the optimal size of a series of bets in order to maximise the logarithm of wealth.

Attached Image (click to enlarge)
Click to Enlarge Name: Screen Shot 2018-01-08 at 9.32.10 AM.png Size: 36 KB


As an example, if a gamble has a 60% chance of winning (p = 0.60, q = 0.40), and the gambler receives 1-to-1 odds on a winning bet (b = 1), then the gambler should bet 20% of their bankroll at each opportunity (f* = 0.20), in order to maximize the long-run growth rate of the bankroll.

I don't always bet on the market... But when I do, it's a Kelly bet or a fractional Kelly bet.
  • #5
  • Jan 9, 2018 6:22am Jan 9, 2018 6:22am
A Correlation Problem

I love the currency market. My favorite thing about it is the liquidity and efficiency of the market. It really is a beautiful thing.

However, there is one issue always facing us when it comes to dealing with the currency market. That issue is, there is a major correlation problem.

Due to how currencies are traded in pairs, every single major pair (28 total) are all correlated with each other. This might not seem like an issue, but it really is for small accounts, and it is also an issue when it comes to evaluating performance in relation to the expected performance after a significant amount of samples.

When it comes to getting an expected result for a system, one thing that has to happen, is you need a large sample of independent trials to evaluate. In an ecosystem where everything is extremely correlated, it is really hard to get independent trades, even on 2 different pairs using 4 different currencies. This causes our results to skew quite a bit when evaluating a batch of trades.

Example:
Say we are trading EURUSD and AUDJPY. 2 Pairs, 4 different currencies.
For simplicity sake, we assume that we have a system that is correct 50% of the time.
Now if these two events are truly independent and we take a trade on each pair, the probability of both being right or wrong would be 25% (.5 * .5 = .25).

However, we know these events are not truly independent because even though these 2 pairs are made up of 4 different currencies, they are still pretty heavily correlated.

EURUSD-AUDJPY Correlation Readings(1 hour, 1 Day, 1 Week, 1 Month, 3 Months, 6 Months, 1 Year):
Attached Image


In this example, let's say each trade lasts a day, so we will use the daily move correlation coefficient of .72 (Shockingly high if you think you are avoiding correlations be trading what appears to be 2 completely separate pairs).

Let's plug that correlation value into the following formula to see what the probability of both being right or wrong is now:
Attached Image


The math works out to be 43%.

So if you are trading this system on two different pairs using 4 different currencies, you may expect the probability of both trades being wrong is only 25%, when in reality, due to the underlying correlations, the probability of being trades being wrong is 43% (Important note: this does not mean the probability of both being right is 1-.43. It's 43%).

Now imagine how this correlation will skew your results in relation to their expected results when you assume your trades are independent, but they really aren't.

Now also imagine how this skew will lead to more losses (or wins) and how that will exaggerate the drawdown (or run up) on small accounts.

Bottom line... If you are trading any 2 pairs, it is basically impossible to have your trades placed on each pair be independent of one another (if that makes sense).
  • #6
  • Jan 14, 2018 10:12pm Jan 14, 2018 10:12pm
Occurrences & Expected Results
DISCLAIMER: In this post I am talking strictly about win rate when I say expected result. I am not talking about PnL.

In the last entry, I talked a bit about how product correlations can really skew the expected results of a system. Another thing that can really skew your results, is the number of occurrences you trade to test your system. This sounds like a no brainer, but I see people on here all the time trading weekly systems or long term daily systems, that then wonder why they aren't getting the expected result after 3-4 weeks of trading.

During this 3-4 weeks they may have taken 12-24 trades. Hell if they took one a day they'd have 20 trades.

Let's assume this systems win rate is 50%.

Our variance in this scenario is 5 ((20)(0.5)(0.5) = 5). Our sigma is 2.236 or 2

68% of the time, this trial will result in 8 wins and 12 losses or vice versa.
95% of the time this trial will result in 6 wins and 14 losses or vice versa.
99% of the time this trial will result in 4 wins and 16 losses.

The point being, you could try this system, do 20 trades, have only 6-7 wins and 13-4 losers, and this performance would still be statically normal based on the number of occurrences. Now factor in the issue with correlations in the last post and say these 20 trades were on different pairs but still correlated, these numbers would be skewed even more. You could easily have only 3 wins and 17 losses due the skew from correlated underlying.

Think how many traders would quite this system and move on the to next, only to get the same lack luster results that are again statistically normal due to the small number of occurrences.

So how can we get closer to our expected result? More independent occurrences!

Thanks to the central limit theorem, as we scale up our occurrences, the variance of our results will gradually shrink as we get closer to our expected result. And with enough occurrences, you will almost surely end up with exactly your expected result. If we were to do 80 trades instead of 20, our variance would cut in half. In other words, if you want to slice your variance in half, you need to do 4 times as many occurrences.

Attached Image


This is how HFTs make money every single day. Even if they have a system that only wins 50% of the time and is profitable, they can be green every day trading the number of occurrences required to get their expected result. It's not all about dark pools and front running...

I think it is crucial for new traders to understand how the number of occurrences and product correlation can really skew your results away from your expected results. If you don't understand how these things work, you will surely dump systems that are actually profitable, and heavily invest in systems that are total garbage, but look great do to the skew working in their favor.
  • #8
  • Jan 16, 2018 12:44pm Jan 16, 2018 12:44pm
Non-Linear Trends and Serial Correlation
This post is essentially just a link to one of my favorite replies on this forum. It is from mzvega and it talks about non-linear trends in the market. I wanted to create this so I and others can easily reference it for later.


Click here to check it out... it really is a gem of a post.
  • #9
  • Jan 18, 2018 7:26am Jan 18, 2018 7:26am
Inserted Video


Time... Is never time at all...
  • #10
  • Edited 8:14am Jan 18, 2018 7:55am | Edited 8:14am
X-Axis Entries/Exits

If you have read some of my old work (some of which is very green and naive, but I am still proud of how my research has progressed), then you likely have read this thread in the recycling bin.

What I am discussing in the first half of the post is what I am focusing on today. I was looking at how price moves from pivot to pivot and the conclusion the data showed is this.

Quote
What this shows us is that no matter what the Entry point TP, and SL levels are, the odds are always that we will break even in the long run. Even if you use 1:2 Risk Reward Ratio(Target Next Pivot Level Or a Further One), the odds are that you will be break even over the long haul & the market is way more balanced than most traders want to believe. It doesn’t matter where we place our entry, TP, and SL AT ALL!

Now, this conclusion still holds true and I still believe it to be true. I can also recreate the experiment on every single pair and get the same results. If you enter the market, place an SL and TP, and just wait for the trade to either be a winner or a loser, you will almost always break even if there were no spread, and obviously have a negative expectancy with the spread.

It took me a really long time to understand why this is the case. For a while, I wasn't even looking for a why, and I never really pursued the why, but just stumbled across it in discussions I would have later with other traders.

The reason is this... when trading like this, there are no X-Axis Exits.

Your entries and exits on the X-Axis are equally as important as your entry and exits on the Y-Axis.
  • #11
  • Jan 18, 2018 4:57pm Jan 18, 2018 4:57pm
Quoting Sis.yphus
X-Axis Entries/Exits If you have read some of my old work (some of which is very green and naive, but I am still proud of how my research has progressed), then you likely have read this thread in the recycling bin. What I am discussing in the first half of the post is what I am focusing on today. I was looking at how price moves from pivot to pivot and the conclusion the data showed is this. {quote} Now, this conclusion still holds true and I still believe it to be true. I can also...
This post really got me thinking about entering positions and how they develop. Are you seeing your entry develop positively in a 'good' amount of time.

Reminded of a Twoblink post where he discussed the development of a position and touched on entries that developed overly quick and positive as lucky and how one should consider exiting.

Also Gann and his angles. Gets one thinking about trajectories.

This could very well not be what you were getting at.
  • #12
  • Jan 18, 2018 8:55pm Jan 18, 2018 8:55pm
Quoting fxfury
{quote} touched on entries that developed overly quick and positive as lucky and how one should consider exiting
There is definitely something to this idea, you can see it in the stats in my first post on this thread if you know what to look for.

Thinking about trajectories is also very important in my opinion. I am always looking for new ways to measure how price gets distributed.
  • #13
  • Jan 19, 2018 7:38am Jan 19, 2018 7:38am
Inserted Video


I really enjoy Jordan Peterson's lectures. They really make you think...
  • #14
  • Jan 19, 2018 10:58am Jan 19, 2018 10:58am
Quoting Sis.yphus
https://www.youtube.com/watch?v=5cTyxJWZC7k I really enjoy Jordan Peterson's lectures. They really make you think...
The frequency dist. charts are a little unclear for me wrt the peak time and pit time.

The video was great, always enjoy a good psychology bit. Just to extrapolate his big rat little rat stuff to the market, ostensibly we are little rats to the markets big rat.

I'd like to think 30% of the time the little guy must win corresponds with 30% of the time the market trends(roughly). As a 'trend' follower you must make the most of the 30% opportunities. I would also like to think your stats show a need to align the price time opportunities.
  • #15
  • Jan 19, 2018 4:08pm Jan 19, 2018 4:08pm
Quoting fxfury
{quote} The frequency dist. charts are a little unclear for me wrt the peak time and pit time. The video was great, always enjoy a good psychology bit. Just to extrapolate his big rat little rat stuff to the market, ostensibly we are little rats to the markets big rat. I'd like to think 30% of the time the little guy must win corresponds with 30% of the time the market trends(roughly). As a 'trend' follower you must make the most of the 30% opportunities. I would also like to think your stats show a need to align the price time opportunities.
I was thinking a similar thought when he talked about that.

I was thinking that when designing a system, you may want to consider having a minimum hit rate of at least 30% otherwise trading the system will be extremely hard to do psychologically on a very instinctual level, even if the system itself is profitable. You conciously know the system is profitable, yet due to the hit rate being super low, say 5%, you will always run into that feeling of maybe this isn't actually working, and I need to change something, and once you stop and question yourself, things can get really bad really quickly.
  • #16
  • Jan 21, 2018 9:46pm Jan 21, 2018 9:46pm
You can probably tell my previous distribution tables were simply set up in Google Sheets. That worked, but whenever I adjusted a few settings in my algorithm, I would have to manually re-select the new ranges to make the charts work again with the newly imported dataset. That was a major headache so this quiet sunday I spent some time coding up my own custom dashboard with JavaScript so now my charts automatically adjust to the newly size dataset when it is imported.

Attached Image (click to enlarge)
Click to Enlarge Name: New Dash 1.PNG Size: 164 KB


Attached Image (click to enlarge)
Click to Enlarge Name: New Dash 2.PNG Size: 155 KB


If you haven't been able to tell already, coding and statistics are two skills/interest I find highly valuable when it comes to trading.

For those wondering how this links all links up with my trading terminal it goes something like this:
MT4 Indicator to Gather Data(MQL4) -> Server to Receive Data from MT4(Node.Js) -> Front-end to Analyze & Display Data (HTML,CSS, JavaScript).
  • #17
  • Jan 23, 2018 3:05pm Jan 23, 2018 3:05pm
Inserted Video


This is hard to watch...

I feel bad for this woman trying to outwit a titan such as Jordan Peterson. She is clearly out of her wheelhouse.

"Jordan: - Cats like to be in warm places... | Cathy: - So you're saying cats are responsible for global warming?"
  • #18
  • Jan 23, 2018 9:06pm Jan 23, 2018 9:06pm
Political Correctness Has Invaded This Forum...

So I was looking to clean up a post, and I saw something interest. It's one of the reasons you can report a post.
Attached Image


I don't understand how nationalism is equated to racism when reporting a post and I think it is very intellectually dishonest to equate the two as if they are one in the same. How is NATIONalism the same thing as having RACIAL prejudice? It's not. News flash people, nationalism is on the rise worldwide. This was in my wall street journal today.

Attached Image (click to enlarge)
Click to Enlarge Name: unnamed.jpg Size: 1.1 MB


People who can accept things that are false by their definitions as the truth have really given up their minds.

But hey, you know what they say...

Attached Image (click to enlarge)
Click to Enlarge Name: Capture2.PNG Size: 522 KB
  • #19
  • Jan 24, 2018 12:15am Jan 24, 2018 12:15am
Quoting Sis.yphus
Political Correctness Has Invaded This Forum... So I was looking to clean up a post, and I saw something interest. It's one of the reasons you can report a post. {image} I don't understand how nationalism is equated to racism when reporting a post and I think it is very intellectually dishonest to equate the two as if they are one in the same. How is NATIONalism the same thing as having RACIAL prejudice? It's not. News flash people, nationalism is on the rise worldwide. This was in my wall street journal today. {image} People who can accept things...
its consequence of so called democracy, where professor vote has same weight as vote of heavy drinker, drugs addicted person, gangster and finally stupid man ....

when what is "a truth" has to be result of voting by the crowd ....
  • #20
  • Edited 12:32pm Jan 25, 2018 12:20pm | Edited 12:32pm
Even When You Should Win, Most Will Lose

If you haven't already, check out this study.

Below is the Abstract
"What would you do if you were invited to play a game where you were given $25 and allowed to place bets for 30 minutes on a coin that you were told was biased to come up heads 60% of the time? This is exactly what we did, gathering 61 young, quantitatively trained men and women to play this game. The results, in a nutshell, were that the majority of these 61 players didn’t place their bets very well, displaying a broad panoply of behaviorial and cognitive biases. About 30% of the subjects actually went bust, losing their full $25 stake. We also discuss optimal betting strategies, valuation of the opportunity to play the game and its similarities to investing in the stock market. The main implication of our study is that people need to be better educated and trained in how to approach decision making under uncertainty. If these quantitatively trained players, playing the simplest game we can think of involving uncertainty and favourable odds, didn’t play well, what hope is there for the rest of us when it comes to playing the biggest and most important game of all: investing our savings?"

The bold statement says it all really.

In the this study, you have $25 to play with an a guaranteed edge of 10% if you only bet heads. With the given timeframe, one could make about 300 bets. If you simply just bet on heads every time, it would be nearly impossible, if not impossible to reach the maximum of $250.00.


Each and every person KNEW this going in, they were told, no matter what, betting heads gives you a 10% edge. If you could find/create this scenario in the market, that is THE holy grail...

A brief summary of the results:
Quote
"Remarkably, 28% of the participants went bust, and the average payout was just $91. Only 21% of the participants reached the maximum. 18 of the 61 participants bet everything on one toss, while two-thirds gambled on tails at some stage in the experiment."

These results are amazing!
1. Nearly 1/3 of the players actually went bust with a 10% edge. Imagine if casinos had 10% edge. These people had nearly a 10x higher edge than a casino, and nearly 1/3 blew all their money and lost...
2. Only 18 of 61 people reached the maximum of $250.00. Basically everyone should have reached the maximum if they simply followed the system.
3. And, this is the most fascinating, nearly 2/3 of the players actually bet on tails at times during the experiment... Nearly 66% of the people took bets that they KNEW had a 10% edge against them.


This just shows, even if you can win and have the system to win, most people will still lose. Most people lose with a guaranteed 10% edge... You can bet your ass most will lose with a smaller edge that is dynamic as the market changes.

Why? People don't do what they are supposed to, even when they know what they are doing is wrong, or won't work.

Even when given a holy grail, 1/3 of the people will lose, and only 1/5 will get the maximum payout. Even when you mathematically shouldn't lose (have the holy grail), a majority of people will still lose... On top of that, 1/3 of people will make bets that have no positive expected values, even when they know that to be true.
0 traders viewing now