Statistical mean of the market [quant corner] | Page 8 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Explore Real (Statistical) Market Edges 21 replies

The Birth of a Quant Trader 107 replies

Quant trading & QuantLib 6 replies

Quant Corner 12 replies

questions from a future quant 39 replies

Attachments: Statistical mean of the market [quant corner]
Exit Attachments

Statistical mean of the market [quant corner]

  • #141
  • Oct 21, 2014 11:14pm Oct 21, 2014 11:14pm
Ok so far, my research is pretty much capped as the problem with the price is that it's too choppy, no matter what filter I put in, those nasty spikes always get around it.

Adding 2-3 filters to filter them out would only cause more lagg and would not improve my performance, so I decided to transform the price itself, using various methods: Renko,CBR,Kagi,Log chart, etc and see if my results do improve or not.

Attached Image (click to enlarge)
Click to Enlarge Name: 60p.png Size: 45 KB

Attached Image (click to enlarge)
Click to Enlarge Name: 240p.png Size: 48 KB


So yes, the 60 pip chart looks much smoother than the H4 chart (240 min), about 70% of the small fluctuating noise is removed without touching decreasing the accuracy much, so the TIME based chart is not that efficient.

Although I would not jump to the tick chart as Adal suggested, because the tick chart contains much more noise than the H4 so, its would be really a stupid idea to do that, so i`ll just use PRICE TRANSFORMATIONS to filter out the noise, and leave the signals to my filters.
"There's a sucker born every minute" - P.T. Barnum
  • #142
  • Oct 22, 2014 12:49am Oct 22, 2014 12:49am
Ok my research got a really big breaktrough, looks like transformed price is really improving my results.

So my new research goal is 25.000 PROXI SCORE, because I think it's far more than enough, as I currently see where my research is going.

So that means that currently I`m at 3% completion, but this is about to change soon
"There's a sucker born every minute" - P.T. Barnum
  • #143
  • Oct 22, 2014 5:27am Oct 22, 2014 5:27am
Ok screw the entire PROXI SCORE benchmark system, it's a useless crap. It is not worth to measure the drawdown, because it only distorts my research.

I have to conclude that the DD is not important, the DD will be adjusted only by the position size, but the expectancy itself is the main player here.

So imagine if i got a system with 2 expectancy and 2% DD, and a system with 1 expectancy and 0.9% DD, based on the PROXI SCORE, the latter one will be the better, however the first one has the better expectancy. So i give up a twice as profitable system just for a little bit of less risk, which is stupid.

The second reason is that the DD is only the intratrade DD, while the consecutive losses are not really incorporated, so either way its useless, it is only useful if i adjust the SL based on this, but we are not there yet, first we need a system...

So, the expectancy is the only option on the desk, so i shall measure only the GROWTH, which will be the inception-to-end growth for the entire backtest.

GROWTH = ((END BALANCE / START BALANCE) * 100 - 100)

Attached Image (click to enlarge)
Click to Enlarge Name: mod.png Size: 49 KB


So here is a modified version of the filter 2.7, a little bit different than the one calculated under the proxi score system, but similar, as it can be seen it has a 73% total growth with a 9 % YoY growth which is pretty lame. Now the LOT size can be scaled so it's not a final number, but a one based on pip growth more or less, as the LOT size is always the minimal, so there is not risk management calculated only raw expectancy.

Unfortunately I have to redo some of my tests in order to be precise as the PROXI SCORE BS screwed up my research a bit.
"There's a sucker born every minute" - P.T. Barnum
  • #144
  • Edited 3:17am Oct 23, 2014 12:12am | Edited 3:17am
Quoting Adal
An interesting thing to note is that the paper was written in 2000. And it is about tick-based "indicators" and non-linear time-frames. 14 years later, retail forex still uses bar-based indicators and wall-clock timeframes. I bet that in 2030 retail forex will still work with M5 and H4 candles, fibos and SMAs So much for retail traders being innovative and thinking outside-the-box (as opposed to academic researchers)
Actually I took the liberty to analyze the TF's again, and yes my conclusion still stands, I just dont see how Adal's claims that tick charts are good, is correct.

Attached Image (click to enlarge)
Click to Enlarge Name: aadasdad.png Size: 14 KB


I`ve tested my basic filter on all timeframes with median and close prices, and still the H4 is the best (amongst classical TF only, because other type of charts could be better)

It seems that as you scale down and down the noise becomes overwhelming and the performance decreases. The Y axis is the % growth measured on those TF's and it can be seen that the peek is at H4 (240). Above the daily I didnt tested because the data was small, and below M5 the calculation was just too slow (over 1 hour), but the decreasing trend is really saying something.

Coupled with this research: http://www.forexfactory.com/showthread.php?t=336715

We can conclude that H4 is a really good TF amongs the classical TF's as it filters out the noise pretty well. Also the M30-H4 range is probably the best one to trade in.

I just don't see how raw tick timeframe is better than this, as the TF itself filters out the noise, even in the H4 you got like 50-60% noise left, then imagine the tick chart about 95% noise?

Ok maybe some non-linear TF that he has been talking about, or a volatility TF build from tick data can be good, but just tick timeframe as it is, i think is just plain rubbish. Most of the moves aren't even bigger than the spread so its not really a good dataset.
"There's a sucker born every minute" - P.T. Barnum
  • #145
  • Oct 23, 2014 2:17am Oct 23, 2014 2:17am
Ok i`ve just reconsidered my goal as i think 10,000% is not really needed, i think 5000% is already better than enough.

Attached Image (click to enlarge)
Click to Enlarge Name: bench.png Size: 44 KB


This is a fake optimization report made on fake historical data, so dont get excited and think that I`ve found the Holy Grail, yet. But it shows that a system that already has 1500% growth percent has a really smooth balance curve, so i guess the 5000% will have a really smooth curve so because of this i think 5000% is more than enough, and probably it will be really hard to find a system with that performance so let's just set 5000% my new goal.

I`ll use this fake report as a benchmark for my future results to compare it to this, obviously if this report were real than it would be a holy grail, so these are the numbers which need to be real in order for a system to be qualified as a real HOLY GRAIL

Even a system which has 1000% total growth is already superior to any average retail system which have 5-10% gain / year, this one has 193%, so see the difference.

Soon i`ll finish my retests and i`ll show my current system so that you can compare it to this fake holy grail, and let's see how my progress is going
"There's a sucker born every minute" - P.T. Barnum
  • #146
  • Oct 25, 2014 2:08am Oct 25, 2014 2:08am
Ok here is my filter 3.0

After transforming the price to a noiseless stream, i succeeded finding a very nice transformation, which filtered so much noise out that it is possible to trade only trends, and all inbetween was filtered out, so a trend follows a trend, and there is no noise or consolidation inbetween, nor any whipsaw.

The preliminary tests confirms this, but i have to run one in MT4 too just to make sure.

The results look like this, this is not fake data anymore, this is genuine candlesticks transformed into a noiseless state using a special kind of transformation.

Attached Image (click to enlarge)
Click to Enlarge Name: asaasa.png Size: 42 KB


While on the price chart it looks like this, very strange looking, but it filters out the noise perfectly

Attached Image (click to enlarge)
Click to Enlarge Name: asdasdadasda.png Size: 29 KB

Attached Image (click to enlarge)
Click to Enlarge Name: asdasdasdadww3.png Size: 35 KB


Looks like a holy grail to me.
"There's a sucker born every minute" - P.T. Barnum
  • #147
  • Oct 25, 2014 2:18am Oct 25, 2014 2:18am
Congrats Proximus nice work, I hope to learn more about coding/trading and do something similar to this one day! You sir are an inspiration! goodluck
A Positive Mind, and Positive Spirit lead to True Happiness
  • #148
  • Oct 25, 2014 2:21am Oct 25, 2014 2:21am
Quoting Rparm
Congrats Proximus nice work, I hope to learn more about coding/trading and do something similar to this one day! You sir are an inspiration! goodluck
Thanks but slow down pal, as this is only a preliminary test, it still has to pass trough MT4 test (because maybe my platform is bugged) and demo forward test.

And then on a real account a live test, so i just cant get excited until that happens.

I`ve found many holy grails which later turned out to be useless junk, and i dont want to get dissapointed anymore.
"There's a sucker born every minute" - P.T. Barnum
  • #149
  • Oct 25, 2014 9:39am Oct 25, 2014 9:39am
Quoting Proximus
Ok here is my filter 3.0 After transforming the price to a noiseless stream, i succeeded finding a very nice transformation, which filtered so much noise out that it is possible to trade only trends, and all inbetween was filtered out, so a trend follows a trend, and there is no noise or consolidation inbetween, nor any whipsaw. The preliminary tests confirms this, but i have to run one in MT4 too just to make sure. The results look like this, this is not fake data anymore, this is genuine candlesticks transformed into a noiseless state using a...
There is something strange on these charts: the price goes straight up but goes down slowly following a line.
Is the distribution symmetric over a long time? Can you generate and post the histogram of the price differences?
No greed. No fear. Just maths.
  • #150
  • Oct 26, 2014 3:27am Oct 26, 2014 3:27am
Quoting Proximus
{quote} That will give you really big headaches, i just dont think its viable to research the NN field if you dont have like a team of PHD mathematicians and econometricists. The Hedge Fund named Renaissance Technologies has atleast 91 PHD's working on that field. What makes you think you can compete with their cumulative knowledge? Besides you would need a supercomputer to constantly test all those parameters for the NN. I`m thinking more into a lower level artificial intelligence...
I hope that you realize that Rentech "edge" has nothing to do with trading at all.
  • #151
  • Oct 27, 2014 4:36am Oct 27, 2014 4:36am
Quoting PipMeUp
{quote} There is something strange on these charts: the price goes straight up but goes down slowly following a line. Is the distribution symmetric over a long time? Can you generate and post the histogram of the price differences?
Here is it. Note that the indexes of the bars might not line up as they are drawn index by index, not by time, since my "holy grail" chart is a pure price charts so there is no time dimension. So since the candles are drawn index by index, there might be some mis-aligning between the 2 charts, but i hope you can spot the stuff you are looking for.

The normal chart is a H4 chart (orange up,blue down), the other is my custom chart (green up,red down)
Attached Image (click to enlarge)
Click to Enlarge Name: a1.png Size: 62 KB

Attached Image (click to enlarge)
Click to Enlarge Name: a2.png Size: 42 KB

Attached Image (click to enlarge)
Click to Enlarge Name: a3.png Size: 56 KB

Attached Image (click to enlarge)
Click to Enlarge Name: a4.png Size: 53 KB

Attached Image (click to enlarge)
Click to Enlarge Name: a5.png Size: 71 KB
"There's a sucker born every minute" - P.T. Barnum
  • #152
  • Oct 27, 2014 8:08am Oct 27, 2014 8:08am
The vertical scale looks strange. Look on your very last screenshot. The time highest high double top appears to match the timeless highest high DT but the low in-between doesn't go as low on the timeless chart. The second next lower high around index 10000 matches the lower high of the timeless chart but the sharp lower high doesn't appear at all on the timeless chart!
No greed. No fear. Just maths.
  • #153
  • Oct 27, 2014 8:47am Oct 27, 2014 8:47am
Quoting PipMeUp
The vertical scale looks strange. Look on your very last screenshot. The time highest high double top appears to match the timeless highest high DT but the low in-between doesn't go as low on the timeless chart. The second next lower high around index 10000 matches the lower high of the timeless chart but the sharp lower high doesn't appear at all on the timeless chart!
Yes you are right, i made an error, a pretty big one as i simulated the trades also on the price chart whereas i should have simulated it on the real chart and only had used the price chart as a filter. Because in real trading you put the trades on whatever price the broker gives you, but you can use any filter you with, and not vice versa.I would wish the market were less choppy in the small scale but thats it.

Hopefully as the market gets bigger and more HFT traders join, the liquidity will raise and less choppyness will be available.Until then i`ll stick to my original charts, or i will try to match them by date.
"There's a sucker born every minute" - P.T. Barnum
  • #154
  • Oct 29, 2014 3:00pm Oct 29, 2014 3:00pm
Your chart gave me an idea. Instead of finding the MA with the lowest lag and highest smoothness why not adapting the price to the MA?
What we dislike with MAs is their lag. When the price gets too far away from the MA I close the bar and start a new one (but I require a minimal range to avoid a lot of 0-1 pip candles). The effect is that more smaller bars are created letting the MA catchup. When the price crosses the MA because of a congestion we want to avoid taking a small loss and reverse the position just to stop again and reverse again. So I require that the bar closes far enough from my MA when the price crosses the line. I modified my range bar chart. Here is the result. The blue line is a LWMA(10) which is used to generate the range bar chart. On the two other pictures I overlaid the H1 chart on it and it over the H1.
Attached Image (click to enlarge)
Click to Enlarge Name: AUDUSD-V20-lwma.png Size: 34 KB

Attached Image (click to enlarge)
Click to Enlarge Name: AUDUSD-V20.png Size: 45 KB

Attached Image (click to enlarge)
Click to Enlarge Name: AUDUSD-H1.png Size: 43 KB
No greed. No fear. Just maths.
  • #155
  • Oct 29, 2014 8:24pm Oct 29, 2014 8:24pm
PipMeUp,

That is really cool. So if I understand, you are using range bars that will only close when price drifts a certain distance from the MA?
By forcing the close of the bar the MA updates and keeps 'hugging' the current price.

What do you use to measure the 'distance' of price from the MA to force the close of the range bar?
What's it all about? It's all about money.
  • #156
  • Edited 4:18am Oct 30, 2014 4:05am | Edited 4:18am
Quoting PipMeUp
Your chart gave me an idea. Instead of finding the MA with the lowest lag and highest smoothness why not adapting the price to the MA? What we dislike with MAs is their lag. When the price gets too far away from the MA I close the bar and start a new one (but I require a minimal range to avoid a lot of 0-1 pip candles). The effect is that more smaller bars are created letting the MA catchup. When the price crosses the MA because of a congestion we want to avoid taking a small loss and reverse the position just to stop again and reverse again. So...
Yes i`ve thought about this, and this is what i was trying to say, that its useless.It doesn't matter how you transform price, because you have to take the trades on a time*price chart what price the broker gives you.

So doesnt matter what filter you use, you can use binary candles or whatever, it must always have to align with the normal time*price chart, and sometimes the price chart can generate fake price levels where the price has never been in the normal time*price chart, so any backtest on it is really useless.

Imagine that your price chart shows a bid of 1.782 and then jumps to 1.783 but the normal time*price chart only jumps to 1.781. So if you backtest solely on the price chart, it will give you fake results.

This is the problem that i face with my backtester, i have to synchronize the price and the price*time chart to make the backtest valid. Otherwise you take trades at price levels that the broker never gave you, you can just transform the price into a nice sine shaped curve, and trade off that, but in reality you trade at the price that the broker gives you, perhaps you can use the filter to guide and give signals but the trade will always be put on the broker's price.

So because of this you can never ever escape the spikes, unless you find a broker which doesn't give you, you can filter it to ignore it, but if the signal is signalled at the spike, then the trade will be executed on the spike, no matter how smooth your MA is. So because of this it's really useless to use 1 dimension price charts, because it will have the same filtering capacity as a MA+ TRIGGER PIPS, type of filter, and that will be more simple to code.
"There's a sucker born every minute" - P.T. Barnum
  • #157
  • Oct 30, 2014 4:18am Oct 30, 2014 4:18am
Quoting Proximus
{quote} Yes i`ve thought about this, and this is what i was trying to say, that its useless.
One could possibly volume/range weight the MA to make it more responsive to sudden price changes / lag less.

k
  • #158
  • Oct 30, 2014 4:28am Oct 30, 2014 4:28am
Quoting kprsa
{quote} One could possibly volume/range weight the MA to make it more responsive to sudden price changes / lag less. k
No you dont understand, the 2 charts don't align, therefore if you backtest only on the price chart you will get a fake backtest result which doesnt match reality. Because only the time*price chart which the broker gives is valid, there you execute the trades at whatever price the broker gives you, not at synthetic bars which you create as you wish.

Attached Image (click to enlarge)
Click to Enlarge Name: AUDUSD-H1.png Size: 59 KB

Attached Image (click to enlarge)
Click to Enlarge Name: AUDUSD-V20.png Size: 62 KB


The 2 charts doesnt match. The trades signalled on the CRB chart, lagg half a day, if you put the trades according to those signals, the broker's price is not there anymore so you will get fake signals, and probably lose money.
"There's a sucker born every minute" - P.T. Barnum
  • #159
  • Oct 30, 2014 4:39am Oct 30, 2014 4:39am
Quoting Proximus
{quote} No you dont understand, the 2 charts don't align.
Hi, sorry for not being clear, I am not talking about your current implementation but about a way to perhaps reduce the lag of the MA on the time-based chart.

k
  • #160
  • Oct 30, 2014 5:41am Oct 30, 2014 5:41am
Quoting kprsa
{quote} Hi, sorry for not being clear, I am not talking about your current implementation but about a way to perhaps reduce the lag of the MA on the time-based chart. k
Possibly but i dont use volume as an input, my historical data doesnt have volume, i have to redownload it again to contain volume. I will test it perhaps later but for now i work with OHLC inputs.

And it's not really about lag, but to find an optimum balance between lag<->accuracy, because if either one of those is too big, the other one get's too. I want to find always a sort of "middle way" between the configurations.
"There's a sucker born every minute" - P.T. Barnum
0 traders viewing now