Systematic Portfolio Diversification - Data Mining Concept | Page 4 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Trading Equity Millipede Concept by Concept 99 replies

Systematic trading 1,576 replies

Anyone use Systematic from Alpari? 1 reply

A Systematic Approach to Markets 68 replies

Systematic Trading vs. Discretionary Trading 14 replies

Attachments: Systematic Portfolio Diversification - Data Mining Concept
Exit Attachments

Systematic Portfolio Diversification - Data Mining Concept

  • #61
  • Sep 13, 2018 7:16am Sep 13, 2018 7:16am
Quoting RondaRousey
{quote} No, Berkshire Hathaway is the biggest and most profitable hedge fund with market cap of US600 billion. They don't use quants or mathematical modelling or programmers or EA or indicators. They made 65 billion last year. Banks go to them to ask for bailouts. Renaissance Technology uses systematic trading, mathematical modelling, huge number of quants, lots of computers & programmers. They make a million trades a year or was it a month. They trade the 1 second timeframe. If you want to emulate them go ahead. I doubt if Renaissance made 65 billion...
>>They trade the 1 second timeframe.

LOL.
  • #62
  • Sep 13, 2018 7:24am Sep 13, 2018 7:24am
Quoting RondaRousey
{quote} No, Berkshire Hathaway is the biggest and most profitable hedge fund with market cap of US600 billion. They don't use quants or mathematical modelling or programmers or EA or indicators. They made 65 billion last year. Banks go to them to ask for bailouts. Renaissance Technology uses systematic trading, mathematical modelling, huge number of quants, lots of computers & programmers. They make a million trades a year or was it a month. They trade the 1 second timeframe. If you want to emulate them go ahead. I doubt if Renaissance made 65 billion...
Wrong; in terms of returns in percentage terms; RenTech (short for Rennaissance Technologies); are much more superior.

That is why their fees are double the standard fees. They charge 4/40% instead of the standard 2/20%.

https://www.afr.com/personal-finance...0161215-gtc5mz
Attached Image (click to enlarge)
Click to Enlarge Name: sc_20180913-122220.png Size: 365 KB
  • #63
  • Sep 13, 2018 7:29am Sep 13, 2018 7:29am
Quoting RondaRousey
{quote} No, Berkshire Hathaway is the biggest and most profitable hedge fund with market cap of US600 billion. They don't use quants or mathematical modelling or programmers or EA or indicators. They made 65 billion last year. Banks go to them to ask for bailouts. Renaissance Technology uses systematic trading, mathematical modelling, huge number of quants, lots of computers & programmers. They make a million trades a year or was it a month. They trade the 1 second timeframe. If you want to emulate them go ahead. I doubt if Renaissance made 65 billion...
Attached Image (click to enlarge)
Click to Enlarge Name: Quant.PNG Size: 1.1 MB
  • #64
  • Sep 13, 2018 9:44am Sep 13, 2018 9:44am
Quoting Copernicus
There is an issue with diversifying into different markets or diversifying into different timeframes as correlations in these two methods are moving feasts. What is correlated this month may not be correlated next month.
The correlation also depends on the sampling frequency: https://www.forexfactory.com/showthr...92#post6916092
No greed. No fear. Just maths.
  • #65
  • Sep 13, 2018 10:12am Sep 13, 2018 10:12am
This might help in the process...

Inserted Video
  • #66
  • Sep 13, 2018 10:28am Sep 13, 2018 10:28am
Quoting PipMeUp
{quote} The correlation also depends on the sampling frequency: https://www.forexfactory.com/showthr...92#post6916092
Thanks Pip. This helps a lot. :-)
  • #67
  • Sep 13, 2018 10:42am Sep 13, 2018 10:42am
Quoting alphadude
{quote} Other ideas I found useful, is to use tools such as Strategy Quant or AdaptiveBuilder to get some ideas from the data mined systems; and add human touch to it through R&D Kind of Toby Crabel style. best
I agree bro. If you've spent years studying the market, it makes no sense to use a raw-strategy from a data-mining engine (unless the strategy meets criteria that relates to your understanding of the market) when the added value comes from the cranium. Leave the raw-strategies for the new inexperienced trader. Not much R&D would be needed anyway if you know edges from experience, the only requirement is to go in the code and tweak it.
  • #68
  • Sep 13, 2018 11:07am Sep 13, 2018 11:07am
Quoting mbrown
{quote} I agree bro. If you've spent years studying the market, it makes no sense to use a raw-strategy from a data-mining engine (unless the strategy meets criteria that relates to your understanding of the market) when the added value comes from the cranium. Leave the raw-strategies for the new inexperienced trader. Not much R&D would be needed anyway if you know edges from experience, the only requirement is to go in the code and tweak it.

This exercise is like throwing a dart on a dart board ,then drawing a circle around it, it does not work in real time , in the future.Highly optimized eas that worked in the past,will fail in the future.

What exactly is purpose of data mining , i.e to find the bulls eye? Copp please explain.
  • #69
  • Edited 1:21pm Sep 13, 2018 12:52pm | Edited 1:21pm
Quoting loozers
{quote} This exercise is like throwing a dart on a dart board ,then drawing a circle around it, it does not work in real time , in the future.Highly optimized eas that worked in the past,will fail in the future. What exactly is purpose of data mining , i.e to find the bulls eye? Copp please explain.
Mate...Remember I am fairly new to the game of data mining but I am approaching it from a risk management perspective where I think that most don't.....and hence the failure rate

I have been working on and off with algorithmic trading for many years mate with a programmer (as I don't have that skill and I prefer to watch him work while I pretend to work on other stuff) but agree with the statement made by many that no EA works across all market conditions. It is a blindingly obvious statement when you understand the nature of different market conditions and understand that an EA is a stationery solution.

No single strat or small selection of strats make a sufficient cashflow to make the game appealing. However when you apply some divergent strats across very long timeframes, you find that some of these divergent solutions (that restrict trading to particular more extreme conditions conditions) at least don't go down the gurgler and can offer positive returns if you are prepared to tolerate the very volatile equity curves and big drawdowns. This is just not appealing to the mass mob looking for instant gratification and a new Bugatti Chiron each year.

However....and this is the key, under a diversified portfolio that slightest edge over the long time can be turned into a real something in a correctly balanced portfolio with a large number of 'weak' solutions that fit the bill. There are no 'strong solutions' that give an appealing equity curve over the long term, but there are some simple strats that do offer slight positive expectancy.... such as some of the simple basic trend following models and simple breakout (momentum) models.....but even these models have their weaknesses and that's why you need lots of different types of weak solution so that collectively the individual weaknesses are plugged.....hence the need to generate lot's of different types of EA that have basic characteristics of a divergent trader (namely tight initial stops with trailing stops and open ended condition) . It is not rocket science mate and in fact you do not want rocket science for this project. The simpler the divergent strat the better as the less variables employed, the less constrained the strategy is and the less the propensity to produce a curve fit result.

Now mate....I am sick to death of discretionary trading, and I am a miser with my capital and hate the thought of going backwards. I need to go fully systematic mate as despite the hard yards in R&D that I invest in projects, I just don't have the patience to sit at the screen for long hours and it bores me *shitless*. Plus systematic traders are sexier than discretionary traders and pull more chicks. Been in this game too long and I am starting to look like a PC. As a result, I am on a mission towards full automation and you can see by this statement that I suck as a standard trader. :-)

So in a nutshell these 'weak' solutions that you may find over the long term horizon come with a painful volatile equity curve on their own that makes the single strat or a few strat game non-appealing to the vast majority of traders.

This is where I kick in with a portfolio solution that takes a bit of grunt to compile correctly, however once achieved can ramp up these returns once I flatten the curve (achieve good risk-weighted returns). Now this can be applied to any trading strategy methodology....but fortunately for me, not many know how to do it correctly :-).

At the moment with some initial trials with this project I am getting close to a workable result, but have a fair bit to go as there is too much volatility still present in the portfolio to get my bang for buck....so I need to 'hit big data with a big portfolio'...hence that is why this is a work in progress...however my experience tells me there is a solution to be found in here by adopting sound PM principles but diversifying like the clappers into only those weak solutions.

I therefore deliberately configure my data mining search to find a vast bundle of weak solutions of non correlated divergent strats wthen correctly balance the portfolio....and voila....I turn a weak solution into a strong long term wealth builder and retire to the beach with my swarm of sirens feeding me grapes as I pump out the music and flex my non-existent abs.

That's about it L :-)

Cheers

C

Attached Image (click to enlarge)
Click to Enlarge Name: Al Pacino.PNG Size: 723 KB
  • #70
  • Sep 13, 2018 1:40pm Sep 13, 2018 1:40pm
Quoting loozers
{quote} This exercise is like throwing a dart on a dart board ,then drawing a circle around it, it does not work in real time , in the future.Highly optimized eas that worked in the past,will fail in the future. What exactly is purpose of data mining , i.e to find the bulls eye? Copp please explain.
It is doing quite well for me. Thank you. If you talk in terms of Unit Root tests, Correlograms etc, I might be able to take you seriously. I am sure your approach has value but don't delude yourself into thinking it is the only approach.

Quoting Copernicus
{quote} .so I need to 'hit big data with a big portfolio'...hence that is why this is a work in progress...however my experience tells me there is a solution to be found in here by adopting sound PM principles but diversifying like the clappers into only those weak solutions. I therefore deliberately configure my data mining search to find a vast bundle of weak solutions of non correlated divergent strats wthen correctly balance the portfolio....and voila....I turn a weak solution into a strong long term wealth builder and retire to the beach with...

Copernicus, your approach works. My portfolio stands at 300 strategies and growing. Frankly, I am very surprised because the drawdowns are very rare indeed.
  • #71
  • Sep 13, 2018 5:22pm Sep 13, 2018 5:22pm
Quoting mbrown
{quote} It is doing quite well for me. Thank you. If you talk in terms of Unit Root tests, Correlograms etc, I might be able to take you seriously. I am sure your approach has value but don't delude yourself into thinking it is the only approach. {quote} Copernicus, your approach works. My portfolio stands at 300 strategies and growing. Frankly, I am very surprised because the drawdowns are very rare indeed.
That gives me hope Brownie. Hopefully I am on the right track with this project.
  • #72
  • Edited 9:29pm Sep 13, 2018 5:22pm | Edited 9:29pm
Quoting RondaRousey
{quote} I use to be fully automated now I am discretionary. Actually 5% automated, 95% discretionary So when are you going to make some live calls so they can be tracked to put your theories in profits or losses More bark than bite: https://www.youtube.com/watch?v=XbGsUwDElZE
It's coming mate and the reason for this thread - A proof of concept over the next couple of years.

The calls will be coming from EA's so they can all be independently validated and portfolio results will be generated from an FX Blue consolidated portfolio operating over the next couple of years. I am price following as opposed to price predicting so base this success of the concept on the Law of Large Numbers. I have a few things to do to set this project up to finalise the monitoring tools which I will post on this thread and we will be working from a universe of EA's operating in a 6 month incubator account..so this takes some time before we get a feel for whether the assumptions used by me are right or wrong.

Rome wasn't built in a day....heh, heh heh :-)

Inserted Video


PS Refer to this chart again to see where I am coming from in regards to this approach. The curve represented below is the performance result of the best of the best in the professional divergent space. Namely the equal weighted result of the top 64 funds who play with systematic diversified momentum approaches towards the longer term timeframes and across asset classes (eg. trend following and rotational/absolute momentum strategies) including the likes of Winton, Dunn Capital, Eckhardt Trading Co, Chesapeake etc etc etc. So as good as I think I am...how good do you really think I am?. In the hard world of reality land let's try to get as close to this benchmark of CAGR/Draw = 0.66 and average returns of approx 9% or so.

Attached Image (click to enlarge)
Click to Enlarge Name: TF Index.PNG Size: 390 KB


The red rectangles above are strongly divergent market conditions across asset classes. You can see that there is a lot of grinding with protracted periods of *relatively slow but building* drawdown between these euphoric moments where other approaches are sticking the finger to us.....but in the long run, this style of approach seems to work. As Nick Radge says, don't focus on the next trade.....focus on the next 1000.

Better still let's try to aim towards the benchmark result of the best risk-weighted returns from the Top 10 in this Index space for a smoother ride in the long term from the likes of Transtrend, Fort LP, Crabel etc. etc. with a CAGR/Draw of 1.25 and average returns of also approx 9%.

What the Hell let's go out on a limb here. Given that these industry benchmarks are net of fund management fees and expenses and represent the NAV curve, we might even find that we can outperform them as modest guys we are.

Attached Image (click to enlarge)
Click to Enlarge Name: Top 10.PNG Size: 166 KB


We could of course strive towards pink unicorns like we do on this forum...but I am choosing to use a verified and audited industry benchmark as a guide to track this project's progress as opposed to the informed commentary of very respected no-names on this forum who all are excellent traders. I don't know about you...but I have trust issues with many on this forum. Probably just another one of those stupid biases....talk about crazy.

Attached Image
  • #73
  • Sep 14, 2018 1:22am Sep 14, 2018 1:22am
Quoting RondaRousey
{quote} My benchmark: funds under management 1-10million, 50% returns funds 1 billion-100billion, 30% Funds 100 billion+ , 15% is excellent The larger the funds, the more difficult to get returns. Noobs on FF mostly have $1000 accounts or under, what can you do with 9% p.a on 1k? Small account trading is a joke The size of your account matters https://www.youtube.com/watch?v=Y-uPT8KbgXU
Nice benchmarks RR but you clearly are a slightly heavier hitter than me....but then you are a martial artist after all. Good to see some realism applied when you have significant skin in the game. A breath of fresh air
  • #74
  • Sep 14, 2018 1:26am Sep 14, 2018 1:26am
Marketing in the industry misled human beings. the one marketing that keeps me questioning myself from broker firms, when they say you can start trading as little as 5 dollars yeah and fire your boss!!!
Will be back in 5years
  • #75
  • Edited 1:57am Sep 14, 2018 1:29am | Edited 1:57am
Quoting Satraderma
Marketing in the industry misled human beings. the one marketing that keeps me questioning myself from broker firms, when they say you can start trading as little as 5 dollars yeah and fire your boss!!!
I tried that once...but after a few months I had to claw my way back into my boss's good graces and in doing so as punishment had to shove my head up my ****** and became 3 feet shorter. Who said trading can't give you back problems for life?:-)
  • #76
  • Sep 14, 2018 1:45am Sep 14, 2018 1:45am
just querying why
Attached Image (click to enlarge)
Click to Enlarge Name: lulzmaster2.jpg Size: 40 KB
  • #77
  • Edited 3:12am Sep 14, 2018 3:01am | Edited 3:12am
Quoting loozers
just querying why {image}
I reckon it is something like an acceptance that learning never ends. He certainly recognized in his career that he was no whiz at maths, but his creative passion for deeper understanding and refusal to merely accept current bias and dogma is what set him apart from the crowd.

Perhaps he was emphasizing that true learning only ends when your ego tells you that you have mastered 'what needs to be learnt'.

To categorize yourself as 'being talented' requires you to have an opinion that you have mastered something when in fact, that mastery is a subjective line drawn in the sand based on an ego statement as opposed to an empirical measure. As soon as you say you have talent, that admission eliminates the passionate drive towards further understanding as in your personal opinion being talented means there is little left to learn.

Physicists (eg, Lord Kelvin) without Einstein's creative genius thought the game was over in the late 19th Century for physics in that all was very well understood except for a handful of small remaining unresolved issues...The resolution of these handful of outstanding issues led to major theoretical revolutions of which Einstein played a central part. Was he talented?....well compared to the eminent scientists of the day that probably thought they were to make such a bold and assumptive statement.....clearly he wasn't like them. Look where they ended up with talent, and look where Einstein ended up by simply questioning every assumption of the day. What he did do however was learn the current models of the day to identify their weaknesses and find solutions to extend their application to a wider domain. That is a bit different to a guy that thinks he is a creative genius with nothing to formally back up his challenging statements for peer review apart from a biased ego itself. They tend to be the problem 'black or white type' guys that have an indoctrinated narrow opinion and are not worth conversing with.
  • #78
  • Sep 14, 2018 5:09am Sep 14, 2018 5:09am
Quoting Copernicus
This research thread takes us into the world of Data Mining and explores an approach that seeks to compile a portfolio of divergent strategies that are robust in terms of their ability to navigate a broad range of market conditions and have a track record that demonstrates a small edge.
this is a wonderful idea.
I am afraid we may need a new trading platform in order to build this kind of thing.
From my experience data mining is very familiar with AI.
Profit growth tells you the trader ability.
  • #79
  • Sep 14, 2018 5:21am Sep 14, 2018 5:21am
Quoting mbrown
{quote} It is doing quite well for me. Thank you. If you talk in terms of Unit Root tests, Correlograms etc, I might be able to take you seriously. I am sure your approach has value but don't delude yourself into thinking it is the only approach. {quote} Copernicus, your approach works. My portfolio stands at 300 strategies and growing. Frankly, I am very surprised because the drawdowns are very rare indeed.

same here; running a portfolio of 80 strategies diversified in all aspects. all mean reverting though.

i used to run momentum strats but gave up due to high death rate.

all 100% systematic. I spend most of my time doing research, DIY work around the house, contributing to FF, fighting scammers on FF and evaluating FX educators.

as am writing this am in a metro train in Barcelona going shopping before heading back home tomorrow. whilest the EA soldiers are fighting volatility
  • #80
  • Edited Sep 15, 2018 2:23am Sep 14, 2018 5:35am | Edited Sep 15, 2018 2:23am
Quoting voketexpert
{quote} this is a wonderful idea. I am afraid we may need a new trading platform in order to build this kind of thing. From my experience data mining is very familiar with AI.
Cheers V. Good to have you aboard.

I am currently preparing some tools that I will be sharing on the initial post that will assist in this endeavour as you need to interrogate lots of data and need to be very organised. The data mining products like Strategy Quant, Adaptrade and EA Studio etc. go a part of the way in streamlining processes at the strategy but are heavily biased towards convergent outcomes but with some specific parameter tweaks and some coding tweaks (as Brownie suggested earlier) you can enforce a divergent methodology into the outline.... but to diversify divergent strategies at the portfolio compilation stage you need some heavy duty tools that are not off the shelf products like a heavy duty database to house all your efforts and a spreadsheet model to apply some of the things that a database does not do well but can be derived from outputs delivered by the database.

We will be like bower birds collecting our twigs (desired class of strategies) and storing them in a database so that when we get to compiling the puzzle that is at the portfolio, we can do lots of reshuffling and rearranging of return distributions to achieve a powerful risk-weighted outcome.

That's why it is taking some time before we can officially start as I need to finalise these tools in a manner that make them easy to use for those not familiar with using them :-)

Stay tuned

C
0 traders viewing now