Quantitative Trading | Page 3 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Classic Quantative Hedge Fund Method 1 reply

Day Trading vs. Swing Trading vs. Position Trading 86 replies

Attachments: Quantitative Trading
Exit Attachments

Quantitative Trading

  • #41
  • Edited 5:43am Jun 16, 5:33am (26 hr ago) | Edited 5:43am
OK I think I am ready to delve into the interesting subject of gating using machine learning to identify essentially 3 market states, Long, Short and Range. I have a price feed into my Ubuntu server and will probably develop using Python, any tips or ideas?
  • #42
  • Jun 17, 4:14am (3 hr ago) Jun 17, 4:14am (3 hr ago)
My first tries are a complete fail, tried training a model to predict the next day of EUR/USD up or down, the results were essentially random.
  • #43
  • Jun 17, 4:36am (3 hr ago) Jun 17, 4:36am (3 hr ago)
At this point my honest conclusion is that although these machine learning models might be very powerful, at the retail level we do not have access to the type of data that would allow them to identify an edge.

Public model + public data = 0

Public model + institutional level data = probably >0

Private model + private data = I suspect there is an edge

I'd love to be proven wrong on this, anybody have any thoughts ?
  • #44
  • Jun 17, 4:47am (3 hr ago) Jun 17, 4:47am (3 hr ago)
Quoting Guy.Baker
My first tries are a complete fail, tried training a model to predict the next day of EUR/USD up or down, the results were essentially random.
Yep - be grateful though, if it'd returned anything it'd be so fragile as to not work long term

My understanding of ML is that it's for discovering patterns in structured stable data. Price isn't that, so ML is rarely used for this purpose. So in plain English example. Say you have a signal, and want to gate by best conditions. You've created three regimes that are persistent, with meaningful separation of states in trend, vol and stability.
Getting the right thresholds and blend of those three for a band of the best conditions is a perfect ML job. Using ML to create the signal? No. Or to create the regimes - mainly no.
Personally, I've skipped it as a later optimisation layer, not a fundamental one
  • #45
  • Jun 17, 4:47am (3 hr ago) Jun 17, 4:47am (3 hr ago)
I've tried piles of stuff, but I've all but given up on Forex. I've found genuine consistent intra-day mean reversion strategies within forex but they don't survive the cost of trading. Numerous times Ive arrived at the same position via different strategies: multiple trades daily grossing an average of about a pip per trade, unbelievable Calmar ratio, but trading costs kill them all. The only thing I actually trade algorithmically is a basket of ETFs and Gold, which re-balances monthly based on some pretty simple conditions. On back test it nets approx 10% annually with a max 10% draw down over 25 years.

I've generated back tests for so many things - a large amount of anything on FF, and needless to say none of it is profitable over the long term. Ive also come up with plenty of my own harebrained concepts, but nothing that will perform consistently over years.

The last thing left for me is news analysis, but I'm to tight too pay for a reliable consistent historical news dataset!
  • #46
  • Jun 17, 5:00am (2 hr 59 min ago) Jun 17, 5:00am (2 hr 59 min ago)
Quoting Guy.Baker
At this point my honest conclusion is that although these machine learning models might be very powerful, at the retail level we do not have access to the type of data that would allow them to identify an edge. Public model + public data = 0 Public model + institutional level data = probably >0 Private model + private data = I suspect there is an edge I'd love to be proven wrong on this, anybody have any thoughts ?
From my testing, Machine Learning and Neural Networks tend to overfit historical data and rarely survive out-of-sample or walk-forward testing.
I've focused instead on testing publicly available trading models. There's no point in reinventing the wheel if robust strategies already exist. I'm looking for high-conviction strategies that pass a rigorous three-stage validation process. Since I'm not competing on microstructure with firms like Citadel, retail-style systematic strategies are a more realistic edge.

I've tested over 100 hypotheses and built a portfolio of strategies that held up across six years of data. However, the recent market regime shift, driven in large part by Trump's announcements, has created conditions unlike anything in that dataset. As a result, the portfolio is having to adapt to an entirely new environment.

Institutional-quality Level 1 and Level 2 data is now far more accessible than I realised, and Claude was instrumental in helping me find, aggregate, and understand it. My next step is to develop new hypotheses using richer volume-based data to add another layer of insight beyond traditional technical analysis.
After extensively testing existing ideas, I've concluded that the NQ and ES are extremely efficient markets. Genuine inefficiencies are small, and once transaction costs are included, many otherwise promising strategies become unprofitable. When I have built out the next stage of ideas with the Level 1 & 2, I will pay for Level 3.

By feeding an LLM hundreds of strategies and trading concepts, I've been able to identify small statistical edges and develop new ideas that Claude has helped refine further. However, the recent regime change reinforces an important lesson: even robust strategies can fail when market behaviour changes dramatically. I used to question why institutional trading firms spent so much time developing new models, but after seeing it firsthand, it makes perfect sense. When your edge comes from exploiting tiny inefficiencies, a change in market character can render an entire portfolio obsolete almost overnight.

The thing that I learned quickly was that an LLM will not create anything novel. You have to give it a goal. You have to know what it is you are expecting to see at the outcome, how the sausage is made. The LLM can just do all the grunt work and help test it faster. So anyone who has an idea of how they trade well in a discretionary way can probably get the LLM to automate some or all of the trade plan.
  • #47
  • Jun 17, 5:00am (2 hr 59 min ago) Jun 17, 5:00am (2 hr 59 min ago)
Quoting bridgstarr
I've tried piles of stuff, but I've all but given up on Forex. I've found genuine consistent intra-day mean reversion strategies within forex but they don't survive the cost of trading. Numerous times Ive arrived at the same position via different strategies: multiple trades daily grossing an average of about a pip per trade, unbelievable Calmar ratio, but trading costs kill them all. The only thing I actually trade algorithmically is a basket of ETFs and Gold, which re-balances monthly based on some pretty simple conditions. On back test it nets...
I feel you. My only advice would be don't give up, you've already learned so much, you probably will eventually pull something profitable together, have you looked at Darwinex? This platform clearly demonstrates that although gaining an edge at the retail level is rare, it is entirely possible, and I would say especially in Forex.
  • #48
  • Jun 17, 5:02am (2 hr 56 min ago) Jun 17, 5:02am (2 hr 56 min ago)
Quoting triphop
{quote} Yep - be grateful though, if it'd returned anything it'd be so fragile as to not work long term My understanding of ML is that it's for discovering patterns in structured stable data. Price isn't that, so ML is rarely used for this purpose. So in plain English example. Say you have a signal, and want to gate by best conditions. You've created three regimes that are persistent, with meaningful separation of states in trend, vol and stability. Getting the right thresholds and blend of those three for a band of the best conditions is a perfect...
Sure I was only looking for a filter, with the hope of improving my overall edge, so far no dice....
  • #49
  • Jun 17, 5:04am (2 hr 55 min ago) Jun 17, 5:04am (2 hr 55 min ago)
Quoting Rfastrat
{quote} Institutional-quality Level 1 and Level 2 data is now far more accessible than I realised
Then my guess is any edge detected will be fleeting.
  • #50
  • Jun 17, 5:08am (2 hr 51 min ago) Jun 17, 5:08am (2 hr 51 min ago)
Quoting Guy.Baker
{quote} Then my guess is any edge detected will be fleeting.
that is the cool thing about going the data route. You can take all the guesswork out.
  • #51
  • Jun 17, 5:12am (2 hr 47 min ago) Jun 17, 5:12am (2 hr 47 min ago)
Quoting Rfastrat
{quote} that is the cool thing about going the data route. You can take all the guesswork out.
Sure, I am certainly not suggesting not testing it. Testing is all we have.
  • #52
  • Jun 17, 5:14am (2 hr 44 min ago) Jun 17, 5:14am (2 hr 44 min ago)
Quoting Guy.Baker
{quote} Sure I was only looking for a filter, with the hope over improving my overall edge, so far no dice....
Wrong tool for the job. Start with direction, vol and X to keep it simple, where X is your wildcard, else you can end up with way too many regimes. A comprehensive regime labelling encompasses ALL conditions, else it's broken before you start.

For X, try efficiency, directional ratios, crowding etc.

I mean even if you avoided trades with low vol, that's an effective filter right there ; low vol regimes last an aggregate of 3 and a bit hours and are far more liable to chop.
  • #53
  • Jun 17, 5:27am (2 hr 32 min ago) Jun 17, 5:27am (2 hr 32 min ago)
Quoting triphop
{quote} Wrong tool for the job. Start with direction, vol and X to keep it simple, where X is your wildcard, else you can end up with way too many regimes. A comprehensive regime labelling encompasses ALL conditions, else it's broken before you start. For X, try efficiency, directional ratios, crowding etc. I mean even if you avoided trades with low vol, that's an effective filter right there ; low vol regimes last an aggregate of 3 and a bit hours and are far more liable to chop.
I'm not convinced though that using ML to detect low volume periods would add any value over a simple algorithmic approach, am I missing the point ?
  • #54
  • Jun 17, 5:32am (2 hr 27 min ago) Jun 17, 5:32am (2 hr 27 min ago)
Quoting Guy.Baker
{quote} I'm not convinced though that using ML to detect low volume periods would add any value over a simple algorithmic approach, am I missing the point ?
Not at all - I mean I wouldn't use ML at this stage. Algo-based regime ID as a base, any fancy ass stuff can hold. Strategy/signal on top. Not the other way around. So your measure of success isn't alpha, it's does my ID encompass all conditions, and statistically separate the market into different states.

Regimes are not sexy at all but the hours - money - they'd save every algo dev here is ridiculous so it's time well spent IMO and reusable forever.
  • #55
  • Jun 17, 5:57am (2 hr 2 min ago) Jun 17, 5:57am (2 hr 2 min ago)
Quoting triphop
{quote} Not at all - I mean I wouldn't use ML at this stage. Algo-based regime ID as a base, any fancy ass stuff can hold. Strategy/signal on top. Not the other way around. So your measure of success isn't alpha, it's does my ID encompass all conditions, and statistically separate the market into different states. Regimes are not sexy at all but the hours - money - they'd save every algo dev here is ridiculous so it's time well spent IMO and reusable forever.
I love the quest for finding a regime. It used to be a side quest, but now I am all in. Because every strategy works, if used in the regime it was designed for.
What I love more is that with 6 to 9 years of data and an LLM I can completely falsify this grifter
Inserted Video
and extract the 1 or 2 bits of real information and add it into my knowledge base.
  • #56
  • Jun 17, 6:03am (1 hr 55 min ago) Jun 17, 6:03am (1 hr 55 min ago)
Quoting Rfastrat
{quote} I love the quest for finding a regime. It used to be a side quest, but now I am all in. Because every strategy works, if used in the regime it was designed for. What I love more is that with 6 to 9 years of data and an LLM I can completely falsify this grifter https://www.youtube.com/watch?v=Z-hU97WO30I&t=296s and extract the 1 or 2 bits of real information and add it into my knowledge base.
Ha you're rare. And totally agree. There's a breakout strategy somewhere here with people trying endless permutations to try and find the "works everyday" combo. They will never find it because it doesn't exist. You're supposed to gate but the memo got lost somewhere.
  • #57
  • Last Post: Edited 7:25am Jun 17, 7:13am (46 min ago) | Edited 7:25am
Quoting triphop
{quote} Not at all - I mean I wouldn't use ML at this stage. Algo-based regime ID as a base, any fancy ass stuff can hold. Strategy/signal on top. Not the other way around. So your measure of success isn't alpha, it's does my ID encompass all conditions, and statistically separate the market into different states. Regimes are not sexy at all but the hours - money - they'd save every algo dev here is ridiculous so it's time well spent IMO and reusable forever.
Before I tested my strategies' performance inside one of these so called regimes, I like to test the reliability of the regime first, and so far I've pulled a blank, maybe I used the wrong data, maybe I used the wrong model, but I'd like to be convinced of ML regime detection and it's superiority to hand coding filters for various market conditions
2 traders viewing now, 1 is registered:
Guy.Baker