Markov Models and Forex | Page 2 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

New historical Forex models 140 replies

Anyone used ARMA + GARCH models in forex trading? 14 replies

Ecn/stp/dma models and slippage 0 replies

Risk Models 0 replies

MT4 VWB Models 1-4 8 replies

Attachments: Markov Models and Forex
Exit Attachments

Markov Models and Forex

  • #21
  • Sep 24, 2025 7:28pm Sep 24, 2025 7:28pm
Quoting W0lfram
{quote} In any case we'll get sample reducer. Quality is debatable. Like in this Hypothetical distribution: {image} {quote} Only the price itself is not stationary, i.e. modelling "close price EOD" etc. is doomed to failure. Bc the market is highly efficient = random walk behavior. Stationarity is easily achievable if needed. Even the count of ticks inside 1m candle has its own distribution and stays the same over the years. {quote} Have you ever seen unfounded claims like that: "Let's say you have a good strategy. With HMM, you could find when...
I talk from my experience and what I heard from my fellow quant trader sits next to me. Yea, you could do lots of things to build a stationary series. There are lots of modeling to try and it's fun indeed. Many CTA ML/Regression/AI models run a good year and fail in the following. That's what I know from the industry. Really good success AI/ML stories are always from HFT/MM. For us CTA traders, I stick to what I said "it's good as a filter rather than a forecaster."
  • #22
  • Sep 24, 2025 10:42pm Sep 24, 2025 10:42pm
Quoting W0lfram
{quote} Well, the links from google weren't very useful. The code in one of them didn't work at all Anyway I was researching and trying to implement HMM filtering. Some brief results: run_up - best unrealized profit during the signal S1 & S2 are from the previous post. And HMM filtering applied to S1 We have slightly shifted mode and higher variance. But no difference between S1 and HMM. - it only reduced the S1 sample by a factor of 2.{image}{image} {quote} Can't agree more. Suppose there are 2 approaches: - random entry, wait...
Hey W0lfram,

I really enjoy the way you always manage to put a nice spin on data analysis (something I don’t quite pull off myself). The idea of testing quantiles instead of just bluntly looking at upruns and downruns feels, to be honest, like a way more scientific approach. Love it!

So what’s your take - how can newbies actually find their own edge in the market? Like I said earlier, that 50:50 RR ratio comes from signals pulled straight out of price action. For me, the few small (and hard-to-spot) edges aren’t about hoping for some permanent trading advantage, but about using price itself to figure out entries and exits: follow price and not hoping for a signal to end the same every time. Even though so many “gurus” preach that trading has to be objective, it seems like - if there really are consistently profitable traders out there - they all trade discretionarily. And whenever you ask these gurus how they break the RRR paradox, I’ve yet to hear a single one explain why they actually win. For instance, TRO just keeps saying: ‘Use money management.’ As if that alone could fix anything …
  • #23
  • Sep 25, 2025 2:04am Sep 25, 2025 2:04am
Quoting zkchyo
{quote} I talk from my experience and what I heard from my fellow quant trader sits next to me. Yea, you could do lots of things to build a stationary series. There are lots of modeling to try and it's fun indeed. Many CTA ML/Regression/AI models run a good year and fail in the following. That's what I know from the industry. Really good success AI/ML stories are always from HFT/MM. For us CTA traders, I stick to what I said "it's good as a filter rather than a forecaster."
So let's try creating this model. In your experience, which inputs are best for the HMM? I've identified, theoretically, the following as the best: log return, realized volatility, high-low range, and ATR. Most academic papers focus on two or three states at most, I plan to try two. For training I found many references to Baum–Welch. How does that sound?

Other ideas are welcome
  • #24
  • Sep 25, 2025 3:14am Sep 25, 2025 3:14am
Log return, realized vol, high-low, ATR
Where there is a will, there is a way. https://www.oyodental.nl/
  • #25
  • Edited 6:23pm Sep 25, 2025 5:54pm | Edited 6:23pm
Quoting AntiCre
{quote} Hey W0lfram, I really enjoy the way you always manage to put a nice spin on data analysis (something I don’t quite pull off myself). The idea of testing quantiles instead of just bluntly looking at upruns and downruns feels, to be honest, like a way more scientific approach. Love it! So what’s your take - how can newbies actually find their own edge in the market? Like I said earlier, that 50:50 RR ratio comes from signals pulled straight out of price action. For me, the few small (and hard-to-spot) edges aren’t about hoping for some permanent...
Quote
So what’s your take - how can newbies actually find their own edge in the market?
I think everyone should go their road (edge of a graph lol) and see all the things that doesn't work (or softer to say - not outperforming each other)
Attached Image (click to enlarge)
Click to Enlarge Name: WolframNB_qL4RsFFQkJ.png Size: 27 KB

I would say leverage isn't there for our benefit. Trading in cash (1:1 per entry) maybe. Forget about SL. Even random entries can yield small profits with very high probability.. is it enough for a hypothetical trader? Probably not, they want to catch swings.

I always look forward for your contributions in other threads. Great stuff!

p.s. but It's risky to overshare here. I once got a message from someone trying to sell me recycled "upgraded" cp's "edge within....",
Air force
  • #26
  • Sep 25, 2025 6:39pm Sep 25, 2025 6:39pm
Quoting Fiox89
{quote} So let's try creating this model. In your experience, which inputs are best for the HMM? I've identified, theoretically, the following as the best: log return, realized volatility, high-low range, and ATR. Most academic papers focus on two or three states at most, I plan to try two. For training I found many references to Baum–Welch. How does that sound? Other ideas are welcome
What are you trying to achieve? Volatility? Directional bias? Based on what you want to do, you can do more than those. Academics are not traders and they don't know what a trader should know. If you have some alternative data, factorize them too. Try to gen more factors and run random forest for feature selection could be helpful. Build a HMM and try to fit it into your strategy for improvement. And you probably don't need to program yourself. Run through LLM models and you'll get the code with some errors. Fix it by yourself. So much easier these days...
  • #27
  • Sep 25, 2025 11:55pm Sep 25, 2025 11:55pm
Quoting W0lfram
{quote} {quote} I think everyone should go their road (edge of a graph lol) and see all the things that doesn't work (or softer to say - not outperforming each other){image}



Quoting W0lfram
I always look forward for your contributions in other threads. Great stuff!
Thank you so much. I'm unfortunately busy with more important stuff atm.
  • #28
  • Oct 15, 2025 9:02am Oct 15, 2025 9:02am
I've looked into this. The models are theoretically sound but practically unreliable for live markets in my experience. If you have any realistic practical experiment with live markets, please share !
  • #29
  • Oct 15, 2025 12:34pm Oct 15, 2025 12:34pm
  •  amvt
  • Joined Feb 2025 | Status: Developer by Choice Trader by Heart | 202 Posts
I have dabbled with Markov Models in my time, and although I don't want to come across as someone who has made a successful strategy using Markov models (I have not) I do want to give some information regarding what I learned from them.

In the markov models world, there are 2 popular options:

1. Hidden Markov Models (HMM): This means, you don't know what the regime is and you are trying to find it

2. Markov Regime Switching (MRS): This means, you know what the regime is

Instinctively and even Practically, I have found out that MRS model generally performs better if you know what you are looking for. My experience with MRS model has been great. I was able to get a a working strategy into MRS, which performed decent but still was not thoroughly tested.

I tested the strategy with the MRS model for 2 years. Which is statistically, enough, but not for my strategy which was very picky. So, the samples I got from 2 years of data was only 150. Samples = Trades.

However, if you are building a Markov model, you will soon understand the "prediction" part is not actually done by the Markov model. Instead, it is done by another model (of your choice) that takes the output from the markov model. It's kind of ensemble within an ensemble. But, it does provide good results if one follows traditional model building practices.

However, my one takeaway from all of this would be the following:

If you are trying to build a model, be it LSTM, ARCH, MRS or even Kernels, pick a model that is designed around the problem or strategy you already have.

If you are building models, hoping that the model itself would find patterns. You are in no-mans land. You are basically trying to find a needle in a haystack. This takes huge amount of time, effort, resources and skills an average quant can't pull of without a team.

It's better to pick a model, that does one thing quite well. That "thing" should be your main strategy or problem you are trying to solve. You are not trying to find an ultimate model, that creates a 100% return strategy out of thin air. You are being practical, and inferring market fundamentals within the model with through you strategy or the model's design.

Fun Fact: I starting building trading algorithms with the Markov Model, because this was the model Jim Simmons published a internal paper on during his work at his government job. I though if Jim Simmons sees something special in this model, it should be a great model by design. But no. You have to design the model yourself. This does not mean you have to literally create it from scratch. This means, you have to design the way Markov Models understand the market. You know the fundamentals of the market. Infer them. In the model. That's the most practical way to do it, for us retail traders.
— Developed by MMQ
  • #30
  • Edited 6:22pm Oct 17, 2025 12:19pm | Edited 6:22pm
Had a month of R&D journey ... ended up in a multi-language experiment.

I began with Mathematica since I'm most comfortable with. The initial focus was on getting the core mechanics right.
Attached Image (click to enlarge)
Click to Enlarge Name: WolframNB_n1joazNBXg.png Size: 6 KB


Wolfram Mathematica (core research and prototyping):
- superior symbolic math, native matrix operations
- rapid prototyping and computational exploration
- fast optimization, but I switched to Python

Python:
- recoded everything
- data pipeline and easy result/model management
- also fats optimization libraries

Pine:
- quick visual validation
- live charting and signal generation


Some references:
- buy/sell after a random candle, close next day
Attached Image (click to enlarge)
Click to Enlarge Name: rhBFUSeBax.png Size: 6 KB


- buy/sell and hold for some period. Useful chart btw {every candle pattern or indicator based trading look like this}
this one for 12 days. I wanted to make a good, comprehensive post but lost in screenshots and workflpw, I'm just not good at it, so, sorry for the mess
Attached Image (click to enlarge)
Click to Enlarge Name: WolframNB_IVTIro3XSS.png Size: 60 KB



Model training. 2 and 5 states on the same sample.

Attached Image (click to enlarge)
Click to Enlarge Name: 6dZm7IOdwg.png Size: 44 KB

Attached Image (click to enlarge)
Click to Enlarge Name: QCJALlezAw.png Size: 83 KB



Feed all available data now. Why these steps are separated: you can train the model on the middle of dataset or pick the recent year as a train set, then feed past
data to it. There's no future for the model, only data points.
Attached Image (click to enlarge)
Click to Enlarge Name: z2uBquQAKL1.png Size: 63 KB

Attached Image (click to enlarge)
Click to Enlarge Name: z2uBquQAKL.png Size: 24 KB
% return is log returns

Obviously these two are awful, but the code is capable of producing good models. I'll show later, too much pictures for one post.
(would these models be just as bad if reverse trade logic were applied?)

I did a stupid mistake with indexing first, so it had a lookahead bias and the report showed a few mln % log returns
Air force
  • #31
  • Oct 17, 2025 12:32pm Oct 17, 2025 12:32pm
One of a decent models I've discovered so far:
median trade duration is 4 trading days.
Able to catch quick volatile moves, proportion of shitty trades is quite low.
Only 512 trades during 24 years
Attached Image (click to enlarge)
Click to Enlarge Name: WolframNB_ORYxoqePhV.png Size: 36 KB


Attached Image (click to enlarge)
Click to Enlarge Name: WolframNB_bVDUJ2ValN.png Size: 25 KB
Air force
  • #32
  • Edited 1:38pm Oct 17, 2025 1:22pm | Edited 1:38pm
Next step was batch testing.
I haven't succeed with Bayesian optimization, so seems like hardcore grid search is my way to go.

But training models even in parallel would take forever, let's just permutate the matrix and other stuff.
And I needed a quick visual validation, so it's Pine's turn to step in.
If the model works in Python and Pine gives similar signals - we have something...
Attached Image (click to enlarge)
Click to Enlarge Name: 9DGoQ0SXNm.png Size: 7 KB



sorting json data... best model here 0.6191 (absolute return for GBP futures)

Attached Image (click to enlarge)
Click to Enlarge Name: WolframNB_GyGEodT5su.png Size: 8 KB


Let's verify on TradingView
6191*6.25 = 38 693.8 without costs not way off. Fees gonna eat ~1/3 of it, but again - this is raw 'entry-exit on opposite signal'. Play with available strategies on TV, almost all I tried are red. SuperTrend was green, but will be red with commissions
idk how they calculate max equity DD, there is clearly no -48K datapoint on equity chart
Attached Image (click to enlarge)
Click to Enlarge Name: TradingView_D0hlJQdUgF.png Size: 55 KB

Train on GU... test on EU
Attached Image (click to enlarge)
Click to Enlarge Name: TradingView_xJ1Gsjf5P5.png Size: 52 KB
Air force
  • #33
  • Edited 11:12am Oct 25, 2025 10:40am | Edited 11:12am
Had some success with genetic optimization using DEAP library (https://deap.readthedocs.io/en/master/). Not as fun as brute-forcing billions of combos but faster

There is a little discrepancy in backtesting idk why - data is exactly the same. Since TV has that obviously false Max. equity DD field (ok, it comes from Buy&Hold. Anyway got more questions about how their back tester works) and Sharpe ratio almost always is negative - I conclude that it's on their side.
Attached Image (click to enlarge)
Click to Enlarge Name: YpIDf4m8T7.png Size: 11 KB


GU Daily
Attached Image (click to enlarge)
Click to Enlarge Name: TradingView_JKzDfCrxVR.png Size: 56 KB



With Hourly data it's much more complicated:
- need 3+ states -> computationally/time/storage expensive
- local data 75k bars, TV provides only last 10k, changing initial state - more problems
If I can get it to work on hourly then I'll automate it
Air force
  • #34
  • Last Post: Nov 3, 2025 9:29am Nov 3, 2025 9:29am
Quoting W0lfram
With Hourly data it's much more complicated: - need 3+ states -> computationally/time/storage expensive - local data 75k bars, TV provides only last 10k, changing initial state - more problems If I can get it to work on hourly then I'll automate it
3 states models look promising but:
- genetic algo picks candidates with ~10 trades in 2 years
- reduced data to 10k rows
- immense parameter space. I had to split it into chanks; 729x4050000 (chunk x combinations) - ~20 minutes/one and the output file for each weights 1.3 Gb

Randomly found good 2-state candidate
This is how it looks like:
Attached Image (click to enlarge)
Click to Enlarge Name: 6B1!_2025-11-03_16-11-22_5377b.png Size: 61 KB

Attached Image (click to enlarge)
Click to Enlarge Name: TradingView_PPBCWgcfdf.png Size: 45 KB




Compare with SuperTrend (the best I could get from it)
Attached Image (click to enlarge)
Click to Enlarge Name: TradingView_hFU9XRivET.png Size: 58 KB
Air force
1 trader viewing now