ML PPO bot training Talk | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

SSS EA v1.0 - Auto Trader Bot 623 replies

You talk the talk I walk the talk 8 replies

Much of Forex talk is illusional esoteric talk 141 replies

EA bot on my strategy 1 reply

Help needed to create forex bot 2 replies

Attachments: ML PPO bot training Talk
Exit Attachments

ML PPO bot training Talk

  • #1
  • First Post: Edited Jun 26, 2025 4:20pm Jun 23, 2025 6:44pm | Edited Jun 26, 2025 4:20pm
ML PPO Bots, Hidden Gem Indicators, and Everything In Between

Hey everyone —

----------------
This will be a log of my journey working on and implementing this bot. Trading NASDAQ futures.
----------------

I'm starting this thread as a place to talk about ML-powered trading bots, specifically ones trained with PPO (Proximal Policy Optimization). I’ve been building and testing bots that use machine learning to learn from price/volume behavior and actually trade like they’ve got a brain — not just follow basic signals.

This is for anyone who's into:

Reinforcement learning (PPO, DQN, A2C, etc.)

Custom indicator ideas — especially ones that aren’t already used by everyone

Strategy creation for Forex, Futures, Crypto — wherever edge matters

Figuring out what features actually matter in a model

Forward testing, walk-forward validation, live deployment... the real stuff


I’m open-sourcing what I can, learning as I go, and hoping this becomes a spot for idea sharing, weird experiments, and maybe even building something great together who knows.

Whether you’re coding bots, trading manually, testing indicators, or just curious how this all works — you’re welcome to jump in. Share your setups, ask questions, post screenshots, vent, brainstorm — whatever.

Thanks chat gpt for the awesome intro.

Anyways I'm starting this thread as it's a new adventure I've been into the past month. I enjoy trading, I enjoy robotics. I work a full time job programming robots at a manufacturing company. I also trade as much as possible on my phone and computer at home when I have the chance.
Designing these bots at FIRST was me trying to create that holy grail. A super smart bot that trades for me! After a few failed attempts I've gotten better and better at setting these up, coding in python through Visual Studio. Now it's become some what of a hobbies and I truly enjoy messing around and doing it. It's so fun to me and so boring to my wife that I feel there has to be others out there that enjoy this type of stuff as much as I do.
Anyone here have the same ML itch that I do?
Trying to learn how to learn how to teach a robot how to learn
  • #2
  • Jun 23, 2025 7:16pm Jun 23, 2025 7:16pm
I think what's so intriguing and interesting is the ENDLESS possibilities of logic you can add to these. I agree that simple is better when trading manual. My manual trading consists of volume profile and price action.
I switch templates as well and trade "Trading Made Simple(r)" by Robinhood. You can find his template in the strategies section~~

Anyways I know simple works manually. But with the bots it not... Ever. Lol

When we trade with simple charts, There is still a LOT of complex things going through our head at any given time. Even when waiting for a simple ema cross..

All those things is what I'm attempting to train these bots.
COUNTLESS possibilities.

There's got to be some awesome combinations you could put together with RR pre calculated, time zones, lot caps. I dunno.
It's been a new really fun hobbie for me as of late
Trying to learn how to learn how to teach a robot how to learn
  • #3
  • Edited 1:15am Jun 24, 2025 12:59am | Edited 1:15am
Quoting Webb1003
I think what's so intriguing and interesting is the ENDLESS possibilities of logic you can add to these. I agree that simple is better when trading manual. My manual trading consists of volume profile and price action. I switch templates as well and trade "Trading Made Simple(r)" by Robinhood. You can find his template in the strategies section~~ Anyways I know simple works manually. But with the bots it not... Ever. Lol When we trade with simple charts, There is still a LOT of complex things going through our head at any given time. Even when waiting...
This might be the most interesting thread here for years. I am not at your level. Don't know anything about machine learning etc. I am Coding in MQ4. Could easily go to code in Python, but I know MQ4 so well and do not want to change now.

I agree that there are countless possibilities. What lack is time to investigate them. And I agree that the more filters in a robot the better. I have several filters in my robots but have not used them all at the same time. I just switched that strategy. As many filters in use as possible. Because it is also vital that you get only few but quality trades.

I will follow this thread. Might not have a lot to say though - let's see. I had decided to keep away from FF because I use way to much time on this, but ---- I can't help it ;-). It is a habit.

I look forward to have Machine Learn and CocAlpha coming here too.
MetaTrader 4 - so many advantages.
  • #4
  • Jun 24, 2025 2:22am Jun 24, 2025 2:22am
Quoting RiskFighter
{quote} This might be the most interesting thread here for years. I am not at your level. Don't know anything about machine learning etc. I am Coding in MQ4. Could easily go to code in Python, but I know MQ4 so well and do not want to change now. I agree that there are countless possibilities. What lack is time to investigate them. And I agree that the more filters in a robot the better. I have several filters in my robots but have not used them all at the same time. I just switched that strategy. As many filters in use as possible. Because it is...

Thank you! i appreciate that.
IM NOT AT ANY LEVEL. Other than a level that knows how to use tools available at my disposal. I definetly use alot of ChatGPT and Claude. They have helped me learn and code these bots along with teaching me how to use python. I do hear alot of people bash AI for giving wrong codes, which is true some of the time but its all in how you prompt it and how many chances you give it to correct code, and how much information you give it.

I havent figured out the correct filters I want in my bots though. Ive been messing with Stuff like Z scores of volume, Z-scores of price. EMA slope %, ATR ratios, MACD histograms, Candle stick body to wick ratios. Along with just adding flags for Supertrend long or shorts. There are SO many and SO many different combinations to add in to add to the bots confluence on when to take a trades.

Adding all of these different indicators together and having the bot cycle through 500,000-1,000,000 iterations of 3-6 months of chart data, it learns the best way to trade given those exact parameters you tell it. After you train the bot you backtest it on OUT OF SAMPLE data(important step) Alot of the times a trained robot will spit out a 80% win rate, then backtested on out of sample data it plumments. Thats when you know it over optimized itself. At first thats all i kept doing was over optimizing. Thats when i started to look at percentages of slope and atr and volume ect. Not EXACTLY sure how or why but it works better when backtesting a trained model. Predictions are more reliable.

Anyways sorry i can ramble on this one. Im trying to put together different ways of telling the bot what to do and how to do it. NOT JUST indicators. Risk to reward ratios. Time frames ofcourse. But everything the bot does, you have to tell it.
My firs 5 bots completely blew my practice account hahahaha. THE FIRST ONE blew my practice account of 150,000 in like 15 minutes trading 1 MNQ LMAO. it was going NUTS. But that one i was just excited that I actually got the stupid thing working, reading chart data, analyzing open high low close and volume, and making a prediction and placing a trade. That was step one. So now i have this template that I can add or subtract different features to my hearts desire, train, backtest, and forward test and see how they work.

Its been a ride. lol
Trying to learn how to learn how to teach a robot how to learn
  • #5
  • Jun 24, 2025 7:11am Jun 24, 2025 7:11am
K mean patterns. Setting the bots to learn these for high probability set up. Then I wanna try order flow analysis with time and sales and volume. And then probably mash the pair together. Got 3 ideas I'm gonna try later today.
Trying to learn how to learn how to teach a robot how to learn
  • #6
  • Jun 24, 2025 8:31am Jun 24, 2025 8:31am
I have stopped backtesting using the automatic feature in Metatrader. It is faulty for once. Another thing is, that I was curvefitting and it did not work in forward trading. So I forward test, which mean that it is a slow process. Therefore my approach is much more like only doing what seems right. And to this day, I have not found the combination that let the robot trade by itself. Each day is different - much different so I have not found the recipe that work in all conditions not even filtering them away. So I watch the market and whenever I find something interesting, I put on a robot to trade it for me.

I have tried a lot. I have no count of the number of robots. Dependent on when you can call it a new robot or a twist, then I must be over 50, maybe 100. I have tried most indicators.

Filters I use for the time being is.

  1. Look left. Take the trade only if the close of bar is beyond the high/low of the last x bars. To prevent ranging conditions.
  2. Squeeze. Using Keltner/and Bollinger. Don't trade in rangings.
  3. MTF CCI.
  4. MTF MA's
  5. Supertrend entry
  6. Manual buttons to filter trades in directions to my liking.
  7. Hidden stoploss. To prevent spikes taking out my SL.
  8. Timeslot of day.
  9. Trading in direction of daily open bar.
  10. MACD filter.
  11. Max Split filter.

Exits

  1. Partial exits on behalf of different CCI values.
  2. MA exit.
  3. Supertrend exit
  4. $ exit per trade.
  5. Percent exit - then call it a day.
  6. Reward.

My robots use Money management risking a percentage to my liking.

All of these features can be chosen On or Off.

I do not have the slope filter for the time being. Tried it, but it gave nothing. I still think it is a good idea though. Specially using it in a RENKO setup. I tried that too. Need to do a second try.

I have a problem. I get new ideas all the time. Should use more time on the robots I make, in stead of hunting new ideas.

I have a mantra. If you can predict the coming bar at a higher timeframe, then you will be better off trading a lower timeframe in that direction. Therefore some kind of pattern recognission in the HTF will be a good idea. I still need to work on this.

MetaTrader 4 - so many advantages.
  • #7
  • Jun 24, 2025 9:09am Jun 24, 2025 9:09am
Quoting RiskFighter
I have stopped backtesting using the automatic feature in Metatrader. It is faulty for once. Another thing is, that I was curvefitting and it did not work in forward trading. So I forward test, which mean that it is a slow process. Therefore my approach is much more like only doing what seems right. And to this day, I have not found the combination that let the robot trade by itself. Each day is different - much different so I have not found the recipe that work in all conditions not even filtering them away. So I watch the market and whenever I...
I really like that idea of somehow training the robot to keep a log of 60-1 minute candles to try to predict What the next candle would be and then filter out noise according to that. I'm definitely been taking a deep dive into more obscure indicators that is hard for us to follow, but easy for robots to pick up patterns with. I don't know I'm still new at this but I really enjoyed dabbling around with them. I like to train over 6 months and then back test over. Completely different 6-months data but the same instrument. It's just an out of sample test. It kind of gives me an idea of what I might be seeing when I forward test. Like I said earlier my first forward test I lost like $2,000 and seconds
Trying to learn how to learn how to teach a robot how to learn
  • #8
  • Jun 24, 2025 9:09am Jun 24, 2025 9:09am
And even if this amounts to nothing, I'm still learning and enjoying myself and having fun
Trying to learn how to learn how to teach a robot how to learn
  • #9
  • Jun 24, 2025 11:38am Jun 24, 2025 11:38am
Quoting Webb1003
{quote} I really like that idea of somehow training the robot to keep a log of 60-1 minute candles to try to predict What the next candle would be and then filter out noise according to that. I'm definitely been taking a deep dive into more obscure indicators that is hard for us to follow, but easy for robots to pick up patterns with. I don't know I'm still new at this but I really enjoyed dabbling around with them. I like to train over 6 months and then back test over. Completely different 6-months data but the same instrument. It's just an out...
And that is where you are in another level than me. I look very much forward to hear how you will be doing that.
MetaTrader 4 - so many advantages.
  • #10
  • Jun 24, 2025 12:56pm Jun 24, 2025 12:56pm
Speaking about filters, does anyone have a sample of trades by one indicator and by combining n of them? Let's say 1000 trades EMA cross vs 1000 Supertrend+EMA Filter, random indicator settings. Are they stat.different?
Air force
  • #11
  • Jun 24, 2025 1:26pm Jun 24, 2025 1:26pm
Quoting RiskFighter
{quote} And that is where you are in another level than me. I look very much forward to hear how you will be doing that.
We'll Since I read that I've been pondering how to do so. I'm thinking maybe something like an extra listener script.
The way my scripts work now is I have the charts being fed from a signal.rs node to my visual studio. It's just printing numbers every tick. Date-time-high-low-open-close-volume. Then in another terminal window I have a script called listener(the ML ears I guess you could say) that is constantly reading that and determining either 2 for buy 1 for hold or 0 for sell.
What if maybe there was another listener script running with the first one that puts together an hours worth of data, and after that one hour it takes its best guess and the likely hood of what the next candle will be given chart patterns or moving averages or something... I'm not sure right now. This will definitely be a trial and error one but I LOVE the idea lol.

Man if only I could hook up claude.ai or chat gpt to a listener script lol.

Oh! No! Have two listener scripts. One with chat gpt and one with Claude, and have them debate each other on the next possible outcome for candles...

Man...
You Got me all thinkin n' shit.
Trying to learn how to learn how to teach a robot how to learn
  • #12
  • Edited 7:50pm Jun 24, 2025 7:32pm | Edited 7:50pm
Quoting Webb1003
{quote} We'll Since I read that I've been pondering how to do so. I'm thinking maybe something like an extra listener script. The way my scripts work now is I have the charts being fed from a signal.rs node to my visual studio. It's just printing numbers every tick. Date-time-high-low-open-close-volume. Then in another terminal window I have a script called listener(the ML ears I guess you could say) that is constantly reading that and determining either 2 for buy 1 for hold or 0 for sell. What if maybe there was another listener script running...
I am 70 years old. I am an Engineer. Have been living a life that was loaded with technique. I have decided, that my otium shall be peacefull without have to use my brain on new techniques and therefore I am not using ChatGpt or anything like it. My code is made from the clean sheet and fully by my own brain. And all the functionallity I have in the robot mentioned in post #6 is done on only 1145 lines of MQL code. Each and every letter comes from my fingertips.

So I don't know, but I guess that these robots are fed using a text field. Well, you can put in the listening data to that text field and have ChatGpt teewing on it. Or not???

Or else....... It is many many years ago that I was any good on the processor technique. Last processor I played with was the 386 - I am old!! But the heart of the processor is still vector based I presume. I made a code at some point which moved the Vector in the operating system to a piece of code that I made which allowed me to mess with the data from the keyboard. Such a code should be able to insert a code from your system in stead of that from the keyboard. Just a wild idea. But it needs an expert.
MetaTrader 4 - so many advantages.
  • #13
  • Jun 24, 2025 7:58pm Jun 24, 2025 7:58pm
Yes - my first PC had a 8086 cpu. Got it in 1980. But before that I had a Sinclair computer.
MetaTrader 4 - so many advantages.
  • #14
  • Jun 24, 2025 9:12pm Jun 24, 2025 9:12pm
Quoting RiskFighter
Yes - my first PC had a 8086 cpu. Got it in 1980. But before that I had a Sinclair computer.
Ah the good old days.
..I stared with ist computer in 1982 which was a challenge for an older person.
Ronnie Renko
  • #15
  • Jun 25, 2025 6:08am Jun 25, 2025 6:08am
Quoting RiskFighter
{quote} I am 70 years old. I am an Engineer. Have been living a life that was loaded with technique. I have decided, that my otium shall be peacefull without have to use my brain on new techniques and therefore I am not using ChatGpt or anything like it. My code is made from the clean sheet and fully by my own brain. And all the functionallity I have in the robot mentioned in post #6 is done on only 1145 lines of MQL code. Each and every letter comes from my fingertips. So I don't know, but I guess that these robots are fed using a text field. Well,...
I bow to you sir. Lol
I am an no way at any level greater than you. Just a level that's measured different as I see it. You have knowledge, know how, you can do this stuff with your brain. I cannot. I am what you would call opportunistic,(quick back story-> I'm 38 years, have 3 kids and a wife, plus a full time job. I do NOT have the time to put in to actually learn this stuff, I did learn some coding a few years back which is the foundation of what helps me out here with these new fancy tools). Ok, back to opportunistic stuff- when I say new fancy tools, I mean the AI. It's 100% a tool to be used with stuff like this, with people like me, and the lack of full blown knowledge when it comes to all this coding and ML things. Just so happens that AI came out and proved to be useful at a time in my life that I have a little knowledge of coding but huge ambitions for me and my family paired with the knowledge I have of trading. That's what I mean by a different level. I have gotten rather good at prompting the AI in very specific ways to get the outcome that I desire. It DOES take practice, same way hitting a nail with a hammer every time and not your thumb does. It's a tool, very useful tool if used correctly. You can use a hammer to build a house, you can also use a hammer to destroy a house. The AI tool can easily be used incorrectly and make your time completely wasted..
I use the AI to code everything I want coded. With my little background of coding I can see where errors are and how to relate those errors to the AI so it will update the code snippets and fix them. That's pretty much my back and forth along with it probably saves me countless hours and days doing it this way. I coded my first, in it's entirety, a ML PPO bot, trained and backtested, along with implementing API features so it reads charts and predicts the next potential move, all in one saturday while holding my baby in my lap cause she was not feeling well and just wanted to chill.
I'm NOT bragging, I'm just laying out an example of how simple the Tool can make these things for someone like ME who does NOT have the type of actual knowledge and skill sets that YOU have in your brain. I WISH I had your level of knowledge with this stuff. Fact is I don't, and I must make do with what I got.

Anyways back to my machine learner thing.
On my docket today after work I have to change the way my bot reads the data. I realized after losing 2000$ in a couple minutes my issue.
I trained the bot on 1 min historic chart data, but my bot reads tick by tick data. It was trained on completely different data than what I'm receiving.
Huge problem of course. Simple fix though I am assuming.
Instead of it reading every tick. I'm gonna make a sub folder that stores 1 min worth of all this data. At the very second the 1 min starts it will log whatever that price was for the open. Throughout the minute whatever the highest price reached will be the high, same with the low, and whatever the last price is on the very last micro second of that minute will be the close. It will accumulate all the volume t received per tick and add it together to give me my full 1 min bar data. After that 1 min bar is logged, the bot will start making decisions. I have a 50 x 1 min bars dwell time at the beginning. So the bot will wait for at least 50 minutes before starting to make any decisions so I can attempt to get a feel of what the market is doing.
I hope.
Haha fun stuff.

Good ol' days to me is my compact computer my parents bought in the late 90s. I used to put AOL free 1hour of internet in so I could download 1 SONG. lol. Them the good ol' days to me.

That's the gist of it pretty much

To one of your questions about putting the chart readings into GPT. First off there is a type of language barrier we have I've noticed. You know a lot of words that have to do with programming that I don't. Anyways I think what you were asking was if I could just load the chart data feed to the AI itself to make the decisions? Yes, I believe you can I just haven't figured out how to do that yet working on it. AI has limits on processing data. And I feel I'd blow that limit in 5 minutes if I was feeding it tick by tick NASDAQ futures chart data lmao. This is on my bucket list to try and figure out one day.
Right now I'm just enjoying this adventure of Machine learning. It's new to me, but I'm diving in head first and seeing where it will take me.
Problem is I've been reading some books and watching videos ECT ect, and it's led me to realize that my competition in this field are mathematicians, physicsts, PhD students with some crazy expertise in fields I could only dream of. I'm some dude with a 700$ laptop tryna make ends meet lol.
Trying to learn how to learn how to teach a robot how to learn
  • #16
  • Jun 25, 2025 8:31am Jun 25, 2025 8:31am
[quote=Webb1003;15289193]{quote} I bow to you sir. Lol I am an no way at any level greater than you. Just a level that's measured different as I see it. You have knowledge, know how, you can do this stuff with your brain. I cannot. I am what you would call opportunistic,(quick back story-> I'm 38 years, have 3 kids and

I know how it is. I had kids too and my wife worked evenings, so for many years I had an interest in trading but not enough time.

Your take on this is very interesting and refreshing. I really hope that somebody will join this thread and work with you. Let us know how it goes.
MetaTrader 4 - so many advantages.
  • #17
  • Jun 25, 2025 3:29pm Jun 25, 2025 3:29pm
Ok so I aggregated my tick by tick chart feed(right side) into 1 minute bars.
You can see on the left side now it's calculating the time open high low close and volume. It's taking every tick within that minute and jotting down the open price, saving the highest achieved price and the lowest achieved price along with the very last tape for the close. It accumulated all the volume into volume for that 1 minute. Now that is what my bot was trained on so we'll see how it goes.
I have the bot set to where it it weights 50 bars until it starts to make decisions. So we'll see what happens after the 51st bar.
Attached Image (click to enlarge)
Click to Enlarge Name: PXL_20250625_222637976.MP~2.jpg Size: 1.9 MB
Trying to learn how to learn how to teach a robot how to learn
  • #18
  • Jun 25, 2025 9:55pm Jun 25, 2025 9:55pm
Quoting Webb1003
Ok so I aggregated my tick by tick chart feed(right side) into 1 minute bars. You can see on the left side now it's calculating the time open high low close and volume. It's taking every tick within that minute and jotting down the open price, saving the highest achieved price and the lowest achieved price along with the very last tape for the close. It accumulated all the volume into volume for that 1 minute. Now that is what my bot was trained on so we'll see how it goes. I have the bot set to where it it weights 50 bars until it starts to make...
So now you have the bars OCHL AND the volume. The volume is very interesting. One thing to remember though about volume is, that you are messuring the volume at your brokers data - nothing more. So volume might be very different at another broker during the same bar. But still a nice parameter to have.
MetaTrader 4 - so many advantages.
  • #19
  • Jun 26, 2025 1:29am Jun 26, 2025 1:29am
Quoting Webb1003
ML PPO Bots, Hidden Gem Indicators, and Everything In Between Hey everyone — I'm starting this thread as a place to talk about ML-powered trading bots, specifically ones trained with PPO (Proximal Policy Optimization). I’ve been building and testing bots that use machine learning to learn from price/volume behavior and actually trade like they’ve got a brain — not just follow basic signals. This is for anyone who's into: Reinforcement learning (PPO, DQN, A2C, etc.) Custom indicator ideas — especially ones that aren’t already used by everyone Strategy...
Hi Webb1003,
you may have a look at https://www.forexfactory.com/thread/...mini-flash-api
maybe you can support each other

Mucky
  • #20
  • Jun 26, 2025 1:35am Jun 26, 2025 1:35am
Quoting RiskFighter
Yes - my first PC had a 8086 cpu. Got it in 1980. But before that I had a Sinclair computer.
... and Yes, I started with an Commodore CBM 8032 Computer at school (6502 8Bit CPU) :-)

sorry for the disruption here

Mucky
0 traders viewing now