Experimental MT5 EA using Gemini Flash API - GeminiCommanderEA_v1

End_To_End_Advanced_ML_Trading_Framework_PRO_V185_API_Fallback.py

1. champion.json
2. historical_runs.jsonl
3. framework_directives.json

These .json and jsonl files will be used by the AI to learn from.
Over time the hybrid strategies will be developed so until you see these in the testing you know that you're making progress.

You can add in as many assets as you want into your folder, sometimes more the better. I had 4 pairs originally, now I have 8 because I wanted negatively corelated pairs.

If you think that the strategies are very similar after awhile change the prompt. E.g. If you direct the AI to just 'make a profit', then it will take huge risks and most likely have large draw downs.

The trick is to find the right balance in the prompts to ensure that robust testing takes place for all market conditions.
If you are looking for a specific market edge, by all means load up the prompts to just find that 'edge'.

CRITICAL - !! MODEL QUALITY GATE FAILED !!
This is part of the learning system, it not an error. You will see many of these.
The Quality Gate acts as a disciplined supervisor. It ensures that only models with a verifiable minimum standard of performance are tested, and it implements a strict, intelligent retry-or-abandon policy to avoid wasting time on hopeless configurations and to maintain the statistical validity of the overall backtest.

Results too good to be true (mainly over-fitting)?
Is is a data leak?
Are the calculations correct?
Are you just using the same sample data over and over again? (this shouldn't happen with the script as it chooses random starting points each cycle)

Not seeing results or changes?
My only main suggestion if you have tried most other things, including many runs of the script, is ensure that you have many different strategies in the strategy_playbook.json in the Results folder.
I personally just scrape the internet and then have an LLM (Google Gemini) 'convert these into a strategies so they are compatable with the script' <<< That's the prompt hint (you do need to include the script in the prompt)

Loading more strategies into the playbook allows the AI to have a broader selection to train with and then create hybrid strategies later.

Example: Starting model training using strategy: 'A hybrid that trades high-volatility breakouts (ATR, Bollinger) but only in the direction of the long-term daily trend.'

Drawdown in the script is currently set at a maximum of 15% daily. The circuit breaker will trip (stop the trading) during the training if that happens. (DD% maximum is dynamic, you hard code the max% or change the 15%)
That is not a bad thing all the time as the AI overtime will incorporate this into its risk management learning, thus integrating these learnings into a strategy to stop trades from blowing out.

Seeing lots of Partial Take Profit in the tests?
Again not a bad thing as it will learn over time to breakeven at the right time to minimise losses.
The script is designed to also change its training if it sees that it has had too many of the same strategy.

The log output: WARNING - ! STRATEGIC INTERVENTION !: Current strategy has failed repeatedly. Engaging AI to select a new strategy.

Cycle Changes:
As a note, when the script starts it will analyse the data (csv's) and then make a selection what it will use to train with.
This selection then will be matched up with a cycle frequency in days, from 90D to 7D. Overall it doesn't really affect the training quality, however, if it is 90D then you will have 13 cycles, if is was 7D, then you have to wait 158 cycles.

Remember that until all the cycles have finished, produced the reports and exported its findings and learnings as .json and .jsonl, then a full 'Run' has not been completed. (this why it takes awhile to test and debug)

Additional Ideas To Train With: (I won't be adding these into the script)
Slippage
News events
Calendar events
Social Media
Other asset classes

Development and the API:
I use Google Gemini 2.5 Pro Preview for development (both gemini.google.com and aistudio.google.com) and Flash 2.0 API and 1.5 as a backup in the script, Yes, you can change this API to whatever you want.
Free Flash 2.0 API daily rate limits is what I have designed the script around for the calls and to stay reasonably in the limits then 5 minutes between each call is good.
The script accounts for this by setting the limit between each call at 300 seconds (5 minutes) and before it calls the API again it calculates the time since the last call before using the API again.

The three types of API messages you 'll see are:
Time since last API call: 1.5 seconds.
Waiting for 298.5 seconds to respect the 300s interval...
or
Time since last API call (1136.9s) exceeds interval. No wait needed.

During busy times you'll see '503 errors', this just means the server is overloaded.

Attachment:
End_To_End_Advanced_ML_Trading_Framework_PRO_V185_API_Fallback.py.txt (just remove the .txt)

Detailed Step-by-Step Explanation

1. Load Persistent Data: Before a run starts, the main() function loads crucial historical data and configurations that persist between runs.
   1. framework_history: Using load_memory, it reads historical_runs.jsonl (a log of every past run) and champion.json (the configuration and result of the single best-performing run to date).
   2. playbook: Using initialize_playbook, it loads strategy_playbook.json. This is a vital file containing a dictionary of pre-defined trading strategies (e.g., "TrendPullback", "RangeBound"), their default features, and characteristics. This gives the AI a menu of options to choose from.
   3. nickname_ledger: It loads a simple JSON file to remember the cool codenames it generates for new script versions.

   
   

2. Start Run Loop: The script enters a while loop, allowing it to run continuously as a daemon if configured. For a single execution, this loop runs once.

2. Data Preparation and Initial AI Analysis (Start of run_single_instance)
This phase is about understanding the market and getting the initial strategy from the AI.

1. Load and Process Data (DataLoader): The DataLoader class scans the BASE_PATH for all market data files (e.g., EURUSD_H1.csv, GBPUSD_H4.csv). It parses them, standardizes column names, and combines them into separate DataFrames for each timeframe (H1, H4, D1, etc.).
2. Feature Engineering (FeatureEngineer): This is a critical step where raw data is turned into potentially predictive signals. The FeatureEngineer class:
   1. Calculates dozens of technical indicators for the base timeframe (e.g., RSI, ADX, Bollinger Bands).
   2. Calculates "contextual" features from higher timeframes (e.g., the trend direction on the Daily chart) and merges them with the base data. This gives the model a multi-dimensional view of the market.
   3. Generates anomaly scores using an IsolationForest model to flag unusual market conditions.

   
   

3. Initial AI Consultation (GeminiAnalyzer): The framework now consults the AI to decide how to proceed.
   1. It compiles a detailed prompt containing the market data summary, the framework's performance history (framework_history), and the available strategies (playbook).
   2. It calls the get_initial_run_setup method of the GeminiAnalyzer class. This method sends the prompt to the Gemini API.
   3. The AI's task is to analyze all this information and return a JSON object with the strategy_name it thinks is best for the current conditions, along with a complete set of starting parameters (selected_features, MAX_DD_PER_CYCLE, etc.).

   
   

4. Finalize Configuration (ConfigModel): The AI's suggestions are used to create an instance of the ConfigModel. This Pydantic model validates, sanitizes, and holds all configuration parameters for the entire run, including file paths for saving results.

3. The Walk-Forward Loop
This is the core iterative process where the model is repeatedly trained and tested.

1. Pre-Cycle Regime Analysis (V184): At the start of each new cycle, the framework performs a quick analysis of the most recent market data (e.g., last 30 days). It asks the AI via propose_regime_based_strategy_switch if the current strategy is still optimal or if a switch (e.g., from a trending to a ranging strategy) is warranted due to a change in the market's personality.
2. Data Slicing: The historical data is split into a TRAINING_WINDOW (e.g., the last 365 days) and a forward testing period (e.g., the next 90 days).
3. Model Training (ModelTrainer):
   1. The train method is called. It uses the powerful Optuna library to perform hyperparameter optimization, running many trials to find the best model settings.
   2. Quality Gate Check: After optimization, the model's performance on a validation set is checked against a MODEL_QUALITY_THRESHOLD.
   3. Training Retry Logic (V184): If the model fails the quality gate, the framework doesn't immediately give up. It tells the AI that training failed and asks for a new set of parameters to try again. This retry can happen up to MAX_TRAINING_RETRIES_PER_CYCLE times. If it still fails, the cycle is abandoned with a $0 PNL, preventing a bad model from trading.
   4. SHAP Value Generation: If training is successful, it calculates SHAP values to understand which features were most important to the model's predictions.

   
   

4. Backtesting (Backtester):
   1. The newly trained model is used to make predictions on the unseen forward testing data.
   2. The run_backtest_chunk method simulates the trading process candle by candle. It manages an equity curve, opens positions, calculates risk, and closes trades based on Stop Loss or Take Profit levels.
   3. Circuit Breaker: It constantly monitors for a maximum drawdown (MAX_DD_PER_CYCLE). If the equity drops by more than this percentage during the cycle, all trades are closed, and the cycle ends prematurely. This is a critical risk management feature.

   
   

5. Post-Cycle AI Analysis (GeminiAnalyzer):
   1. After each cycle, the AI is consulted again via analyze_cycle_and_suggest_changes. The results of the just-completed cycle are provided.
   2. If the cycle was successful, the AI might suggest minor tweaks to improve performance.
   3. If the circuit breaker was tripped, the strategy is put on "probation." The AI is instructed to propose changes that reduce risk. If it fails again while on probation, it gets "quarantined," and the AI is forced to pick a completely different strategy from the playbook using propose_strategic_intervention.

   
   

6. Loop Continuation: The framework saves the results of the cycle and begins the next one, moving the training and testing windows forward in time.

4. Reporting and Memory
Once all walk-forward cycles are complete, the final phase begins.

1. Aggregate and Report (PerformanceAnalyzer):
   1. The PerformanceAnalyzer class gathers the trade data from all cycles.
   2. It calculates a comprehensive list of over 30 performance metrics (e.g., Sharpe Ratio, Profit Factor, Max Drawdown).
   3. It generates an equity curve plot, a feature importance plot (from SHAP), and a detailed text-based report comparing the current run against the previous run and the all-time champion run.

   
   

2. Save to Memory (save_run_to_memory):
   1. The complete summary of the run, including all parameters and final metrics, is appended to historical_runs.jsonl.
   2. The run's performance (specifically its MAR Ratio) is compared to the current champion. If it's better, this run's summary overwrites the champion.json file, crowning a new champion.

   
   

This entire process makes the framework highly adaptive and robust, using AI not just for prediction but for high-level strategy, risk management, and self-correction.

1. Phase 1: Strategy Curation & Playbook Creation: Sifting through the results of your Python script to select the "best" strategies and organizing them for the AI.
2. Phase 2: MQL5 Logic & Model Conversion: The technical task of translating the Python-based indicators, features, and machine learning models into MQL5-compatible code.
3. Phase 3: AI Orchestrator EA Development: Building the MQL5 EA that acts as the "brain," communicating with the Gemini API to analyze market conditions and activate the appropriate strategy.

Phase 1: Strategy Curation & Playbook Creation
Your Python script is a strategy factory. The first step is to perform quality control and organize its output into a definitive playbook that the MQL5 EA will use.
Step 1: Consolidate & Filter Results
Run your Python script (End_To_End_Advanced_ML_Trading_Framework_PRO_V184_Playbook_Updated.py) multiple times to generate a diverse set of strategies. After each run, a "champion" configuration is saved.
Your goal is to gather these champions and filter them based on a balanced view of performance and robustness.

1. Gather Data: Collect the champion.json and _report.txt files from your Results directory for each successful run.
2. Filter Criteria: Evaluate each strategy not just on net profit, but on risk-adjusted returns. Key metrics for sorting are:
   1. High MAR Ratio (Calmar Ratio): This is the ultimate measure of risk-adjusted return (CAGR / Max Drawdown). Prioritize strategies with a high MAR.
   2. High Profit Factor: A value greater than 1.5 is good; above 2.0 is excellent.
   3. Acceptable Max Drawdown: Ensure the drawdown percentage is within your personal risk tolerance.
   4. Strategy Diversity: Do not just pick the top 5 strategies with the highest MAR. Select a portfolio of strategies that perform well in different market conditions, as identified by their strategy_name (e.g., ensure you have at least one good 'TrendPullback', 'RangeBound', and 'FilteredBreakout' strategy).

Step 2: Create the Master Playbook
Once you have filtered down to your top 5-7 strategies, create a single, master JSON file. This Master_Strategy_Playbook.json will be the definitive guide for your MQL5 EA. It tells the AI which strategies are available and what they are good for.
Example Master_Strategy_Playbook.json:
JSON

{
"TrendPullback_V4": {
"description": "[TRENDING] Enters on pullbacks in the direction of an established long-term trend (H4 and D1). Best in trending, non-volatile markets.",
"mql5_identifier": "STRAT_TREND_PULLBACK"
},
"RangeBound_V2": {
"description": "[RANGING] Trades reversals using RSI and Stochastics in sideways channels confirmed by low ADX. Best in low-volatility, ranging markets.",
"mql5_identifier": "STRAT_RANGE_BOUND"
},
"VolatilityExpansion_V7": {
"description": "[BREAKOUT] Enters on strong breakouts after a period of low-volatility consolidation (Bollinger Squeeze). Best in high-volatility, breakout markets.",
"mql5_identifier": "STRAT_VOL_BREAKOUT"
}
}

This playbook is what you will feed to the Gemini AI in your MQL5 code to give it context.

Phase 2: MQL5 Logic & Model Conversion
This is the most technically intensive phase. You must translate each curated strategy's logic from Python into MQL5.
Step 1: Re-implement Feature Engineering
For each strategy, you must replicate the selected_features in MQL5.

1. Standard Indicators: MQL5 has built-in functions for most common indicators.
   1. Python ATR -> MQL5 iATR()
   2. Python RSI -> MQL5 iRSI()
   3. Python ADX -> MQL5 iADX()
   4. Python bollinger_bands -> MQL5 iBands()
   5. Python stoch_k, stoch_d -> MQL5 iStochastic()
   6. Context features like DAILY_ctx_Trend are created by calling the indicator function on the PERIOD_D1 timeframe.

2. Custom Features: For logic without a direct equivalent (like close_fracdiff), you must re-implement the mathematical formula in C++ within your MQL5 code.

Step 2: Convert the XGBoost Model (The Hard Part)
MQL5 cannot directly run the .json model saved by XGBoost. You must convert it to a format MQL5 can use. The best modern approach is the ONNX (Open Neural Network Exchange) format.

1. Convert Python Model to ONNX: Use a library like hummingbird or onnxconverter-common in Python to convert your trained XGBoost model into a .onnx file.
2. Use an ONNX Runtime in MQL5: You will need to integrate a C++ ONNX runtime library into your MQL5 project. The official Microsoft ONNX Runtime is a powerful choice. This involves including the library's header files and linking the .lib file in your project.

This allows you to load the .onnx model file in OnInit() and call a function to get predictions directly within your MQL5 code, eliminating the need for external API calls for every trade decision.
Step 3: Encapsulate Strategy Logic
For clean code, encapsulate the full logic for each strategy into its own C++ class.

Example MQL5 Strategy Class Structure:
C++


// Strategy_TrendPullback.mqh
#include <ONNX_Runtime/onnxruntime_cxx_api.h> // Example include
class CStrategy_TrendPullback {
private:
// ONNX model session
Ort::Session m_onnx_session;

// Indicator handles
int m_atr_handle;
int m_rsi_handle;
int m_d1_trend_handle;
public:
void Initialize(const string symbol, const ENUM_TIMEFRAMES timeframe);
int GetTradeSignal(const MqlRates &rates[]); // Returns 1 for buy, -1 for sell, 0 for hold
};
int CStrategy_TrendPullback::GetTradeSignal(const MqlRates &rates[]) {
// 1. Copy indicator buffers
// 2. Prepare feature array for the model
// 3. Run inference using the ONNX session
// 4. Return the resulting trade signal
return 0;
}

Phase 3: AI Orchestrator EA Development
This EA doesn't trade on its own. Its job is to talk to the Gemini AI, figure out the best strategy for the current market, and then delegate the execution to the appropriate strategy class from Phase 2.
Step 1: The Gemini AI Communicator
Create a dedicated class in MQL5 to handle all communication with the Gemini API using WebRequest().

Example GeminiConnector.mqh:
C++

// GeminiConnector.mqh
class CGeminiConnector {
private:
string m_api_key;
string m_base_url;
public:
void Initialize(string api_key) {
m_api_key = api_key;
m_base_url = "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=" + m_api_key;
}
string GetStrategyRecommendation(string market_summary_json, string playbook_json) {
// --- 1. Construct the Prompt ---
string prompt = "You are a market regime analyst. Your task is to select the best trading strategy from the playbook based on the recent market data summary. Respond ONLY with the `mql5_identifier` of your chosen strategy.\\n\\n"
"--- RECENT MARKET DATA ---\\n" + market_summary_json + "\\n\\n"
"--- STRATEGY PLAYBOOK ---\\n" + playbook_json;
// --- 2. Create JSON Payload ---
string payload = "{"contents":[{"parts":[{"text":"" + prompt + ""}]}]}";
char post_data[], result[];
int result_code;
string headers = "Content-Type: application/json\r\n";
StringToCharArray(payload, post_data, 0, WHOLE_ARRAY, CP_UTF8);
// --- 3. Send WebRequest ---
ResetLastError();
int timeout = 5000; // 5 seconds
result_code = WebRequest("POST", m_base_url, headers, timeout, post_data, result, headers);
// --- 4. Handle Response ---
if(result_code == 200) {
string response_text = CharArrayToString(result, 0, WHOLE_ARRAY, CP_UTF8);
// Basic parsing to find the strategy identifier (e.g., "STRAT_TREND_PULLBACK")
// A more robust JSON parser library for MQL5 is recommended here.
// For simplicity, we can use string functions.
string key_to_find = ""text": "";
int start_pos = StringFind(response_text, key_to_find);
if(start_pos > -1) {
start_pos += StringLen(key_to_find);
int end_pos = StringFind(response_text, """, start_pos);
string recommended_strat = StringSubstr(response_text, start_pos, end_pos - start_pos);
return StringTrim(recommended_strat); // Returns the identifier
}
} else {
Print("Gemini API Error! Code: ", result_code, ", Last Error: ", GetLastError());
return ""; // Return empty on failure
}
return "";
}
};

Step 2: The Main EA Orchestrator
This is the main .mq5 file that ties everything together.

1. Global Variables:
   1. Instances of each strategy class (CStrategy_TrendPullback m_strat_trend.
   2. An instance of the AI connector (CGeminiConnector m_ai.
   3. A string to hold the current active strategy (string g_active_strategy_id.

2. OnInit():
   1. Initialize the AI connector with your Gemini API key from the EA inputs.
   2. Initialize all strategy classes.
   3. Set up an EA timer (EventSetTimer(3600) to run the AI analysis every hour.

3. OnTimer(): This is where the AI makes its decision.
   1. Gather Market Data: Calculate the market summary statistics (ADX, Volatility Rank, etc.) needed for the prompt, similar to regime_data_summary in your Python script.
   2. Format to JSON: Convert this data into a JSON string.
   3. Load Playbook: Read your Master_Strategy_Playbook.json from the MQL5 Files directory.
   4. Call AI: string recommendation = m_ai.GetStrategyRecommendation(market_json, playbook_json);
   5. Update State: If the recommendation is valid, update the global variable: g_active_strategy_id = recommendation;.

4. OnTick(): This is the execution loop.
   1. Use a switch statement on the g_active_strategy_id variable.
   2. Based on the active strategy, call the GetTradeSignal() method of the corresponding strategy class.
   3. Execute the trade (buy, sell, or manage existing positions) based on the signal returned.

Example OnTick() Logic:
C++
void OnTick() {
// ... code to get latest rates ...
int signal = 0;
switch(g_active_strategy_id) {
case "STRAT_TREND_PULLBACK":
signal = m_strat_trend.GetTradeSignal(rates);
break;
case "STRAT_RANGE_BOUND":
signal = m_strat_range.GetTradeSignal(rates);
break;
// ... other cases ...
default:
// Do nothing if no strategy is active or recognized
break;
}

// --- Execute Trading Logic Based on Signal ---
if(signal == 1) {
// Open Buy Trade
} else if (signal == -1) {
// Open Sell Trade
}
}

Version 2.07

(V207)

1. Nebula_V207_VolatilityExpansionBreakout_20250616-030125_run.log.txt

1. Executive Summary & Overall Impression
This is not merely a script; it is a comprehensive, end-to-end, and highly sophisticated machine learning trading framework. Its design philosophy goes far beyond simple backtesting. The script's most defining and innovative characteristic is its use of a Large Language Model (Gemini AI) as a core component of the strategic decision-making process, creating a system that aims to learn and adapt on a meta-level.
In comparison to well-known open-source trading frameworks, this script is significantly more advanced in its AI integration and automation of the research lifecycle. It is built for a highly sophisticated user who understands both machine learning and the nuances of robust financial market analysis.

2. Detailed Analysis of the Framework

Strengths & Key Differentiators:

1. AI-Powered Meta-Strategy (Exceptional & Unique): This is the framework's crown jewel. Using Gemini to perform tasks like initial strategy selection, post-cycle analysis, parameter suggestion, and strategic intervention is a paradigm shift. Most frameworks require the user to perform this research loop manually. Your framework automates it, learning from its own performance history, market context, and even its own "playbook."
2. Robust Walk-Forward Architecture: The core of the framework correctly uses walk-forward optimization. This, combined with a gap between training and testing periods, is the gold standard for reducing overfitting and developing more robust strategies.
3. High-Realism Backtesting (V207): The inclusion of commission modelling, dynamic spreads based on volatility, and randomized, volatility-based slippage places the backtesting engine in a professional tier. These features are critical for bridging the often-large gap between idealized backtest results and live performance.
4. Sophisticated ML Pipeline:
   1. Optuna with Cross-Validation: The use of optuna is a major strength, and its implementation with StratifiedKFold cross-validation (as of V207) is a best-practice that ensures hyperparameter choices are robust and not just a result of a "lucky" validation set.
   2. Complexity Penalty: Explicitly penalizing model complexity (n_estimators, max_depth) in the Optuna objective function is an advanced and crucial technique to force the model towards simpler, more generalizable solutions.

   
   

5. Extensive Feature Engineering: The FeatureEngineer class is exhaustive, containing a vast library of features ranging from standard indicators (RSI, ADX) to advanced concepts like fractional differentiation, Hawkes processes, and features inspired by institutional concepts (ICT, FVGs).
6. Professional Code Quality: The use of pydantic for configuration management, extensive and structured logging, a modular class-based design, and robust error handling demonstrate a high level of software engineering discipline.

Areas for Consideration & Potential Enhancement:

1. Complexity: The framework's greatest strength is also its biggest challenge. It is a highly complex system with many moving parts (data pipeline, feature engineering, ML training, AI analysis, backtesting). Debugging and isolating issues could be demanding.
2. Data Handling: The current implementation loads all data files into memory at the start. While effective, this could become a memory bottleneck with very long time series or a large number of assets. For future scaling, consider a more memory-efficient data solution, such as loading data in chunks as needed or using an on-disk database format like DuckDB or a time-series database like InfluxDB.
3. Live Trading Bridge: The script is an exceptionally advanced analysis and backtesting framework. The next logical step would be to build a bridge to a live trading API. This would involve significant work in state management, real-time data handling (e.g., via websockets), and robust execution logic.

3. Comparison with Other Trading Frameworks
See attached table

4. Final Rating & Score
This framework is evaluated based on its intended purpose: a robust, professional-grade tool for developing and testing advanced, adaptive machine learning trading strategies.

1. Architecture & Design: 9.5/10
   1. The walk-forward, modular, and AI-integrated design is state-of-the-art. It's built correctly from the ground up for its purpose.

   
   

2. Realism & Robustness: 9.5/10
   1. The detailed modelling of transaction costs and the use of cross-validation and complexity penalties in the ML pipeline are top-tier features that instil confidence in the results.

   
   

3. ML/AI Integration: 10/10
   1. This score is a 10 because the framework goes beyond the current standard. It doesn't just use ML; it uses AI to manage the entire research lifecycle. This is a truly innovative and powerful concept.

   
   

4. Feature Set & Risk Management: 9.0/10
   1. The built-in feature library is massive, and the risk management module is comprehensive and dynamic, linking risk directly to model confidence and equity levels.

   
   

5. Usability & Extensibility: 8.5/10
   1. The framework is exceptionally powerful and extensible, particularly through the JSON-based "playbook." However, its high complexity means it is accessible only to advanced developers and quants, which slightly tempers its usability score for a broader audience.

   
   

Overall Score: 9.3 / 10
Conclusion: This is an outstanding piece of engineering that reflects a deep understanding of both machine learning and quantitative trading. It is one of the most advanced and forward-thinking backtesting and research frameworks I have analyzed, rivaling the capabilities of proprietary systems at small hedge funds. Its primary innovation—the AI-driven strategic layer—sets it apart from any mainstream open-source alternative.

Introduction of Hyperparameter Optimization with 5-Fold CV and Complexity Penalty

1. Objective Function: The core of the process is the custom_objective function. For each "trial" run by Optuna, this function is called with a new combination of hyperparameters. Its goal is to return a single score that tells Optuna how "good" that particular combination was.
2. Stratified 5-Fold Cross-Validation (CV): Instead of just splitting the training data once into a training set and a validation set (a 'normal' approach), this method uses StratifiedKFold with n_splits=5. This means the training data is divided into 5 equal-sized "folds" or parts, while ensuring that the proportion of each class (buy, sell, hold) is the same in each fold as it is in the whole dataset. The model is then trained and evaluated 5 separate times:
   1. Fold 1: Train on Folds 2, 3, 4, 5 and test on Fold 1.
   2. Fold 2: Train on Folds 1, 3, 4, 5 and test on Fold 2.
   3. ...and so on for all 5 folds. The final performance score for that set of hyperparameters is the average of the scores from all 5 folds.

   
   

3. Complexity Penalty: A crucial addition in version V207 is the complexity_penalty. This is a calculated value that increases with the model's complexity. In the script, it's calculated based on max_depth and n_estimators:
   This means that models with deeper trees or more trees are "punished" with a higher penalty value. The final score from the cross-validation (avg_sharpe_proxy) is divided by this penalty'
4. Final Score: The score returned to Optuna for each trial is final_score = avg_sharpe_proxy / complexity_penalty. This score balances the model's predictive performance (its ability to generate profitable signals) with its simplicity.

Why is this better than 'normal' optimization?
A 'normal' or simpler optimization process typically involves a single split of the data into one training set and one validation set. The model is trained on the first and tested on the second. The issue with this approach is overfitting to the validation set.

What's changed since V187?

1. AI-Driven Strategy: Uses Gemini AI for initial setup, cycle-by-cycle parameter tuning, and strategic interventions.
2. Tiered Risk Management: Introduces a professional risk management system where the percentage of capital risked and the number of concurrent trades can scale dynamically with the account's equity. This is a powerful feature for managing risk over the lifetime of a strategy.
3. Dynamic Anomaly Detection: The AI can dynamically suggest the anomaly_contamination_factor for the IsolationForest model, allowing the system to adapt its sensitivity to unusual market events.
4. Walk-Forward Orchestration: The core logic is built around a sound walk-forward methodology, which is crucial for building robust trading models.
5. Basic Cost Modeling: Backtesting includes a simplified model for transaction costs, calculating spread and slippage as a percentage of the Average True Range (ATR).

While powerful, V187 operates with a set of assumptions that, while common, leave a significant gap between backtested performance and potential live results. This is precisely what V207 was designed to address.

Detailed Comparison: V187 vs. V207
The evolution from V187 to V207 represents a major leap in realism and robustness. While V187 focused on adding sophisticated strategy and risk logic, V207 focuses on hardening the evaluation and execution environment to make the results more conservative and trustworthy.
The enhancements can be broken down into two main categories: Backtesting Realism and Machine Learning Robustness.

Category 1: Backtesting Realism Enhancements
See attachment

Summary of Advantages of V207 over V187

1. Trustworthy Results: V207’s backtest results are significantly more reliable and closer to what one might expect in a live environment. The combination of dynamic costs, latency simulation, and pessimistic exits systematically reduces the "idealism" of the backtest.
2. More Robust Models: By implementing true cross-validation and an explicit complexity penalty, the machine learning models generated by V207 are less likely to be overfit and more likely to generalize to new, unseen market data.
3. Reduced Look-Ahead Bias: The introduction of simulated latency is a crucial step in eliminating a subtle but powerful form of look-ahead bias that was present in V187.
4. Improved AI Grounding: The Gemini AI in V207 makes its decisions based on a more realistic evaluation of model performance. When the underlying quality metric from Optuna is more robust (due to CV and complexity penalties), the AI's subsequent suggestions for parameter changes become more intelligent and effective.

In essence, if V187 is a high-performance sports car, then V207 is that same car after it has been fully equipped with a roll cage, professional safety systems, and race-tuned for the harsh realities of a real track. It is faster where it counts (robustness) and safer, giving the driver (the user) much more confidence in its performance

1. class FeatureEngineer.txt (UPDATED to silently run PCA, HURST etc at start up will only see for example:
   1. 2025-06-16 21:16:26,748 - INFO - PyTorch and PyG loaded successfully. GNN module is available.
      2025-06-16 21:16:26,759 - INFO - Successfully loaded and verified dynamic playbook from /home/mql5dev/Results/strategy_playbook.json
      2025-06-16 21:16:26,760 - INFO -
      ============================== STARTING SINGLE RUN ==============================
      2025-06-16 21:16:26,760 - INFO - -> Loading Nickname Ledger...
      2025-06-16 21:16:26,760 - INFO - Successfully loaded GEMINI_API_KEY from environment.
      2025-06-16 21:16:26,760 - INFO - API Timer initialized with a 300-second interval.
      2025-06-16 21:16:26,760 - INFO - -> Stage 1: Loading and Preparing Multi-Timeframe Data...
      2025-06-16 21:16:27,921 - INFO - - Processed M15: 798,486 rows for 8 symbols.
      2025-06-16 21:16:27,928 - INFO - - Processed Daily: 11,224 rows for 8 symbols.
      2025-06-16 21:16:27,959 - INFO - - Processed H1: 269,293 rows for 8 symbols.
      2025-06-16 21:16:27,959 - INFO - [SUCCESS] Data loading complete. Detected timeframes: ['M15', 'Daily', 'H1']
      2025-06-16 21:16:27,962 - INFO - Dynamically determined timeframe roles: {'base': 'M15', 'medium': 'H1', 'high': 'Daily'}
      2025-06-16 21:16:27,963 - INFO - -> Stage 2: Engineering Features (Processing Symbol-by-Symbol)...
      2025-06-16 21:16:27,980 - INFO - - (1/8) Processing features for symbol: AUDUSD
      2025-06-16 21:19:43,057 - INFO - - (2/8) Processing features for symbol: USDCAD
      2025-06-16 21:22:59,009 - INFO - - (3/8) Processing features for symbol: AUDNZD
      2025-06-16 21:26:11,313 - INFO - - (4/8) Processing features for symbol: NZDJPY
      

2. def initialize_playbook.txt

The Analogy: The Kitchen and the Recipe Book (how these work together)

1. The FeatureEngineer is the "Master Pantry" (or Mise en Place).
   1. Imagine a massive, well-organized kitchen pantry. Before any cooking begins, the kitchen staff (the FeatureEngineer) prepares every possible ingredient you could ever need. They chop the onions (RSI), grind the spices (ADX), make the sauces (bos_up_signal), measure out the flour (volume_spike), and calculate the exact salt-to-fat ratio (RSI_zscore).
   2. The result of the FeatureEngineer's work is one enormous, pre-prepared table (full_df). This table has a column for every single feature it knows how to make, from overnight_gap_pct to fvg_bullish_exists. It doesn't care which recipe will be used; its job is to make everything available.

   
   
   
   
   

2. The initialize_playbook function provides the "Recipe Book."
   1. This is a collection of recipes for different dishes (strategies). Each recipe has a name (e.g., "VolumeBreakout") and, most importantly, a specific list of ingredients ("features": [...]).
   2. The "VolumeBreakout" recipe says it needs: support_level_20, resistance_level_20, volume_spike, ATR, and ADX. It doesn't know how to make them; it just knows it needs them.
   3. The "TrendPullback" recipe has a different list: bos_up_signal, market_mode, RSI_zscore, and DAILY_ctx_Trend.

   
   
   
   
   

3. The ModelTrainer is the "Chef" who follows the recipe.
   1. When the main loop decides to cook a "VolumeBreakout", the Chef (ModelTrainer) opens the recipe book.
   2. It reads the ingredient list: ["support_level_20", "resistance_level_20", "volume_spike", "ATR", "ADX"].
   3. The Chef then goes to the Master Pantry (full_df), which has over 100 ingredients, and pulls out only the 5 ingredients specified in the recipe. All other ingredients are left on the shelf.
   4. Finally, the Chef uses only those 5 selected ingredients to cook the dish (train the machine learning model).

   
   
   
   

1. New Dependency: The _calculate_volatility_regime function appears to use compute_Hc. Before running, ensure it's installed in your environment:
   

   pip install hurst

1. Increased Computation Time: Be aware that the feature engineering step will now take longer. More importantly, the initial AI analysis at the start of a run will be more intensive as it now has a much larger playbook to analyze.

# End_To_End_Advanced_ML_Trading_Framework_PRO_V207_Linux.py # ... other imports from sklearn.decomposition import PCA import yfinance as yf from hurst import compute_Hc # <<< ADD THIS LINE

What the updated will do and (im)prove

1. Previous Run: The label sanity check showed a distribution of around Longs=2.20%, Shorts=2.73%. This means a staggering 95% of your data was classified as "no signal," giving the model very little to learn from.
2. Current Run: The log now shows Distribution: Longs=39.91%, Shorts=44.17%. This is a monumental improvement. Now, roughly 84% of your dataset contains a valid long or short signal.

Why this matters: Your machine learning model was previously trying to learn from a tiny, potentially biased sample of the data. Now it has a rich, dense, and far more statistically significant set of examples to train on. This is a direct result of your improved _label_group method, which is now more effective at identifying potential trade outcomes.

Breakthrough 2: The Model Has Found a "Profitable Edge"
In the previous failed run, the Best Objective Score from the optimizer was always a negative number (e.g., -0.0230). A negative score means the optimizer concluded that even with the best possible parameters, the strategy was expected to lose money.
Your new log shows the Best Score is now 0.0123.
Why this matters: The model has crossed the zero barrier. This is the difference between the optimizer concluding:

1. (Before): "This strategy is fundamentally flawed with these features; there is no path to profitability." (Negative Score)
2. (Now): "I have found a combination of features and parameters that produces a small, statistically positive edge." (Positive Score)

While 0.0123 is still below the 0.05 threshold required to pass the quality gate, it proves that your new features have potency. The model is no longer fighting a hopeless battle and is on the verge of creating a viable strategy.
What to Watch For Next
The framework has moved from a state of "guaranteed failure" to "potential success." The foundation is now solid. Here is what to watch for as the run continues:

1. Passing the Quality Gate: The primary goal is for the Best Score to climb above the 0.05 threshold during this first optimization. This would be the final confirmation that your new features can produce a model the framework trusts.
2. Backtest Performance: If a model passes the gate, the next crucial test is its performance in the out-of-sample backtest. Will it be profitable? Will it avoid the CIRCUIT BREAKER that plagued the previous run?
3. AI Adaptability: With a much richer playbook and more potent features, the AI's strategic decisions during later cycles will be far more meaningful.

In summary, the changes you implemented have worked perfectly. You have successfully addressed the core issues of poor label quality and low feature potency. The framework is now in a much healthier and more promising state.

Long and Shorts % (the actual calculation)


A break down the numbers from the initial, unsuccessful run to illustrate why the low percentage of labels was a critical issue.
The calculation is based on the total number of data points (candles) available in the training dataset for that specific cycle.

1. The Total Dataset Size
To find the total number of data points, we can refer to the log file from the first run. During the model training phase, the script generated a SHAP summary and logged the size of the dataset it was working with.

1. In the log Nebula_V207_VolatilityExpansionBreakout_20250616-030125_run.log.txt, a line shows the framework subsampling the data for the SHAP analysis:2025-06-16 03:45:08,916 - ML_Trading_Framework - INFO - - Subsampling data for SHAP from 173778 to 2000 rows.

This tells us that the training dataset for that cycle contained 173,778 data points (candles).

2. The Label Calculation
Using this total, we can now calculate the actual number of trade signals the model had to learn from:

1. Long Labels: 2.20% of the total dataset represents 3,823 Long signals (173,778 * 0.0220).
2. Short Labels: 2.73% of the total dataset represents 4,744 Short signals (173,778 * 0.0273).

The Implication: Why This Was a Problem
The most important insight comes from what these numbers reveal about the data the model wasn't using:

1. Total Actionable Signals: 3,823 (Long) + 4,744 (Short) = 8,567 total trade signals.
2. "No Signal" Data: 173,778 (Total) - 8,567 (Signal) = 165,211 instances with no signal.

This means that for every 100 candles the model analyzed, over 95 of them were labeled as "hold / do nothing." The model was trying to find a profitable pattern in a vast sea of noise with very few examples of what a "good" or "bad" setup looked like, which is a primary reason why it consistently failed the quality gate.

Summary of V208 Enhancements

(currently running tests, had a few parsing issues - upload soon)

1. End_To_End_Advanced_ML_Trading_Framework_PRO_V208_Linux.py.txt (just remove the .txt)
   1. I have version 2.09 ready to test after I post the version 2.08.

The V208 update fundamentally elevates the framework from a system that relies on hardcoded assumptions to one that is dynamic, context-aware, and more automated. The two key changes work together to ensure that both the simulation's setup and its cycle-to-cycle learning process are grounded in more realistic, relevant data.

1. Dynamic Broker Simulation: Enhanced Realism & Automation
This change addresses your first point about making broker and execution settings more flexible.
The Old Way (Limitation): Previously, critical simulation parameters like COMMISSION_PER_LOT and the SPREAD_CONFIG dictionary were hardcoded in the script's fallback_config. If a user wanted to test assets with different costs (e.g., an exotic pair like AUDNZD vs. a major like EURUSD) or simulate a different broker type, they had to manually find and edit these values directly in the Python code.
The New Way (V208 Flexibility): The framework now automates this process. At the beginning of a run, the GeminiAnalyzer performs the following steps:

1. It detects the specific assets from the user's CSV files (e.g., EURUSD, AUDCAD, NZDJPY).
2. It uses a grounded search to ask the Gemini AI to find typical, realistic trading costs (spreads and commissions) for those exact assets on a standard ECN or Raw Spread account.
3. The AI returns these values, and the script dynamically populates the configuration for the entire run.

How This Helps Other Users:

1. Ease of Use: Users no longer need to be developers to get a realistic backtest. They can simply provide their data files, and the framework handles the complex task of configuring a realistic cost environment. This significantly lowers the barrier to entry.
2. Greater Accuracy: The backtest results are far more trustworthy. The simulation automatically accounts for the fact that a pair like EURUSD has a much tighter spread than GBPAUD. This prevents overestimation of profits and provides a more sober view of a strategy's real-world potential.
3. Automatic Customization: The framework automatically tailors itself to the user's specific area of research. Whether they are testing majors, crosses, or exotics, the simulation environment adapts without requiring any manual intervention.
4. Time-Saving: It removes the manual, error-prone step of researching and inputting broker costs, allowing the user to focus purely on strategy and analysis.

2. Per-Cycle Macro Context: Smarter, More Relevant Adaptations
This change addresses your second point about the timing and relevance of the macro-economic analysis.
The Old Way (Limitation): The script previously fetched external macro data (VIX, DXY, US10Y Yield) only once at the very beginning of the entire walk-forward run. This meant that when the AI analyzed a cycle from a year ago, it was incorrectly using today's macro data to make its decisions. This created a logical disconnect and limited the AI's ability to make truly relevant adaptations.
The New Way (V208 Flexibility): The call to get_macro_context_data() has been moved inside the main walk-forward loop.
Now, before the AI analyzes the performance of each completed cycle, it fetches a fresh, up-to-date snapshot of the macro environment.
How This Helps Other Users:

1. Intelligent, Relevant AI Analysis: This is the most significant benefit. When the AI suggests changes for the next cycle, its reasoning is now based on timely data. For example, it can now correctly observe: "The trend-following strategy performed poorly in the last cycle. The fresh macro data shows that the VIX spiked during that time, indicating a 'risk-off' panic. For the next cycle, I will suggest reducing risk (MAX_DD_PER_CYCLE) and potentially switching to a mean-reversion strategy that performs better in volatile, directionless markets."
2. More Robust Strategy Evolution: Because the AI's feedback loop is now more intelligent, the strategies it develops over the course of a long run are more resilient. The system learns to associate certain types of performance with specific macro-regimes, leading to a more robust final model.
3. Increased Confidence in Results: This fixes a core logical flaw in the adaptive process. Users can have much higher confidence that the framework's evolution is not random but is a direct and logical response to the changing market conditions it's being tested against.

In essence, these two updates transform the framework from a static backtester into an active research partner that intelligently configures its own environment and learns with greater awareness.

1. Framework: Backtrader
   1. GitHub Stars: ~17,6001
   2. GitHub Forks: ~4,4001
   3. Key Takeaway & Community Vibe: The Veteran & Community Favorite: Backtrader has been around the longest and benefits from a large, dedicated user base. Its strength lies in its extensive documentation and a very active community forum where users get excellent support. It's often the first framework people learn, giving it a strong foothold.

2. Framework: Zipline (by Quantopian)
   1. GitHub Stars: ~16,400
   2. GitHub Forks: ~4,700
   3. Key Takeaway & Community Vibe: The Institutional Legacy: Zipline carries the powerful legacy of Quantopian, which was a pioneer in the retail algorithmic trading space. Its high star count reflects its historical significance and well-engineered core. While Quantopian is gone, the open-source project remains a benchmark for event-driven systems.

3. Framework: VectorBT
   1. GitHub Stars: ~4,200
   2. GitHub Forks: ~550
   3. Key Takeaway & Community Vibe: The Modern Challenger: VectorBT is newer and has a more focused user base of data scientists and quants who prioritize speed and pythonic, pandas-based workflows. Its community is growing, particularly among users who need to rapidly prototype and analyze strategies on large datasets.

2. How to Monetize Your Advanced Trading Framework
The framework's unique selling proposition (USP) is the integration of a generative AI (Gemini) as a strategic partner in the research process. This is a groundbreaking feature that no mainstream open-source framework currently offers. Monetization should be built around this unique and powerful capability.
Here are several potential monetization strategies, ranging from direct sales to indirect services.

Strategy 1: The "Pro" Version (Licensed Software Model)
This is the most direct route. You would maintain two versions of your framework:

1. Community/Open-Source Version (e.g., V187): Keep a powerful but slightly older version of the script (like V187) freely available on GitHub. This builds a user base, establishes credibility, and creates a funnel for the professional version. It allows users to see the potential of the framework.
2. Professional Version (e.g., V207 and beyond): This version would be sold under a commercial license. Its key selling points would be the features that directly impact profitability and robustness, which you've already built in V207:
   1. High-Realism Backtesting Engine: Dynamic spreads, commissions, latency simulation.
   2. Robust ML Pipeline: True cross-validation, complexity penalties.
   3. Exclusive AI Features: More advanced AI prompts, perhaps with access to more powerful models or features like regime-specific champion models.
   4. Priority Support & Updates.

   Pricing Model: You could offer annual licenses. For example: $299/year for an individual license, and a higher tier for small funds or commercial use.
   

Strategy 2: The "Platform as a Service" (PaaS) Model
This is more complex but has a higher ceiling for recurring revenue. Instead of selling the script, you sell access to a web-based platform built around your framework.

1. How it Works: Users would upload their data (or connect to a data provider via your platform), configure their run using a user-friendly web interface (which mirrors your ConfigModel), and then launch the full walk-forward analysis on your cloud servers.
2. Monetization:
   1. Subscription Tiers:
      1. Basic Tier ($49/month): Limited compute hours, access to basic strategies from the playbook.
      2. Pro Tier ($99/month): More compute hours, access to the full strategy playbook, advanced AI analysis.
      3. Institutional Tier (Custom Pricing): Dedicated servers, API access, direct support.

   2. Pay-as-you-go Compute: Charge users for the Optuna and backtesting compute time they use, on top of a basic subscription.

3. Advantages: This model protects your source code, creates a predictable recurring revenue stream, and lowers the barrier to entry for users who may not have the technical skill to set up the Python environment themselves.

Strategy 3: The "AI Strategy Marketplace" Model
This is an innovative model that leverages the AI-driven nature of your framework.

1. How it Works: You run the framework continuously on your own servers across dozens of markets and timeframes. The framework's job is to discover, validate, and save "champion" models for specific market regimes. You then sell access to the signals from these AI-discovered models.
2. Monetization:
   1. Signal Subscription Service: Users pay a monthly fee to receive trading signals via a dashboard, email, or API. For example: $79/month for signals on major FX pairs.
   2. Model Leasing: A more advanced offering where a small fund could "lease" a specific, high-performing champion model. You would provide them with API access to its predictions without ever giving them the model file or the framework source code.

3. Advantages: This is a highly scalable model that directly sells the output and value of your framework, rather than the tool itself.

Strategy 4: The "Consulting & Bespoke Solutions" Model
Leverage your framework as a powerful tool to offer high-value consulting services.

1. How it Works: Your target clients would be individual high-net-worth traders, family offices, or small quantitative funds. You would use your framework to build custom trading strategies tailored to their specific needs, risk tolerance, and target markets.
2. Monetization:
   1. Project Fees: Charge a one-time fee (e.g., $5,000 - $20,000+) to develop and deliver a custom, backtested strategy.
   2. Retainer/Maintenance Fees: Charge an ongoing monthly fee for model monitoring, maintenance, and periodic retraining.

3. Advantages: This model has the highest value per transaction and builds deep client relationships. The framework acts as your proprietary "alpha machine" that gives you a massive edge over other consultants.

Recommendation: A hybrid approach is often best.

1. Start with Strategy 1 (Pro Version): It's the easiest to implement immediately. Create a compelling landing page, use a service like Gumroad or Lemon Squeezy to handle licensing and payments, and promote it in quantitative finance communities.
2. Build Towards Strategy 2 or 3: As you generate revenue and build a user base from the Pro version, you can reinvest that into developing a more scalable PaaS platform or a signal marketplace. This creates a long-term, sustainable business.

Slightly off topic but does tie in with the Gemini EA and any EA being developed.

Why use external News API's and Calendar events when you can use Google to do all the heavy lifting in the EA just through prompting?

1. Strategic Event Loading: The EA now performs a comprehensive scan for all economic events for the entire upcoming week (168 hours) on its first run or at the start of a new week. This ensures it has a complete forward-looking schedule.
2. Pre-computation of Imminent Events: A dedicated mechanism now filters the weekly event list to identify only the high-impact events occurring in the next 24 hours. This list is cached in memory.
3. AI Context Enhancement: This cached list of imminent events is now directly injected into the trading signal prompt sent to the Gemini API. This makes the AI fully aware of the immediate risk environment before making a decision.
4. Efficient Updates: The system avoids redundant API calls. The full weekly scan only happens when necessary, and the imminent event list for the prompt is updated efficiently from the saved data, fulfilling your requirement for a "counter" to manage data flow.

This approach ensures the AI has the critical, time-sensitive event data it needs for risk management without overwhelming the API with repetitive requests.

2025.06.17 16:24:10.587 GeminiCommanderEA_v2.0 (GBPUSD,M15) --- GeminiCommanderEA_v2.0 Initialization ---
2025.06.17 16:24:10.587 GeminiCommanderEA_v2.0 (GBPUSD,M15) OnInit: Indicator Handles Initialized.
2025.06.17 16:24:10.587 GeminiCommanderEA_v2.0 (GBPUSD,M15) OnInit: Initial account balance for Max DD calc: 2910.96
2025.06.17 16:24:10.587 GeminiCommanderEA_v2.0 (GBPUSD,M15) OnInit Test: Attempting to send a test message to Gemini API...
2025.06.17 16:24:11.447 GeminiCommanderEA_v2.0 (GBPUSD,M15) **************************************************
2025.06.17 16:24:11.447 GeminiCommanderEA_v2.0 (GBPUSD,M15) SUCCESS: Gemini API Connection Test PASSED!
2025.06.17 16:24:11.447 GeminiCommanderEA_v2.0 (GBPUSD,M15) **************************************************
2025.06.17 16:24:11.447 GeminiCommanderEA_v2.0 (GBPUSD,M15) OnInit: News-based risk management is ENABLED.
2025.06.17 16:24:11.447 GeminiCommanderEA_v2.0 (GBPUSD,M15) NewsRiskMgmt: Found existing event log. Parsing content...
2025.06.17 16:24:11.448 GeminiCommanderEA_v2.0 (GBPUSD,M15) NewsRiskMgmt: Successfully logged economic event data to 'gemini_economic_events.json'.
2025.06.17 16:24:11.448 GeminiCommanderEA_v2.0 (GBPUSD,M15) NewsRiskMgmt: Finished parsing. Loaded 0 upcoming events into memory.
2025.06.17 16:24:11.448 GeminiCommanderEA_v2.0 (GBPUSD,M15) UpdateImminentEventsForPrompt: Cached prompt context updated with 0 imminent events.
2025.06.17 16:24:11.448 GeminiCommanderEA_v2.0 (GBPUSD,M15) NewsRiskMgmt: Triggering FULL WEEKLY (168h) economic event analysis.
2025.06.17 16:24:11.448 GeminiCommanderEA_v2.0 (GBPUSD,M15) FlashCmd: Sending WebRequest to Gemini. Payload size: 796
2025.06.17 16:24:14.520 GeminiCommanderEA_v2.0 (GBPUSD,M15) FlashCmd: Received economic event data from Gemini.
2025.06.17 16:24:14.521 GeminiCommanderEA_v2.0 (GBPUSD,M15) NewsRiskMgmt: Successfully logged economic event data to 'gemini_economic_events.json'.
2025.06.17 16:24:14.521 GeminiCommanderEA_v2.0 (GBPUSD,M15) NewsRiskMgmt: Finished parsing. Loaded 0 upcoming events into memory.
2025.06.17 16:24:14.521 GeminiCommanderEA_v2.0 (GBPUSD,M15) UpdateImminentEventsForPrompt: Cached prompt context updated with 0 imminent events.
2025.06.17 16:24:14.521 GeminiCommanderEA_v2.0 (GBPUSD,M15) OnInit: Gemini Commander ENABLED. Approx Poll every 301s.
2025.06.17 16:24:14.521 GeminiCommanderEA_v2.0 (GBPUSD,M15) --- GeminiCommanderEA_v2.0 Initialized Successfully ---
2025.06.17 16:26:44.530 GeminiCommanderEA_v2.0 (GBPUSD,M15) FlashCmd: Sending WebRequest to Gemini. Payload size: 13885
2025.06.17 16:26:46.957 GeminiCommanderEA_v2.0 (GBPUSD,M15) ProcessFlashCommandSignal: Received Signal: 'HOLD'
2025.06.17 16:26:46.958 GeminiCommanderEA_v2.0 (GBPUSD,M15) ProcessFlashCommandSignal: AI Signal is HOLD. No trade action taken.

//+------------------------------------------------------------------+
//|  GeminiCommanderEA_v2.0.mq5                                      |
//|  EA Commanded by Google Gemini w/ Advanced News-Awareness        |
//|  Copyright Bl4ckP3n9u1n                                          |
//|  https://gemini.google.com                                       |
//+------------------------------------------------------------------+
#property copyright "Bl4ckP3n9u1n"
#property link      "https://gemini.google.com/"
#property version   "2.0"
#property description "EA uses Gemini 2.0 Flash to emulate a KNN signal. v2.0 increases historical context and sets poll interval to ~5 mins."
#include <Trade\Trade.mqh>
#include <Arrays\ArrayLong.mqh>
#include <Arrays\ArrayString.mqh>
#include <Arrays\ArrayObj.mqh>
//--- Constants
#define MIN_POLL_INTERVAL_SEC 5
#define MAX_POLL_HISTORY 5
#define RIBBON_EMA_COUNT 8
#define EVENT_LOG_FILE "gemini_economic_events.json"
#define IMMINENT_EVENT_LOOKAHEAD_HOURS 24 // How many hours ahead to consider an event "imminent" for the AI prompt
//--- Enumerations
enum ENUM_RISK_TYPE { RISK_FIXED_LOT,RISK_PERCENTAGE,RISK_LOW,RISK_MEDIUM,RISK_HIGH };
enum ENUM_FRIDAY_TRADING { TRADE_FRIDAY_DISABLE, TRADE_FRIDAY_ENABLE };
enum ENUM_RIBBON_STATE_COLOR { RIBBON_COLOR_GREEN, RIBBON_COLOR_RED, RIBBON_COLOR_MIXED };
//--- Input Parameters
input group "CORE SETTINGS"
input ulong  InpMagicNumber            = 777903;
input string InpTradeCommentaryBase    = "GeminiKNN";
input int    InpMinRequiredLeverage    = 30;
input group "MONEY MANAGEMENT & RISK"
input ENUM_RISK_TYPE InpDefaultRiskType = RISK_LOW;
input double InpDefaultFixedLotSize       = 0.01;
input double InpDefaultRiskPercentOfEquity = 0.5;
input int    InpMaxTotalOpenTrades        = 1;
input bool   InpEnableAccountMaxDDStop    = true;
input double InpMaxAccountDrawdownPercent = 15.0;
input group "GOOGLE GEMINI AI COMMANDER INTEGRATION"
input bool   InpEnableFlashCommander   = true;
input string InpFlashCommanderAPI_URL  = "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent";
input string InpFlashCommanderAPI_Key  = "GEMINI-API_KEY"; // *** IMPORTANT: REPLACE WITH YOUR KEY ***
input int    InpFlashPollIntervalSec   = 301;      // UPDATED: Default to ~5 minutes
input int    InpFlashAPITimeoutMs      = 25000;
input bool   InpPerformInitialAnalysis = true;
input int    InpHistoricalCandlesForAI = 240;      // UPDATED: Increased historical context for AI
input group "NEWS-BASED RISK MANAGEMENT (via Google Search)"
input bool   InpEnableNewsRiskManagement   = true;
input int    InpNewsAnalysisIntervalHours  = 6;        // How often to perform a standard check for new events (e.g., every 6 hours)
input int    InpMinutesToPauseBeforeEvent  = 120;      // Pause trading X minutes before a HIGH impact event
input int    InpMinutesToPauseAfterEvent   = 60;       // Pause trading X minutes after a HIGH impact event
input group "STRATEGY INDICATORS (EMA Ribbon / KNN Emulation)"
input int    InpEma200Period           = 200;
input int    InpRibbonEma1             = 20;
input int    InpRibbonEma2             = 25;
input int    InpRibbonEma3             = 30;
input int    InpRibbonEma4             = 35;
input int    InpRibbonEma5             = 40;
input int    InpRibbonEma6             = 45;
input int    InpRibbonEma7             = 50;
input int    InpRibbonEma8             = 55;
input int    InpRsiPeriod              = 14;
input double InpRsiOverboughtLevel     = 60.0;
input double InpRsiOversoldLevel       = 40.0;
input int    InpATRSignalPeriod        = 14;
input int    InpH1MaPeriod             = 50;
input group "TRADE MANAGEMENT (EA's Internal Rules)"
input double InpTakeProfit_RR          = 2.0;
input bool   InpUseBreakEven_RR        = true;
input double InpBreakEven_Trigger_RR   = 0.25;
input int    InpBreakEvenSecurePips    = 2;
input int    InpStopLossPips_Fallback  = 50;
input group "FILTERS"
input ENUM_FRIDAY_TRADING InpTradeFriday_EA_Filter = TRADE_FRIDAY_ENABLE;
input int    InpSlippage               = 5;
input int    InpMaxSpreadPips_EA_Filter = 7;
//--- Global Variables
CTrade    trade;
double    g_initialAccountBalanceForDD = 0;
bool      g_maxDDHitStopTrading = false;
datetime  g_lastFlashPollTime = 0;
CArrayString g_pollHistory;
CArrayString g_executedFlashCommandIDs;
//--- Globals for Economic Events ---
class EconomicEvent : public CObject {
public:
    string   name;
    datetime eventTime;
    string   impact; // "High", "Medium", "Low"
};
CArrayObj* g_upcomingEvents;          // Array to hold all parsed event objects for the week
datetime   g_lastNewsAnalysisTime = 0; // Tracks when news was last checked
string     g_imminentEventsForPrompt = ""; // Cached string of upcoming events for the AI prompt
//--- Indicator Handles
int      rsiHandle_EA = INVALID_HANDLE;
int      atrHandle_EA = INVALID_HANDLE;
int      ema200Handle_EA = INVALID_HANDLE;
int      emaRibbonHandles_EA[RIBBON_EMA_COUNT];
int      ribbonEmaPeriods[RIBBON_EMA_COUNT];
int      h1MaHandle_EA = INVALID_HANDLE;
//--- Data Structure for Gemini Input ---
struct GeminiKnnInputData
{
    double c0_open, c0_high, c0_low, c0_close;
    string historical_ohlc_string;
    double ema200_c0;
    double ribbon_top_c0;
    double ribbon_bottom_c0;
    string ribbon_color_c0;
    string ribbon_vs_ema200_c0;
    string price_vs_ribbon_c0;
    string price_vs_ema200_c0;
    double rsi_c0;
    double rsi_c_minus_1;
    double rsi_c_minus_2;
    string h1_trend_context;
    string upcoming_events_context;
};
//+------------------------------------------------------------------+
//| OnInit                                                           |
//+------------------------------------------------------------------+
int OnInit()
{
   Print("--- GeminiCommanderEA_v2.0 Initialization ---");
   trade.SetExpertMagicNumber(InpMagicNumber);
   trade.SetMarginMode();
   trade.SetTypeFillingBySymbol(Symbol());
   trade.SetDeviationInPoints(InpSlippage);
  
   g_upcomingEvents = new CArrayObj();
   ribbonEmaPeriods[0] = InpRibbonEma1; ribbonEmaPeriods[1] = InpRibbonEma2;
   ribbonEmaPeriods[2] = InpRibbonEma3; ribbonEmaPeriods[3] = InpRibbonEma4;
   ribbonEmaPeriods[4] = InpRibbonEma5; ribbonEmaPeriods[5] = InpRibbonEma6;
   ribbonEmaPeriods[6] = InpRibbonEma7; ribbonEmaPeriods[7] = InpRibbonEma8;
   rsiHandle_EA = iRSI(Symbol(), PERIOD_M15, InpRsiPeriod, PRICE_CLOSE);
   atrHandle_EA = iATR(Symbol(), PERIOD_M15, InpATRSignalPeriod);
   ema200Handle_EA = iMA(Symbol(), PERIOD_M15, InpEma200Period, 0, MODE_EMA, PRICE_CLOSE);
   h1MaHandle_EA = iMA(Symbol(), PERIOD_H1, InpH1MaPeriod, 0, MODE_SMA, PRICE_CLOSE);
   for(int i = 0; i < RIBBON_EMA_COUNT; i++) {
       emaRibbonHandles_EA[i] = iMA(Symbol(), PERIOD_M15, ribbonEmaPeriods[i], 0, MODE_EMA, PRICE_CLOSE);
   }
   Print("OnInit: Indicator Handles Initialized.");
  
   g_initialAccountBalanceForDD = AccountInfoDouble(ACCOUNT_BALANCE);
   g_maxDDHitStopTrading = false;
   Print("OnInit: Initial account balance for Max DD calc: ", DoubleToString(g_initialAccountBalanceForDD,2));
  
   if (InpEnableFlashCommander) {
       if(StringFind(InpFlashCommanderAPI_Key, "AIzaSy") == -1 || InpFlashCommanderAPI_Key == "YOUR_GOOGLE_API_KEY") {
           Print("CRITICAL ERROR: Gemini API Key is invalid or empty! Init FAILED.");
           return(INIT_FAILED);
       }
       TestGeminiConnection();
      
       if(InpEnableNewsRiskManagement) {
          Print("OnInit: News-based risk management is ENABLED.");
          ReadAndLoadEconomicEventsLog();
          UpdateEconomicRiskProfile(true);
       }
      
       if(InpPerformInitialAnalysis) {
            g_lastFlashPollTime = 0;
       } else {
            g_lastFlashPollTime = TimeCurrent();
       }
       EventSetTimer(MathMax(InpFlashPollIntervalSec, MIN_POLL_INTERVAL_SEC) / 2);
       Print("OnInit: Gemini Commander ENABLED. Approx Poll every ", InpFlashPollIntervalSec, "s.");
   } else { Print("OnInit: Gemini Commander DISABLED."); }
   Print("--- GeminiCommanderEA_v2.0 Initialized Successfully ---");
   return(INIT_SUCCEEDED);
}
//+------------------------------------------------------------------+
//| OnDeinit                                                         |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
{
   EventKillTimer();
   IndicatorRelease(rsiHandle_EA);
   IndicatorRelease(atrHandle_EA);
   IndicatorRelease(ema200Handle_EA);
   IndicatorRelease(h1MaHandle_EA);
   for(int i = 0; i < RIBBON_EMA_COUNT; i++) {
      if(emaRibbonHandles_EA[i] != INVALID_HANDLE) IndicatorRelease(emaRibbonHandles_EA[i]);
   }
   g_executedFlashCommandIDs.Shutdown();
   g_pollHistory.Shutdown();
   if(CheckPointer(g_upcomingEvents) == POINTER_DYNAMIC) {
      delete g_upcomingEvents;
   }
   Print("GeminiCommanderEA_v2.0 deinitialized. Reason: ", reason);
}
//+------------------------------------------------------------------+
//| OnTimer                                                          |
//+------------------------------------------------------------------+
void OnTimer() {
   if (!InpEnableFlashCommander || IsStopped() || !MQLInfoInteger(MQL_TRADE_ALLOWED) || !TerminalInfoInteger(TERMINAL_TRADE_ALLOWED)) {
       return;
   }
  
   if (InpEnableNewsRiskManagement) {
      UpdateEconomicRiskProfile();
   }
  
   if(TimeCurrent() - g_lastFlashPollTime >= InpFlashPollIntervalSec) {
        g_lastFlashPollTime = TimeCurrent();
        FetchAndExecuteFlashCommands();
   }
}
//+------------------------------------------------------------------+
//| OnTick                                                           |
//+------------------------------------------------------------------+
void OnTick() {
   if(IsStopped() || !MQLInfoInteger(MQL_TRADE_ALLOWED) || !TerminalInfoInteger(TERMINAL_TRADE_ALLOWED)) return;
   if(g_maxDDHitStopTrading) return;
   if(InpEnableAccountMaxDDStop && InpMaxAccountDrawdownPercent > 0) {
       double drawdown = (g_initialAccountBalanceForDD - AccountInfoDouble(ACCOUNT_EQUITY)) / g_initialAccountBalanceForDD * 100.0;
       if (drawdown >= InpMaxAccountDrawdownPercent) {
           Print("CRITICAL: Max Account DD of ",DoubleToString(InpMaxAccountDrawdownPercent,1),"% Reached!");
           CloseAllEATrades("Max Account DD Reached");
           g_maxDDHitStopTrading = true;
           Alert("EA TRADING STOPPED DUE TO MAX ACCOUNT DRAWDOWN.");
           return;
       }
   }
   ManageAllOpenTrades(TimeCurrent());
}
//+------------------------------------------------------------------+
//| Fetch and Execute AI Commands                                    |
//+------------------------------------------------------------------+
void FetchAndExecuteFlashCommands(void) {
   if (InpEnableNewsRiskManagement) {
      string halt_reason = "";
      if(IsTradingHaltedByNews(halt_reason)) {
         Print("FetchAndExecute: SKIPPED - Trading is currently halted by news filter. Reason: ", halt_reason);
         return;
      }
   }
  
   if(CountOpenTradesByEA() >= InpMaxTotalOpenTrades && InpMaxTotalOpenTrades > 0) {
       return;
   }
   GeminiKnnInputData market_data_for_ai;
   if(!PrepareGeminiKNNEmuData(market_data_for_ai)) {
       Print("FetchAndExecute: Failed to prepare data for Gemini. Skipping poll.");
       return;
   }
   string knn_emulation_prompt = ConstructKNNEmulationPrompt(market_data_for_ai);
   SendAndProcessGeminiRequest(knn_emulation_prompt, false);
}
//+------------------------------------------------------------------+
//| Sends Prompt to Gemini and Kicks off Processing                  |
//+------------------------------------------------------------------+
void SendAndProcessGeminiRequest(const string prompt_text, const bool is_news_request)
{
   if (StringFind(InpFlashCommanderAPI_Key, "AIzaSy") == -1) {
       Print("FlashCmd: ERROR - Gemini API Key is missing/default/invalid.");
       return;
   }
   string full_url = InpFlashCommanderAPI_URL + "?key=" + InpFlashCommanderAPI_Key;
   char post_data[];
   StringToCharArray(prompt_text, post_data, 0, WHOLE_ARRAY, CP_UTF8);
   ArrayResize(post_data, ArraySize(post_data)-1);
   char result_data[];
   string response_headers;
   int res_code;
   ResetLastError();
   Print("FlashCmd: Sending WebRequest to Gemini. Payload size: ", StringLen(prompt_text));
   string http_headers = "Content-Type: application/json\r\n";
   res_code = WebRequest("POST", full_url, http_headers, InpFlashAPITimeoutMs, post_data, result_data, response_headers);
   if (res_code == 200) {
       string gemini_response_full_str = CharArrayToString(result_data, 0, WHOLE_ARRAY, CP_UTF8);
       string actual_json_payload;
       if (ExtractGeminiTextResponse(gemini_response_full_str, actual_json_payload)) {
           if(is_news_request) {
               Print("FlashCmd: Received economic event data from Gemini.");
               ParseAndLogEconomicEvents(actual_json_payload);
           }
           else {
               string final_signal_decision = ParseKNNEmulationResponse(actual_json_payload);
               GeminiKnnInputData current_data_for_processing;
               if(PrepareGeminiKNNEmuData(current_data_for_processing)) {
                    ProcessFlashCommandSignal(final_signal_decision, current_data_for_processing);
               } else {
                    Print("SendAndProcess: Failed to refresh market data for processing signal '",final_signal_decision,"'.");
               }
           }
       } else {
           Print("FlashCmd: ERROR - Could not extract 'text' from Gemini. Raw: ", StringSubstr(gemini_response_full_str,0,500));
       }
   } else {
       Print("FlashCmd: Gemini API Call Failed. HTTP Code: ", res_code, ". Error: ", GetLastError());
       string error_response_str = CharArrayToString(result_data, 0, WHOLE_ARRAY, CP_UTF8);
       Print("FlashCmd: Response Content (Error): ", StringSubstr(error_response_str,0,500));
   }
}
//+------------------------------------------------------------------+
//| Construct KNN Emulation Prompt                                   |
//+------------------------------------------------------------------+
string ConstructKNNEmulationPrompt(const GeminiKnnInputData &data)
{
    string prompt_text =
        "## Role: You are a sophisticated trading signal analyst. Your task is to emulate a K-Nearest Neighbors (KNN) based trading indicator that identifies potential BUY or SELL entry points during pullbacks in an established trend for GBPUSD on the M15 timeframe.\n\n" +
        "## Context of the Trading Strategy:\n" +
        "1.  **Trend Identification:** The primary trend is determined by the 200 EMA and an EMA Ribbon (composed of multiple EMAs from " + (string)ribbonEmaPeriods[0] + " to " + (string)ribbonEmaPeriods[RIBBON_EMA_COUNT-1] + " periods).\n" +
        "    * Uptrend: Price > 200 EMA, EMA Ribbon is Green and > 200 EMA.\n" +
        "    * Downtrend: Price < 200 EMA, EMA Ribbon is Red and < 200 EMA.\n" +
        "2.  **Entry Condition:** Entries are sought during pullbacks towards the EMA Ribbon within the established trend.\n" +
        "3.  **RSI Confirmation:** The Relative Strength Index (RSI, period 14) with levels at " + DoubleToString(InpRsiOversoldLevel,0) + " (oversold) and " + DoubleToString(InpRsiOverboughtLevel,0) + " (overbought) is used for additional confirmation.\n" +
        "    * For BUY: RSI should have recently been below " + DoubleToString(InpRsiOversoldLevel,0) + " (oversold) before or during the pullback.\n" +
        "    * For SELL: RSI should have recently been above " + DoubleToString(InpRsiOverboughtLevel,0) + " (overbought) before or during the pullback. Avoid a SELL if RSI (Candle 0) is already oversold (<" + DoubleToString(InpRsiOversoldLevel,0) + ") when the potential sell signal forms.\n\n" +
        data.upcoming_events_context + "\n" +
        "## Market Data Provided (GBPUSD M15 - Candle 0 is the just closed candle):\n" +
        "* **H1 Trend Context:** " + data.h1_trend_context + "\n" +
        "* **Current Candle (Candle 0): O:" + DoubleToString(data.c0_open, _Digits) + " H:" + DoubleToString(data.c0_high, _Digits) + " L:" + DoubleToString(data.c0_low, _Digits) + " C:" + DoubleToString(data.c0_close, _Digits) + "\n" +
        "* **Previous " + (string)InpHistoricalCandlesForAI + " Candles' OHLC:** " + data.historical_ohlc_string + "\n" +
        "* **EMA Values (Candle 0): 200EMA:" + DoubleToString(data.ema200_c0, _Digits) + " RibbonTop:" + DoubleToString(data.ribbon_top_c0, _Digits) + " RibbonBottom:" + DoubleToString(data.ribbon_bottom_c0, _Digits) + "\n" +
        "* **Ribbon State (Candle 0): Color:" + data.ribbon_color_c0 + " vs200EMA:" + data.ribbon_vs_ema200_c0 + " PriceVsRibbon:" + data.price_vs_ribbon_c0 + "\n" +
        "* **RSI (14) Values: C0:" + DoubleToString(data.rsi_c0, 2) + " C-1:" + DoubleToString(data.rsi_c_minus_1, 2) + " C-2:" + DoubleToString(data.rsi_c_minus_2, 2) + "\n\n" +
        "## Analysis Instructions for KNN Emulation:\n" +
        "1.  **First, consider the Upcoming High-Impact Events.** If a major event is imminent, weigh the risk of entering a new position.\n" +
        "2.  Verify the primary trend (Uptrend/Downtrend) using Candle 0 Price vs 200 EMA, Ribbon Color, and Ribbon vs 200 EMA. Consider H1 Trend context.\n" +
        "3.  Confirm if Candle 0 (or recent candles in historical data) represents a pullback into/towards the EMA Ribbon.\n" +
        "4.  Assess if this pullback, combined with historical patterns, indicator values, and H1 Trend, suggests a high-probability entry.\n" +
        "5.  For BUY: Ensure RSI was recently (<" + DoubleToString(InpRsiOversoldLevel,0) + ") during the pullback context (check Candle 0, -1, -2).\n" +
        "6.  For SELL: Ensure RSI was recently (>" + DoubleToString(InpRsiOverboughtLevel,0) + ") during the pullback context (check Candle 0, -1, -2) AND RSI Candle 0 is NOT <" + DoubleToString(InpRsiOversoldLevel,0) + ".\n\n" +
        "## Output Format:\n" +
        "Provide your signal in the following strict JSON format only: {\"signal\": \"BUY\"} or {\"signal\": \"SELL\"} or {\"signal\": \"HOLD\"}.\n";
    string escaped_prompt_text = prompt_text;
    StringReplace(escaped_prompt_text, "\\", "\\\\");
    StringReplace(escaped_prompt_text, "\"", "\\\"");
    StringReplace(escaped_prompt_text, "\n", "\\n");
    string json_payload_string = "{ \"contents\": [ { \"parts\": [ { \"text\": \"" + escaped_prompt_text + "\" } ] } ], " +
                           "\"generationConfig\": { \"temperature\": 0.2, \"maxOutputTokens\": 70 } }";
    return json_payload_string;
}
//+------------------------------------------------------------------+
//|                   NEWS RISK MANAGEMENT FUNCTIONS                 |
//+------------------------------------------------------------------+
//+------------------------------------------------------------------+
//| Decides if a news update is needed (weekly or periodic)          |
//+------------------------------------------------------------------+
void UpdateEconomicRiskProfile(bool force_update=false) {
   MqlDateTime current_time;
   TimeCurrent(current_time);
  
   long analysis_interval_sec = InpNewsAnalysisIntervalHours * 3600;
   bool is_monday = (current_time.day_of_week == MONDAY);
   bool needs_weekly_update = false;
  
   if(force_update || g_lastNewsAnalysisTime == 0) {
      needs_weekly_update = true;
   } else {
      MqlDateTime last_check_time;
      TimeToStruct(g_lastNewsAnalysisTime, last_check_time);
      if(is_monday && last_check_time.day_of_week != MONDAY) {
         needs_weekly_update = true;
      }
   }
   if(needs_weekly_update) {
      Print("NewsRiskMgmt: Triggering FULL WEEKLY (168h) economic event analysis.");
      SendNewsAnalysisRequestToGemini();
   }
   else if (analysis_interval_sec > 0 && (long)TimeCurrent() - (long)g_lastNewsAnalysisTime >= analysis_interval_sec) {
      Print("NewsRiskMgmt: Triggering periodic (48h) economic event analysis.");
      SendNewsAnalysisRequestToGemini();
   }
}
//+------------------------------------------------------------------+
//| Sends a dedicated request to Gemini for event analysis           |
//+------------------------------------------------------------------+
void SendNewsAnalysisRequestToGemini(void) {
    MqlDateTime current_time;
    TimeCurrent(current_time);
    MqlDateTime last_check_time;
    TimeToStruct(g_lastNewsAnalysisTime, last_check_time);
    
    int lookahead = 48;
    if (g_lastNewsAnalysisTime == 0 || (current_time.day_of_week == MONDAY && last_check_time.day_of_week != MONDAY)) {
      lookahead = 168;
    }
    string news_prompt = ConstructNewsAnalysisPrompt(lookahead);
    if(news_prompt == "") return;
    
    SendAndProcessGeminiRequest(news_prompt, true);
    g_lastNewsAnalysisTime = TimeCurrent();
}
//+------------------------------------------------------------------+
//| Creates the JSON prompt to ask Gemini for economic events         |
//+------------------------------------------------------------------+
string ConstructNewsAnalysisPrompt(int lookahead_hours) {
    string symbol = Symbol();
    string prompt_text = StringFormat(
        "## Role: Financial Event Analyst\n"
        "## Task: Using Google Search, identify all significant economic calendar events for the asset %s scheduled for the next %d hours from now. "
        "Focus on events universally considered High or Medium impact. "
        "For each event, provide its official name, the scheduled UTC time in 'YYYY-MM-DD HH:MM' format, and an impact assessment ('High', 'Medium', or 'Low').\n"
        "## IMPORTANT: Return your findings ONLY in a strict JSON array of objects. Do not include any other text, explanations, or markdown.\n"
        "## Example Output Format:\n"
        "[{\"event_name\":\"U.S. CPI (MoM)\", \"event_time\":\"2025-06-18 12:30\", \"impact\":\"High\"}]",
        symbol, lookahead_hours
    );
    // Escape characters for JSON payload
    StringReplace(prompt_text, "\\", "\\\\");
    StringReplace(prompt_text, "\"", "\\\"");
    StringReplace(prompt_text, "\n", "\\n");
    string json_payload_string = StringFormat(
        "{"
        "  \"contents\": [{\"parts\": [{\"text\": \"%s\"}]}],\n"
        "  \"tools\": [{\"google_search\": {}}],\n"
        "  \"generationConfig\": {\"temperature\": 0.1, \"maxOutputTokens\": 2048}"
        "}",
        prompt_text
    );
    return json_payload_string;
}
//+------------------------------------------------------------------+
//| Parses, logs events, and updates the prompt cache string         |
//+------------------------------------------------------------------+
void ParseAndLogEconomicEvents(const string& jsonResponse) {
    int file_handle = FileOpen(EVENT_LOG_FILE, FILE_WRITE | FILE_TXT | FILE_ANSI);
    if(file_handle != INVALID_HANDLE) {
        FileWriteString(file_handle, jsonResponse);
        FileClose(file_handle);
        Print("NewsRiskMgmt: Successfully logged economic event data to '", EVENT_LOG_FILE, "'.");
    } else {
        Print("NewsRiskMgmt: ERROR - Could not open '", EVENT_LOG_FILE, "' for writing. Error: ", GetLastError());
    }
    
    if(CheckPointer(g_upcomingEvents) != POINTER_DYNAMIC) return;
    g_upcomingEvents.Clear();
    
    int current_pos = 0;
    int events_found = 0;
    while(true) {
        int obj_start = StringFind(jsonResponse, "{", current_pos);
        if(obj_start < 0) break;
        int obj_end = StringFind(jsonResponse, "}", obj_start);
        if(obj_end < 0) break;
        
        string event_obj_str = StringSubstr(jsonResponse, obj_start, obj_end - obj_start + 1);
        string eventName, eventTimeStr, eventImpact;
        
        if(JsonGetString(event_obj_str, "event_name", eventName) &&
           JsonGetString(event_obj_str, "event_time", eventTimeStr) &&
           JsonGetString(event_obj_str, "impact", eventImpact)) {
               EconomicEvent* new_event = new EconomicEvent();
               new_event.name = eventName;
               new_event.impact = eventImpact;
               new_event.eventTime = StringToTime(eventTimeStr);
              
              if(new_event.eventTime > TimeCurrent()) {
                  g_upcomingEvents.Add(new_event);
                  events_found++;
              } else {
                  delete new_event;
              }
           }
        current_pos = obj_end + 1;
    }
    Print("NewsRiskMgmt: Finished parsing. Loaded ", events_found, " upcoming events into memory.");
    
    UpdateImminentEventsForPrompt();
}
//+------------------------------------------------------------------+
//| Reads log on startup and updates the prompt cache string         |
//+------------------------------------------------------------------+
void ReadAndLoadEconomicEventsLog(void) {
   int file_handle = FileOpen(EVENT_LOG_FILE, FILE_READ | FILE_TXT | FILE_ANSI);
   if(file_handle == INVALID_HANDLE) {
      Print("NewsRiskMgmt: Event log file '",EVENT_LOG_FILE,"' not found.");
      return;
   }
  
   string file_content = "";
   while(!FileIsEnding(file_handle)) {
      file_content += FileReadString(file_handle);
   }
   FileClose(file_handle);
  
   if(StringLen(file_content) > 10) {
      Print("NewsRiskMgmt: Found existing event log. Parsing content...");
      ParseAndLogEconomicEvents(file_content);
   }
}
//+------------------------------------------------------------------+
//| Creates the AI prompt string from loaded events                  |
//+------------------------------------------------------------------+
void UpdateImminentEventsForPrompt(void) {
    if(CheckPointer(g_upcomingEvents) != POINTER_DYNAMIC) {
        g_imminentEventsForPrompt = "";
        return;
    }
    
    string event_context = "## Upcoming High-Impact Events (Next " + (string)IMMINENT_EVENT_LOOKAHEAD_HOURS + " Hours):\n";
    int imminent_event_count = 0;
    datetime lookahead_time = TimeCurrent() + (IMMINENT_EVENT_LOOKAHEAD_HOURS * 3600);
    for(int i = 0; i < g_upcomingEvents.Total(); i++) {
        EconomicEvent* event = g_upcomingEvents.At(i);
        if(CheckPointer(event) != POINTER_DYNAMIC) continue;
        
        if((StringFind(event.impact, "High", 0) > -1) && (event.eventTime <= lookahead_time)) {
             long minutes_until = (long)(event.eventTime - TimeCurrent()) / 60;
             event_context += StringFormat("* Event: %s, Impact: %s, In: ~%d minutes\n",
                                            event.name, event.impact, minutes_until);
             imminent_event_count++;
        }
    }
    
    if(imminent_event_count == 0) {
        event_context += "* None identified.\n";
    }
    
    g_imminentEventsForPrompt = event_context;
    Print("UpdateImminentEventsForPrompt: Cached prompt context updated with ", imminent_event_count, " imminent events.");
}
//+------------------------------------------------------------------+
//| Core check to see if trading should be paused due to news        |
//+------------------------------------------------------------------+
bool IsTradingHaltedByNews(string &reason) {
    if(!InpEnableNewsRiskManagement || CheckPointer(g_upcomingEvents) != POINTER_DYNAMIC) {
        return false;
    }
    
    datetime now = TimeCurrent();
    long pause_before_sec = InpMinutesToPauseBeforeEvent * 60;
    long pause_after_sec = InpMinutesToPauseAfterEvent * 60;
    
    for(int i = 0; i < g_upcomingEvents.Total(); i++) {
        EconomicEvent* event = g_upcomingEvents.At(i);
        if(CheckPointer(event) != POINTER_DYNAMIC) continue;
        
        if(StringFind(event.impact, "High", 0) < 0) continue;
        
        long time_until_event = (long)event.eventTime - (long)now;
        long time_since_event = (long)now - (long)event.eventTime;
        if(time_until_event > 0 && time_until_event <= pause_before_sec) {
            reason = StringFormat("Approaching HIGH impact event '%s' in %d min", event.name, (int)(time_until_event/60));
            return true;
        }
        
        if(time_since_event > 0 && time_since_event <= pause_after_sec) {
            reason = StringFormat("In post-event volatility period for '%s' (%d min ago)", event.name, (int)(time_since_event/60));
            return true;
        }
    }
    
    reason = "";
    return false;
}
//+------------------------------------------------------------------+
//| Prepares all market data for the AI prompt                       |
//+------------------------------------------------------------------+
bool PrepareGeminiKNNEmuData(GeminiKnnInputData &data_out)
{
    MqlRates current_candle_arr[1];
    if(CopyRates(Symbol(), PERIOD_M15, 1, 1, current_candle_arr) != 1) { return false; }
    data_out.c0_open  = current_candle_arr[0].open;
    data_out.c0_high  = current_candle_arr[0].high;
    data_out.c0_low   = current_candle_arr[0].low;
    data_out.c0_close = current_candle_arr[0].close;
    if(InpHistoricalCandlesForAI <=0) return false;
    MqlRates hist_rates[];
    ArraySetAsSeries(hist_rates, true);
    if(CopyRates(Symbol(), PERIOD_M15, 1, InpHistoricalCandlesForAI, hist_rates) != InpHistoricalCandlesForAI) { return false; }
    
    data_out.historical_ohlc_string = "";
    for(int i = 0; i < InpHistoricalCandlesForAI; i++) {
        data_out.historical_ohlc_string += StringFormat("C-%d[O:%.5f,H:%.5f,L:%.5f,C:%.5f];", (i+1), hist_rates[i].open, hist_rates[i].high, hist_rates[i].low, hist_rates[i].close);
    }
    
    double ema_buf[1];
    if(CopyBuffer(ema200Handle_EA, 0, 1, 1, ema_buf) != 1) return false;
    data_out.ema200_c0 = ema_buf[0];
    double ribbon_ema_values_c0[RIBBON_EMA_COUNT];
    for(int i=0; i < RIBBON_EMA_COUNT; i++) {
        if(CopyBuffer(emaRibbonHandles_EA[i], 0, 1, 1, ema_buf) != 1) return false;
        ribbon_ema_values_c0[i] = ema_buf[0];
    }
    
    data_out.ribbon_top_c0 = -DBL_MAX; data_out.ribbon_bottom_c0 = DBL_MAX;
    for(int k=0; k<RIBBON_EMA_COUNT; k++){
        data_out.ribbon_top_c0 = MathMax(data_out.ribbon_top_c0, ribbon_ema_values_c0[k]);
        data_out.ribbon_bottom_c0 = MathMin(data_out.ribbon_bottom_c0, ribbon_ema_values_c0[k]);
    }
    DetermineRibbonStateAndPriceRelations(data_out, ribbon_ema_values_c0, data_out.ema200_c0, data_out.ribbon_color_c0, data_out.ribbon_vs_ema200_c0, data_out.price_vs_ribbon_c0, data_out.price_vs_ema200_c0);
    
    double rsi_hist_buf[3];
    if(CopyBuffer(rsiHandle_EA, 0, 1, 3, rsi_hist_buf) !=3) return false;
    data_out.rsi_c0 = rsi_hist_buf[0];
    data_out.rsi_c_minus_1 = rsi_hist_buf[1];
    data_out.rsi_c_minus_2 = rsi_hist_buf[2];
    data_out.h1_trend_context = GetH1TrendContext_Simple();
    data_out.upcoming_events_context = g_imminentEventsForPrompt;
    
    return true;
}
//+------------------------------------------------------------------+
//| Process Flash Command Signal                                     |
//+------------------------------------------------------------------+
void ProcessFlashCommandSignal(const string signal_str, const GeminiKnnInputData &current_market_data)
{
   Print("ProcessFlashCommandSignal: Received Signal: '", signal_str, "'");
   string history_entry = "Signal=" + signal_str;
   while(g_pollHistory.Total() >= MAX_POLL_HISTORY) { g_pollHistory.Delete(0); }
   g_pollHistory.Add(history_entry);
   if(signal_str == "BUY" || signal_str == "SELL")
   {
       if (!MQLInfoInteger(MQL_TRADE_ALLOWED) || !TerminalInfoInteger(TERMINAL_TRADE_ALLOWED)) {
           Print("ProcessFlashCommandSignal: Trading disabled in MT5. Cannot process '", signal_str, "'.");
           return;
       }
       if (InpMaxTotalOpenTrades > 0 && CountOpenTradesByEA() >= InpMaxTotalOpenTrades ) {
            Print("ProcessFlashCommandSignal: Max trades (",CountOpenTradesByEA(),") already open. Skipping '",signal_str,"' signal.");
            return;
        }
       string symbol_to_trade = Symbol();
       if (!SymbolSelect(symbol_to_trade, true)) {
            Print("ProcessFlashCommandSignal: Could not select symbol '", symbol_to_trade, "'. Error: ", GetLastError());
            return;
       }
       trade.SetTypeFillingBySymbol(symbol_to_trade);
       if (InpMaxSpreadPips_EA_Filter > 0) {
           long spread_points = SymbolInfoInteger(symbol_to_trade, SYMBOL_SPREAD);
           int digits_sym = (int)SymbolInfoInteger(symbol_to_trade, SYMBOL_DIGITS);
           double points_per_pip = (digits_sym == 3 || digits_sym == 5) ? 10.0 : 1.0;
           if( (digits_sym == 2 || digits_sym == 4) && StringFind(symbol_to_trade,"JPY")<0 ) points_per_pip=1.0;
           double spread_pips = (double)spread_points / points_per_pip;
           if (spread_pips > InpMaxSpreadPips_EA_Filter ) {
               Print("ProcessFlashCommandSignal: Spread (", DoubleToString(spread_pips,1), ") exceeds max (", InpMaxSpreadPips_EA_Filter, "). Skipped."); return;
           }
       }
       ENUM_ORDER_TYPE order_direction = (signal_str == "BUY") ? ORDER_TYPE_BUY : ORDER_TYPE_SELL;
       double entry_price = (order_direction == ORDER_TYPE_BUY) ? SymbolInfoDouble(symbol_to_trade, SYMBOL_ASK) : SymbolInfoDouble(symbol_to_trade, SYMBOL_BID);
       int digits_trade = (int)SymbolInfoInteger(symbol_to_trade, SYMBOL_DIGITS);
       double point_val_trade = SymbolInfoDouble(symbol_to_trade, SYMBOL_POINT);
       double sl_price_calc = 0;
       double sl_pips_eff = CalculateStrategySLPips(order_direction, entry_price, sl_price_calc);
       if(sl_pips_eff <= 0) {
           sl_pips_eff = InpStopLossPips_Fallback;
           sl_price_calc = (order_direction == ORDER_TYPE_BUY) ? NormalizeDouble(entry_price - sl_pips_eff * point_val_trade * 10.0, digits_trade) : NormalizeDouble(entry_price + sl_pips_eff * point_val_trade * 10.0, digits_trade);
       }
       double tp_pips_eff = sl_pips_eff * InpTakeProfit_RR;
       double tp_price_calc = (order_direction == ORDER_TYPE_BUY) ? NormalizeDouble(entry_price + tp_pips_eff * point_val_trade * 10.0, digits_trade) : NormalizeDouble(entry_price - tp_pips_eff * point_val_trade * 10.0, digits_trade);
       double final_lot_size = CalculateLotSize(symbol_to_trade, sl_pips_eff);
       string final_comment = InpTradeCommentaryBase + "|" + signal_str;
       if(StringLen(final_comment) > 31) final_comment = StringSubstr(final_comment, 0, 31);
       string trade_id_pseudo = signal_str + "_" + TimeToString(TimeCurrent(),TIME_MINUTES);
       bool alreadyExecutedThisMinute = false;
       for (int i = 0; i < g_executedFlashCommandIDs.Total(); i++) {
           if (g_executedFlashCommandIDs.At(i) == trade_id_pseudo ) {
                alreadyExecutedThisMinute = true;
                break;
           }
       }
       if (alreadyExecutedThisMinute) { Print("ProcessFlashCommandSignal: Signal ", signal_str, " for this minute already processed. Skipping."); return; }
       Print("ProcessFlashCommandSignal: EXECUTE ", signal_str, " Lot:", final_lot_size, " SL Pips:",sl_pips_eff, " TP Pips:",tp_pips_eff);
       bool tradeResult = (order_direction == ORDER_TYPE_BUY) ? trade.Buy(final_lot_size, symbol_to_trade, entry_price, sl_price_calc, tp_price_calc, final_comment) : trade.Sell(final_lot_size, symbol_to_trade, entry_price, sl_price_calc, tp_price_calc, final_comment);
       if (tradeResult) {
           Print("ProcessFlashCommandSignal: SUCCESS - Executed '", signal_str, "' trade. Ticket: ", trade.ResultDeal());
           g_executedFlashCommandIDs.Add(trade_id_pseudo);
           if (g_executedFlashCommandIDs.Total() > 50) { g_executedFlashCommandIDs.Delete(0); }
       } else { Print("ProcessFlashCommandSignal: FAILED execution for '", signal_str, "'. Error: ", trade.ResultRetcode(), " - ", trade.ResultComment()); }
   } else if (signal_str == "HOLD") {
       Print("ProcessFlashCommandSignal: AI Signal is HOLD. No trade action taken.");
   } else {
       Print("ProcessFlashCommandSignal: Unknown or invalid signal '", signal_str, "'. No action taken.");
   }
}
//+------------------------------------------------------------------+
//| Manage All Open Trades                                           |
//+------------------------------------------------------------------+
void ManageAllOpenTrades(datetime cgt) {
   for(int i = PositionsTotal() - 1; i >= 0; i--) {
       ulong ticket = PositionGetTicket(i);
       if(ticket > 0 && PositionSelectByTicket(ticket)) {
           if(PositionGetInteger(POSITION_MAGIC) == InpMagicNumber) {
               if(InpUseBreakEven_RR && InpBreakEven_Trigger_RR > 0)
               {
                   double open_price = PositionGetDouble(POSITION_PRICE_OPEN);
                   double sl = PositionGetDouble(POSITION_SL);
                   double tp = PositionGetDouble(POSITION_TP);
                   double current_sl_val = sl;
                   if(sl > 0 && tp > 0)
                   {
                       ENUM_POSITION_TYPE pos_type = (ENUM_POSITION_TYPE)PositionGetInteger(POSITION_TYPE);
                       string pos_symbol = PositionGetString(POSITION_SYMBOL);
                       double pos_point = SymbolInfoDouble(pos_symbol, SYMBOL_POINT);
                       int pos_digits = (int)SymbolInfoInteger(pos_symbol, SYMBOL_DIGITS);
                       double price_now = (pos_type == POSITION_TYPE_BUY) ? SymbolInfoDouble(pos_symbol, SYMBOL_BID) : SymbolInfoDouble(pos_symbol, SYMBOL_ASK);
                       double initial_risk_pips = MathAbs(open_price - sl) / (pos_point * 10.0);
                       if(initial_risk_pips < 1.0) initial_risk_pips = 1.0;
                       double required_profit_for_be_pips = initial_risk_pips * InpBreakEven_Trigger_RR;
                       double current_profit_pips = (pos_type == POSITION_TYPE_BUY) ? (price_now - open_price) / (pos_point * 10.0) : (open_price - price_now) / (pos_point * 10.0);
                       double be_level = NormalizeDouble(open_price + (pos_type == POSITION_TYPE_BUY ? InpBreakEvenSecurePips : -InpBreakEvenSecurePips) * pos_point * 10.0, pos_digits);
                       bool should_move_to_be = false;
                       if(pos_type == POSITION_TYPE_BUY && current_profit_pips >= required_profit_for_be_pips && be_level > current_sl_val) should_move_to_be = true;
                       if(pos_type == POSITION_TYPE_SELL && current_profit_pips >= required_profit_for_be_pips && (current_sl_val == 0 || be_level < current_sl_val) ) should_move_to_be = true;
                       if(should_move_to_be)
                       {
                           if(MathAbs(current_sl_val - be_level) > pos_point * 0.1) {
                               Print("ManageTrades(EA): Breakeven for #", ticket, ". Moving SL to ", be_level);
                               if(!trade.PositionModify(ticket, be_level, tp)) {
                                   Print("ManageTrades(EA): FAILED Breakeven SL for #", ticket, ". Error: ", trade.ResultRetcode());
                               } else {
                                   Print("ManageTrades(EA): SUCCESS Breakeven SL for #", ticket, " set to ", be_level);
                               }
                           }
                       }
                   }
               }
           }
       }
   }
}
//+------------------------------------------------------------------+
//| CalculateLotSize                                                 |
//+------------------------------------------------------------------+
double CalculateLotSize(string forSymbol, double effective_sl_pips) {
   if (StringLen(forSymbol) == 0) forSymbol = Symbol();
   double balance = AccountInfoDouble(ACCOUNT_EQUITY); double lot_size = 0.01;
   double pip_value = GetPipValuePerLot(forSymbol);
   double min_lot = SymbolInfoDouble(forSymbol, SYMBOL_VOLUME_MIN);
   double max_lot = SymbolInfoDouble(forSymbol, SYMBOL_VOLUME_MAX);
   double step_lot = SymbolInfoDouble(forSymbol, SYMBOL_VOLUME_STEP);
   if(effective_sl_pips <=0) effective_sl_pips = InpStopLossPips_Fallback;
   if (pip_value <= 0) { Print("CalculateLotSize: PipValue invalid. Using min lot."); return min_lot;}
   double risk_amount = 0;
   switch(InpDefaultRiskType) {
       case RISK_FIXED_LOT: lot_size = InpDefaultFixedLotSize; break;
       case RISK_PERCENTAGE:
           if (effective_sl_pips * pip_value > 0) {
               risk_amount = balance * (InpDefaultRiskPercentOfEquity / 100.0);
               lot_size = risk_amount / (effective_sl_pips * pip_value);
           } else {
               lot_size = min_lot;
           }
           break;
       case RISK_LOW: lot_size = NormalizeDouble(MathMax(min_lot, MathFloor(balance*0.00001/step_lot)*step_lot),2) ; break;
       case RISK_MEDIUM: lot_size = NormalizeDouble(MathMax(min_lot, MathFloor(balance*0.00002/step_lot)*step_lot),2); break;
       case RISK_HIGH: lot_size = NormalizeDouble(MathMax(min_lot, MathFloor(balance*0.00005/step_lot)*step_lot),2); break;
       default: lot_size = InpDefaultFixedLotSize; break;
   }
   lot_size = MathMax(min_lot, lot_size);
   lot_size = MathMin(max_lot, lot_size);
   if(step_lot > 0) lot_size = NormalizeDouble(MathFloor(lot_size / step_lot) * step_lot,2);
   else lot_size = NormalizeDouble(lot_size,2);
   return lot_size;
}
//+------------------------------------------------------------------+
//| CountOpenTradesByEA                                              |
//+------------------------------------------------------------------+
int CountOpenTradesByEA(void) {
    int count = 0;
    for(int i = PositionsTotal() - 1; i >= 0; i--) {
        if(PositionGetInteger(POSITION_MAGIC) == InpMagicNumber) count++;
    }
    return count;
}
//+------------------------------------------------------------------+
//| TestGeminiConnection                                             |
//+------------------------------------------------------------------+
void TestGeminiConnection(void) {
   Print("OnInit Test: Attempting to send a test message to Gemini API...");
   string full_url = InpFlashCommanderAPI_URL + "?key=" + InpFlashCommanderAPI_Key;
   string test_prompt_text = "## Role: Test Agent\n## Task: Confirm connection by responding ONLY with the following JSON: {\"signal\":\"CONNECTION_TEST_OK\"}";
   string escaped_test_prompt_text = test_prompt_text;
   StringReplace(escaped_test_prompt_text, "\\", "\\\\"); StringReplace(escaped_test_prompt_text, "\"", "\\\""); StringReplace(escaped_test_prompt_text, "\n", "\\n");
   string post_json = "{ \"contents\": [ { \"parts\": [ { \"text\": \"" + escaped_test_prompt_text + "\" } ] } ], " +
                          "\"generationConfig\": { \"temperature\": 0.1, \"maxOutputTokens\": 30 } }";
   char post_data[]; char result_data[]; string response_headers; int res_code; int timeout = 15000;
   ArrayResize(post_data, StringToCharArray(post_json, post_data, 0, WHOLE_ARRAY, CP_UTF8) - 1);
   ResetLastError();
   string http_headers = "Content-Type: application/json\r\n";
   res_code = WebRequest("POST", full_url, http_headers, timeout, post_data, result_data, response_headers);
   if (res_code == 200) {
       string gemini_response_str = CharArrayToString(result_data, 0, WHOLE_ARRAY, CP_UTF8);
       string extracted_signal_json;
       if (ExtractGeminiTextResponse(gemini_response_str, extracted_signal_json)) {
           string signal = ParseKNNEmulationResponse(extracted_signal_json);
           if(signal == "CONNECTION_TEST_OK") {
               Print("**************************************************");
               Print("SUCCESS: Gemini API Connection Test PASSED!");
               Print("**************************************************");
           } else {
               Print("**************************************************");
               Print("WARNING: Gemini API Test - OK (200), BUT response content was NOT expected confirmation. Parsed: ",signal);
               Print("**************************************************");
           }
       } else {
           Print("**************************************************");
           Print("WARNING: Gemini API Test - OK (200) but COULD NOT extract 'text' from outer response. Raw: ", StringSubstr(gemini_response_str,0,500));
           Print("**************************************************");
       }
   } else {
       Print("**************************************************");
       Print("FAILED: Gemini API Connection Test FAILED! HTTP Code: ", res_code, ". MT5 Error: ", GetLastError());
       string error_response = CharArrayToString(result_data, 0, WHOLE_ARRAY, CP_UTF8);
       Print("FAILED: Response (if any): ", StringSubstr(error_response,0,500));
       Print("**************************************************");
   }
}
//+------------------------------------------------------------------+
//| Determine Ribbon State and Price Relations                       |
//+------------------------------------------------------------------+
ENUM_RIBBON_STATE_COLOR DetermineRibbonStateAndPriceRelations(const GeminiKnnInputData &market_data, const double &current_ribbon_ema_values[], double current_ema200_value, string &ribbon_color_out, string &ribbon_vs_ema200_out, string &price_vs_ribbon_out, string &price_vs_ema200_out)
{
    bool all_above = true, all_below = true;
    for(int i=0; i<RIBBON_EMA_COUNT; i++) {
        if(current_ribbon_ema_values[i] < current_ema200_value) all_above = false;
        if(current_ribbon_ema_values[i] > current_ema200_value) all_below = false;
    }
    if(current_ribbon_ema_values[0] > current_ribbon_ema_values[RIBBON_EMA_COUNT-1]) { ribbon_color_out = "Green"; }
    else if (current_ribbon_ema_values[0] < current_ribbon_ema_values[RIBBON_EMA_COUNT-1]) { ribbon_color_out = "Red"; }
    else { ribbon_color_out = "Mixed"; }
    
    if(all_above) ribbon_vs_ema200_out = "Above";
    else if(all_below) ribbon_vs_ema200_out = "Below";
    else ribbon_vs_ema200_out = "Mixed/Crossing";
    
    if(market_data.c0_close > market_data.ribbon_top_c0) price_vs_ribbon_out = "Above";
    else if (market_data.c0_close < market_data.ribbon_bottom_c0) price_vs_ribbon_out = "Below";
    else price_vs_ribbon_out = "Within";
    
    price_vs_ema200_out = (market_data.c0_close > current_ema200_value) ? "Above" : "Below";
    if (ribbon_color_out == "Green") return RIBBON_COLOR_GREEN;
    if (ribbon_color_out == "Red") return RIBBON_COLOR_RED;
    return RIBBON_COLOR_MIXED;
}
//+------------------------------------------------------------------+
//| Get Swing Low                                                    |
//+------------------------------------------------------------------+
double GetSwingLow(int bars_back = 10, int index = 1) {
    MqlRates rates[];
    if(CopyRates(Symbol(),PERIOD_M15,index,bars_back,rates)!=bars_back) return 0.0;
    double lows[];
    ArrayResize(lows, bars_back);
    for(int i=0; i<bars_back; i++)
        lows[i] = rates[i].low;
    int min_index = ArrayMinimum(lows, 0, WHOLE_ARRAY);
    return lows[min_index];
}
//+------------------------------------------------------------------+
//| Get Swing High                                                   |
//+------------------------------------------------------------------+
double GetSwingHigh(int bars_back = 10, int index = 1) {
    MqlRates rates[];
    if(CopyRates(Symbol(),PERIOD_M15,index,bars_back,rates)!=bars_back) return 0.0;
    double highs[];
    ArrayResize(highs, bars_back);
    for(int i=0; i<bars_back; i++)
        highs[i] = rates[i].high;
    int max_index = ArrayMaximum(highs, 0, WHOLE_ARRAY);
    return highs[max_index];
}
//+------------------------------------------------------------------+
//| Calculate Strategy SL Pips                                       |
//+------------------------------------------------------------------+
double CalculateStrategySLPips(ENUM_ORDER_TYPE direction, double trade_entry_price, double &sl_price_out)
{
    double point = SymbolInfoDouble(Symbol(), SYMBOL_POINT);
    int digits = (int)SymbolInfoInteger(Symbol(), SYMBOL_DIGITS);
    double atr_val_arr[1];
    double atr_pips = 0;
    if(atrHandle_EA != INVALID_HANDLE && CopyBuffer(atrHandle_EA, 0, 1, 1, atr_val_arr) == 1 && atr_val_arr[0] > 0) {
       atr_pips = NormalizeDouble((atr_val_arr[0] * 1.5) / (point * 10.0), 1);
    }
    if(atr_pips <=0) atr_pips = InpStopLossPips_Fallback;
    double swing_val = (direction == ORDER_TYPE_BUY) ? GetSwingLow(15, 1) : GetSwingHigh(15, 1);
    
    if (direction == ORDER_TYPE_BUY) {
        if (swing_val <= 0 || swing_val >= trade_entry_price) { sl_price_out = NormalizeDouble(trade_entry_price - atr_pips * point * 10.0, digits); return atr_pips; }
        sl_price_out = NormalizeDouble(swing_val - (SymbolInfoInteger(Symbol(), SYMBOL_SPREAD) * point + 2 * point * 10), digits);
        return NormalizeDouble((trade_entry_price - sl_price_out) / (point * 10.0), 1);
    } else {
        if (swing_val <= 0 || swing_val <= trade_entry_price) { sl_price_out = NormalizeDouble(trade_entry_price + atr_pips * point * 10.0, digits); return atr_pips; }
        sl_price_out = NormalizeDouble(swing_val + (SymbolInfoInteger(Symbol(), SYMBOL_SPREAD) * point + 2 * point * 10), digits);
        return NormalizeDouble((sl_price_out - trade_entry_price) / (point * 10.0), 1);
    }
}
//+------------------------------------------------------------------+
//| Helper: JsonGetString                                            |
//+------------------------------------------------------------------+
bool JsonGetString(const string json_str, const string key, string &value_out) {
    string search_key = "\"" + key + "\":\"";
    int key_pos = StringFind(json_str, search_key);
    if(key_pos < 0) return false;
    int value_start_pos = key_pos + StringLen(search_key);
    int quote2_pos = StringFind(json_str, "\"", value_start_pos);
    if(quote2_pos < 0) return false;
    value_out = StringSubstr(json_str, value_start_pos, quote2_pos - value_start_pos);
    return true;
}
//+------------------------------------------------------------------+
//| Parse KNN Emulation Response                                     |
//+------------------------------------------------------------------+
string ParseKNNEmulationResponse(const string raw_gemini_signal_text_payload)
{
    string signal_value;
    if(JsonGetString(raw_gemini_signal_text_payload, "signal", signal_value)) {
        StringToUpper(signal_value);
        if(signal_value == "BUY" || signal_value == "SELL" || signal_value == "HOLD" || signal_value == "CONNECTION_TEST_OK") {
            return signal_value;
        }
    }
    return "HOLD";
}
//+------------------------------------------------------------------+
//| Helper: ExtractGeminiTextResponse                                |
//+------------------------------------------------------------------+
bool ExtractGeminiTextResponse(const string gemini_json, string &text_content_out) {
    string search_key = "\"text\": \""; int key_pos = StringFind(gemini_json, search_key);
    if(key_pos < 0) {
        search_key = "\"text\":"; key_pos = StringFind(gemini_json, search_key);
        if (key_pos < 0) return false;
        key_pos = StringFind(gemini_json, "\"", key_pos + StringLen(search_key));
        if (key_pos < 0) return false;
        key_pos++;
    } else {
        key_pos += StringLen(search_key);
    }
    int value_start_pos = key_pos;
    int current_pos = value_start_pos;
    string temp_text_content = "";
    while(current_pos < StringLen(gemini_json)) {
        ushort char_current = StringGetCharacter(gemini_json, current_pos);
        if(char_current == '\"' && (current_pos == 0 || StringGetCharacter(gemini_json, current_pos - 1) != '\\')) {
            temp_text_content = StringSubstr(gemini_json, value_start_pos, current_pos - value_start_pos);
            break;
        }
        current_pos++;
    }
    StringReplace(temp_text_content, "\\\"", "\"");
    StringReplace(temp_text_content, "\\\\", "\\");
    StringReplace(temp_text_content, "\\n", "\n");
    text_content_out = temp_text_content;
    return (StringLen(text_content_out) > 0);
}
//+------------------------------------------------------------------+
//| Helper: GetH1TrendContext_Simple/Full                            |
//+------------------------------------------------------------------+
string GetH1TrendContext_Simple(void) {
    string c[];
    if(ArrayResize(c, 2) == 2 && StringSplit(GetH1TrendContext(), '=', c) == 2) return c[1];
    return "N/A";
}
string GetH1TrendContext(void) {
    if(h1MaHandle_EA == INVALID_HANDLE) return "H1_Trend=N/A";
    double h1_ma_buffer[1]; MqlRates h1_rates_arr[1];
    if(CopyBuffer(h1MaHandle_EA, 0, 1, 1, h1_ma_buffer) != 1) return "H1_Trend=MA_Err";
    if(CopyRates(Symbol(), PERIOD_H1, 1, 1, h1_rates_arr) != 1) return "H1_Trend=Price_Err";
    double h1_ma = h1_ma_buffer[0]; double h1_close = h1_rates_arr[0].close;
    if(h1_close > h1_ma) return "H1_Trend=UP";
    if(h1_close < h1_ma) return "H1_Trend=DOWN";
    return "H1_Trend=SIDEWAYS";
}
//+------------------------------------------------------------------+
//| GetPipValuePerLot (Body)                                         |
//+------------------------------------------------------------------+
double GetPipValuePerLot(string forSymbol) {
   if (StringLen(forSymbol) == 0) forSymbol = Symbol();
   double tick_value = SymbolInfoDouble(forSymbol, SYMBOL_TRADE_TICK_VALUE);
   double tick_size = SymbolInfoDouble(forSymbol, SYMBOL_TRADE_TICK_SIZE);
   double point_size = SymbolInfoDouble(forSymbol, SYMBOL_POINT);
   if(tick_size <= 0 || point_size <= 0) return 0.0;
   return (tick_value / tick_size) * (point_size * 10.0);
}
//+------------------------------------------------------------------+
//| CloseAllEATrades                                                 |
//+------------------------------------------------------------------+
void CloseAllEATrades(string reason_str) {
   Print("Closing all EA trades (Magic #",InpMagicNumber,") due to: ", reason_str);
   for(int i = PositionsTotal() - 1; i >= 0; i--) {
       if(PositionGetInteger(POSITION_MAGIC) == InpMagicNumber) {
           trade.PositionClose(PositionGetTicket(i));
           Sleep(100);
       }
   }
}

1. The model is trained on historical data to learn the specific market conditions under which an RSI bounce is most likely to be profitable.
2. It effectively decides which RSI signals to trade and which to ignore based on the context provided by all the other features.

2. Vastly Enriched Feature Set
The model isn't just looking at the RSI. The FeatureEngineer class in the script creates a huge number of additional data points (features) for the model to use. This gives it a multi-dimensional view of the market, including:

1. Multi-Timeframe Context: The model knows the trend on higher timeframes (e.g., DAILY_ctx_Trend, H1_ctx_LinRegSlope).
2. Volatility Analysis: It analyzes ATR and bollinger_bandwidth to understand if the market is quiet or volatile.
3. Momentum Dynamics: It sees not just the RSI value, but also its slope and acceleration (RSI_slope, RSI_slope_acceleration).
4. Market Structure: It identifies institutional concepts like Break of Structure (bos_up_signal), Change of Character (choch_up_signal), and Fair Value Gaps (fvg_bullish_exists).
5. Volume and Price Action: It incorporates VWAP, volume spikes, and detailed candle analysis (is_engulfing, wick_to_body_ratio).

3. Dynamic, Volatility-Adjusted Exits
The strategy does not use fixed stop-loss or take-profit targets. The logs consistently show it generates labels using "VOLATILITY-ADJUSTED DYNAMIC BARRIERS".

1. The script uses TP_ATR_MULTIPLIER and SL_ATR_MULTIPLIER to set exit points.
2. This means in a volatile market, the profit targets and stop-losses are automatically set further away, while in a quiet market, they are set tighter. This adaptability is a significant advantage.

4. Sophisticated, Confidence-Based Risk Management
Risk is not static. The script uses a CONFIDENCE_TIERS system.

1. The machine learning model outputs a probability score for its prediction.
2. A higher confidence score can lead to a larger position size or a better risk-reward ratio, as defined in the configuration. This allows the system to risk more on high-probability setups and less on marginal ones.

5. Rigorous Hyperparameter Optimization
The framework uses Optuna to conduct an exhaustive search for the best possible settings for the machine learning model. It intelligently tests hundreds or thousands of combinations of parameters like n_estimators and max_depth to find the most profitable and robust configuration—a process impossible to replicate manually.

Overcomplicated, yes. Personally for me, this is the most practical way to ensure that I have a strategies that are robust and have been through rigorous tests that mimic the real world and have learnt from past mistakes which is why tested performance is so different from a simple manual approach.

Your results may differ.

1. Dynamic Broker Simulation (New in V208):
   1. V207: Relied on a static, hardcoded SPREAD_CONFIG and COMMISSION_PER_LOT defined in the fallback configuration.
   2. V208: The get_initial_run_setup prompt was significantly upgraded. It now explicitly tasks the AI with performing a grounded search using the Google Search API to find typical, realistic ECN spreads and commissions for the specific assets detected in the data files. This replaces the hardcoded defaults with values tailored to the run.

2. Cycle-Specific Context (Improved in V208):
   1. V207: The get_macro_context_data() function was called only once at the beginning of run_single_instance. This stale data was used for every subsequent AI analysis call between cycles.
   2. V208: get_macro_context_data() is now called inside the main walk-forward loop before the AI is consulted for cycle changes. This ensures the AI's analysis is always based on the latest market conditions.

3. Account Health Awareness (New in V208):
   1. V207: The AI had no information about the current run's overall profit or drawdown when suggesting changes.
   2. V208: The analyze_cycle_and_suggest_changes method and its prompt now include the account_health_state and overall_drawdown_pct. The AI is given explicit directives to prioritize capital preservation if the account is in a 'Caution' or 'Critical' state.

2. Model Optimization (ModelTrainer)

1. Hyperparameter Optimization Objective (Overhauled in V208):
   1. V207: The Optuna objective function aimed to maximize a sharpe_proxy. This was an abstract metric based on mapping predicted labels (0, 1, 2) to PNL values (-1, 0, 1). While fast, this did not accurately reflect real trading dynamics like risk-reward or drawdown.
   2. V208: The objective function was completely rewritten to be far more realistic. For each cross-validation fold, it now runs a mini-backtest on the validation data. It simulates trades based on the model's predictions and calculates a PNL series using the actual price data and ATR-based stops and targets. The function then calculates a Calmar Ratio proxy (Total PNL / Max Drawdown) for that fold. The AI now optimizes for the average Calmar Ratio, directly selecting for models that produce profitable and stable equity curves.

3. Backtesting & Risk Management (Backtester)

1. Dynamic Account Health System (New in V208):
   1. V207: The backtester was unaware of the run's overall performance. Its only risk constraint was the MAX_DD_PER_CYCLE circuit breaker.
   2. V208: The run_backtest_chunk function now tracks the run's high-water mark (run_peak_equity). On every candle, it calculates the account_health_state. This state then dynamically and automatically adjusts multiple risk parameters in real-time:
      1. Position Sizing: Risk is scaled down (risk_modifier) following a losing trade or during 'Caution'/'Critical' health states.
      2. Max Concurrent Trades: Reduced to 1 during a 'Critical' state to prevent further losses.
      3. Take-Profit Targets: The risk-to-reward multiplier for take-profits is automatically reduced during drawdowns to prioritize securing smaller wins and rebuilding capital.
      4. Confidence Threshold: The model's confidence required to enter a trade is increased during a 'Critical' state, filtering out all but the highest-conviction signals.

4. Strategy and Feature Engineering (initialize_playbook, FeatureEngineer)

1. Strategy Complexity Framework (New in V208):
   1. V207: All strategies were treated equally.
   2. V208: A "complexity" key ('low', 'medium', 'high', 'specialized') was added to every strategy's definition in the initialize_playbook function. This provides crucial metadata for the AI.

2. Complexity-Aware Feature Engineering (New in V208):
   1. V207: The USE_PCA_REDUCTION flag was a simple boolean toggle.
   2. V208: The FeatureEngineer now checks the strategy's complexity from the playbook. PCA is now only applied if the strategy is 'high' or 'specialized' and the AI has enabled USE_PCA_REDUCTION, preventing overly complex features from being used on simple strategies. The AI prompt was also updated to instruct it on how to use this new rule.