Different settings per pair = Curve Fitting

I developed an EA that has some simple parameters, and watching some youtube videos on backtesting/optimizing seems to point to the best scenario being one setting for all 12 pairs that I'm trading. Problem is I'm having difficulty finding a setting that works on all 12 pairs for the 5 year backtest.

I can, however, optimize each pair positively. My concern is that having 12 different settings for 12 different pairs may mean that I've curve fitted. I guess I'm just paranoid about curve fitting, which is why I only really have a few parameters...mainly on the ratios between SL and TP, and a couple of filters.

Or am I being paranoid?

Some believe that a strategy with a real edge, should at least generate profits on a few other pairs. Profit might be mediocre for these other pairs, but it should be seen as a confirmation that your strategy has an actual edge.

Personally I am not entirely convinced about this theory. If your algorithm extracts alpha from the behavior of one specific currency, then the above statement might be true for a few other pairs that contain that same currency. However, if your algorithm extracts alpha from the interaction between two specific currencies (which are two economies/countries trading with each other), then why should we expect it to work on other pairs that represent other economies/countries? That doesn't make sense to me. So I guess it depends on what edge your strategy has.

Good if your strategy works on more pairs! It is now up to you what to do next. You could trade only the best performing pair and forget about the others, you could optimize them each individually, or try to find 1 setting that works for all. Each pair has its own "personality". The average range per day can be very different. Some pairs can make large trends, others are fairly ranging. Pairs can show more volatility at other times of the day.

If you want to trade more pairs, then it depends a bit on the design of your strategy whether you can use 1 setting for all pairs, of have separate settings per pair. Are SL and TP expressed in absolute pips or points? In that case it would be unreasonable to expect the settings to be the same for all pairs. If they are expressed as a percentage that is already better. If expressed in ATRs even better! In that case you might have a chance to find 1 setting to works for all pairs.

"Only the paranoid survive" -- Andy Grove

Curve fitting it the most challenging problem in strategy development. With todays' computers it is easy to quickly find thousands of strategies that give a rising equity curve. But which of these do really have an edge? So it is good that you are careful!

Already wanted to post this in your other thread How often do you re-optimize but it "fits" better here:

You are not. Curve-fitting is the result of two things: (1) using an inproper optimization criterion and (2) an invalid selection of the best optimization result. If either one of these topics is not solved you will experience curve-fitted results, there's no way around it. What you have seen in the past will not repeat in the future.

The first topic was already discussed at your other thread. The solution to the second topic are 3D surface diagrams. 3D diagrams are the only way to properly select a stable/robust parameter set. Don't chose the set with the best results (peaks), chose the one with multiple similar results nearby.

The best result in the following picture (peak top-left) will most probably be spurious and not repeat in the future. Robustness is much better on the plateau middle-right.


Creating and interpreting 3D surface diagrams is an art of itself. In between most retail platforms offer "some" suppport of 3D surface diagrams. It depends on what your platform offers and your skill level to create your own diagrams. MT5 offers built-in and since a few years also custom 3D surface diagrams (after more than 10 years finally a reason to install it). MT4 doesn't but there are following workarounds (if you can write code):

1. There's a JavaScript library "vis.js" which creates nice interactive 3D surface diagrams. It uses the canvas functionality built-in to your browser. You can feed/include your optimization results as a JSON file and are free as to what features to use/implement. Countless possibilities.
2. MT5's canvas functionality can be used to create interactive 3D surface diagrams from external data sources. You may use as well your MT4 optimization results. For example you could read your data from a .TXT or .CSV file into an MQL5 EA which uses the canvas. On Youtube is a short video showing a custom 3D renderer with mouse interaction.

As it's so important, I'd like to repeat the statement from the beginning: Curve-fitting is the result of two things: (1) using an inproper optimization criterion and (2) an invalid selection of the best optimization result. If either one of these topics is not solved you will experience spurious and curve-fitted results.

Links:

1. vis.js Graph3d Example (use your mouse to move around/zoom in/out)
2. How to create 3D graphics using DirectX in MetaTrader 5

Regarding those recommendations: Not knowing what videos you watched, assumed they didn't solve the two issues explained in my previous post then they are quite lost. How to achieve stable/robust results then? The only way I can imagine is to find a parameter set that magically "fits all". "Fitting all pairs" is stability by definition. That's very lucky and luck is indeed the best scenario one can encounter. Unsurprisingly it's hard to find..

Another pure JavaScript implementation (no libraries needed) of 3D surface diagrams:

http://www.grvisualisation.50webs.co...face_plot.html

Yes you are curve fitting.

When you make a backward test on real data, then you can find the parameters that fits the best for each pair. But when you tetst on another time slot those parameters will not fit. And they will not fit at other pairs either. The reason is the market behavier. It is different each and every day. Of cause you will then have periods where the market fit with your parameters and visa versa.

I do it like this. Make the best strategy possible. Be sure that it is logically a fine strategy in the type of market that you would like to trade. (Trending , ranging, News, not News, Asian - Euro or US etc.)

Then find pairs on a daily basis that seems to have the conditions you need.

I make robots for trending markets. Not too volatile. I find those markets by top/down analyses and then the robots are set to trade in the direction of the trend.

So I know, that my logic is working in a trending market and I HAVE curvefitted to this condition. My robots are only working if I think they will fit the market at that time and I am ready to stop them. I trade all 28 normal Forex pairs and cose the best for the day.

Here is my Sandbox for today.