Curve fitting problem in systems | Forex Factory

Options

Permalinks
Print Thread

Similar Threads

Curve Fitting: The Ultimate Conundrum 17 replies

EA Backtesting and Curve Fitting Concern 2 replies

(curve)-fitting and overfitting, good examples wanted 0 replies

why curve fitting doesn't work? 23 replies

"curve fitting" or just improving?? 5 replies

Curve fitting problem in systems

  • Poll

  • Which equities must be tested by your ea before live test?
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  • #1
  • First Post: Edited 6:57am Feb 21, 2011 6:28am | Edited 6:57am
So,
finally, you have a system,
and it backtested on EURUSD, for since 1999 till today,
-Make consistent profits
-Maintain a healthy balance between risk and reward
-Produces at least 5 trades per week (The higher the trading frequency the smaller the chance of having a losing month.)
-Fully automated
-Have a high percentage of winning trades
-Few variables, and There is no sharp drop in performance due to small changes in the values of system variables
-Few rules:The more rules you have, the more likely you "curve-fitted" your trading system to the past, and such an over-optimized system is very unlikely to produce profits in real markets.
-SL/TP is set by volatility

you are happy :-)

but the ea will not work same in live!
past performance does not predict future performance !
because with all the optimizations,
it may have "curve-fitted" to the past!!!
in order to solve curve fitting problem,
you have to test it on other symbols and equities as well,
so it perform good results as well.
my question.
on which symbols you must test your ea
before running it live?
majors, crosses, gold, sp500, oil?

please vote!
  • #2
  • Feb 21, 2011 6:43am Feb 21, 2011 6:43am
Disclaimer - I'm not an expert!

Curve fitting is dangerous assuming you did curve fit, e.g. you optimised your parameters for the time period you tested. Those parameters then only apply to that period of time. When I design systems I build them around smaller periods of time then backtest them against a wider spectrum of market conditions to see if they hold up.

For instance I may design a breakout system on the daily charts using 4 random 3 month periods across a few years. I'll then backtest this across a couple of years exhibiting different market conditions and assuming it shows promise I'll perform a more extensive backtest.

We all have a number of cognitive biases that influence our testing. Try your best to test like a machine and even be pessimistic with your results. Take out some of the big winners, what does the expentency drop to? Is it worth it?
  • #3
  • Edited 8:47am Feb 21, 2011 8:34am | Edited 8:47am
if you try to avoid curve fitting by optimizing on one length of time and back-testing on another. Some people call this a "forward walk."
Of course, if you do this 10,000 times and pick the best result you are no better than the automatic optimizer on all period possible of the symbol.
so what difference does it make?
don't you fill the need to backtest on other symbols
so your strategy will be robust?

sorry for my bad english...
what is expentency ?
  • #4
  • Feb 21, 2011 9:06am Feb 21, 2011 9:06am
Quoting lolik2020
if you try to avoid curve fitting by optimizing on one length of time and back-testing on another. Some people call this a "forward walk."
Of course, if you do this 10,000 times and pick the best result you are no better than the automatic optimizer on all period possible of the symbol.
so what difference does it make?
don't you fill the need to backtest on other symbols
so your strategy will be robust?

sorry for my bad english...
what is expentency ?
I've heard it called Forward Testing before in that you don't need to wait for live market conditions. This isn't optimising though, you don't change the parameters. The paremeters you set on the original test must remain static through the wider backtest so I'm not sure about your "10,000 times" question.

Different symbols have different behaviour. I don't think it's valid that one strategy should support all symbols. I think that a strategy is a reflection of your interpretation of the behaviour of a particular symbol.

Expectancy is really just the average return per $ risked. I like to see at least $.50 per $1.00 risked.
  • #5
  • Feb 22, 2011 1:49am Feb 22, 2011 1:49am
Expert avisor that is tested succesfully on all major pairs? Not sure this is possible.

You want my opinion? It will always be a gamble, no way you can ever be sure unless the EA works on all pairs with a good profit margin. And that I have yet to see
  • #6
  • Feb 22, 2011 2:40am Feb 22, 2011 2:40am
Hi,

Was just reading this the other day and seems like a solid method to avoid falling for many of the curve fitting traps that are present with system building. http://www.forexfactory.com/showthread.php?t=4446. Oh and forward test. If it does work for 10 years straight then 3 months of forward testing should not be too demanding for it.

Cheers
  • #7
  • Feb 22, 2011 2:55am Feb 22, 2011 2:55am
An EA System is like trying to train a computer to draw the Mona Lisa.....it's not gonna happen. Trading is so much more complicated than that....EA systems are like a slap in the face to a true trader. Theoretically, the ultimate EA system would allow a person off the street to trade FX profitably immediately even though they have never traded any instrument before in their lives.......it will never happen folks.
  • #8
  • Feb 22, 2011 3:10am Feb 22, 2011 3:10am
Quoting Troikaone1
An EA System is like trying to train a computer to draw the Mona Lisa.....it's not gonna happen. Trading is so much more complicated than that....EA systems are like a slap in the face to a true trader. Theoretically, the ultimate EA system would allow a person off the street to trade FX profitably immediately even though they have never traded any instrument before in their lives.......it will never happen folks.
Must say I agree with you on this... EA on itself will eventually fail... But what it can do is improve your manual trading capabilities. A combination of both is the ultimate in my opinion.

But that brings up the question, why are there "systems" if they don't work? No human can trade a mechanical system as good as an EA can. After years of trading your brain just starts to "see" it, mechanical rules are just confirmation. That is the error people make, they tend to follow a system where as the system should be the confirmation not the decision.
  • #9
  • Feb 22, 2011 3:37am Feb 22, 2011 3:37am
guys,
generic, robust ea is possible.
do not give up.
Reasons:
1. if men walked on the moon, why men would not be able to create the perfect ea
2. 20 years ago, men did not think that there will be a chess software that will beat KASPAROV, but it does.
3. there are already running ea that make billions. but they are not for sale.
  • #10
  • Feb 22, 2011 4:24am Feb 22, 2011 4:24am
Quoting lolik2020
guys,
generic, robust ea is possible.
do not give up.
Reasons:
1. if men walked on the moon, why men would not be able to create the perfect ea
2. 20 years ago, men did not think that there will be a chess software that will beat KASPAROV, but it does.
3. there are already running ea that make billions. but they are not for sale.
How long have you been developing EA's? How many EA's have you written up to date?

To my knowledge banks employ traders for a reason, the human brain is too complex cannot see how software will ever achieve the same level.

Believe me I really want to believe EA's work but I have programmed just about every combination of strategies out there and they all failed sooner or later.

Even Ronald Raygun said an EA needs human evaluation, no EA can run 100% on its own.

If you can convince me otherwise please do so
  • #11
  • Feb 22, 2011 7:24am Feb 22, 2011 7:24am
Quoting lolik2020
guys,
generic, robust ea is possible.
do not give up.
Reasons:
1. if men walked on the moon, why men would not be able to create the perfect ea
2. 20 years ago, men did not think that there will be a chess software that will beat KASPAROV, but it does.
3. there are already running ea that make billions. but they are not for sale.
The reason why EA systems can never replace humans is because no computer in the world can eyeball trendline resistance/support. I'm not talking about obvious trendline resistance, I am talking about the complicated types of trendlines that the average inexperienced trader may not see. These trendlines have a unique "Y=mX+B" formula that cannot be comprehended by any algorithm because an algorithm cannot outthink or outreason a human. An algorithm is designed to calculate, which is something it does far better than any human. You might be able to account for horizontal lines of support and resistance in an EA but not trendlines. Once an EA system is "programmed" to account for complex trendlines, hell you won't need them anymore.....just rely on your brain.
  • #12
  • Feb 22, 2011 8:31am Feb 22, 2011 8:31am
Quoting Troikaone1
The reason why EA systems can never replace humans is because no computer in the world can eyeball trendline resistance/support. I'm not talking about obvious trendline resistance, I am talking about the complicated types of trendlines that the average inexperienced trader may not see. These trendlines have a unique "Y=mX+B" formula that cannot be comprehended by any algorithm because an algorithm cannot outthink or outreason a human. An algorithm is designed to calculate, which is something it does far better than any human. You might be able...
Troikaone1, what is your trading style? I agree with you on this issue although I think it is more complicated than just trend lines. Complex patterns are easy for a human to recognize, for a computer it is another story.
  • #13
  • Feb 22, 2011 8:39am Feb 22, 2011 8:39am
Quoting Saidar
Troikaone1, what is your trading style? I agree with you on this issue although I think it is more complicated than just trend lines. Complex patterns are easy for a human to recognize, for a computer it is another story.
Hey Saidar,

I trade simple H&S patterns, with the assistance of channels and trendlines. You are correct, it is definitely more complicated than just trendlines. But as I mentioned before, I don't use the obvious trendlines that are taught in most trading books.....the best trendlines are hidden from view (on purpose). And once you factor in trendlines in different timeframes...it becomes even more complex.
  • #14
  • Feb 22, 2011 12:00pm Feb 22, 2011 12:00pm
Quoting Troikaone1
Hey Saidar,

I trade simple H&S patterns, with the assistance of channels and trendlines. You are correct, it is definitely more complicated than just trendlines. But as I mentioned before, I don't use the obvious trendlines that are taught in most trading books.....the best trendlines are hidden from view (on purpose). And once you factor in trendlines in different timeframes...it becomes even more complex.
Sounds interesting, if this info is available somewhere please share the link if you don't mind. I'm always open to new ideas.
  • #15
  • Feb 22, 2011 1:05pm Feb 22, 2011 1:05pm
lol, 33,000 hours of hard work along with a lot of blood, sweat and tears!! It's all about thinking outside the box.
  • #16
  • Last Post: Feb 22, 2011 1:10pm Feb 22, 2011 1:10pm
Quoting Troikaone1
lol, 33,000 hours of hard work along with a lot of blood, sweat and tears!! It's all about thinking outside the box.
Mmm yes that is a lot of work, I could have learned to fly if I spent 33000 hours trying LOL.

Glad you made a success good job
0 traders viewing now