I have been curious about this subject, and I'm interested in hearing some educated opinions on the matter:
Curve fitting is the major pitfall of optimization - anyone who has tried to move on to forward-testing knows this.
The way I try to avoid curve fitting is by optimizing on one length of time and back-testing on another. Some people call this a "forward walk." Of course, if you do this 10,000 times and pick the best result you are no better than the automatic optimizer
.
I try to find value ranges within my variables that, when tested with a "forward walk," produce a normal distribution (statistically) of +profit.
Another obvious practice to avoid curve fitting is to minimize the number of variables being optimized.
I am sure that much more sophisticated methods than this exist; what are they? What statistical tests / optimization practices are relevant to the avoidance of curve fitting?
Curve fitting is the major pitfall of optimization - anyone who has tried to move on to forward-testing knows this.
The way I try to avoid curve fitting is by optimizing on one length of time and back-testing on another. Some people call this a "forward walk." Of course, if you do this 10,000 times and pick the best result you are no better than the automatic optimizer
.
I try to find value ranges within my variables that, when tested with a "forward walk," produce a normal distribution (statistically) of +profit.
Another obvious practice to avoid curve fitting is to minimize the number of variables being optimized.
I am sure that much more sophisticated methods than this exist; what are they? What statistical tests / optimization practices are relevant to the avoidance of curve fitting?