backtesting, The past... | Page 2 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Backtesting programs 20 replies

Past news indicator for backtesting 4 replies

Best backtesting Period 3 replies

Custom backtesting code 9 replies

Backtesting vs. Thought Process 10 replies

backtesting, The past...

  • #21
  • Oct 7, 2006 3:46am Oct 7, 2006 3:46am
Quoting quathar
Ok, now it's getting interesting.
I have for instance < 10 parameters (a backtest in java), and although I have done some "manual genetic programming" to roughly optimize the parameters, full parameter space calculations takes a lot of processing time (I'm testing with a 2 million minute time series).
I don't really get you would accomplish it in excel though...
Also could you give examples on linear, and non-linear trajectories?
Quathar - I agree that for more than 4 or 5 parameters pair wise visualisations are probably not feasible. Myself and my colleagues only have 2 parameters and yet modelling for just two has been the major challenge for us for several years now. A few years ago we wrote a dynamic visualisation program that plotted the performance of our system over time and over the full parameter space. Unfortunately we failed to see any clear pattern that allowed us to predict future performance from one time period to the next. It did however convince us that our system parameters behaved in a non-linear fashion and that performance was not reliably stable in one area from one time period to the next and that cherry picking hot/warm or cold spots from the previous period was not going to work without a fundamental grasp of the underlying mathematics.

Re: examples of non-linear systems - there's a good description on wikipedia under nonlinearity. As you'll see it explains that non-linear systems are very difficult to predict and are inherently unstable. Weather patterns are well known examples of non-linear systems. You'll know the common saying that the best predictor for tomorrow's weather is today's weather. In essence this is the equivalent to using a basic form of re-optimising every time period. Whilst this is probably not a bad crude predictor of the weather, meteorologists need non-linear modelling techniques to really understand what's going on. You could also look at the wikipedia entry for a lorenz attractor - this shows a nice pic of the kind of trajectory I suspect our parameters will be following.

I think a genetic programming approach makes a lot of sense as GP's are claimed to be capable of modelling non-linear systems.

My final disclaimer is that all my experience has been for our particular system parameters - others may have a better underlying system that produces better behaving parameters.
  • #22
  • Last Post: Oct 7, 2006 4:48am Oct 7, 2006 4:48am
Quoting richp
Quathar - I agree that for more than 4 or 5 parameters pair wise visualisations are probably not feasible. Myself and my colleagues only have 2 parameters and yet modelling for just two has been the major challenge for us for several years now. A few years ago we wrote a dynamic visualisation program that plotted the performance of our system over time and over the full parameter space. Unfortunately we failed to see any clear pattern that allowed us to predict future performance from one time period to the next. It did however convince us that our system parameters behaved in a non-linear fashion and that performance was not reliably stable in one area from one time period to the next and that cherry picking hot/warm or cold spots from the previous period was not going to work without a fundamental grasp of the underlying mathematics.

Re: examples of non-linear systems - there's a good description on wikipedia under nonlinearity. As you'll see it explains that non-linear systems are very difficult to predict and are inherently unstable. Weather patterns are well known examples of non-linear systems. You'll know the common saying that the best predictor for tomorrow's weather is today's weather. In essence this is the equivalent to using a basic form of re-optimising every time period. Whilst this is probably not a bad crude predictor of the weather, meteorologists need non-linear modelling techniques to really understand what's going on. You could also look at the wikipedia entry for a lorenz attractor - this shows a nice pic of the kind of trajectory I suspect our parameters will be following.

I think a genetic programming approach makes a lot of sense as GP's are claimed to be capable of modelling non-linear systems.

My final disclaimer is that all my experience has been for our particular system parameters - others may have a better underlying system that produces better behaving parameters.
Yes, I know (more or less) what linear vs non-linear dynamic systems are, I just couldn't visualize a nonlinear trajectory.
Btw isn't a "non-repeating pattern" (as it says in the beggining of the lorenz attractor wikipedia article) a misnomer?
How would you distinguish a pattern if it didn't repeat itself?
Anyway, since all markets are basically chaotic, you would have to filter out a lot of noise to get an linear approximation of parameter consistency.

Heh, my "manual genetic programming" was more of a joke as perhaps the "manual" part would indicate. What I basically do is run seperate backtests with different parameters, roughly discerning where the best results are. That way I narrow the possibilites down. This obviously only works if parameter adjustment results in uniform changes in the systems effectivness. So far, so good.
0 traders viewing now