Menu

Options

Bookmark Thread

First Page First Unread Last Page Last Post

Print Thread

Similar Threads

Trade another day through Systematic Trading 203 replies

Nonparametric Association Measures in Systematic Trading 57 replies

Anyone use Systematic from Alpari? 1 reply

A Systematic Approach to Markets 68 replies

Systematic Trading vs. Discretionary Trading 14 replies

Attachments: Systematic trading
Exit Attachments

Systematic trading

  • #1,101
  • Quote
  • Edited 8:11am Sep 16, 2010 8:01am | Edited 8:11am
Quoting pvpn
Thank you.

I thought GA algorithms were long forgotten. They were a big hit in the 90s when I was still programming for living, but haven't seen much talk about them lately. You certainly motivated me to dig in this field again and see how much trading AI I can squeeze using them.

I've been thinking a little about what your achievements are currently and the smoothness of the cumulative "earnings" curve you managed to obtain and the reasons behind it.

My take is that by having a large number of (profitable) algorithms you're dividing the risk...
IMHO Gp/ga works simply because evolution works but it's damn hard to get it working correctly so that it doesn't overfit the results (ie. finding good fitness algo is crucial and very hard).

About my systems, it actually doesn't matter what kind of systems one uses - what really matters is that you have systems that fit to your criteria and how clustered your DD is (this is more imporant than most people realize).

Simple stats I have are posted to the top of the pic of the curve - all of the systems are not long-tail systems and to be honest I really don't care about details that much anymore I have decided that I'm kind of the decisionmaker who decides what the rules and benchmarks are and the system will take care of the rest. I know the conenctions between systems and calculated risk profiles between systems are too complicated for me to understand totally so I basically only monitor in-sample vs out-sample results and as long as out-sample looks at least a bit similar to in-sample I'm all happy. At the moment my fx strat collection seems to be as near ideal as it's possible with my current framework so I'm looking into diversification to different instruments and/or different timeframes (or changing the system to something a bit different, a kind of genetic/event driven model hybrid).
Quoting mikkom
IMHO Gp/ga works simply because evolution works but it's damn hard to get it working correctly so that it doesn't overfit the results (ie. finding good fitness algo is crucial and very hard).
....
In understand what you mean (also the rest of your post). I tend to agree with your approach. At some point empirical solutions are cheaper to develop as long as proper metrics are applied and controlled.

I remember that I had a problem with excessive clustering of my population. Good individuals were quite similar and thus the whole population tended to cluster towards some local maximum. To be honest I didn't try hard enough to spread it. My experiments were also focused on finding optimum settings for multi-variable strategies (with the obvious result of overfitting).

I guess a fitness function that takes care of several parameters (DD/profit etc) is only a part of the solution.

Perhaps the population constructor must make sure that there's enough variety of individuals even if some of them have lower fitnesses.

It would be nice the the GA could find rules that work by itself, but this is another story.

I think I have an idea for trading multiple systems and still keep margin requirements down.... must verify some things before I'm ready to share it.

pvpn.
Quoting pvpn
Perhaps the population constructor must make sure that there's enough variety of individuals even if some of them have lower fitnesses.
Why observe indirectly if you can observe directly?

Sorry for being vague. I just think you have to think a bit further what you really want from your results.
mikkom and others: what's your experience from going live with systems that looked promising from testing?
I noticed that the built-in random number generator in VB wasn't giving me even/consistent results so I did some investigation and found a new algo that I'm using to generate random numbers that are more uniform called the Mersenne Twister (VBA code). What random number algos are you guys using? I've seen algorithms for generating standard normal numbers but haven't found much use in it in developing GA/GP. If you have differing experience, I'd love to hear about it. One benefit of using a better random number generator is that I can do fewer tests and get more even results (that previously required more samples) - fewer tests means faster results for a monte carlo type algo. I'm sure there are other benefits for GA/GP but to be honest I'm still getting my code to where I can use it in a more meaningful way (rewriting now).

http://www.math.sci.hiroshima-u.ac.j...37arVBcode.txt
Quoting knukk
mikkom and others: what's your experience from going live with systems that looked promising from testing?
Mostly disappointing, but it really depends on what you mean by 'promising'.
The breaking of a wave cannot explain the whole sea.
Quoting FXEZ
What random number algos are you guys using?
http://www.boost.org/doc/libs/1_44_0...ct_list.random
The breaking of a wave cannot explain the whole sea.
My understanding, which may be wrong, is that a random number is generated from some seed value. If the seed is some deterministic source like the current time then it wont be random. Maybe polling twitter tweets would be random enough for a seed?
Economists have forecast 9 out of the last 5 recessions
Quoting FXEZ
I noticed that the built-in random number generator in VB wasn't giving me even/consistent results ...
You are using Windows. The best random generator in this case is this: http://en.wikipedia.org/wiki/CryptGenRandom

It's suitable even for cryptographic purposes.

@markmm: CryptGenRandom uses all kinds of inputs to it's mixing pool, including hardware events (keyboard presses, hardware IRQs, time took by various disk operations)

Using the current time as a seed to a good generator (like mersene) is good enough for non-cryptographic purposes.
Quoting Craig
http://www.boost.org/doc/libs/1_44_0...ct_list.random
Im finding C++/Linux/Boost/QT creator/Git is an excellent toolset for implementing my system, everything is so fast.
Economists have forecast 9 out of the last 5 recessions
@FXZES: I don't think that the quality of the random series is of so much importance in what we're doing. It's more important for security reasons to avoid predictions, but the quality of a population depends on much more than the quality of the random number generator.

However if you're serious about it here's a clue: http://en.wikipedia.org/wiki/Hardwar...mber_generator

@knukk: I think there's nothing special of mechanical trading algorithms versus manual systems. All of the systems that seem to work in backtest (or forward tests if you prefer) can fail at any time for various reasons (like being a result of luck/optimization or loosing their edge).

pvpn
Quoting markmm
Im finding C++/Linux/Boost/QT creator/Git is an excellent toolset for implementing my system, everything is so fast.
Yep, it runs pretty good, and boost stops you having to worry about low level detail, works for me.
The breaking of a wave cannot explain the whole sea.
Quoting markmm
Im finding C++/Linux/Boost/QT creator/Git is an excellent toolset for implementing my system, everything is so fast.
Could you post a screen-shot of the GUI you're working on? Eventually with the sensitive stuff blocked out.
Quoting knukk
mikkom and others: what's your experience from going live with systems that looked promising from testing?
My out-sample testing results seem to be almost the same as live.
Quoting Adal
Could you post a screen-shot of the GUI you're working on? Eventually with the sensitive stuff blocked out.
I am working on the library at the moment, i.e. all the classes for strategies. so far I only have a tabbed window in QT. Once the UI is further developed I will post a screenshot.
Economists have forecast 9 out of the last 5 recessions
  • #1,116
  • Quote
  • Sep 20, 2010 11:14am Sep 20, 2010 11:14am
Quoting knukk
mikkom and others: what's your experience from going live with systems that looked promising from testing?
I have gone live with my first system about a month ago. (see attachment). Since I have gone live the curve has been moving sideways(typical) On a positive note it is still behaving within the performance parameters. This is a complete different ball game as every loss hurts. One of the problems is that I am using IB and they have a minimum transaction cost so I am forced to use a decent size. On the moment I am in a slight loss thanks to the transaction cost.

Very curious how this one developes... If this developes as I am hoping I will be adding many of these babies.
Attached Image (click to enlarge)
Click to Enlarge Name: live.png Size: 20 KB
Markmm, do you have any advise on how to get started with QT Creator, I've browsed though the documentation, but it's pretty basic...I can't really find anything on how one would import existing C++ libs (for example).
The breaking of a wave cannot explain the whole sea.
Quoting Craig
Markmm, do you have any advise on how to get started with QT Creator, I've browsed though the documentation, but it's pretty basic...I can't really find anything on how one would import existing C++ libs (for example).
You can create an empty folder to house your projects like a solution in vs, and the create projects inside it, so eg after creating the dir mysys, start qt creator and create a qt gui project(mysys_gui) placing it inside the mysys folder, then close the project and then create a cpp lib project (mysys_lib) also inside the mysys folder

Now create a mysys.pro using a text editor in the mysys folder with something like the following


TEMPLATE = subdirs
SUBDIRS = mysys_lib mysys_gui
mysys_lib.subdir = myLib
mysys_gui.subdir = myGui
mysys_gui.depends = mysys_lib

You can now open this in qt creator, open project and point to mysys.pro.

Your projects will all be there under the mysys, so you could right click add existing files to your lib project, (after copying the various .h and .cpp files to mysysmysys_lib folder.

Hope that makes sense , and it should all build correctly based on the depends bits. If you have boost installed it will just find it, and you have code completion to help navigate.

Let me know if you need any more info or if that didn't make sense.


Mark
Economists have forecast 9 out of the last 5 recessions
Cheers for that, it does make sense.
Managed to get some basic projects up and running, but will have to keep exploring the GUI, it is however much nicer than what I'm using now.
The breaking of a wave cannot explain the whole sea.
Quoting Craig
Cheers for that, it does make sense.
Managed to get some basic projects up and running, but will have to keep exploring the GUI, it is however much nicer than what I'm using now.
No problem, once I had my projects setup and had git installed (this is on Linux), I cd'ed into my project folder and did
git init
git add .
git commit

this will add all your projects to git and QT Creator has git integration.

Mark
Economists have forecast 9 out of the last 5 recessions
0 traders viewing now