Systematic trading | Page 38 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Trade another day through Systematic Trading 203 replies

Nonparametric Association Measures in Systematic Trading 57 replies

Anyone use Systematic from Alpari? 1 reply

A Systematic Approach to Markets 68 replies

Systematic Trading vs. Discretionary Trading 14 replies

Attachments: Systematic trading
Exit Attachments

Systematic trading

  • #741
  • Apr 16, 2010 2:48pm Apr 16, 2010 2:48pm
Quoting spekkiefx
Could you give us an update on your strategies that you described with your alpha factory, are they profitable.
Yes they are. However as I said in one of the first posts, I use a limited search landscape (limited types of strategies and limited types of filters to those strategies) and I have worked a lot with speed optimization which I consider to be one of the biggest issues with ga/gp. When I have the next version up the speed should be even faster.

I should note that I only use 4 pairs (EURUSD, GBPUSD, USDJPY and AUDUSD) and the stats I have shown include commissions (I padded spread a bit more than necessary to include slippage as well).

The problem with only 4 markets is that there is quite a lot of correlation between them so I don't get as smooth curve as I would like to and as everyone knows (or should know ) profit is a direct derivate of dd you want to have (I have currently set my to 20% which might be too little).

I think I also have quite a solid fitness algo which is IMHO one of the most important things. I'm still not sure if it's even near optimal - trading is not exactly the easiest data to optimize to.

I'm moving to different architecture in my testing (totally event driven) after that I think I can speed the optimization at least 10-100x (see my earlier post about this). After that I think I'll have much more interesting pics, my current tests are about 10000-20000 iterations.

I'm quite intersted, those people who have event driven platforms already, have you done optimal method searches and what kind of results have you gotten in-sample vs out-sample.
  • #742
  • Apr 16, 2010 3:03pm Apr 16, 2010 3:03pm
mikkom,
Do you have multiple dimensions to your fitness function? If you do, how do you handle scaling so one dimension does not dominate the others?
The breaking of a wave cannot explain the whole sea.
  • #743
  • Edited 3:32pm Apr 16, 2010 3:20pm | Edited 3:32pm
Quoting Craig
mikkom,
Do you have multiple dimensions to your fitness function? If you do, how do you handle scaling so one dimension does not dominate the others?
Sorry I don't quite follow, what do you mean by multiple dimensions? Time?

At this point I'm not even sure if it's the fitness function (I use very simple one now, just a simple check that few of the parameters are above treshold) or the portfolio selection algo (that is a bit more complex) that is more important or if it's the combination of both.

Here is a shot of 100% out of sample data, about 3-5 hours running (I'm not sure, had a few beers). 2009 was a hard year but I'm still not at all satisfied to MAR of ~2. (data until first of this month so 2010 looks a lot better). That clustered dd at a bit over mid of 2009 is awful, I have to find a way to get rid of points in time where that happens and only way I can think of is to get more markets into the mix (or more timeframes, I'll have to see what I can do).

This is with maximum DD of 20% on training period (2002-2008)
Attached Image (click to enlarge)
Click to Enlarge Name: Screenshot-AlphaFactory-85.png Size: 16 KB
  • #744
  • Apr 16, 2010 3:29pm Apr 16, 2010 3:29pm
Quoting mikkom
Sorry I don't quite follow, what do you mean by multiple dimensions? Time?

At this point I'm not even sure if it's the fitness function (I use very simple one now, just a simple check that few of the parameters are above treshold) or the portfolio selection algo (that is a bit more complex) that is more important or if it's the combination of both.
One has to develop a ranking or 'fitness function' for the selection process, this could be something simple like the Sharpe Ratio for instance, this would be a 1-dimensional function, or to put it another way the fitness function is a function of one variable. One could define a 2-variable fitness function, say making it a linear combination of Sharpe Ratio and Profit Factor (as a completely arbitrary example). The problem arises when ones fitness function is a linear combination of a number of different factors and those factors have different scales, as the larger factor will obviously dominate the process. Sorry if this doesn't make much sense, I'm trying to explain without giving anything away
The breaking of a wave cannot explain the whole sea.
  • #745
  • Apr 16, 2010 3:35pm Apr 16, 2010 3:35pm
Quoting Craig
One has to develop a ranking or 'fitness function' for the selection process, this could be something simple like the Sharpe Ratio for instance, this would be a 1-dimensional function, or to put it another way the fitness function is a function of one variable. One could define a 2-variable fitness function, say making it a linear combination of Sharpe Ratio and Profit Factor (as a completely arbitrary example). The problem arises when ones fitness function is a linear combination of a number of different factors and those factors have different...
I do understand what you are saying, I have been experimenting with different combinations a lot (and I mean a lot, I did have a personal server running GA/GP stuff for stocks in our companys server room 24h some 10 years ago ) - that didn't lead anywhere btw.

In the end, at least for me most simple ones have worked the best because you don't want the peaks of landscape - you want the ones that have the least correlation but that work on the whole dataset. That's why I think lots of data is so important. Also my fitness function is a combination of one-market and multimarket metrics <- there is a hint that anyone using evolutionary algos should read

Just for fun, here is latest gp pic - it improves all the time but there is a limit because of the few markets that are used.
Attached Image (click to enlarge)
Click to Enlarge Name: Screenshot-AlphaFactory-86.png Size: 16 KB
  • #746
  • Apr 16, 2010 3:39pm Apr 16, 2010 3:39pm
Quoting mikkom
Yes they are.
What are you using to execute your live trades?
The breaking of a wave cannot explain the whole sea.
  • #747
  • Apr 16, 2010 3:40pm Apr 16, 2010 3:40pm
Quoting Craig
What are you using to execute your live trades?
At the moment still mt4 with "parser" ea. urgh.

Want to get out asap.
  • #748
  • Apr 16, 2010 3:42pm Apr 16, 2010 3:42pm
This is I believe the point I am at. I have a large database with strategies and results. Some of them do well for certain periods of time. I now need to design a function to pick the right strategies for the right time. Quite a task.
  • #749
  • Apr 16, 2010 3:43pm Apr 16, 2010 3:43pm
Quoting spekkiefx
This is I believe the point I am at. I have a large database with strategies and results. Some of them do well for certain periods of time. I now need to design a function to pick the right strategies for the right time. Quite a task.
I might as well tell that I don't do that - I let the strategies themselves filter when they want to trade or not.

Evolution works
  • #750
  • Apr 21, 2010 5:23am Apr 21, 2010 5:23am
Hi guys,

Ive got the first piece of my system up and running, the quote engine, which is an interface to a quote priovider. At the moment its hooked up to MBs quote API and it is saving quotes to my DB.

I intend to build a health monitor system to make sure each part of the system is up, and then onto my Order Engine.

How is everyone else going?

PS: I tried doing this in C++ but I ended up spending days getting simple things working, I have moved completely to C# and .NET with SQL server and Windows services/WCF (named pipes) for the various engines.

Mark
Economists have forecast 9 out of the last 5 recessions
  • #751
  • Apr 21, 2010 8:58am Apr 21, 2010 8:58am
Quoting markmm
I tried doing this in C++ but I ended up spending days getting simple things working, I have moved completely to C# and .NET with SQL server and Windows services/WCF (named pipes) for the various engines.
Wise choice. Unless you need to meet specific low latency requirements or C++ simply just happens to be your most productive language, there really are no particularity reasons for using C++ instead of a higher level language IMO.
  • #752
  • Edited 9:33am Apr 21, 2010 9:03am | Edited 9:33am
Quoting BlackMage
Wise choice. Unless you need to meet specific low latency requirements or C++ simply just happens to be your most productive language, there really are no particularity reasons for using C++ instead of a higher level language IMO.
I agree, its fast enough for what I am building and I can produce stuff in 10th of the time.

My day job is and has been developing in C#, .NET for the last 6-7 years. The productivity difference is from using C++ is just too great.
Economists have forecast 9 out of the last 5 recessions
  • #753
  • Apr 21, 2010 9:35am Apr 21, 2010 9:35am
Quoting markmm
PS: I tried doing this in C++ but I ended up spending days getting simple things working, I have moved completely to C# and .NET with SQL server and Windows services/WCF (named pipes) for the various engines.
I think a few posts back you said that you had a Linux box ready for your systems?

Me, I've played a little with WPF together with IronPython. Very nice. Better than PyQT.
  • #754
  • Apr 21, 2010 10:16am Apr 21, 2010 10:16am
Quoting Adal
I think a few posts back you said that you had a Linux box ready for your systems?

Me, I've played a little with WPF together with IronPython. Very nice. Better than PyQT.
Yeah I was all set to use Linux/C++/Mysql but I am back on Windows Server 2008/C#/SQL server 2008, Its what I use very day, so although I do love the Linux platform its not my natural habitat.
Economists have forecast 9 out of the last 5 recessions
  • #755
  • Apr 21, 2010 1:47pm Apr 21, 2010 1:47pm
I use C++, but it's also the language I use at my day job. I'm now live with IB & IQFeed, so far the IB gateway has worked flawlessly (touch wood), the only trouble I'm having is processing the amount of data I'm getting via IQFeed, which is currently 112 tick streams, I can guarantee you won't get that kind of throughput in C#
The breaking of a wave cannot explain the whole sea.
  • #756
  • Apr 21, 2010 2:47pm Apr 21, 2010 2:47pm
Quoting markmm
Yeah I was all set to use Linux/C++/Mysql but I am back on Windows Server 2008/C#/SQL server 2008, Its what I use very day, so although I do love the Linux platform its not my natural habitat.
I think you made the right choice. When I read your first comment I thought hmmm. Performance is initially not the main problem. When you use the environment you used to this will save you a lot of time. Finding something that works is the main problem. Unless you use C++ daily this is a fiddly language.

Saying that I do believe linux/unix has to be at the core of it. I use fedora wich has java/eclipse completely integrated. Developing/running things in this environment is just rock solid and way better than any windoze environment.
I use oracle as a database which has come in handy with features like partitioned and compressed tables.
Java is a lovely language and is also a main interface for brokers including IB and dukascopy,currenex. Once you get the hang of it , it is pretty quick. Sure maybe not as quick as C++ but probably quicker than a badly written C++ program. I believe there are ways to port JAVA to C but I haven't really spend to much time on that.

Anyway you have increased your chances of success
  • #757
  • Apr 21, 2010 3:06pm Apr 21, 2010 3:06pm
Quoting Craig
I use C++, but it's also the language I use at my day job. I'm now live with IB & IQFeed, so far the IB gateway has worked flawlessly (touch wood), the only trouble I'm having is processing the amount of data I'm getting via IQFeed, which is currently 112 tick streams, I can guarantee you won't get that kind of throughput in C#
I think dropping down to C++ for critical areas like the one you suggested is always going to be on the cards. I am only processing (just saving) 25 forex tick streams at the minute, but I am using threading to make sure the socket is always attended to.

I agree though for that many tick streams C++/C is the right choice.
Economists have forecast 9 out of the last 5 recessions
  • #758
  • Apr 21, 2010 3:15pm Apr 21, 2010 3:15pm
Quoting spekkiefx
I think you made the right choice. When I read your first comment I thought hmmm. Performance is initially not the main problem. When you use the environment you used to this will save you a lot of time. Finding something that works is the main problem. Unless you use C++ daily this is a fiddly language.

Saying that I do believe linux/unix has to be at the core of it. I use fedora wich has java/eclipse completely integrated. Developing/running things in this environment is just rock solid and way better than any windoze environment.
I use oracle...

Thanks Spekkie,

I think Windows server specially later ones are as stable as Linux, as as Sql server and the .NET framework.

http://uptime.netcraft.com/up/today/top.avg.html

The only downside is the updates when you have to reboot, but these can be don on weekends when he market is closed.
Economists have forecast 9 out of the last 5 recessions
  • #759
  • Apr 21, 2010 3:40pm Apr 21, 2010 3:40pm
Quoting markmm
Thanks Spekkie,

I think Windows server specially later ones are as stable as Linux, as as Sql server and the .NET framework.

http://uptime.netcraft.com/up/today/top.avg.html

The only downside is the updates when you have to reboot, but these can be don on weekends when he market is closed.
I am not just talking about stability. In my view it is just a superior environment in terms of "achieving" things. From simple things like cron and shell scripts. I also appreciate that you can achieve all the same things on a windoze environment but seems less instinctive. No surprise that any serious database I have worked on in my career is always unix based. This is for a reason. On top of that linux(fedora) provides the rest ssh vnc gnome yum java eclipse etc..
  • #760
  • Apr 21, 2010 3:47pm Apr 21, 2010 3:47pm
Something I have noticed on the whole Linux/Windows thing is that numerical code generated by g++ runs an order of magnitude faster than code generated by VS2008. But you can't beat VS2008 for a development/debug environment, I develop on Windows then compile the code on my Linux box, the code differences between the two platforms are small when you use Boost.
The breaking of a wave cannot explain the whole sea.
1 trader viewing now