Systematic trading | Page 43 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Trade another day through Systematic Trading 203 replies

Nonparametric Association Measures in Systematic Trading 57 replies

Anyone use Systematic from Alpari? 1 reply

A Systematic Approach to Markets 68 replies

Systematic Trading vs. Discretionary Trading 14 replies

Attachments: Systematic trading
Exit Attachments

Systematic trading

  • #841
  • May 25, 2010 3:20pm May 25, 2010 3:20pm
Quoting Craig
You can do C++ on both Win32 and Linux with very few of differences in the source code (e.g. where to log), certainly (for me) using VS to debug is a lot easier than gdb. If you use boost you should not really have to write any platform specific code, the only differences should be your configuration values.

Also, unless you really, really know what you are doing I would avoid multi-threading, I handle 100's of quote streams in a single thread no problem, if I need more, I scale out (extra processes).
QT creator is great for debugging just like VS, I also find GDB to be quite powerful. Multi-threading is something I find its speeds up networking based operations, but as you say it can be difficult if you don't know what you are doing. Keeping it simple is key like having a thread read the socket and one processing the data (using the proper locks on the shared data), so 2 or 2 threads max, nothing crazy like one thread per stream.

There is a great set of lectures from Stanford University on C and a few are on multi-threading. Very useful as we move to multicore world.

Mark
Economists have forecast 9 out of the last 5 recessions
  • #842
  • May 25, 2010 3:20pm May 25, 2010 3:20pm
Quoting mikkom
About the smoothness, it is mostly a phenomenon of quite good combination algo, individual strategy curves are not even nearly as smooth as total curve (note the winning percentage of 57.9% and see below for example - that is one of the strategies included in the combination (risk on the pic below is not the same as it would be for the strategy in the combined curve before)
I'm quite intrigued by the concept of a combination algo, but admit that I have little to no experience or frame of reference. Are there some basic pointers, principles or a general philosophy to designing a successful algo or does it come down to trial and error? Offhand is the principle that profits beget profits a good place to start? Any references? Thanks!
  • #843
  • May 25, 2010 3:36pm May 25, 2010 3:36pm
Quoting FXEZ
I'm quite intrigued by the concept of a combination algo, but admit that I have little to no experience or frame of reference. Are there some basic pointers, principles or a general philosophy to designing a successful algo or does it come down to trial and error? Offhand is the principle that profits beget profits a good place to start? Any references? Thanks!
In your explorations you can find multiple profitable systems - for example a trending one and a mean reversion one. By trading both at the same time you'll get in general a smoother ride.

If you want some theory read about portofolio construction, for example here (chapter 6): http://books.google.com/books?id=pEM...page&q&f=false
  • #844
  • May 25, 2010 3:58pm May 25, 2010 3:58pm
The Algorithmic Trading & DMA book also has a very accessible chapter on portfolio construction.
The breaking of a wave cannot explain the whole sea.
  • #845
  • May 25, 2010 4:16pm May 25, 2010 4:16pm
Quoting markmm
QT creator is great for debugging just like VS, I also find GDB to be quite powerful.
I keep meaning to have a look at QT Creator, because Coding Blocks is rubbish.
The breaking of a wave cannot explain the whole sea.
  • #846
  • May 26, 2010 1:00am May 26, 2010 1:00am
There are lots of resources about portfolio construction, one starting point could be to read about modern/post-modern portfolio theory and continue from there.
  • #847
  • May 26, 2010 6:28am May 26, 2010 6:28am
Quoting markmm
Hi Guys, good to see the progress and people having results live....
Mark I admire your persistence with C/C++. The fact that you are up and forward between c# indicates it is somewhat of a struggle and I truly wander if it is really necessary. I believe approach is more important than speed of language (this is where the real time is saved). Also once you know a language well, you know what not to do.

I am pretty sure you can do 100s of symbols with java (especially because multithreading is at the core of the language) and I would also like to point out that java is the much preferred api for a lot of Forex Brokers. (IB,DukasCopy,Currenex).

I now have the setup I desire. I have a remote linux vps that collects data into a database and executes strategies. I had to do this as my broadband connection can be flaky at times (also latency is much better plus they provide redundant networks). I then copy the data into my local overclocked watercooled power datawarehouse database where I can do the analysis. The remote database is as if it is local using ssh tunnels so I can manipulate it any way I want.
With most linux flavours you can install eclipse with yum/apt-get and you have the ideal development environment. Java comes with a jconsole which enables you to find bottlenecks.

KISS is the motto. The most important thing is to find something that works. Unless you are an experienced C programmer it is not unlikely you will end up that (-:
  • #848
  • May 26, 2010 7:24am May 26, 2010 7:24am
Quoting spekkiefx
Mark I admire your persistence with C/C++. The fact that you are up and forward between c# indicates it is somewhat of a struggle and I truly wander if it is really necessary. I believe approach is more important than speed of language (this is where the real time is saved). Also once you know a language well, you know what not to do.

I am pretty sure you can do 100s of symbols with java (especially because multithreading is at the core of the language) and I would also like to point out that java is the much preferred api for a lot of Forex...
I have already written most of the quote stuff in C, I don't see why what the broker uses is important, I prefer to have a decoupled system using interfaces or services that are implementation independent. I prefer to rely only on some standard interface/protocol that I know will not change than a brokers badly written API.

I would never use Java, I find it extremely slow, I know pointers and memory management can be scary to some but I find it satisfying when I run my program through Valgrind and know there are know memory problems, and my system will run extremely efficiently. I find C/C++ easy to program with especially when using good libraries such as Boost.

I would rather spend a little more time developing to save millions of clock cycles later.
Economists have forecast 9 out of the last 5 recessions
  • #849
  • May 26, 2010 7:38am May 26, 2010 7:38am
Quoting markmm
I have already written most of the quote stuff in C
Fair enough. The quote engine is the easy part, how are you progressing with some type of workable strategy/ideas? Have you done any concrete research yet? I use a database not because it is fast but it helps speed up the R&D process. This is the same reason I use JAVA. Many times I have tried things just to come to a dead end. Worrying about pointers and cpu cycles is not my priority but everybody to their own.
  • #850
  • May 26, 2010 7:49am May 26, 2010 7:49am
Quoting spekkiefx
Fair enough. The quote engine is the easy part, how are you progressing with some type of workable strategy/ideas? Have you done any concrete research yet? I use a database not because it is fast but it helps speed up the R&D process. This is the same reason I use JAVA. Many times I have tried things just to come to a dead end. Worrying about pointers and cpu cycles is not my priority but everybody to their own.
A better Analogy would be using object database as its easier to program against than a relational database from an OO language (no impedance mismatch). But people keep using relational DB because they are faster, I will accept the fact that I need to map objects to tables etc, or handle memory (which can be eliminated if you use scoped_ptr etc). The benefits Java gives over C++ in terms of productivity is exaggerated, many of the concepts already in C++ took many version to come to Java (albeit running slower).
Economists have forecast 9 out of the last 5 recessions
  • #851
  • May 26, 2010 8:04am May 26, 2010 8:04am
Quoting markmm
I prefer to rely only on some standard interface/protocol that I know will not change than a brokers badly written API.
Does this mean you use the FIX protocol?
  • #852
  • May 26, 2010 8:21am May 26, 2010 8:21am
Quoting spekkiefx
Does this mean you use the FIX protocol?
Yeah, but I plan to only implement the parts that I need. What broker do you use spekkie? I want to use IB but the whole 25k min for day trading stocks is a bit steep. Wonder if the law is the same in the UK.
Economists have forecast 9 out of the last 5 recessions
  • #853
  • May 26, 2010 8:45am May 26, 2010 8:45am
Quoting markmm
Yeah, but I plan to only implement the parts that I need. What broker do you use spekkie? I want to use IB but the whole 25k min for day trading stocks is a bit steep. Wonder if the law is the same in the UK.
I use IB but I only focus on Forex. I don't think it applies for the UK exchange but because of the stamp duty(0.5% on shares) I doubt they are interesting. When buying on US exchange (from UK account) I assume it is the same. Perhaps the german/french exchanges are interesting, I don't believe they have stamp duty or rules like that (not sure though). You could also use options/futures instead of stocks, those rules do not apply for them as far as I know.
  • #854
  • Edited 9:16am May 26, 2010 9:06am | Edited 9:16am
Has any of you traded live with Dukascopy?

I like the fact that they have the most ticks of all the brokers I tested, and that they also offer 10 levels of market depth for each pair. But I don't like the fact that they don't seem to be FIFO, since they allow positions in both directions at the same time. And of course, they still didn't receive theirs banking license.

I think that I will start recording their depth of market, so that I will have data to work with in the future. I need a live account for that, since they show demo orders in the demo market.

HotspotFX seems really nice, they offer full event stream data, you can see each order come in, each cancel and so on. And all quotes/volumes are executable. But I couldn't find any broker which offers HotspotFX access, at least for retail level (~$20.000 account).

http://data.hotspotfxi.com:9090/faqs.jsp
http://data.hotspotfxi.com:9090/sample.zip - sample tick data (both aggregated and event stream)

EDIT: Sample event stream data:
Inserted Code
18:01:44.392,10627,N,1,1.48647,1000000
18:01:44.392,10628,N,1,1.48637,2000000
18:01:44.393,10629,N,1,1.48617,2000000
18:01:44.393,10630,N,1,1.48587,2000000
18:01:44.393,10542,C,,,
18:01:44.393,10631,N,1,1.48655,5000000
18:01:44.393,10543,C,,,
18:01:44.393,10632,N,2,1.48700,5000000
18:01:44.394,10604,C,,,
18:01:44.396,10633,N,2,1.48705,100000
18:01:44.396,10561,C,,,
18:01:44.396,10562,C,,,
18:01:44.396,10563,C,,,
18:01:44.396,10634,N,1,1.48662,2500000
18:01:44.396,10635,N,1,1.48667,1500000
18:01:44.396,10636,N,2,1.48692,1000000
18:01:44.397,10586,C,,,
....................
18:12:40.874,18582,N,2,1.48702,1000000
18:12:40.875,18583,N,2,1.48712,1000000
18:12:40.933,18580,C,,,
18:12:40.933,18581,C,,,
18:12:40.934,18584,N,1,1.48690,2000000
18:12:40.934,18585,N,2,1.48720,2000000
18:12:40.940,18584,M,,,1880000
18:12:40.941,18585,C,,,
18:12:40.941,18584,C,,,
18:12:40.961,18578,C,,,
18:12:40.961,18586,N,1,1.48665,100000
18:12:40.963,18587,N,2,1.48740,4000000
18:12:40.963,18588,N,2,1.48710,100000
18:12:41.024,18540,C,,,
18:12:41.024,18589,N,1,1.48660,5000000
18:12:41.106,18582,C,,,
18:12:41.106,18583,C,,,
18:12:41.277,18590,N,2,1.48703,1000000
18:12:41.282,18591,N,2,1.48713,1000000
18:12:41.306,18579,C,,,
18:12:41.306,18592,N,1,1.48657,900000
18:12:41.979,18571,C,,,
18:12:41.980,18593,N,1,1.48652,1000000
18:12:46.869,18594,N,1,1.48650,100000
18:12:46.871,18565,C,,,
18:12:46.871,18595,N,1,1.48665,250000
18:12:46.887,18594,C,,,
18:12:46.887,18596,N,1,1.48655,100000
All the events during two periods of EURUSD. C is Cancel Order with ID in the previous field, N is new order, M is modify order There are 93000 events in this sample hour, which means 25 events/sec on average. Of course, you have to build your own order book from this raw data, but like they say, you don't have to wait 50 ms for their servers to build the book and send you the updates.

In my opinion only systems working on this kind of data need maximum efficiency (C++, ...)
  • #855
  • Edited 1:29pm May 26, 2010 1:12pm | Edited 1:29pm
Quoting Adal
In my opinion only systems working on this kind of data need maximum efficiency (C++, ...)
25 events in a second is not that much in any language and if one really needs a lot of processing power, in the end cluster is your only option and therefore language doesn't matter that much (architecture in both hardware and software does).

I'm not saying c/++ is not a good choice but I wouldn't use it because of performance reasons - good analogy is to look at what languages [mobile] telephone centers for example are programmed - you would be surprised (one of those languages is one of my favourites, erlang which is far from the most efficient language out there)

edit: by the way if someone hasn't seen this, here is an interesting link
  • #856
  • May 26, 2010 1:28pm May 26, 2010 1:28pm
Quoting mikkom
25 events in a second is not that much in any language and if one really needs a lot of processing power, in the end cluster is your only option and therefore language doesn't matter that much (architecture does).

I'm not saying c/++ is not a good choice but I wouldn't use it because of performance reasons - good analogy is to look at what languages [mobile] telephone centers for example are programmed - you would be surprised (one of those languages is one of my favourites, erlang which is far from the most...
Well, I'm with you on this, my trading system is written in Python, with manual execution, so that I can double-check the market, to make sure it doesn't trade during a spike or something.

I was just saying that I could understand wanting to use C++ for such a system. Also, this was 18:00. I'm sure that on NFP release there are a lot more than 25 events/sec.

Charting on the other hand, might require something more performant. I've written a small charting class, which can scroll instantly through 1.5 mil M1 candles. CPU usage doesn't pass 10% when scrolling (on a quad). It's written in C++/CLI, and drawn with Direct2D. The scrolling speed remains the same for a small window or for a maximized window, since Direct2D is hardware accelerated. It's currently faster than NinjaTrader charting, which is the fastest from what I've tested, but I am missing some features. In NinjaTrader you can see the scrolling speed drop when you maximize the window.

C++/CLI is nice because I can write usual C++ stuff, and do only a small glue class which will be called from IronPython. In the past I used ctypes to invoke C++, and it's more work. Besides, I can now work in 64 bit, unlike with CPython which has some problems here (at least on Windows). 64 bit is nice since I can map all the ticks/bars into memory and not worry about running out of address space.
  • #857
  • May 27, 2010 7:15am May 27, 2010 7:15am
This might be of interest to some of people here..

http://www.tradeworx.com/TWX-SEC-2010.pdf
  • #858
  • May 27, 2010 8:08am May 27, 2010 8:08am
Quoting mikkom
This might be of interest to some of people here..

http://www.tradeworx.com/TWX-SEC-2010.pdf
Very nice. Where do you get this papers?

Quote
Small returns are appealing to HFT because they are very plentiful:
* typical HF trade: expected return = 0.1 cps after costs, standard deviation = +/- 5 cents
* after 100 such trades: expected return = 0.1 cps; standard deviation = +/- 0.5 cps
* if you do 100 such trades per day, for full year: sharpe ratio of 3.2
* if you do 10,000 such trades per day, for full year: sharpe ratio of 32
This is exactly the reason I'm trying to increase the frequency of my system. I'm trying to go from 1-3 trades per week (per instrument) to 5 per day. Long way to go till 10,000/day
  • #859
  • May 27, 2010 12:04pm May 27, 2010 12:04pm
Quoting Adal
Very nice. Where do you get this papers?

This is exactly the reason I'm trying to increase the frequency of my system. I'm trying to go from 1-3 trades per week (per instrument) to 5 per day. Long way to go till 10,000/day
Need a very good edge for find 10000 good trades/day
  • #860
  • May 27, 2010 7:42pm May 27, 2010 7:42pm
Increasing the frequency of returns stabilizes the probability distribution of your profits, in turn, this allows one to exploit smaller edges due the the increased confidence in having a positive expectation. This is the beauty of automation.
The breaking of a wave cannot explain the whole sea.
0 traders viewing now