Systematic trading | Page 16 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Trade another day through Systematic Trading 203 replies

Nonparametric Association Measures in Systematic Trading 57 replies

Anyone use Systematic from Alpari? 1 reply

A Systematic Approach to Markets 68 replies

Systematic Trading vs. Discretionary Trading 14 replies

Attachments: Systematic trading
Exit Attachments

Systematic trading

  • #301
  • Aug 28, 2009 11:35am Aug 28, 2009 11:35am
Quoting Craig
Most of the strategies in the PLAT papers mentioned above disregard historical price, so for those types of strategy one can visualize a reduced state space. Obviously if we add book data to our existing information, this is not the case.
Ok, I haven't read the papers.. Basically order book tells everything assuming all/most orders are entered via pending orders. It would be very interesting to do some data mining based on longer term book data [especially s/r levels].
  • #302
  • Aug 28, 2009 11:35am Aug 28, 2009 11:35am
Quoting wtmckee
I've spent a good part of my Friday, and numerous beers, reading this thread....
I'm sorry dude, but it takes years there is no shortcut, not any lasting one.
Speaking only for myself, I've been basically working 2 jobs for 4 years now to get this stuff off the ground, many false starts, endless frustration. I went though all the same useless rubbish that fills this and other sites in search of good information, it took years. It comes back to the same old cliche, 'how bad do you want it'?.

To answer your question, I'm not familiar with the FXCM systems, but as a rule of thumb, if they are selling it or giving it away, it's worthless.
The breaking of a wave cannot explain the whole sea.
  • #303
  • Aug 28, 2009 11:40am Aug 28, 2009 11:40am
Quoting mikkom
Ok, I haven't read the papers.. Basically order book tells everything assuming all/most orders are entered via pending orders. It would be very interesting to do some data mining based on longer term book data [especially s/r levels].
Indeed! I have heard conflicting opinions over whether edges exist in book data, some people have said there are none due to the highly nonlinear price impact of orders. But also I have heard via trusted sources that this all all some people do. Back testing is obviously more problematic due to the sheer volume of data, have you seen the jbooktrader project?
The breaking of a wave cannot explain the whole sea.
  • #304
  • Aug 28, 2009 11:44am Aug 28, 2009 11:44am
Quoting Craig
I'm sorry dude, but it takes years there is no shortcut, not any lasting one.
Speaking only for myself, I've been basically working 2 jobs for 4 years now to get this stuff off the ground, many false starts, endless frustration. I went though all the same useless rubbish that fills this and other sites in search of good information, it took years. It comes back to the same old cliche, 'how bad do you want it'?.
I totally agree with this - there are no shortcuts. I have been trading a little less than 10 years [I did my first genetic algo test some 8 years ago or something, my computer ran about 100% for 2 weeks or soemthing] and founded 2 companies and worked as a CTO at 2 places during that time - that should count at least 3 jobs simultaneously

All I can say is that you have to love this or it's pointless - I do this because I really love trading, market is a gigantic puzzle for me [and I love puzzles]. I could easily found another company and make some good money (we sold most of our current company 2 years ago) but this is really what I want to do long term.
  • #305
  • Aug 28, 2009 11:47am Aug 28, 2009 11:47am
Quoting Craig
Indeed! I have heard conflicting opinions over whether edges exist in book data, some people have said there are none due to the highly nonlinear price impact of orders. But also I have heard via trusted sources that this all all some people do. Back testing is obviously more problematic due to the sheer volume of data, have you seen the jbooktrader project?
I have been trading manually book open [IB] and I would claim that you can see s/r levels from book data (there is a huge concentration of orders sitting there). What I'm interested if there is an edge in there and how big the order deviation must be from the normal for a short term bounce.

No, I haven't looked at jbooktrader at all.

by the way, I was looking at this some weeks ago, it's probably very expensive but it's quite interesting anyway at least for data mining purposes
http://www.eurexchange.com/market/hi...erbook_en.html
  • #306
  • Aug 28, 2009 11:51am Aug 28, 2009 11:51am
Quoting mikkom
market is a gigantic puzzle for me [and I love puzzles]. I could easily found another company and make some good money (we sold the last one 2 years ago) but this is really what I want to do long term.
I am in a similar position, I get payed spectacularly well for my 'day job', and I simply will not quit until my trading exceeds these income levels consistently and at the moment I have had a pretty flat summer (winter here) so there is still more work to do. I think what you mention about puzzles etc. is important. One has to have a passionate curiosity and a willingness to work and be wrong.
The breaking of a wave cannot explain the whole sea.
  • #307
  • Aug 28, 2009 1:00pm Aug 28, 2009 1:00pm
Here is another blog I read... http://puppetmastertrading.com/blog/
This guy has also been developing a trading platform, there are many interesting articles in the archives.
The breaking of a wave cannot explain the whole sea.
  • #308
  • Aug 28, 2009 1:45pm Aug 28, 2009 1:45pm
I was cleaning and going through some old piles I had taken from from the office and found this, it's worth reading for everyone who haven't seen it. (pdf download at download link)

http://papers.ssrn.com/sol3/papers.c...ract_id=888805
  • #309
  • Aug 29, 2009 1:18am Aug 29, 2009 1:18am
This was posted to another thread (behind price-orderflow, http://www.forexfactory.com/showthre...=193795&page=3), research about various things including market microstructure and seems worth reading.

http://people.brandeis.edu/~cosler/ (research-link)
  • #310
  • Edited 5:36am Aug 29, 2009 2:15am | Edited 5:36am
Little bit more eyecandy with random variation of some algos for 4 pairs, I'm wondering what stats are relevant to the full picture - maybe sortino - but this is already quite good.

So the next step in my plan is to run gazillions of algos for multiple pairs and adjust the algo collection based on clustering of algos [and possibly by some other aspects]. After the collection I'll run a separate test on period that is not included on the first part (so called validation period) and see what the results are - now this is getting very interesting.

And it's still about 1 sec per algo
Attached Image (click to enlarge)
Click to Enlarge Name: Screenshot-AlphaFactory-8.png Size: 49 KB
  • #311
  • Aug 29, 2009 11:22am Aug 29, 2009 11:22am
I assume you have written that stuff in Java? What are you using to draw the graphs?

I have a console application written in C#, I have tried porting this to Mono but that project still has a way to go, is it possible to give an old C++ hack some tips on where to start with GUI work in Java?
The breaking of a wave cannot explain the whole sea.
  • #312
  • Edited 3:28am Aug 30, 2009 1:36am | Edited 3:28am
Quoting Craig
I assume you have written that stuff in Java? What are you using to draw the graphs?

I have a console application written in C#, I have tried porting this to Mono but that project still has a way to go, is it possible to give an old C++ hack some tips on where to start with GUI work in Java?
Yes, that is java, I just open a window (jframe/jpanel in java) and use the basic graphics functions to draw everything.

The best place to start is suns Java tutorial,
http://java.sun.com/docs/books/tutorial/2d/index.html
and
http://java.sun.com/docs/books/tutor...ing/index.html
  • #313
  • Edited 10:59am Aug 30, 2009 3:04am | Edited 10:59am
I have been quite busy

Here is one of the first runs [edit:updated a new pic] with testing/verification period separated. All algos are random variations of something I know works and are tested/filtered using pre 2005 data, curve >= 2005 is so called out of sample period. (the dots on the right panel show all algos profit(y)/dd(x) in graphical mode)

Not so bad and tells that at least some methods do work on longer term data - most of the algos are so simple and trade so many times that curvefitting is very hard - IMHO the most important thing in doing algorithmic algo discovery is to eliminate all possibility of curve fitting, easier said than done.

My biggest problem right now is that when I have a lot of algos, the profit always goes to infinity and messes screen up - I have to invent some kind of common relative risk adjustment algo on top of this.

Man I love having my own framework and being able to tweak everything, I should have ditched the awful mt4 tester a long time ago.
Attached Image (click to enlarge)
Click to Enlarge Name: Screenshot-AlphaFactory-24.png Size: 97 KB
  • #314
  • Edited 11:21am Aug 30, 2009 8:51am | Edited 11:21am
Here is one more screenshot from the testing platform - I want to show something here.

This is 2009 data (a hard year by all aspects), this simulation uses algos filtered by their properties during 2001 - 2005-1-1 (eurusd, gbpusd, usdjpy, usdchf) - so 2009 is "out of sample" and is not curvefitted. What I want to show here is that when you have simple enough algos, the edge is there even with harder markets - good edges don't just disappear overnight.

I have to be honest here, I really didn't thought the discovery method would work this well - I assumed out of the sample periods would give much worse results than in sample but it seems there really isn't much difference which is quite unbelievable.

The best thing with this is that now it's the computer that does basically the same work I have been doing manually with mt4 and it's a lot faster than me.
Attached Image (click to enlarge)
Click to Enlarge Name: Screenshot-AlphaFactory-23.png Size: 27 KB
  • #315
  • Aug 30, 2009 11:29am Aug 30, 2009 11:29am
This all looks very nice, but I'm still not altogether sure what you mean by 'algorithmic discovery', are you permuting parameters, or are you actually permuting the algorithm? I'll completely understand if you don't want to answer this question
The breaking of a wave cannot explain the whole sea.
  • #316
  • Aug 30, 2009 12:02pm Aug 30, 2009 12:02pm
Quoting Craig
This all looks very nice, but I'm still not altogether sure what you mean by 'algorithmic discovery', are you permuting parameters, or are you actually permuting the algorithm? I'll completely understand if you don't want to answer this question
I'm doing both.
  • #317
  • Aug 30, 2009 12:06pm Aug 30, 2009 12:06pm
So do you have a preset collection of strategies (breakout etc), which you then permute parameters for, or are you actually constructing algos on the fly?
The breaking of a wave cannot explain the whole sea.
  • #318
  • Aug 30, 2009 12:26pm Aug 30, 2009 12:26pm
Quoting Craig
So do you have a preset collection of strategies (breakout etc), which you then permute parameters for, or are you actually constructing algos on the fly?
Sorry but this is getting a bit too specific.
  • #319
  • Edited 12:47pm Aug 30, 2009 12:34pm | Edited 12:47pm
Quoting mikkom
Sorry but this is getting a bit too specific.
Fair enough, the results look great, if you are not in fact cherry picking the out of sample runs then you have in essence solved the biggest problem in back testing (i.e. will it work out of sample).
Edit: On the subject of algo construction, I once implemented a genetic programming algo generator which used GA to combine rules, but it was the same old story OOS.
The breaking of a wave cannot explain the whole sea.
  • #320
  • Edited 1:10pm Aug 30, 2009 12:45pm | Edited 1:10pm
Quoting Craig
Fair enough, the results look great, if you are not in fact cherry picking the out of sample runs then you have in essence solved the biggest problem in back testing (i.e. will it work out of sample).
I think it's the power of simplicity and the fact that I dump all algos that don't work on all the pairs I test with.

I'm still unsure about the results, this kind of testing is always problematic because I have done some really serious caching so there might be some bug somewhere that allows algos to cheat [this has happened few times in the past], I think I'm wiser few weeks from now. In some way this seems too good to be accurate.

edit: I found one bug that made the results better than they actually are, let's see how this goes from now.. So the pics I posted were not totally accurate. (The in-sample and out-sample periods still look the same)
1 trader viewing now