Systematic trading

LOL. This world is smaller than you think.

Has anyone experimented with bayesian filtering or directional graphs (contextual network graphs for example) for clustering price information / mapping possible outcomes? Basically you should be able to determine what events are more likely to happen next assuming there is some correlation.

I have used them and some other clustering methods in the past with quite good success in the past with a very different kind of a project so I was thinking how well they could manage with trading.

Hi Mikkom, quick question about build strategy in java scripts, we need write sub-functions to analysis Equity/balance/win percentage/average loss/average win/ ?

If so, it's an good idea to open an open-source project to write the sub-functions, so that some of the engineers in this thread can contribute.

I'm not Mikkom and I'm also not 100% sure what you are asking for, but if you are looking for an open source, Java based trading platform try googling 'jsystemtrader'.

Thanks Craig,

I am looking for backtesting enviroment, my appoach is close to Mikkom, "try to test long period of time data(10 years) to testify a strategy"

It should monitor multi pairs(portforlio), and can define my strategy just like MT4.

There are some commercial package available, but I am not ready to pay for the licence, so I am searching a backtest tool.

As a hardware engineer, I am good at scripts(Perl, Tcl, C), But if there is some function library(in any scripts language) that we commonly used in building our strategy, I am happy to use it.

Mikkom said that "The framework is still quite rough" in http://www.forexfactory.com/showpost...&postcount=260
There may be some common fuctions we all used in our stategy, such as (placeStopOrder, trailingStop), and some sub-fuctions we used in analysis the strategy performance( we need get the Total trades, win ratio, drawdown, averge win/loss, expectation)

If we can creat a project, share some of the common fuctions, it can save lot of time. so we can concentrate on the most critical problem


I am surfing around in google group JSystemTrader

These two might be of your interest as well

http://quantlib.org/index.shtml
http://www.marketcetera.com/site/

At the moment at least I have no plans to open my platform.

(I added all these to the first post, I try to keep some kind of a simple link library there)

There are also many commercial packages that can do portfolio backtesting, some include

http://www.smartquant.com/openquant.php
http://www.rightedgesystems.com/

Interesting tick movement Looks like few algos fighting each other

One of my experimental algos ones did this by mistake...so it might also be a broken algo. By the way, what feed is this?

On a more constructive note, I recently "rediscovered" the Penn-Lehman
Automated Trading Project:

http://www.cis.upenn.edu/~mkearns/projects/plat.html

I don't recall that it has been mentioned before in this thread. There are tons of interesting information on this site. I especially like that they emphasizes a game theoretical approach to designing trading strategies (think about who/what your algos are up against).

Here is the main article:

http://www.cis.upenn.edu/~mkearns/papers/plat.pdf

Too bad the project was shut down (not to speak about Lehman...).

Excellent post! I have all those papers printed in a pile under my desk, they are very interesting. The problem of course with doing this in forex is that you are only seeing the brokers order book. But I guess it is game on for futures or stocks. I while ago before I had my ATS I tried implementing the SOBI strategy using NinjaTrader on IB, I never really reached any firm conclusions.
I've often thought the book had good potential for AI applications, since it has a much smaller state space than price.

Time to put on the thinking cap.

BlackMage, thanks for the link - looks like excellent stuff. My feed is ActivTrades "interbank" account (basically some kind of ecn bridge with commission included in the spread, I'm not 100% sure where they connect to - I have asked them about this and they mentioned multiple big banks so I guess it's some liquidity pool), they basically forced me to move there from their "bucket" side because they didn't like taking opposite positions to my trades haha

Here is something different for people not understanding how much correlation means when using multiple systems / pairs - below is a copy from my testing framework results for a very simple algo with 3 pairs

[java] 3 [main] DEBUG com.af.AlphaFactory - ***** ALPHA FACTORY *****
[java] 441 [main] DEBUG com.af.AlphaFactory - Running strategy..
[java] 480 [main] DEBUG com.af.StrategyRunner - Loading data for eurusd
[java] 5596 [main] DEBUG com.af.ChartLoader - Loaded 2865258 candles (20010103 000100 - 20090701 235900)
[java] 7908 [main] DEBUG com.af.StrategyRunner - Running strategy for eurusd
[java] 10139 [main] DEBUG com.af.AlphaFactory - eurusd [1] 12.44% DD 2.88% (201 trades)
[java] 10140 [main] DEBUG com.af.AlphaFactory - TOTAL (201): 12.44% DD 2.88%
[java] 10176 [main] DEBUG com.af.StrategyRunner - Running strategy for eurusd
[java] 12626 [main] DEBUG com.af.AlphaFactory - eurusd [2] 6.94% DD 4.21% (171 trades)
[java] 12627 [main] DEBUG com.af.AlphaFactory - TOTAL (372): 20.24% DD 6.96%
[java] 12658 [main] DEBUG com.af.StrategyRunner - Running strategy for eurusd
[java] 15449 [main] DEBUG com.af.AlphaFactory - eurusd [3] 5.35% DD 3.78% (145 trades)
[java] 15450 [main] DEBUG com.af.AlphaFactory - TOTAL (517): 26.68% DD 10.13%
[java] 15478 [main] DEBUG com.af.StrategyRunner - Running strategy for eurusd
[java] 18709 [main] DEBUG com.af.AlphaFactory - eurusd [4] 5.8% DD 3.5% (126 trades)
[java] 18710 [main] DEBUG com.af.AlphaFactory - TOTAL (643): 34.03% DD 13.14%
[java] 18742 [main] DEBUG com.af.StrategyRunner - Running strategy for eurusd
[java] 23287 [main] DEBUG com.af.AlphaFactory - eurusd [5] 6.11% DD 3.12% (113 trades)
[java] 23289 [main] DEBUG com.af.AlphaFactory - TOTAL (756): 42.23% DD 15.85%
[java] 23325 [main] DEBUG com.af.StrategyRunner - Running strategy for eurusd
[java] 28305 [main] DEBUG com.af.AlphaFactory - eurusd [6] 2.9% DD 4.01% (110 trades)
[java] 28306 [main] DEBUG com.af.AlphaFactory - TOTAL (866): 46.35% DD 18.84%
[java] 28325 [main] DEBUG com.af.StrategyRunner - Running strategy for eurusd
[java] 32315 [main] DEBUG com.af.AlphaFactory - eurusd [7] 3.49% DD 3.65% (97 trades)
[java] 32316 [main] DEBUG com.af.AlphaFactory - TOTAL (963): 51.46% DD 21.54%
[java] 32332 [main] DEBUG com.af.StrategyRunner - Running strategy for eurusd
[java] 36977 [main] DEBUG com.af.AlphaFactory - eurusd [8] 5.7% DD 2.55% (86 trades)
[java] 36978 [main] DEBUG com.af.AlphaFactory - TOTAL (1049): 60.09% DD 23.54%
[java] 36996 [main] DEBUG com.af.StrategyRunner - Running strategy for eurusd
[java] 42164 [main] DEBUG com.af.AlphaFactory - eurusd [9] 6.49% DD 2.8% (79 trades)
[java] 42165 [main] DEBUG com.af.AlphaFactory - TOTAL (1128): 70.49% DD 25.68%
[java] 42179 [main] DEBUG com.af.StrategyRunner - Running strategy for eurusd
[java] 47926 [main] DEBUG com.af.AlphaFactory - eurusd [10] 7.06% DD 2.71% (73 trades)
[java] 47927 [main] DEBUG com.af.AlphaFactory - TOTAL (1201): 82.53% DD 27.65%
[java] 47942 [main] DEBUG com.af.StrategyRunner - Loading data for gbpusd
[java] 55570 [main] DEBUG com.af.ChartLoader - Loaded 2827681 candles (20010103 000100 - 20090701 235900)
[java] 57538 [main] DEBUG com.af.StrategyRunner - Running strategy for gbpusd
[java] 58786 [main] DEBUG com.af.AlphaFactory - gbpusd [1] 5.64% DD 4.88% (218 trades)
[java] 58787 [main] DEBUG com.af.AlphaFactory - TOTAL (1419): 92.83% DD 27.31%
[java] 58836 [main] DEBUG com.af.StrategyRunner - Running strategy for gbpusd
[java] 60421 [main] DEBUG com.af.AlphaFactory - gbpusd [2] 1.36% DD 4.99% (189 trades)
[java] 60422 [main] DEBUG com.af.AlphaFactory - TOTAL (1608): 95.46% DD 27.83%
[java] 60436 [main] DEBUG com.af.StrategyRunner - Running strategy for gbpusd
[java] 62278 [main] DEBUG com.af.AlphaFactory - gbpusd [3] 4.7% DD 3.07% (152 trades)
[java] 62278 [main] DEBUG com.af.AlphaFactory - TOTAL (1760): 104.64% DD 28.05%
[java] 62292 [main] DEBUG com.af.StrategyRunner - Running strategy for gbpusd
[java] 64557 [main] DEBUG com.af.AlphaFactory - gbpusd [4] 4.05% DD 2.2% (135 trades)
[java] 64558 [main] DEBUG com.af.AlphaFactory - TOTAL (1895): 112.92% DD 27.63%
[java] 64573 [main] DEBUG com.af.StrategyRunner - Running strategy for gbpusd
[java] 67252 [main] DEBUG com.af.AlphaFactory - gbpusd [5] 4.95% DD 1.97% (118 trades)
[java] 67253 [main] DEBUG com.af.AlphaFactory - TOTAL (2013): 123.46% DD 26.5%
[java] 67306 [main] DEBUG com.af.StrategyRunner - Running strategy for gbpusd
[java] 70603 [main] DEBUG com.af.AlphaFactory - gbpusd [6] 4.69% DD 2.24% (109 trades)
[java] 70604 [main] DEBUG com.af.AlphaFactory - TOTAL (2122): 133.93% DD 26.26%
[java] 70620 [main] DEBUG com.af.StrategyRunner - Running strategy for gbpusd
[java] 74583 [main] DEBUG com.af.AlphaFactory - gbpusd [7] 4.34% DD 2.21% (100 trades)
[java] 74584 [main] DEBUG com.af.AlphaFactory - TOTAL (2222): 144.08% DD 25.56%
[java] 74599 [main] DEBUG com.af.StrategyRunner - Running strategy for gbpusd
[java] 79291 [main] DEBUG com.af.AlphaFactory - gbpusd [8] 3.54% DD 2.27% (97 trades)
[java] 79292 [main] DEBUG com.af.AlphaFactory - TOTAL (2319): 152.73% DD 25.62%
[java] 79307 [main] DEBUG com.af.StrategyRunner - Running strategy for gbpusd
[java] 84671 [main] DEBUG com.af.AlphaFactory - gbpusd [9] 2.14% DD 2.51% (92 trades)
[java] 84671 [main] DEBUG com.af.AlphaFactory - TOTAL (2411): 158.14% DD 26.32%
[java] 84685 [main] DEBUG com.af.StrategyRunner - Running strategy for gbpusd
[java] 90756 [main] DEBUG com.af.AlphaFactory - gbpusd [10] 2.91% DD 2.2% (82 trades)
[java] 90757 [main] DEBUG com.af.AlphaFactory - TOTAL (2493): 165.64% DD 27.05%
[java] 90773 [main] DEBUG com.af.StrategyRunner - Loading data for usdjpy
[java] 97625 [main] DEBUG com.af.ChartLoader - Loaded 2880037 candles (20010103 000300 - 20090701 235900)
[java] 99097 [main] DEBUG com.af.StrategyRunner - Running strategy for usdjpy
[java] 101647 [main] DEBUG com.af.AlphaFactory - usdjpy [1] 8.83% DD 5.21% (205 trades)
[java] 101648 [main] DEBUG com.af.AlphaFactory - TOTAL (2698): 189.09% DD 27.47%
[java] 101674 [main] DEBUG com.af.StrategyRunner - Running strategy for usdjpy
[java] 104708 [main] DEBUG com.af.AlphaFactory - usdjpy [2] 8.5% DD 4.06% (176 trades)
[java] 104710 [main] DEBUG com.af.AlphaFactory - TOTAL (2874): 213.67% DD 28.07%
[java] 104734 [main] DEBUG com.af.StrategyRunner - Running strategy for usdjpy
[java] 108369 [main] DEBUG com.af.AlphaFactory - usdjpy [3] 5.79% DD 2.69% (149 trades)
[java] 108371 [main] DEBUG com.af.AlphaFactory - TOTAL (3023): 231.83% DD 28.91%
[java] 108395 [main] DEBUG com.af.StrategyRunner - Running strategy for usdjpy
[java] 112544 [main] DEBUG com.af.AlphaFactory - usdjpy [4] 6.98% DD 2.7% (126 trades)
[java] 112546 [main] DEBUG com.af.AlphaFactory - TOTAL (3149): 254.99% DD 29.89%
[java] 112570 [main] DEBUG com.af.StrategyRunner - Running strategy for usdjpy
[java] 117312 [main] DEBUG com.af.AlphaFactory - usdjpy [5] 7.79% DD 2.49% (109 trades)
[java] 117313 [main] DEBUG com.af.AlphaFactory - TOTAL (3258): 282.65% DD 30.68%
[java] 117354 [main] DEBUG com.af.StrategyRunner - Running strategy for usdjpy
[java] 122569 [main] DEBUG com.af.AlphaFactory - usdjpy [6] 5.27% DD 3.55% (101 trades)
[java] 122570 [main] DEBUG com.af.AlphaFactory - TOTAL (3359): 302.81% DD 31.67%
[java] 122602 [main] DEBUG com.af.StrategyRunner - Running strategy for usdjpy
[java] 127542 [main] DEBUG com.af.AlphaFactory - usdjpy [7] 5.82% DD 2.72% (93 trades)
[java] 127543 [main] DEBUG com.af.AlphaFactory - TOTAL (3452): 326.25% DD 32.36%
[java] 127562 [main] DEBUG com.af.StrategyRunner - Running strategy for usdjpy
[java] 133220 [main] DEBUG com.af.AlphaFactory - usdjpy [8] 5.1% DD 3.36% (86 trades)
[java] 133221 [main] DEBUG com.af.AlphaFactory - TOTAL (3538): 347.99% DD 33.19%
[java] 133240 [main] DEBUG com.af.StrategyRunner - Running strategy for usdjpy
[java] 139496 [main] DEBUG com.af.AlphaFactory - usdjpy [9] 5.24% DD 2.84% (80 trades)
[java] 139497 [main] DEBUG com.af.AlphaFactory - TOTAL (3618): 371.49% DD 34.18%
[java] 139516 [main] DEBUG com.af.StrategyRunner - Running strategy for usdjpy
[java] 146783 [main] DEBUG com.af.AlphaFactory - usdjpy [10] 4.83% DD 2.62% (76 trades)
[java] 146784 [main] DEBUG com.af.AlphaFactory - TOTAL (3694): 394.28% DD 35.02%

[number] means an algo variation (and not the best algo at all, just some testing stuff), all algos inside one pair are correlated, not totally but quite correlated anyway. Every trade risk is 0.5%. Note how profit percentage goes up when new pairs are added but the drawdown remains the same or even goes lower. Note especially how profit% doubles when gbp is added but max DD remains about the same.

Just something to ponder for those people who haven't thought this before.

Here is some eyecandy from the tester I'm working with This is one of the first runs where this nice visual profit curve thing is working (this is not monte carlo, the gray lines are algo variations - this is just a test algo, the same one as in previous post with some more variation - 60 algo variations per pair, 4 pairs).

White line is balance for combined exponential risk (risk based on current balance), blue line is combined balance with fixed risk always based on initial saldo - fixed risk shows the curve much better, exponential hides the begin of the curve (as can be seen from the pic).

So now this is getting quite interesting - next thing will be the discovery algorithm.

Nice, just goes to show that once you have the freedom to really do large scale testing, the sky is the limit! In a certain sense all we are really doing is developing search engines. I'd like to post screen-shots, but it would give away too much. One of the fun things about writing such a project is thinking of a name, 'Alpha Factory' is pretty cool, I still havn't thought of a good one.

The name was a bit of a joke (you have to make something up for package names and so on and I don't want to refactor evetrything after few months) but I kind of like it so it's probably there to stay.

By the way I mentioned donchian 4 week/1 week stop (also initial stop, I haven't implemented atr yet) some time ago, here is a curve for eurusd, gbpusd, usdjpy and usdchf just to show what it looks like and to keep you away from the awful mql4 (0.5% risk per trade)

edit: and a disclaimer here, this framework might still be full of bugs, especially with my very hardcore caching so don't trust these blindly (not that I think anyone would)..

edit2: Whoops forget this curve - the exit is not turtle, it's something else that I forgot on..

Are you planning any sort of execution engine?

Signing up.

Lenoxer

Probably yes but first I have planned to code some kind of algo to mt4 translator so I can jump right into trading and separate the development framework from execution framework.

I took a look at FIX but it was so fragmented [ib fix was very different from for example MBT] that I think it's probably easier to have some kind of platform independent framework with multiple pluggable execution implementations.

That is what I have, strategies don't refer to FIX, only to execution interfaces, this way I can retrofit different execution engines if need be.
As I said before, I don't do explicit back-testing, but if I wanted to, once again this would just be a matter of retrofitting the various interfaces to supply data from the database as opposed to MB.

The same state as with the price is still there even if you view just the book, you just have more information about the current price.

Talking of book, does anyone know if there are any sources that provide longer term book data?

I've spent a good part of my Friday, and numerous beers, reading this thread. And while I wish I could get that into a technical system, and while i researched some of the elements you mentioned (softwares, books, strategies, websites, theories, etc.) I doubt I'm going to have time it would take... I mean, the years it would take to get to where you are...

I trade, "discretionally", with the basic FXCM platform. I like trading on the charts, you see. And I recently put some funds in their "system selector" program. I just look at the histories and the P/L... After reading this thread I'm looking at the "start dates", understanding that systems can have a market life.

Are these systems (generic systems for all I know) worth two cents?

Is there a better way to go for a layman like myself? A better platform? Better systems?

I thank you for the interesting eye opening thread. Now I'll just go back to programming gas pumping stations... which is, now i understand, the easy stuff

Most of the strategies in the PLAT papers mentioned above disregard historical price, so for those types of strategy one can visualize a reduced state space. Obviously if we add book data to our existing information, this is not the case.