Forex Factory (https://www.forexfactory.com/)
-   Trading Journals (https://www.forexfactory.com/forum/49-trading-journals)
-   -   Veloce algo trading (https://www.forexfactory.com/thread/230231-veloce-algo-trading)

Adal Apr 7, 2010 5:27pm | Post# 1

Veloce algo trading
 
-

Adal Apr 7, 2010 5:42pm | Post# 2

Tick downloaders
 
My FXCM and Dukascopy tick downloaders working (click for full size):

http://dl.dropbox.com/u/190212/ff/tick.png

The dbFX tick downloader is very similar to the FXCM one from above, since it's using the same API.

Now the GAIN Capital (forex.com) history data .zip downloader:

http://dl.dropbox.com/u/190212/ff/tick_gain.png

And the GAIN Capital .zip to tick convertor:

http://dl.dropbox.com/u/190212/ff/ti..._convertor.png

This concludes my tick handlers.

Adal Apr 7, 2010 6:07pm | Post# 3

Generating M1 candles
 
In the last months I have downloaded tons of tick data. Over 2 billion ticks on 150 instruments (but some, like EUR/USD, are counted multiple times - once for each broker).

The tick data occupies 11 GB in compressed form. If I would convert that to normal .CSV files it would consume 187 GB.

I've decided to generate only M1 candlestick data from the ticks, just like MetaTrader 5. The other time frames will be dynamically generated when needed.

http://dl.dropbox.com/u/190212/ff/generate_m1.png

I prefer working in Python when possible. The tick downloaders are 100% Python. But computing the M1 candles for just one instrument, like in the picture above took 10 minutes. So I wrote a small C++/Python extension which implements the performance-critical candle generation code. With this, I can build the full M1 data for 3 years of EUR/USD in 20 seconds. That's 30 times faster

The compressed M1 candlestick data for all my ticks weighs in at 863 MB. Much easier to handle than the full tick data.

markmm Apr 7, 2010 6:07pm | Post# 4

Hi Adal looking forward to your thread, I used a php script to get all the tick and 1 min data from dukascopy, from this guy . Good luck with your trading.

Adal Apr 7, 2010 6:19pm | Post# 5

Hi Adal looking forward to your thread, I used a php script to get all the tick and 1 min data from dukascopy, from this guy . Good luck with your trading.
Thanks!

I know that site. In fact if it wasn't for him, I wouldn't have found out that Dukascopy data is so good and so easy to download (the easiest to work with from my 4 tick sources).

spekkiefx Apr 7, 2010 6:44pm | Post# 6

Tick Data
 
When testing your strategies are you going to use the tick data or the 1 min bars or both?

Just wandering.

Adal Apr 7, 2010 6:53pm | Post# 7

When testing your strategies are you going to use the tick data or the 1 min bars or both?

Just wandering.
Since the M1 data is built from the tick data, I can use M1 for most purposes. I store for each candle the real O H L C values for both the bid and the asks.

So when I test I can truly work on real market prices. I have one more trick here: I added to each candle two more flags, which store what happened first: the low or the high. In this way I know the exact order in which the 4 points were hit.

Obviously, the lower the time frame I use and the smaller the profit target gets, the more important is working with real tick data. But I will limit myself to maximum M5 for start.

This is only when developing my system.

Before deploying the algo for real, I will obviously test it with full tick data.

spekkiefx Apr 7, 2010 7:03pm | Post# 8

Since the M1 data is built from the tick data, I can use M1 for most purposes. I store for each candle the real O H L C values for both the bid and the asks.

So when I test I can truly work on real market prices. I have one more trick here: I added to each candle two more flags, which store what happened first: the low or the high. In this way I know the exact order in which the 4 points were hit.

Obviously, the lower the time frame I use and the smaller the profit target gets, the more important is working with real tick data. But I will limit...
There is no point to build OHLC for bid and ask, just use the mid. If you want to be as realistic as possible (real market prices) then use the tick data. Obviously this is rather resource hungry when testing but the only thing you don't take into account then is slippage of execution. This is as close as it gets.

Adal Apr 7, 2010 7:22pm | Post# 9

There is no point to build OHLC for bid and ask, just use the mid. If you want to be as realistic as possible (real market prices) then use the tick data. Obviously this is rather resource hungry when testing but the only thing you don't take into account then is slippage of execution. This is as close as it gets.
I will use FXCM or dbFX as my broker. They both have variable spread. If I store only the mid point I wouldn't know at what price to buy/sell. Also, before news, the spreads widen dramatically. 30 seconds before NFP/FOMC the spreads go to 20 points. 30 seconds after the news release they come back down, but still 2-3 points wider than usual for another 1 minute. But I probably wouldn't want my system to trade around news anyway.

The first system I want to test, an H4 with average holding period of 1-3 days, has take profit targets of 100 points, and stop losses of 300 with very good winning rates. So I really don't care even for 10 point real price difference/slippage for now.

BTW, I'm on a little known broker at the moment, CMC Markets (very ugly FIFO GUI, but which I've grown to like), and in hundreds of EUR/USD trades I think I was slipped only 3 or 4 times. Unfortunately they don't have any API, and the spreads are not exactly top-notch (3 points on EUR/USD). Hopefully my future broker will be just like that (non slipping)

The Fool May 11, 2010 4:42am | Post# 10

I'll use this thread to document my work on algo trading.
Hello Adal,

I am looking forward to lurking her to see what I can learn from you.

Happy Trading!

sebastionay May 12, 2010 2:56am | Post# 11

Adal please enable your PM function, I want to mail you.

Adal May 13, 2010 3:26am | Post# 12

Done. I was not aware it was disabled.

fxpilot Jul 30, 2010 6:25am | Post# 13

Hi Adal,

How is your research going? Very interesting topic. Looking forward to following along.

Adal Jul 31, 2010 7:48am | Post# 14

Hi Adal,

How is your research going? Very interesting topic. Looking forward to following along.
I post status updates here: http://www.forexfactory.com/showthread.php?p=3884271


© Forex Factory