Veloce algo trading
My FXCM and Dukascopy tick downloaders working (click for full size):
The dbFX tick downloader is very similar to the FXCM one from above, since it's using the same API.
Now the GAIN Capital (forex.com) history data .zip downloader:
And the GAIN Capital .zip to tick convertor:
This concludes my tick handlers.
Generating M1 candles
In the last months I have downloaded tons of tick data. Over 2 billion ticks on 150 instruments (but some, like EUR/USD, are counted multiple times - once for each broker).
The tick data occupies 11 GB in compressed form. If I would convert that to normal .CSV files it would consume 187 GB.
I've decided to generate only M1 candlestick data from the ticks, just like MetaTrader 5. The other time frames will be dynamically generated when needed.
I prefer working in Python when possible. The tick downloaders are 100% Python. But computing the M1 candles for just one instrument, like in the picture above took 10 minutes. So I wrote a small C++/Python extension which implements the performance-critical candle generation code. With this, I can build the full M1 data for 3 years of EUR/USD in 20 seconds. That's 30 times faster
The compressed M1 candlestick data for all my ticks weighs in at 863 MB. Much easier to handle than the full tick data.
Hi Adal looking forward to your thread, I used a php script to get all the tick and 1 min data from dukascopy, from this guy . Good luck with your trading.
I know that site. In fact if it wasn't for him, I wouldn't have found out that Dukascopy data is so good and so easy to download (the easiest to work with from my 4 tick sources).
When testing your strategies are you going to use the tick data or the 1 min bars or both?
So when I test I can truly work on real market prices. I have one more trick here: I added to each candle two more flags, which store what happened first: the low or the high. In this way I know the exact order in which the 4 points were hit.
Obviously, the lower the time frame I use and the smaller the profit target gets, the more important is working with real tick data. But I will limit myself to maximum M5 for start.
This is only when developing my system.
Before deploying the algo for real, I will obviously test it with full tick data.
The first system I want to test, an H4 with average holding period of 1-3 days, has take profit targets of 100 points, and stop losses of 300 with very good winning rates. So I really don't care even for 10 point real price difference/slippage for now.
BTW, I'm on a little known broker at the moment, CMC Markets (very ugly FIFO GUI, but which I've grown to like), and in hundreds of EUR/USD trades I think I was slipped only 3 or 4 times. Unfortunately they don't have any API, and the spreads are not exactly top-notch (3 points on EUR/USD). Hopefully my future broker will be just like that (non slipping)
I am looking forward to lurking her to see what I can learn from you.
Adal please enable your PM function, I want to mail you.
Done. I was not aware it was disabled.
How is your research going? Very interesting topic. Looking forward to following along.
© Forex Factory