Systematic trading | Page 73 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Trade another day through Systematic Trading 203 replies

Nonparametric Association Measures in Systematic Trading 57 replies

Anyone use Systematic from Alpari? 1 reply

A Systematic Approach to Markets 68 replies

Systematic Trading vs. Discretionary Trading 14 replies

Attachments: Systematic trading
Exit Attachments

Systematic trading

  • #1,441
  • Aug 25, 2011 4:45am Aug 25, 2011 4:45am
Mikkom (and other GA users),
is Pareto optimality and fitness sharing incorporated in your fitness algorithms? Why yes/no?
Thanks
  • #1,442
  • Sep 13, 2011 11:16am Sep 13, 2011 11:16am
Hi,

is this thread still active? I have browsed it quickly and it seems that I am now taking the road that Mikkom has already taken two years ago. After lots of experiments with MetaTrader 4 and several EAs that I have coded, I have come to the conclusion that its backtesting features are somewhat limited. It's comfortable, it works for simple strategies, but it is also extremely slow and doesn't work if you want to change your strategy more than just exchanging parameters.

I also want to try genetic programming, so I have started to build my own simulator. So far, I have a database with M1 candles and about 4M entries and can generate tick data from it. Right now, I am working on a trade manager.

The next step obviously would be to implement a backtester and then the framework for genetic programming. I am planning to use TA-LIB for all the math functions.

Since I'm doing this in my (limited) spare time and out of fascination and personal interest, not with the aim of generating real money from it, I'm progressing slowly but consistently. Needless to say, I would enjoy the exchange with people who have chosen the same path or are at a similar position.

If anybody's out there, let me know.
  • #1,443
  • Edited 2:32pm Sep 14, 2011 2:13pm | Edited 2:32pm
tigloo, if you're a C++ programmer keen in distributed computing, neural network, or genetic algo, please join the T0009 (www.hemist.com/T0009) group of programmers in the continual development of their system. I've joined them. We share codes and we're onto our next stage of development: adding more layers, auto walk-forward, multiple instruments, and distributed computing to shorten the search life-cycle on each run. If you like to contribute your coding time to the group, feel free join us through Skype. It's been my privilege and pleasure to find a group whose members withholding neither prior failure nor success experience back.

PS: tigloo, unless you're being very lucky (survivorship bias) like the T0009, you're still in the infant stage of your backtesting or bot development. I still remember back at the days when I was throwing the questions regarding the essential meaning or irony (if any) of mikkom's walk-forward test. The thinking process was painstaking but I enjoyed it. And eventually, I came to realize my own understanding of the "truth" (ever evolving though)!
  • #1,444
  • Sep 14, 2011 3:40pm Sep 14, 2011 3:40pm
I will try to get in touch with you - need to reactivate my Skype account. I used to be a professional (freelancing) C/C++ programmer for more than 10 years, I haven't really coded a lot in the past 6 years though, because I'm not programming anymore in my regular job. I'm a little rusty if you know what I mean, but the skills are there.

My personal backtester is not even in its infancy yet, I'm still working on it. I have been working heavily with MT4's capabilities and built a nice codebase for an EA infrastructure there but ultimately came to the conclusion that MT4 is more of a "convenience framework" for EA developers rather than a serious development environment if you need number crunching and fast execution. I created a few EAs ... the code was nice but I was not able to turn my strategies into a profitable system, most likely because I was working by myself and was lacking solid feedback.

I'm not planning to use neural networks with my current approach, my plan is to employ genetic algorithms. My technical know-how is not the problem, but I lack the understanding of the Forex market so I thought it would be great to get in touch with people to share experiences and ideas. What I want to avoid is the situation I've already been in, "great idea, great code, somehow isn't profitable".
  • #1,445
  • Sep 14, 2011 3:53pm Sep 14, 2011 3:53pm
Hi usalchemist, can you send me your Skype contact info or contact me via PM or email? As a Junior Member, I do not seem to able to contact you directly. I tried messaging Tom Wilkinson via Skype but also seem to be blocked there, my message is not going to be delivered for some reason. I never use Skype, I suspect it is because I am not on his contact list.
  • #1,446
  • Sep 14, 2011 5:10pm Sep 14, 2011 5:10pm
tigloo, anyway, his Skype is published on the T0009 performance page on FxStat.
  • #1,447
  • Edited 3:35am Sep 16, 2011 3:16am | Edited 3:35am
Quoting usalchemist
tigloo, if you're a C++ programmer keen in distributed computing, neural network, or genetic algo, please join the T0009 (www.hemist.com/T0009) group of programmers in the continual development of their system. I've joined them. We share codes and we're onto our next stage of development: adding more layers, auto walk-forward, multiple instruments, and distributed computing to shorten the search life-cycle on each run. If you like to contribute your coding time to the group, feel free join us through Skype....
Haven't posted here for a while but I just have to after seeing that curve..

Have you done any longer term tests with multiple pairs? Your groups curve looks very nice (well I can't see trade risk level so I can't be sure) but to be honest, it looks too good. If you can really get ~+150% per month in profit with drawdown under 15% long term then all I can say is wow , unbelievable results.

How does simulation look longer term? (5-10 years)? How does the result look like if run on multiple currencies? How does the result look in different volatility/trend conditions?

edit: Also, why in earth would you sell this "EA" if it works? Sorry about being sceptical but too good to be true stats usually are that.
  • #1,448
  • Sep 17, 2011 3:53pm Sep 17, 2011 3:53pm
@mikkom, the T0009, as told in the forum, is a lucky survivor among the many failed (or was it told in there?!). The rest of the failed live trading tests that generated by the same system were not published. Also, as I also mentioned it few times before, EA selling, with the right publisher, makes a lot more money than trading your own money or even private fund that ranges in a few millions. And what? 150%?! You must have read it wrong. On average, it returns a mere few percents a month. The FxStat presentation of the yearly and overall return were somehow misleading. Portfolio training is still missing in the system. And it's not the killing issue. What's killing right now is the training takes too long. Also, an auto walk-forward is being developed. After that, we're considering having the whole system re-built in a distributive computing environment allowing many PCs' free CPU time to work at the same time to shorten the training time. The goal is to re-train as frequently (daily or even hourly) and still able to produce good-looking curve.
  • #1,449
  • Sep 19, 2011 12:55am Sep 19, 2011 12:55am
Quoting usalchemist
Also, as I also mentioned it few times before, EA selling, with the right publisher, makes a lot more money than trading your own money or even private fund that ranges in a few millions. And what? 150%?! You must have read it wrong.
Okay then, that sounds reasonable. few percent per month system sounds much more reasonable than 165%

(the page you linked says Monthly: 165.3498% at the moment).

> After that, we're considering having the whole system re-built in a distributive computing environment allowing many PCs' free CPU time to work at the same time to shorten the training time. The goal is to re-train as frequently (daily or even hourly) and still able to produce good-looking curve.

That's a very good idea, distributing this kind of calculation is pretty straightforward.
  • #1,450
  • Sep 19, 2011 1:50am Sep 19, 2011 1:50am
Quoting mikkom
(the page you linked says Monthly: 165.3498% at the moment).
LMHO. As I said, the presentation on FxStat is misleading. The summary column simply adds everything up to created an wow effect.
  • #1,451
  • Sep 19, 2011 7:52am Sep 19, 2011 7:52am
Mikkom, since you're active again here: I'm looking for a thread to discuss my own (genetic algorithm) framework development. I have just finished the basics, now I'm dealing with the hard stuff such as fitness functions, evolution strategies, performance optimization, etc.

I feel that it would fit perfectly to this thread, but I do not want to hijack it.

Please let me know if I should open a different thread or if you would like to see it discussed here. My first post would be a summary about my current work, then I would maybe post weekly or so about any results I come up with in the hope of getting a fruitful discussion going.
  • #1,452
  • Sep 21, 2011 12:35pm Sep 21, 2011 12:35pm
This thread is on LinkedIn under the "Automated Trading Strategies" group. Pretty good discussion on some attempts on algo trading development, successful and failed ones. The thread is started by Henrik that goes like this:

Using genetic programming to create trading strategies. We have over the last 4 years developed an automated system to create trading strategies using GP technology. The software is integrated in CUDA, so we can process data quickly using the latest graphics cards. However, we have not managed to get good results in live trading. Curve fitting is a huge problem that we have worked on for the last 2 years. A couple of weeks ago I decided to fire all my programmers, because our ideas ran out. Is there anyone out there that have any thoughts on GP and curve fitting? In theory I am convinced that GP technology is the way to go, but I simply can’t find the right setup. Do you have any ideas?

You might need to get an LinkedIn account and perhaps joining the ATS group to view that thread.
  • #1,453
  • Sep 22, 2011 10:33am Sep 22, 2011 10:33am
Mikkom and everyone

I was reading optimization articles from Amibroker officila website and I stumbled upon other optimization techniqes one is swarm particle optimization http://www.swarmintelligence.org/links.php second one is Covariance Matrix Adaptation Evolutionary Strategy
http://www.swarmintelligence.org/links.php how do these compare to genetic programming ? Are these easier to code?
  • #1,454
  • Sep 23, 2011 12:32am Sep 23, 2011 12:32am
Quoting tigloo
Mikkom, since you're active again here: I'm looking for a thread to discuss my own (genetic algorithm) framework development. I have just finished the basics, now I'm dealing with the hard stuff such as fitness functions, evolution strategies, performance optimization, etc.

I feel that it would fit perfectly to this thread, but I do not want to hijack it.

Please let me know if I should open a different thread or if you would like to see it discussed here. My first post would be a summary about my current work, then I would maybe post weekly or...
Sure, go ahead. No need to create a new thread.
  • #1,455
  • Sep 24, 2011 3:05pm Sep 24, 2011 3:05pm
Over the past two weeks, I imported some sample data (MetaTrader history of EURUSD M1) into an SQL database and then proceeded to build my own quote engine on top of that, together with a small backtesting framework.

The backtester feeds the prices to a formula, which will cause sell or buy orders to be issued based on its output. The input values to the formula as well as the formula itself is evolved using a genetic algorithm.

My assumption is that if I test over a long enough time span and use a time frame that's not so prone to noise (such as H4 or D1), I can eventually come up with a set of strategies that may be curve-fit to certain degree but will work long enough on out-of-sample data to be profitable, until the next GA run is completed.

Right now I'm working on code for proper visualization to see how the different strategies trade. Next up are the tricky things such as fitness function and strategy selection/combination.

My question right now is: is anyone live with a similar setup and profitable? The thread seemed to be very active a while ago, but after a few announcements of "going live" the discussion kind of ended, which is not always a good sign.
  • #1,456
  • Sep 24, 2011 3:25pm Sep 24, 2011 3:25pm
Some facts about my current toolbox: I'm developing in C++/Qt with QtCreator. I use GALib for evolution and TA-Lib for indicators. I'm using Postgresql as database backend but all I do for now is load the bar data to memory, then replay all ticks from there.
  • #1,457
  • Sep 27, 2011 1:55am Sep 27, 2011 1:55am
Quoting tigloo
...The backtester feeds the prices to a formula, which will cause sell or buy orders to be issued based on its output...
What elements (terminals and functions) is the formula composed of? Thanks
  • #1,458
  • Sep 27, 2011 3:10am Sep 27, 2011 3:10am
As terminal set, I currently use time series access (OHLC) as well as a set of simple indicators such as RSI, MA, CCI, etc. The indicators are calculated on the fly, their parameters are evolved with the genetic algorithm.

My functional set is simple math only: +, -, *, /

I am hoping to get a curve-fit genome over a testing period of 6 months M1 data to prove that this works successfully. If it doesn't, I will try to restrict the degrees of freedom. Right now I am unsure if the genetic algorithm is able to simply remove unnecessary degrees of freedom or if it will put too much strain on the evolutionary process.
  • #1,459
  • Oct 4, 2011 11:29pm Oct 4, 2011 11:29pm
Quoting tigloo
As terminal set, I currently use time series access (OHLC) as well as a set of simple indicators such as RSI, MA, CCI, etc. The indicators are calculated on the fly, their parameters are evolved with the genetic algorithm.
Garbage in, garbage out.
The breaking of a wave cannot explain the whole sea.
  • #1,460
  • Oct 11, 2011 1:26pm Oct 11, 2011 1:26pm
First off I want to thank everyone for all of the excellent information on this website. I have been reading the forums here for years but for some reason I have not joined until now.

My question is for all of the R users here. I'm trying to get away from using metatrader for backtesting and charting so I have been trying to teach myself R. I'm new to R but I have a decent amount of programming experience and am used to using Stata for all of my quantitative analysis. I can't for the life of me figure out how to stream a live data feed into R. If anyone could point me in the right direction it would be much appreciated.

-whale
1 trader viewing now