Systematic trading | Page 74 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Trade another day through Systematic Trading 203 replies

Nonparametric Association Measures in Systematic Trading 57 replies

Anyone use Systematic from Alpari? 1 reply

A Systematic Approach to Markets 68 replies

Systematic Trading vs. Discretionary Trading 14 replies

Attachments: Systematic trading
Exit Attachments

Systematic trading

  • #1,461
  • Oct 11, 2011 10:27pm Oct 11, 2011 10:27pm
Quoting whale2001
First off I want to thank everyone for all of the excellent information on this website. I have been reading the forums here for years but for some reason I have not joined until now.

My question is for all of the R users here. I'm trying to get away from using metatrader for backtesting and charting so I have been trying to teach myself R. I'm new to R but I have a decent amount of programming experience and am used to using Stata for all of my quantitative analysis. I can't for the life of me figure out how to stream a live data feed into R....
R is an excellent piece of software. I am not aware any free resource that would allow you to import forex data to R. Quantmod package have the capacity to import yahoo finance eod equity data. You can do the same using zoo package and read.csv command, if you are familiar with yahoo finance csv data download. You can get info on that by googling. For forex you can save data from metatrader as csv and then export to R. Although I am unsure how reliable metatrader data will be. You can also look at Dukascopy csv data export on the web.
  • #1,462
  • Oct 12, 2011 7:46am Oct 12, 2011 7:46am
Thanks Pillager,

Quantmod seems like a great package. If I manage to find an FX feed/tick data source, do you think it would be possible to write a script using some "while" loops to have the data object and chart updated at specified intervals? Before I make the effort to learn the ins and outs of this software I want to make sure that it is possible to do what I want with it.

Do you need a special EA or anything to export metatrader data as csv? I haven't messed around a lot lately because I use a Mac now but I can still run it in Parallels but its not the smoothest setup.
  • #1,463
  • Oct 12, 2011 9:03pm Oct 12, 2011 9:03pm
Quote
Quantmod seems like a great package. If I manage to find an FX feed/tick data source, do you think it would be possible to write a script using some "while" loops to have the data object and chart updated at specified intervals? Before I make the effort to learn the ins and outs of this software I want to make sure that it is possible to do what I want with it.

Not sure about this. If you can save your feed as csv, you can import that in R.

Quote
Do you need a special EA or anything to export metatrader data as csv? I haven't messed around a lot lately because I use a Mac now but I can still run it in Parallels but its not the smoothest setup.

Just do File > Save as.. This should give you a csv. For more control you can definitely write an EA.
  • #1,464
  • Oct 21, 2011 9:47am Oct 21, 2011 9:47am
This might interest people reading this thread..

(click the upper link to go to youtube so you don't have to watch in this tiny box)

Inserted Video
  • #1,465
  • Oct 22, 2011 1:17pm Oct 22, 2011 1:17pm
Excellent document!

Quoting mikkom
This might interest people reading this thread..

(click the upper link to go to youtube so you don't have to watch in this tiny box)
  • #1,466
  • Edited 9:04pm Nov 9, 2011 1:33am | Edited 9:04pm
Overfitting is believed to be the problem for every machine learning quest. But is it really? We start investigating into this. Before we try to deal with overfitting, we must first confirm that it really exists. The thought is that if improper fitness function is used, any kind of fitting will not exist let alone overfitting. In other words, the nonexistence of overfitting can potentially disqualify a fitness function. The traditional understanding of overfitting has certain form. We're trying to find out of our system's out-sample validation conform to the form.

Here's a graphical update of what T0009 group is working on to find out if the system is really suffering from overfitting. (We were trying to overcome it. But we stop now. And we want to find out if it really exists first! If the form doesn't even exist, the fitness function used for the training might have been unqualified in the first place.)

http://www.hemist.com/capture/Screen...9_15.30.35.png
  • #1,467
  • Edited 4:28pm Nov 11, 2011 3:00pm | Edited 4:28pm
Oh the beauty of noncorrelation and huge list of equities.

Here is the first screenshot I have posted anywhere of my new stock/etf trader.
Attached Image (click to enlarge)
Click to Enlarge Name: Screenshot-Lyzer.png Size: 246 KB
  • #1,468
  • Nov 15, 2011 4:44pm Nov 15, 2011 4:44pm
Quoting mikkom
Oh the beauty of noncorrelation and huge list of equities.

Here is the first screenshot I have posted anywhere of my new stock/etf trader.
Nice work; thanks for sharing. Are the (non-white) curves etf equities or strategies running on equities? I'd assume the latter since many of the curves have huge jumps towards the end. If by chance they are individual equities, can you tell us what the blue and green outlying equities are? Thanks.
  • #1,469
  • Nov 16, 2011 12:41am Nov 16, 2011 12:41am
Quoting jamjamjam
Nice work; thanks for sharing. Are the (non-white) curves etf equities or strategies running on equities? I'd assume the latter since many of the curves have huge jumps towards the end. If by chance they are individual equities, can you tell us what the blue and green outlying equities are? Thanks.
All trade same equity list. The strategies are directional so some are long strategies and some are short strategies. There was a down period at the point where some of the strategies went up and some down, the main thing is that the composite curve is good.

Fx doesn't need separate long/short strategies as it's direction neutral. Stocks are not.
  • #1,470
  • Nov 21, 2011 11:05am Nov 21, 2011 11:05am
By the way if anyone is interested, I use open source ta-lib http://ta-lib.org/ on this equity platform. The documentation is awful but they have separated implementations from interfaces allowing all kinds of interesting things - ie, I can wrap most of the Indicator functions inside one global IndicatorCalculator function dynamically (without having to code a separate functionality for each indicator, pattern etc). Same is true for all price overlays (bands, different averages etc).

It's worth checking out if you are willing to dig into source code.. there are almost no docs except source code and this http://tadoc.org/
  • #1,471
  • Nov 22, 2011 2:47pm Nov 22, 2011 2:47pm
Yes, I am really happy that TA-Lib exists, there are no docs, but with some examples and looking to source code you can understand it pretty fast. I use it for my forex strategy backtester, for stock pair screener and for my stock pair trading backtester. The library is coded in C so it is really fast.
Also the advantage is - you can set unstable period for almost all indicators, which is not really common (useful for moving averages like EMA, which in theory work on infinite number of previous samples).

If you are interested in systematic trading of stock (statistical arbitrage), you can test my online backtester: http://www.pairtradinglab.com/
I use TA-Lib to calc all kind of moving averages there.
  • #1,472
  • Nov 30, 2011 6:30am Nov 30, 2011 6:30am
Worth reading.. I have gotten more than good (so good I believe there must be a bug somewhere) results from my 500+ eq systems and this is one of the reasons:

http://www.automated-trading-system....on-free-lunch/
  • #1,473
  • Jan 31, 2012 11:37am Jan 31, 2012 11:37am
  •  rayzeel
  • | Joined Nov 2008 | Status: Automated/DiscretionaryTrader/Coder | 37 Posts
Hello!
Ive red this thread few times and now I have some questions to You all, especialy for those have eqperience like Mikkom. You guys have a big knowledge and I have some dilemma about choosing aproperiate parametric set for strategy on the given time.

I have a strategy and optimalising it WalkForward method... Is it a good method or is there something better like ... optimizing separately uptrend downtrend and sideway and then deciding using technical and fundametal analysis (predicting) how the market will behave in next days...

I think that WalkForward is a good method, but maybe it will be better to use different method.

If I have a few optimized sets for each type of markets and an example:
we have an uptrend and some sets are working for this type of market, and then the parameters are starting to change and we see that program starting to loosing more than previous. So it should be an indicator of changing market and we should change the parameters for sideway or downtrend...

How to chooosing this sets? Analysing market and predicting or rely on walkforward ?

Hope You guys understend me. If not please ask.
Greets!!
  • #1,474
  • Feb 1, 2012 2:24am Feb 1, 2012 2:24am
How can we systematically define up/download/sideway trend though? I went though the very same thought process before, but I had no luck finding the right way to define up/down/sideway trend systematically. Even some claim there's trend indicators, those are lagging. If trends change more frequently then what the system would have you tuned to, then the change will kill the system, still. How can we know or define the rate of change in trends even if we know how to define trends? More and more questions... Perhaps someone else can chip in simpler answer then my adding confusion! Cheers!
  • #1,475
  • Feb 1, 2012 4:05am Feb 1, 2012 4:05am
After the fact, traders tend to artificially categorize markets as having been bullish, bearish, trending, or anything else we can come up with. However, for the most part, we tend classify nothing but the product of randomness.

What you're saying is that you want to try to optimize for different types of markets. For this to work, this implies that you need to be able to predict market behavior before the fact. Unfortunately, given the high degree of randomness, you will probably have difficulty making accurate prediction of whether the market will be bullish, bearish, or ranging. In fact, if you would succeed, this would offer an "edge" by itself.

What traders do agree on, is that volatility is sticky. Volatility conditions would definitely be worth it to consider when optimizing your strategy, specifically when you work with limit orders.

Now lets make the assumption that autocorrelation is sticky. This means that periods of negative autocorrelation tend to be followed by periods of negative autocorrelation, and the same for positive autocorrelation. (Where negative autocorrelation means that if prices rose today, there is a greater probability that prices will fall tomorrow.)
If your methods do not take this into account, you will probably see positive autocorrelation in your results. (For example, in "ranging markets" your method performs poorly, while in "trending markets" it flourishes.) If there is positive autocorrelation in your results, you should be able to optimize your money management to take this into account. (e.g. make a rule that if your previous trades were successful, increase the (relative) lot size of your next trade).

If you have multiple strategies running with such an arbitrary lot sizing strategy, and the strategies perform rather independently or are negatively correlated, this implies that your methods on aggregate will automatically be switching from one "mode" to another.
The nail that sticks out gets hammered back in
  • #1,476
  • Feb 1, 2012 5:59am Feb 1, 2012 5:59am
  •  rayzeel
  • | Joined Nov 2008 | Status: Automated/DiscretionaryTrader/Coder | 37 Posts
This approach is more complex. If I have some sets working good on different market maybe I won't need to predict the market. I should only observe the DD or parameters. But it will need to do optimization in real time process... Aproperiate tools are needed. And when I see that some parameters are different in optimization I will consider set change... Its hard to explain this and I dont know all details of this approach. Actually I'm doing researches.

The goal of searching is to find the best set of parameters for future

It is easy to say, but task is very hard and it depends on how the algorithm is behaving, how robust it is...

Some says that choosing a good set is same as flipping the coin.

I have to make more research. My question was if someone using similar method and have some advices what to focus on...

Thanks!
  • #1,477
  • Feb 13, 2012 2:35am Feb 13, 2012 2:35am
I am reading The Quants: http://www.amazon.com/Quants-Whizzes.../dp/0307453375

And come across the term of PDT(process driven trading)

Anyone could provide some info about PDT, or give an example of PDT.
  • #1,478
  • Feb 13, 2012 3:22am Feb 13, 2012 3:22am
  •  rayzeel
  • | Joined Nov 2008 | Status: Automated/DiscretionaryTrader/Coder | 37 Posts
I want to quick back to the post I have written above. Actually I throw up analyzing trends and started doing similar things as mikkom. I've started to developing software to combining sets and strategies searching best performance together and trying to get some continue. Actually doing it in Java. The speed is now enough for me but I know I will need more.

Cheers!
  • #1,479
  • Feb 13, 2012 3:41am Feb 13, 2012 3:41am
Quoting rayzeel
Actually doing it in Java. The speed is now enough for me but I know I will need more.
Hint: For speed use arrays of primitives anywhere where you can (no object arrays as they are basically pointers). That way cpu cache starts kicking in. For example my 1 min equity data looks like this:

Inserted Code
    public int barCount;
    public char[] open;
    public char[] close;
    public char[] high;
    public char[] low;
    public int[] volume;
  • #1,480
  • Feb 13, 2012 6:24am Feb 13, 2012 6:24am
  •  rayzeel
  • | Joined Nov 2008 | Status: Automated/DiscretionaryTrader/Coder | 37 Posts
Thanks for a hint. I making tests in mt4 and generating equity curve in csv and then analyzing files in my soft. Im reading the file to primitive arrays as You wrote. Its seems to work well now. Its really simple method and in future I will build my own tester as You mikkom. Now I can get 1.000.000 combinations without repeating ( k! / n!(k-n)! ) in 8 hours and I think its enough for now ( test on cpu 1.6; 2gb ram).

It was easy task to make soft for analyzing csv files. If I succeed making good working combinations of strategies and its parameters I will go further.

Maybe I used wrong words "something similar". I mean it's similar in goal we trying to achieve.

Please point out any disadvantage You see in this method.

Oh, and thanks for this thread anyway. Reading it opened my eyes for where to search a solution and I think directed me in the right way (I hope)
1 trader viewing now