Systematic trading | Page 68 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Trade another day through Systematic Trading 203 replies

Nonparametric Association Measures in Systematic Trading 57 replies

Anyone use Systematic from Alpari? 1 reply

A Systematic Approach to Markets 68 replies

Systematic Trading vs. Discretionary Trading 14 replies

Attachments: Systematic trading
Exit Attachments

Systematic trading

  • #1,341
  • Apr 8, 2011 3:16pm Apr 8, 2011 3:16pm
Quoting PipMasterMik
To start with, it is highly unlikely that you would proceed to run your system live if #3 is true (unless you all know something that I don't!)

If #2 is true, then your system has failed when verified on the "out-sample" data....
Yes, that is correct if you do it as you state above. However I do it a bit differently, I use the out-sample to verify that my fitness algo provides me results that work both in-sample and out-sample. I use the out-sample to make sure that every time after I have run my framework for a day the results are similar - not to test that my framework makes perfect out-sample.

If after a week every day I run my system and reset, the result is similar both in-sample and out-sample, then I shut the system down and reset it, let it run for a week and see if the result is still similar. If this is true, I repeat it few times and if the result is still similar, I can be sure that I have a system that produces right types of algos. After that I can reset the system once more and run it with FULL data (all the points I have) and move to live [I'm in this point with my current system btw, I'm just trying to find the right broker that I can use]

Out-sample is the future. It is worthless with many kinds of systems but with the kind of what I run it's priceless. I don't have to wait for 2 years to prove my system works, I can do it in a second.

Does this make any sense?

ps. funny thing is that with the correct framework it doesn't matter what kinds of rules I add to the mix. If I can find a new type of rule (I'm testing one today), I'll just add it to the mix and run the framework and see if it behaves the same way or better (more variability between algos) and if there is faster/more distributed results etc I'm quite happy. I don't see any reason to test any kind of sytem manually, I much rather let the system/rule type to be tested by my framework that can find all kinds of different statistics that I would never be able to find.
  • #1,342
  • Apr 8, 2011 3:22pm Apr 8, 2011 3:22pm
Quoting jamjamjam
I think it was mentioned that you can test the strategy against other instruments (for robustness), which is another approach, but for similar reasons to the options above, I don't really think it's most ideal.
Big problem with multi-instrument testing is that volatility tends to cluster across the instrument scope.
  • #1,343
  • Apr 8, 2011 4:43pm Apr 8, 2011 4:43pm
Quoting mikkom
I use the out-sample to verify that my fitness algo provides me results that work both in-sample and out-sample...
This all makes sense to me.

As part of my fitness algo "pondering" my plan was to simply make the verification part of the system. Hence, while the system does not train using the verification data, it does use it to either accept or reject a set of parameters (i.e set fitness criteria to 0 for a parameter set if the verification results are not up to expectations). In the end I anticipate that the final parameters won't necessarily be wildly different from what you'd get if you simply trained over the entire data set (i.e. using the verification data in training).

I agree with Mikkom's comment about putting new rules in the system. I'm currently using a set of events that I have never tested manually; I'm sure that some of these events add no value at any time to the system, but who cares? Let the system do the work!
  • #1,344
  • Apr 13, 2011 2:53am Apr 13, 2011 2:53am
PipMasterMik seems to have understood what mikkom was talking about but I'm not entirely sure. I'd like to confirm my understanding.

Suppose I have the data of 2010.

First I pretend that I only have the January data as in-sample data. We use the Jan data to train. The optimized values for the search parameters is plugged into the walk forward trading on the 1st week of Feb. Trades are generated. Profit or loss is booked for the week. It becomes part of the equity curve.
After that, I pretend that the 1st week of Feb is new incoming data. I'm going to retrain my objective function using the Jan and the new data. The new optimized values is plugged into the walk forward trading on the 2nd week of Feb. Trades are generated. Profit or loss is booked for the week. It becomes an extended part of the equity curve.
This process repeat until the data of 2010 is exhausted.

In other words, we assume that all robust objective function has to be retrained regularly at a certain fixed interval. Therefore, to examine the robustness of the objective function, we do the above back to the future stunt.
  • #1,345
  • Apr 13, 2011 4:51am Apr 13, 2011 4:51am
FANN, a NN library: http://leenissen.dk/fann/wp/
C++ ANN lib: http://www.cimne.com/flood/Examples.asp
Java NN lib: http://sourceforge.net/projects/joone/
R Project: http://www.r-project.org/
C# ANN lib: http://franck.fleurey.free.fr/NeuralNetwork/
Is this an ANN?!: http://www.cs.umd.edu/~mount/ANN/
ANN training stuff: http://www.heatonresearch.com/

Paper on using ANN to forcast Forex: http://citeseerx.ist.psu.edu/viewdoc...=rep1&type=pdf

Flexible Neteurl Tree (FNT) called 2nd generation ANN by the researchers: http://www.softcomputing.net/chen-neucom2.pdf
More paper about FNT written by the same author: http://academic.research.microsoft.c...e-neural-trees
Later he published this book: http://www.amazon.com/exec/obidos/AS...8973/acmorg-20
  • #1,346
  • Edited 4:35am Apr 14, 2011 1:15am | Edited 4:35am
Quoting Craig
This started to make me uncomfortable, in the end there were easier ways to make money.
Unless you have a 8 figure balance in your trading account, it makes more sense to sell a system then to use it to trade your own money. The top 5 bots selling on ClickBank, albeit their mediocre performance, are pulling in millions in sales annually. This is not an exaggerated claim. Check the top selling bots on cbEngine (a 3rd party stat platform that track sales figure of Clickbank) or buy one of the stat package from ClickbankCSV. You'll see that their gravity (average number of affiliates that made a sales for the product daily) shots over 100. Say a gross profit is 50 USD per transaction, a bot is pulling in at least 5000 USD daily profit. That's about 2 million annual profit, just for a single bot. Although ClickBank offers unconditional 60 money-back full refund guarantee, few took advantage of it to refund the underperforming bot. The refund rate is under 5% as I've seen on the file provided by ClickbankCSV. I've even heard marketers said that direct selling products that's sold on ClickBank are rarely put into practice. Perhaps that's the reason why they don't refund. They bot buyers were simply in it to buy the dream. They forget about it after making the purchase.
  • #1,347
  • Apr 15, 2011 1:32am Apr 15, 2011 1:32am
Quoting usalchemist
PipMasterMik seems to have understood what mikkom was talking about
Simply said, I use out-sample for verification of method itself, not as part of the method.

I don't think I can explain it any clearer than I have.
  • #1,348
  • Apr 15, 2011 4:51am Apr 15, 2011 4:51am
Quoting mikkom
Simply said, I use out-sample for verification of method itself, not as part of the method.
I don't think I can explain it any clearer than I have.
Okay. Thank you for your further commenting. I think I understand now. You will modify or even abandon the objective function itself if the equity curve of the out-of-sample data doesn't look good.
  • #1,349
  • Apr 17, 2011 4:01am Apr 17, 2011 4:01am
Quoting mikkom
Simply said, I use out-sample for verification of method itself, not as part of the method.

I don't think I can explain it any clearer than I have.
Isn't this equivalent to just extending the in-sample period? If you just remove strategies/population generated in-sample that don't work out-of-sample then your're effectively extending the sample period.

A proper way to do it would be to find in-sample strategy that works on 95% (to say) of out-of-sample data.

pvpn.
  • #1,350
  • Apr 18, 2011 7:46am Apr 18, 2011 7:46am
Quoting pvpn
If you just remove strategies/population generated in-sample that don't work out-of-sample then your're effectively extending the sample period.
As I already said, I don't do that and yes, that would make out-sample in-sample.
  • #1,351
  • Apr 22, 2011 11:48pm Apr 22, 2011 11:48pm
System guys, feel free to chip in...
http://www.elitetrader.com/vb/showth...6&pagenumber=1
The breaking of a wave cannot explain the whole sea.
  • #1,352
  • Apr 23, 2011 7:24am Apr 23, 2011 7:24am
Quoting Craig
System guys, feel free to chip in...
http://www.elitetrader.com/vb/showth...6&pagenumber=1
You could always go with the empirical option of doing a Monte Carlo simulation of various scenarios, using a kernel to sample from the (highly) non normal distribution...
  • #1,353
  • Apr 23, 2011 1:55pm Apr 23, 2011 1:55pm
Quoting realize
You could always go with the empirical option of doing a Monte Carlo simulation of various scenarios, using a kernel to sample from the (highly) non normal distribution...
I did consider that, but I feel my subsequent idea of using distributions as PDFs is simpler as this makes no assumptions on the normality of the distributions.
The breaking of a wave cannot explain the whole sea.
  • #1,354
  • May 2, 2011 11:58am May 2, 2011 11:58am
Hello!

Thanks to mikkom for an excellent thread! I am also dedicated to the quant trading (in the research phase), but not using GP at the moment - I chose a different way for now - I am trying to use a hybrid model based on SOM and SVM (with genetically optimized input sets and parameters) to predict market time series.
I am quite at the start - about a month ago a developed a special framework containing a bridge to Dukascopy historical data API, a custom API for generating features from OHLC data (using various technical indicators and transformations - using TA-LIB implementation too), GHSOM implementation and SVM/SVR implementation.
I am at the final stage of testing the framework and implementing some new features...also had few runs on analyzing some FX market data...results are not bad so far, but not great either...I will now focus on tuning parameters and adding more features for analyzing properties of clusters gained by SOM runs....I will post more results or remarks here if you are interested (and if mikkom approves

Ending with a question - is anybody other here using SVM, SOM, or ANN?
  • #1,355
  • May 3, 2011 9:42am May 3, 2011 9:42am
Can you give an example of the result?
  • #1,356
  • May 4, 2011 11:10am May 4, 2011 11:10am
Quoting sals3r0
Hello!
I am quite at the start - about a month ago a developed a special framework containing a bridge to Dukascopy historical data API, a custom API for generating features from OHLC data....
...
...
...
Ending with a question - is anybody other here using SVM, SOM, or ANN?
How far back the data were you able to get through the API method? I wasn't aware of it before your mentioning it.
What library are you using in constructing SVM or SOM?

I'm trying my luck by implementing ANN. But I'm still at the stage of working out the requirement to outsource though.
  • #1,357
  • May 4, 2011 1:32pm May 4, 2011 1:32pm
I think it is about 3 years
  • #1,358
  • May 5, 2011 3:20am May 5, 2011 3:20am
Quoting foracy
Can you give an example of the result?
I can but it won't be very obvious what it does mean, as I don't really play with trading strategy now, I am just focusing on evaluating a prediction performance of the system.

I usually have 6 training sets and 6 validation sets. I train the SOM+SVM on those training sets and evaluate performance on validation sets.
So validation sets are out-of-sample for SOM/SVM but in-sample for GA (which takes care of optimizing inputs and SVM parameters). So in order to confirm I will need additional out-of-sample tests.
I use NMSE as a performance metric for now. The goal of GA is to minimize average NMSE on validation sets.

I try to predict the trend direction and magnitude of next 5 bars to future, using 4h and 3h timeframe.

Attached validation results of USDJPY prediction (6 sets). The number at the top of images is the NMSE.
For those who are not familiar with NMSE - in general what is <0.9 is excellent for this purpose, anything < 1 is good, what is >1.2 usually sucks for trading....so here some validation results are good, some are so-so
Attached Image(s) (click to enlarge)
Click to Enlarge Name: usdjpy_validation_1.png Size: 48 KB
Click to Enlarge Name: usdjpy_validation_2.png Size: 49 KB
Click to Enlarge Name: usdjpy_validation_3.png Size: 41 KB
Click to Enlarge Name: usdjpy_validation_4.png Size: 45 KB
Click to Enlarge Name: usdjpy_validation_5.png Size: 51 KB
Click to Enlarge Name: usdjpy_validation_6.png Size: 50 KB
  • #1,359
  • May 5, 2011 3:24am May 5, 2011 3:24am
Quoting usalchemist
How far back the data were you able to get through the API method? I wasn't aware of it before your mentioning it.
What library are you using in constructing SVM or SOM?

I'm trying my luck by implementing ANN. But I'm still at the stage of working out the requirement to outsource though.
Depends on the symbol and timeframe. For instance for EURUSD (listing oldest datetimes available per timeframe) :
102003-07-27 16:26
602003-08-08 15:35
3002005-04-16 00:00
6002003-07-18 17:10
9002004-10-25 00:00
18002005-04-16 00:00
36002003-05-04 00:00
144002005-04-16 00:00
864001997-11-02 00:00
6048001986-05-12 00:00

I use http://www.ifs.tuwien.ac.at/~andi/ghsom/download.html for SOM, LIBSVM for SVM and TA-LIB for technical indicator calculations.
  • #1,360
  • May 5, 2011 5:04am May 5, 2011 5:04am
Quoting sals3r0
I can but it won't be very obvious what it does mean, as I don't really play with trading strategy now, I am just focusing on evaluating a prediction performance of the system.

I usually have 6 training sets and 6 validation sets. I train the SOM+SVM on those training sets and evaluate performance on validation sets.
So...
Thanks sals,

Very impressive and I am interested to see how you progress. The key problem I see in your approach that it isn't about being able to predict the future price. What is more important is the win/loss distribution and that is not covered in what you have shown. If you have large loss clusters it will kill you no matter how good your price prediction may be.
1 trader viewing now