Machine Learning with algoTraderJo | Page 30 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Algorithmic Quant Trading (Machine Learning + Stat-Arb) 25 replies

Forex with Machine Learning Software Project 1 reply

Machine Learning + Retail Forex = Profitable? (Quant) 1 reply

Potential new machine learning style software. 79 replies

My most recent advancements into machine learning 16 replies

Attachments: Machine Learning with algoTraderJo
Exit Attachments

Machine Learning with algoTraderJo

  • #581
  • Aug 6, 2015 12:19pm Aug 6, 2015 12:19pm
Quoting ForexSamsam
in your last example i understand : at 14h GMT compute return for the last 9 day bar return at the same hour (14h gmt return in pip or percent) -- >this is the input like [a,b,c,d,e,f,g,h,i] Output is the return from now + 70 hours (delta from now) and your add 1 is TP is reach first or 0 is SL is reach first (2 output vector) like [X,Y] this for 225day.
I always use GMT +1/+2 this is extremely important, use GMT with the wrong shift or without DST and you won't be able to reproduce.

the input is the return of the past 9 bars defined as (Open[N]-Open[N-1])/Open[N-1] where N=0 is the currently used entry bar.

The output contains two values, one for the profit/loss of a short trade and another for the profit/loss of a long trade. It is not simply 0/1, it is the actual value in absolute price terms, using absolute values and doing regression instead of classification is critical. So for example an output vector might be 0.0034,-0.0043 which means that a long trade would have hit a TP of 34 pips and a short trade a stop loss at 43 pips. As I said before it is of vital important not to use classifiers but regression models, this will become even more important later on.

The linear regression model should be retrained on each bar where a trade decision is taken. I repeat, do not use classifiers or you won't be able to reproduce these results.
  • #582
  • Aug 6, 2015 12:39pm Aug 6, 2015 12:39pm
Quoting algoTraderJo
{quote} I always use GMT +1/+2 this is extremely important, use GMT with the wrong shift or without DST and you won't be able to reproduce. the input is the return of the past 9 bars defined as (Open[N]-Open[N-1])/Open[N-1] where N=0 is the currently used entry bar. The output contains two values, one for the profit/loss of a short trade and another for the profit/loss of a long trade. It is not simply 0/1, it is the actual value in absolute price terms, using absolute values and doing regression instead of classification is critical. So for example...

That totally clear ! thanks i try this
  • #583
  • Aug 7, 2015 8:44am Aug 7, 2015 8:44am
The EU and GU results can be combined in a portfolio to give better performance than the individual systems:

Attached Image (click to enlarge)
Click to Enlarge Name: machinelearning_sltp_portfolio1.png Size: 92 KB
  • #584
  • Aug 7, 2015 11:06am Aug 7, 2015 11:06am
For the target: if the SL and TP is not reach at the end of horizon for both, what value are you using ?
  • #585
  • Aug 7, 2015 5:08pm Aug 7, 2015 5:08pm
Quoting ForexSamsam
For the target: if the SL and TP is not reach at the end of horizon for both, what value are you using ?
This means that the example is not valid so it is skipped.
  • #586
  • Aug 8, 2015 4:48pm Aug 8, 2015 4:48pm
Hi Jo,

I will try to get the setup going next week, but how do these results compare to your previous posts? Do you prefer the approach with the ATR TP / SL over the MSE/MLE?
I tried some variations with the ATR but never having the SL different to the TP - need to change things in my framework. It seems the returns are much smaller but get more consistent R2?

Cheers
KaBo

P.S. Would be exiting if you come back to the ensemble subject sometime ;-)
  • #587
  • Edited 8:05am Aug 9, 2015 7:48am | Edited 8:05am
Quoting KaBo
Hi Jo, I will try to get the setup going next week, but how do these results compare to your previous posts? Do you prefer the approach with the ATR TP / SL over the MSE/MLE? I tried some variations with the ATR but never having the SL different to the TP - need to change things in my framework. It seems the returns are much smaller but get more consistent R2? Cheers KaBo P.S. Would be exiting if you come back to the ensemble subject sometime ;-)

  1. If you want to compare you can do so by looking at my previous posts. I don't have enough time available to help you with this task.
  2. Don't ask me about my preferences. I am just showing you different machine learning approaches, it is up to you to compare them, build new ones and choose the one that you think is best for you. Also never ignore mining bias, some of the approaches exposed so far lead to very high levels of data mining bias. Always test the data mining bias of any system you use before deciding what to trade.
  3. You can get high R2 values with a variety of different approaches, including many different ones I have posted so far.
  4. I will get back to ensembles later on.

  • #588
  • Aug 9, 2015 8:03am Aug 9, 2015 8:03am
You can also modify the trading system in the following ways:

  1. Take a take long trade whenever the long prediction is positive (value of the short prediction does not matter)
  2. Take a short trade whenever the short prediction is positive (value of the long prediction does not matter)
  3. All trades are left alone until they reach the SL or TP.
  4. New signals do not affect previous trades since new trades are always open on new signals provided either the long or short prediction is positive.

You can get highly linear results with this as well on the EURUSD:

Inserted Code
SL=90% daily ATR20
TP=320% daily ATR20
A=4
B=125
C=8
D=190

Attached Image


See how we can also get profitable results from a trading system where multiple simultaneous positions are possible. Also see how this system can get very nice trading results during the past 3-4 years. There exist many roads to the implementation of a successful machine learning based system, each one with different advantages and disadvantages.
  • #589
  • Aug 9, 2015 11:22am Aug 9, 2015 11:22am
Hi algoTraderJo,

Thanks for your answers.

Can you post your data prepare function (SL/TP) to avoid more questions
  • #590
  • Aug 10, 2015 7:18am Aug 10, 2015 7:18am
Quoting levkly
Hi algoTraderJo, Thanks for your answers. Can you post your data prepare function (SL/TP) to avoid more questions
A couple of Asirikuy members are helping me double check the functions before they are added to the F4 unstable branch this week (I think). I will post them when they are ready. I don't want to post messy code here.
  • #591
  • Aug 10, 2015 7:32am Aug 10, 2015 7:32am
Quoting algoTraderJo
{quote} A couple of Asirikuy members are helping me double check the functions before they are added to the F4 unstable branch this week (I think). I will post them when they are ready. I don't want to post messy code here.
algoTraderJo

Asirikuy is only C++ code ? if i don't know C++ i can subscribe or not ?
  • #592
  • Aug 10, 2015 8:14am Aug 10, 2015 8:14am
Quoting surfeur
{quote} algoTraderJo Asirikuy is only C++ code ? if i don't know C++ i can subscribe or not ?
Systems in the F4 framework are coded in C/C++ if you want to code using the framework you naturally need to know these languages.
  • #593
  • Aug 10, 2015 9:38am Aug 10, 2015 9:38am
Some USDJPY results using the same modifications as in this post :

Inserted Code
SL=90%
TP=350%
A=12
B=50
C=6
D=180

Attached Image


This is just a simple example after a small search... much more linear results are indeed possible...

This machine learning algo can create acceptable systems for many different pairs (as does the single position variant I showed first). But does it have a low enough data mining bias? That's up to you to find out!

Putting together many of these different systems (those with low data mining bias) allows you to create highly diversified portfolios that trade an important variety of symbols with very good back-testing results. After all this information it is not very difficult to imagine how being consistently profitable becomes a real possibility... Many machine learning algos, many symbols, low data mining bias (this is key)... And off you go...
  • #594
  • Aug 10, 2015 12:14pm Aug 10, 2015 12:14pm
Quoting algoTraderJo
{quote} Systems in the F4 framework are coded in C/C++ if you want to code using the framework you naturally need to know these languages.
Yes framework is c/c++ but there a Python bindings for trading and backtest ?
  • #595
  • Aug 10, 2015 12:23pm Aug 10, 2015 12:23pm
Quoting surfeur
{quote} Yes framework is c/c++ but there a Python bindings for trading and backtest ?
There are several front-ends you can use. You can trade/backtest the systems in MT4/MT5 or in the programs developed by the community (Asirikuy Trader and Strategy Tester). Using the Asirikuy Trader you can live trade using the Oanda or Dukascopy APIs. Both the Asirikuy Trader and Strategy Tester are coded in python and you get access to all their source code when you get in. I personally only use the Asirikuy Trader and the Strategy Tester, I have never used the F4 framework with MT4/MT5.
  • #596
  • Aug 10, 2015 12:27pm Aug 10, 2015 12:27pm
Quoting surfeur
{quote} Yes framework is c/c++ but there a Python bindings for trading and backtest ?
One of the things I like the most is that you can use the Asirikuy Trader and Strategy Tester programs from Linux. This is really a must for me, to be able to live trade and backtest on Linux. The strategy tester also supports OpenMP/MPICH which is what I use to carry out my data mining bias analysis using a decently sized computer cluster. There is no other backtesting software I know that allows for this to work.
  • #597
  • Aug 10, 2015 2:35pm Aug 10, 2015 2:35pm
Quoting surfeur
{quote} Yes framework is c/c++ but there a Python bindings for trading and backtest ?
I use python Pandas with scikit Learn.
  • #598
  • Aug 10, 2015 2:47pm Aug 10, 2015 2:47pm
Quoting ForexSamsam
{quote} I use python Pandas with scikit Learn.
You can use many things, those two are excellent python libraries. My main issue with implementing my own framework is that no one helps you debug, so bugs may go unattended for long periods of time. These bugs may cost you a lot of money! I have seen this happen. Having real support for a development framework and live trading implementation is very important to me. The fact that many others are using the same global implementation and that updates are constantly being made and bugs are constantly being fixed gives me more confidence to live trade.

Another huge point is the live trading. The Trader has a virtual order wrapper that allows you to seamlessly trade strategies on a single account that may open any number of long/short positions without having to worry about the Oanda FIFO and non-hedge limitations. I haven't found any other library or program to trade with Oanda that implements this. I currently trade 50-100 machine learning strategies on the same account so for me it's super important to be able to have this type of functionality (for example to manage 20 long EURUSD and 10 short EURUSD trades simultaneously).

Honestly I just prefer to spend my time developing my systems, not worrying about having to also develop and debug my own framework implementation, and back+live trading platform. But if you do so, then the more power to you!
  • #599
  • Aug 10, 2015 3:19pm Aug 10, 2015 3:19pm
Take a look at these results on the USDJPY, using multiple positions as shown here (remember that the ML model is retrained on every bar using the last B examples):

Inserted Code
SL=80%
TP=60%
A=22
B=200
C=7
D=60

Attached Image


This is the first case I present where we have a TP that is actually lower than the SL. A machine learning strategy that does not behave so much as a "trend follower". Using this type of multiple position ML strategy using examples based on candle returns as inputs and SL/TP outcomes as ouputs to train a linear regression model we can create systems where we enter long and short trades at the same time... Expecting to profit from both!

Ohhh the many things we can do in the world of ML... But be careful! Always measure your mining bias! -- read previous posts about this on the thread
  • #600
  • Aug 10, 2015 3:26pm Aug 10, 2015 3:26pm
Quoting algoTraderJo
{quote} One of the things I like the most is that you can use the Asirikuy Trader and Strategy Tester programs from Linux. This is really a must for me, to be able to live trade and backtest on Linux. The strategy tester also supports OpenMP/MPICH which is what I use to carry out my data mining bias analysis using a decently sized computer cluster. There is no other backtesting software I know that allows for this to work.

Just a question, the OpenMP/MPICH allow to spread the internal calculations of the ML algorithms (gradient descent, SVM...) through a cluster, or just parallel optimization of non-dependent input BT? I mean, the cluster will be useful to speed up the every bar re-training in a single setup BT?
1 trader viewing now