Machine Learning with algoTraderJo | Page 34 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Algorithmic Quant Trading (Machine Learning + Stat-Arb) 25 replies

Forex with Machine Learning Software Project 1 reply

Machine Learning + Retail Forex = Profitable? (Quant) 1 reply

Potential new machine learning style software. 79 replies

My most recent advancements into machine learning 16 replies

Attachments: Machine Learning with algoTraderJo
Exit Attachments

Machine Learning with algoTraderJo

  • #661
  • Edited 5:24am Sep 27, 2015 4:51am | Edited 5:24am
Jo is back with some more machine learning fun. I will be posting for a few days before leaving again. I am terribly busy with my own trading and coding, so there is not much time for this FF thread. If you're familiarized with this thread you will notice that I just post a few times every few months, when I get the time to do so. I will try to answer questions while I am posting but I will concentrate on showing some more things before leaving.

Last time we were discussing ensembles that were created using strategies with multiple entries and SL+TP structures. This time I want to share another family of machine learning strategies that can be coded using the same philosophy of predicting the trade outcome but this time based on a simple type of trailing stop mechanism, single entries and no TP. This mechanism is different to the ones you will find on Asirikuy today in their machine learning mining (what they call "function based trailing stops") but this mechanism is much simpler and adds no external parameters.

The system trades like this:

  1. Trade decisions are only made on a single hour during the trading day (A)
  2. The machine learning method is retrained every time a decision is made
  3. If the ML algo predicts a long/short we enter a long/short if no positions are open
  4. If there is a position open in the same direction then reset the SL as if the position had just been opened
  5. If there is a position in the opposite direction then stop and reverse
  6. If a position is open check at the start of every 1H bar whether moving the SL to a distance equal to the initial SL from the current open is more favorable than the current SL. If it is then move the SL to this position.

The ML algo works like this:

  1. Train the ML algo every time a trading decision is made using a linear regression model. Use the last (B) examples for training.
  2. Each example is built starting on the same decision hour (A) in the past, using the last (C) hourly returns as inputs
  3. For the outputs evolve a long and short trade forward a (D) number of bars. This means moving the virtual SL for the trade as specified in the system rules above. Record the trade exit for both and assign each to a separate output. Each example has C inputs and 2 outputs.

This is an example on the EUR/USD:

Inserted Code
SL = 110% daily ATR20
A=8 (I remind you that this is GMT +1/+2)
B=260
C=14
D=100

Attached Image
  • #662
  • Sep 27, 2015 4:54am Sep 27, 2015 4:54am
Quoting fajst_k
Well I asked for demo account statement not real account. It would confirm efficiency of such methods and calculation of DMB as according to statements of guys in ties its kind of impossible. Krzysztof
It is very difficult and it takes a lot of energy and effort to do DMB analysis, then perform demo/live tests. Sorry but if you want information you will need to put this energy in yourself. I won't do the hard work for you. As I said, I'm not trying to prove anything to anyone, just sharing a bit of what I do so that it might benefit those who might find it useful.
  • #663
  • Sep 27, 2015 5:07am Sep 27, 2015 5:07am
It seems that cloud mining for machine learning has started giving some fruit : http://mechanicalforex.com/2015/09/c...ning-bias.html. Those of you interested in doing DMB analysis might find that post interesting. Look at the GBP/USD case, tons and tons of bias. If those systems had been traded without DMB analysis the probability of failure would have been very high. With the DMB analysis the decision not to trade them can be made very easily.
  • #664
  • Sep 27, 2015 5:14am Sep 27, 2015 5:14am
For those of you with access to F4 I will email Daniel the functions to create a system as described on this post, he will probably debug and polish them further and should add them to the next F4 update.
  • #665
  • Sep 27, 2015 6:03am Sep 27, 2015 6:03am
Talking about this type of system, the exit we decide to use in trades and the ML algo is one of the salient points in the successful implementation of ML strategies of this kind. Since we try to predict each trades' result, the exit will set to some degree if this prediction can or cannot be made successfully. Ask yourself if the exit tactic might be relevant to the level of DMB as well? Do you think different exits can lead to different DMB levels? What type of exit is best to use? Is it better to use just an SL? A trailing stop? A take profit? What kind of trailing stop? Give all these questions ample thought, getting the answers is necessary if you want to get as much as you can from this type of ML algo trading.

Are any of you doing DMB analysis? Share your experience! I'm really curious!
  • #666
  • Edited 6:48am Sep 27, 2015 6:34am | Edited 6:48am
Quoting algoTraderJo
{quote} It is very difficult and it takes a lot of energy and effort to do DMB analysis, then perform demo/live tests. Sorry but if you want information you will need to put this energy in yourself. I won't do the hard work for you. As I said, I'm not trying to prove anything to anyone, just sharing a bit of what I do so that it might benefit those who might find it useful.
Well I put the effort. My system based on ML trades 8 instruments on 1 min charts. From attache screenschoots u can see equity curve from demo account from 15.09 for total account and separate symbols. (around 10 000 trades done for 2 weeks). The system is being retrained for every 24h, the same methodology is applied to all symbols. From screenshoots for last 2 weeks u can see that is was profitable and efficient for 1st week than lost efficiency. Backtest is as usual is profitable. This example shows that even big number of trades and averaging performance from portfolio don't guarantee future performance. Of course some symbols were profitable at this time, some not.

Comparing to methodology form this thread, your methodology generates very little trades (due to daily TF), you usually show results from one instrument and because of this I believe is much more biased than mine. At the top of it demo account results are not presented and backtesting on like 10-20 years back data is in my opinion meanless as at this time market was very different, different players and market trading techniques and technologies.

I hope u understand why i asked for real trading results.

Krzysztof
Attached Image(s) (click to enlarge)
Click to Enlarge Name: Image1.jpeg Size: 264 KB
Click to Enlarge Name: Screenshot1.jpeg Size: 271 KB
Attached File(s)
File Type: zip results.zip   3 KB | 320 downloads
  • #667
  • Sep 27, 2015 9:48am Sep 27, 2015 9:48am
Quoting fajst_k
{quote} Well I put the effort. My system based on ML trades 8 instruments on 1 min charts. From attache screenschoots u can see equity curve from demo account from 15.09 for total account and separate symbols. (around 10 000 trades done for 2 weeks). The system is being retrained for every 24h, the same methodology is applied to all symbols. From screenshoots for last 2 weeks u can see that is was profitable and efficient for 1st week than lost efficiency. Backtest is as usual is profitable. This example shows that even big number of trades and...
Have you tested whether backtesting on 10-20 years of data is really meaningless? This is an opinion you have. Consider that it might be very wrong. Look at the 1H data distribution of returns from 10 years ago and from this year. Do you see them being very different? Have you actually tried backtesting for 25 years? Many times people say "long term backtests are meaningless" simply because they have a hard time producing profitable backtests over 25 years or because it's very expensive for them computationally. If you do not want to do long term backtests then I suggest you find a strong mathematical reason not to do so. You may be falling into a trap because of holding an opinion you haven't challenged! Do not simply make a trading decision because you "believe" something, get some data that actually supports you so that you can have a real mathematical reason to do things.

Also have you tested the DMB of your method? If you haven't your method might have a hopelessly high mining bias and what you're doing might be an exercise in futility. IMHO having a low DMB and doing constant retraining (after every trade) is critical to achieving good results.

I want to emphasize that I'm not saying that "mine is better than yours" or anything like that. I'm just sharing what I do. If you do not like it or you think your method is better then continue using it and do not use my suggestions. I want to say it one more time: I am just sharing what I do. Anyone should do as they please with their own money, time and resources. Again, I am not trying to prove anything to anyone. Read the thread and take whatever you find useful for yourself, discard the rest.
  • #668
  • Sep 27, 2015 9:58am Sep 27, 2015 9:58am
Highly stable, strong compounding results are common with this trailing stop modification. This is another example:

Inserted Code
SL = 110% 1D/ATR20
A= 6 (GMT +1/+2)
B= 205
C= 6
D= 120

Attached Image


See the SL value of 110%. How does it compare with values for similar systems I have showed before? Can you explain the difference?
  • #669
  • Sep 27, 2015 5:05pm Sep 27, 2015 5:05pm
Quoting algoTraderJo
{quote} Have you tested whether backtesting on 10-20 years of data is really meaningless? This is an opinion you have. Consider that it might be very wrong. Look at the 1H data distribution of returns from 10 years ago and from this year. Do you see them being very different? Have you actually tried backtesting for 25 years? Many times people say "long term backtests are meaningless" simply because they have a hard time producing profitable backtests over 25 years or because it's very expensive for them computationally. If you do not want to do...
Can you give any reference with officially released research paper which will use your method of calculation DMB ?? Or it is just your method ??
I only know 2 papers related to this and they are attached.

Krzysztof
Attached File(s)
File Type: pdf backtest-prob.pdf   1.1 MB | 1,001 downloads
File Type: pdf deflated-sharpe.pdf   1,024 KB | 1,052 downloads
  • #670
  • Sep 27, 2015 5:11pm Sep 27, 2015 5:11pm
A test on the USD/JPY shows useful potential systems as well. See this one as an example:

Inserted Code
SL = 90% 1D/ATR20
A= 1 (GMT +1/+2)
B= 215
C= 5
D= 100

Attached Image


I like the lack of additional parameters of this type of trailing. We do not make our DMB search more complicated and yet we do enhance our system results. Exit tactics that do not depend on additional parametrization are nice! Can you think of any other trailing stop types that do not introduce more parameters? Share them with us
  • #671
  • Sep 27, 2015 5:25pm Sep 27, 2015 5:25pm
Quoting fajst_k
{quote} Can you give any reference with officially released research paper which will use your method of calculation DMB ?? Or it is just your method ?? I only know 2 papers related to this and they are attached. Krzysztof {image} {image}
I advice you read babelproofre's posts on the matter. Read his blog as well. He has already pointed to several sources where you can read more about this type of DMB analysis. I have been using this analysis for the past several years and so have several colleagues of mine. It is a statistically sound test for the null hypothesis it is testing. FYI it has given me the best results when building distributions of the Sharpe ratio, although I know others use Sortino, Martin, etc. IMHO it is very expensive but very necessary. I would never trade a machine learning based system that has not been showed to have a low mining bias.

I would also deem it necessary but not sufficient. For me things did not start working until I started using very long term backtests (with retraining after every trade) to ensure I could cover as many market conditions as possible. Very long term backtesting plus DMB analysis was the real endgame for me. I'm not saying this is the only way, just what worked out for me.
  • #672
  • Sep 28, 2015 7:14am Sep 28, 2015 7:14am
There are other "light" machine learning method besides linear regression... Any ideas?
  • #673
  • Sep 28, 2015 6:24pm Sep 28, 2015 6:24pm
Quoting algoTraderJo
{quote} I advice you read babelproofre's posts on the matter. Read his blog as well. He has already pointed to several sources where you can read more about this type of DMB analysis. I have been using this analysis for the past several years and so have several colleagues of mine. It is a statistically sound test for the null hypothesis it is testing. FYI it has given me the best results when building distributions of the Sharpe ratio, although I...
Yes I know dekalog blog chapter about DMB. So i understand that there is no research paper where your method of measuring DMB is described ?? Is it correct ??

Krzysztof
  • #674
  • Sep 28, 2015 6:55pm Sep 28, 2015 6:55pm
Good discussion here. Curious on others thoughts about algo testing periods. I have been testing different algo systems on shorter time frames that are optimized against much more recent data windows, all with a statistically significant data size (open to debate on what this is). It is like comparing a system that works well over 10 years of data versus one that works well over one month, where each system has the same number of trades.... After looking at this for years, I am starting to think the latter is a better approach for algo based trading, primarily because patterns usually last at least few months and the system can change and adjust accordingly thereafter, whereas the system fitted for 10 years of data is fixed and slow to respond to market changes.
  • #675
  • Sep 29, 2015 2:49am Sep 29, 2015 2:49am
Hi Joe,

great to see you back with more ml-ideas.
I have been developing and testing the past months. As you say, there is a lot of effort required. Even when you have a working framework there are still a million ways to adjust the input-output-system entry-exit structure and computing time frustrates on top of it.

Quoting algoTraderJo
Ask yourself if the exit tactic might be relevant to the level of DMB as well? Do you think different exits can lead to different DMB levels? What type of exit is best to use? Is it better to use just an SL? A trailing stop? A take profit? What kind of trailing stop? Give all these questions ample thought, getting the answers is necessary if you want to get as much as you can from this type of ML algo trading. Are any of you doing DMB analysis? Share your experience! I'm really curious!
On one of my systems, which is very loosely a variation of the systems you presented in this thread, I did measure the DMB. However, at this time not to the extend I wish I could. I calculated the original on 28 years of data and afterwards the entire process on 20 bootstrapped versions of the original instrument - I wish I could have done 50 or 100, but these 20 already took my 2 computers 2 weeks! I use my own metric, which is a combination of profit, drawdown and R2 and save values above 2.5 as relevant systems.

Original instrument: 24,000 > 2.5 // 2864 > 3.5 // 232 > 4.5 // 17 > 5.5
Bootstrapped instruments (all 20): 21 > 2.5 // 0 > 3.5

The results are exciting, however, even with these great metric results any system can be months in drawdown. All of the systems have very low success rates of 30%-35%. So I will not know if the move from theory to reality is successful until next year
Portfolios of one instrument are much correlated. Several instruments look better, but with my method I get good results only on three pairs. E.g. anything with GBP does not work at all, no matter what I tried

TP killed any of my systems - I just don't get any good results with it. Surprisingly, at this time it seems like going with just a variation of SL works best (at least for my approach).

I have not checked out your last approach, you get some interesting values - e.g. 260 samples or 6 o'clock - these are rather unusual parameter values (from my research). So I will certainly check it out. Thanks!

Cheers
KaBo
  • #676
  • Sep 29, 2015 6:15am Sep 29, 2015 6:15am
Quoting fajst_k
{quote} Yes I know dekalog blog chapter about DMB. So i understand that there is no research paper where your method of measuring DMB is described ?? Is it correct ?? Krzysztof
I have never searched for one so I cannot tell you if there is. There might or might not be one, you would have to search for it.
  • #677
  • Sep 29, 2015 6:18am Sep 29, 2015 6:18am
Quoting nonlinear
Good discussion here. Curious on others thoughts about algo testing periods. I have been testing different algo systems on shorter time frames that are optimized against much more recent data windows, all with a statistically significant data size (open to debate on what this is). It is like comparing a system that works well over 10 years of data versus one that works well over one month, where each system has the same number of trades.... After looking at this for years, I am starting to think the latter is a better approach for algo based trading,...
In the systems I have posted retraining is done after every trade. This means that patterns are constantly being redefined in the 25 year backtests.

For me it is very different to have a trading system with a 10 year backtest with 1000 trades than a system with 2 months of trading with 1000 trades. The system that traded for 10 years faced a much more varied set of market conditions, the system that traded for 2 months is much more curve fitted (faced less conditions). I agree that adapting is key, this is why retraining the machine learning methods constantly is so important. IMHO it is also very important to remove curve fitting as much as possible, that means test on as many conditions as possible. Of course, for the lower time frames this might be different. The lowest I trade is 15M, so I don't know if the above makes sense for things below that.
  • #678
  • Sep 29, 2015 6:26am Sep 29, 2015 6:26am
Quoting KaBo
Hi Joe, great to see you back with more ml-ideas. I have been developing and testing the past months. As you say, there is a lot of effort required. Even when you have a working framework there are still a million ways to adjust the input-output-system entry-exit structure and computing time frustrates on top of it. {quote} On one of my systems, which is very loosely a variation of the systems you presented in this thread, I did measure the DMB. However, at this time not to the extend I wish I could. I calculated the original on 28 years of data...
The GBP instruments are difficult. Even if you find something profitable the DMB is going to be high most of the time. I just have about 5-10 GBP/USD systems because of this. What you say all matches my experience, the TP tends to be bad (results without the TP are almost always better) and different kinds of stops can lead to systems that compliment each other better. You're definitely on a good track. Consider testing lower timeframes (15M-30M) to get that bias lowered on the GBP/USD and get creative with the way that you design stops.

It is rather interesting to see how almost always low DMB systems have low success rates with very high risk to reward ratios. This resonates strongly with "the trend is your friend" and "cut your losses short and let your winners run"....
  • #679
  • Sep 29, 2015 6:38am Sep 29, 2015 6:38am
Quoting algoTraderJo
{quote} In the systems I have posted retraining is done after every trade. This means that patterns are constantly being redefined in the 25 year backtests. For me it is very different to have a trading system with a 10 year backtest with 1000 trades than a system with 2 months of trading with 1000 trades. The system that traded for 10 years faced a much more varied set of market conditions, the system that traded for 2 months is much more curve fitted (faced less conditions). I agree that adapting is key, this is why retraining the machine learning...

If the trade count of each system is the same and of a statistically significant amount, then I am not so certain that a daily system that traded for 10 years is much more curve fitted than the system that traded for 2 months on M15. The thought is that that the system only needs to see a statistically relevant number of trades over a given test window, whatever that window may be (I don't know this to be true yet). All this keeps leading me to question how many trades are a statistically significant sample size.
  • #680
  • Sep 29, 2015 7:17am Sep 29, 2015 7:17am
Hi AlgoTraderJo,

1. Do you paper trade the strategies before going live? if yes for how long?
2. How you build the portfolios? Equal weight ?
1 trader viewing now