ETMA_The Evolution

Yeah, good logical Framework to work with, i replicated most of his models for another platform with 90% precision and it works even with the same settings on other Data Feeds, robust is the word for it but its still early to Judge .

Using OpenKantu in Practice: Finding systems on the AUD/USD

When building systems using OpenKantu you will notice that it is rather easy to find strategies that generate high R² results on symbols like the EUR/USD and the USD/JPY while it is generally significantly hard to get positive results on symbols like the AUD/USD. I have done a lot of research about why this is the case – you can read more about that here – but finding systems on symbols where it has been traditionally difficult for me to find any has always been an interesting goal. Today I want to show you how you can actually use the OpenKantu software to find some highly historically stable strategies on the AUD/USD, we will discuss why this is the case and the potential problems that are present when this is done.

When you use OpenKantu to mine trading strategies on the 1D charts and you choose a simple stop-loss as the only way to control position closes what you have is that most systems found by OpenKantu behave somewhat like trend followers. This is due to the fact that the generated systems have their SL updated whenever a signal is generated in the same direction as an open trade – this is done to avoid trade chain dependency – and a trade is only closed either when the SL is touched or when a signal in the opposite direction is triggered. What happens is that the SL tends to behave like a “trailing stop” within the profitable systems that are generated and the systems end up benefiting from long term momentum, in the end you end up with systems that are in some senses trend followers.
When you mine systems like this on the EUR/USD, USD/JPY, USD/CHF or GBP/USD you tend to find a large number of strategies but on other symbols, especially a symbol like the AUD/USD, the landscape is usually terribly barren. This is because on these first symbols there is an abundance of strong momentum through their history while in the AUD/USD there does not seem to be enough of this to generate strategies of this type at all. The solution to avoid generating this type of system and instead find some highly linear strategies is basically to simply enable the use of the take-profit and use a space where the max shift is restricted to be lower than 100.

Doing the above allows you to find systems with R²>0.95 with a frequency of more than 10 trades per year using daily data from 1987 to 2016. The produced strategies are alike the strategy showed above where historical results are quite consistent through the entire period. Although as you can see in the options the SL and TP are allowed to vary between 0.5 and 5 in 0.1 steps we actually do not find very extreme SL to TP ratios but as a matter of fact – as showed below for a sample of mined systems – there are both SL>TP and TP<SL cases but most actually have R:R ratios between 0.8 and 1.3. There is only one extreme case where the R:R is 0.56 where the TP is 1.3 and the SL is 2.7 (the most extreme ratio is the samples I mined).
Of course the introduction of the TP implies the introduction of an additional degree of freedom which implies that the mining bias of the process has been increased. It is possible that all of the generated results are therefore the result of pure mining bias – rather than the presence of true historical inefficiencies – reason why it’s important to repeat the exact same mining process on data generated using bootstrapping with replacement to ensure that the mining bias of the process is low and that the systems generated have a low probability to come from random chance.

The above results also support some of my previous observations regarding reward to risk ratios and how establishing different reward to risk ratios forces the creation of trading systems that are very fundamentally different in nature (see here). Not only do the reward to risk ratio can affect the resulting data mining bias of the process – because the number of systems that can be found just by random chance chances – but it can also increase the number of systems that are in fact found on the real data. Not all pairs might be suited for all reward to risk ratios and finding systems on pairs where it has been traditionally harder might only be a matter of changing the system character we’re asking for.

Copyright Mechanical Forex - Trading in the FX market using mechanical trading strategies

OpenKantu System Generator

Through the past two years at Asirikuy we have developed a substantial amount of knowledge in the generation, evaluation and live trading of automatically generated trading strategies. The first of our software development efforts in this field, Kantu, has now become deprecated within our community (see why below) and I have therefore decided to release this code into the open community in order to avoid wasting all this work and instead encourage others to also explore the possibilities of algorithmic system generation using an already built open framework.
Kantu is a trading system generator that creates trading strategies based on price-action derived rules (comparison of different open/high/low/close values) using OHLC data. Using it you can search for strategies within a selected logic space, finding those that match the statistical characteristics imposed by the user (for example you can search for systems which have a given Sharpe, reward to risk ratio, winning percentage, etc). You can see how the program looks on the image below:
–
http://mechanicalforex.com/wp-conten...21_7-20-48.jpg
http://mechanicalforex.com/wp-conten...21_7-22-14.jpg

–
These are the main characteristics of the software:

1. This and all future versions of OpenKantu will be available for free with fully open source code :o)
2. Coded in FreePascal/Lazarus , full source code available under the GPL v2 license.
3. Manual included.
4. Simply export data from your MT4 history center to use it with OpenKantu
5. Multi-platform support. Precompiled binaries available for Windows but the software can be compiled from source on Windows, Linux and MacOSX.
6. Fast simulations, a 25 year test using daily data takes only 3 milliseconds while a 25 year 1H test can take around 60-80 milliseconds. This allows you to perform millions of tests within a realistic amount of time.
7. Multi-core support, you can perform tests using as many computer cores as your computer allows
8. Configure the system creation process to search systems with or without SL/TP within a set rule complexity (maximum shift, maximum rule number, etc)
9. Configure an out-of-sample window if this is desired
10. You can search for strategies on any financial instrument.
11. Filter systems using the pre-built statistics or a custom built filtering rule
12. Get trade-by-trade system results
13. Simulate portfolios made up of different generated systems
14. Get a mathematical expectancy analysis (MAE-MFE) for long/short trades for all generated systems
15. Get balance graphs with trade results showing on an OHLC graph of the data (see where the system has traded)
16. Export generated strategies to MT4

I would also like to point out that OpenKantu is NOT a holy grail generator and that use of the program without a good understanding of potential sources of bias (curve-fitting bias, data-mining bias) is bound to lead to losing strategies in forward/live trading. Remember that past performance is never a guarantee of future results. Although OpenKantu is coded in good faith end users are responsible for all uses of the software. The software is provided as-is, with no guarantees, implicit or implied. OpenKantu is also provided without any support, please refer to the manual for instructions on how to use the software. You can download windows binaries and the program’s source files by using the following links:

Current program version is v2.40.

http://mechanicalforex.com/wp-conten...2_15-16-29.png
http://mechanicalforex.com/wp-conten...21_7-13-16.jpg
http://mechanicalforex.com/wp-conten...2_15-19-37.png
notes on building from source: If building from source make sure you install Lazarus, then install the synapse and ZMSQL packages included within the github repository before attempting to build the software. If you would like to make code contributions please contact me (leave a comment on this page) and we’ll coordinate so that you can contribute on github. All contributions that improve the software for the open community are welcome.
Remember that to reproduce the same testing results obtained in OpenKantu on MT4 simulations you need to use the exact same data within the generation process and the MT4 backtesting. Along the same lines your live/demo broker’s GMT, DST and weekly opening/closing times need to exactly match those of the data used within the generation process. Using different data leads to unpredictable changes in simulation results between the programs.
You may wonder why we decided to discard use of Kantu within our community (given all the above positive characteristics) we decided to move to pKantu as it has many advantages over our initial Kantu (now OpenKantu) implementation. With pKantu we have the following features:

1. Coded in OpenCL/Python with speed as the top priority
2. Explicit evaluation of the entire logic space (openKantu uses random sampling instead which can lead to important issues when attempting to evaluate some sources of statistical bias)
3. Extremely fast simulations using GPUs, around 100-1000x faster than OpenKantu
4. Evaluation of data mining bias using automatic random data generation
5. Community cloud mining implementation that allows us to sum up all our system creation and bias evaluation efforts
6. Many alternative stop loss evolution mechanisms (which means we have access to more advanced exit techniques)

If you’re interested in learning more about sources of statistical bias in automatic system generation and using pKantu, our latest automated system generation software please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general.

Copyright Mechanical Forex - Trading in the FX market using mechanical trading strategies

Interesting D1 Triple Bottom with Divergences and a Vic Sperandeo 1-2-3 Pattern on the H4, signs of a Reversal .

Strategy Testing - Violating These Steps Will Damage Your Account

One of the most rewarding experiences for a trader is to pick up a performance report that proves their great strategy idea is indeed a profitable strategy. Strategy testing done properly, as is outlined in this article, can verify the efficacy of your trading strategy and give you confidence to start trading it. But be forewarned, strategy testing done improperly can lead you toward financial destruction.
Strategy testing done incorrectly can result in false hope in a losing strategy. Here is a dramatic example.

Strategy Performance Report BEFORE Proper Back Testing
http://www.customizedtrading.com/tmp/Grail_Before.JPG

Same Strategy - Performance Report AFTER Proper Back Testing Followed
http://www.customizedtrading.com/tmp/Grail_After.JPG


A trader recently shared his experience of getting great results from strategy testing his idea, but after trading it live in the market, he was losing money every day. He was baffled about what he did wrong. Having gotten excellent results on his back-testing performance report, he wondered why his promising strategy was draining his trading account. The problem was he violated several of the proper steps necessary for reliable strategy testing.
With the knowledge of how to get an accurate performance report you will be able to trust your strategy in live trading and protect your trading account. In order to properly test a strategy, there are 5 main steps that are vital to follow; configure TradeStation, "in-sample data" testing, "out-of-sample data" testing, live forward testing on the simulator account, and real live trading execution.

Step 1: Configuration
Before you begin testing your data, you must configure TradeStation so that the data it pulls onto your performance report will be accurate. Follow these 3 critical items to properly configure TradeStation.
(a) In your platform menu, go to “format symbol” and give a starting and ending date to test. This historical date range is called the "in-sample data." Do not include the most recent six months in this “in-sample data.” The most recent six months is called the "out-of-sample data," and it will be used later during your "out-of-sample data" testing step.
(b) Next, in your platform menu, go to "format strategy" and select "properties for all." Now select the "general" tab and enter the commissions and slippage (be as realistic as possible, or estimate too high if you are not sure). If this step is skipped, then the strategy testing performance report will be meaningless. If this is not done you might have a good looking performance report equity curve, but as soon as you enter the commissions and slippage figures the equity curve can reverse into an underwater equity curve.
(c) The last configuration step is under "properties for all" under the "general" tab. Look in the bottom left section called "strategy testing resolution." Check the "look-inside-bar back-testing" option and then select the smallest time frame available for your chart style to make the strategy testing more closely resemble live data. When strategy testing, testing uses the open, high, low, and closing data, thus the larger the time frame bar, the more distorted the strategy testing performance report can be. This "look-inside bar back-testing" option will make the computer do a lot more strategy testing calculations. This may really slow down your performance report generation, so please be patient. For an accurate performance report you must use the "look-inside bar back-testing" option.
These configuration steps are critical to getting an accurate performance report, so be sure this is completed precisely before continuing. Once TradeStation has been configured correctly, you can begin testing your strategy.

Step 2: In-Sample Data Testing (also called “Back testing”)
You are now ready to start testing your strategy idea. We will begin with testing the “in-sample data” that you set up for testing during the configuration steps. Begin with bringing up a trading performance report. Right now I have a performance report in front of me that I will refer to, but you will be looking at your own performance report to analyze your own numbers. This is what we will be referring to in the steps below. There are 7 sub-steps to “in-sample data” testing, as follows:
First, look at how many trades the strategy made. To reduce strategy testing errors, where error is defined by [ error = 1 / Square Root (Number Trades In Test) ], you want at least 400 trades to reduce the margin for error to 5% in your strategy testing results. At 100 trades you have a 10% margin for error.
Critical Note: The greater the number of inputs in your strategy that you optimize, the greater the number of trades required to keep from over optimizing your strategy.
Also look at how many times the strategy traded on average per day. The more often a strategy trades the more profit it can generate.
In the performance report that I am looking at, it traded 397 trades in the last 3 1/2 months, averaging 5.3 trades per day.
Second, look at the “Average Trade Amount.” It needs to be large enough that slow order fills and/or larger than normal slippage does not kill the profitability of the strategy.
In my report the “Average Trade Amount” is $162.32. The commissions and slippage amount as defined in the set up steps is already subtracted in this performance report.
65% of the time this strategy trades 1 contract.
35% of the time this strategy trades 3 contracts.
10% of the time this strategy trades 5 contracts.
Third, look to see if the “Profit Factor” and “Ratio Average Win-Average Loss” are both above 1.5 and the percentage of winning trades around 45% or better
This strategy had a “Profit Factor” of 1.83.
This strategy had a “Ratio Average Win-Average Loss” of 2.28 (2.28 means breakeven is around 28% “Percent Winning Trades”)
On this strategy the “Percent Winning Trades” was 44.58%.
Fourth, look at the trade list page and assess the profit run ups and draw downs column. Notice how many trades made money and how much money they made before the trade exit occurred. Looking at what amount of money was made in relation to the profit run up and draw down, you want to know if managing the trades could generate more profits. The example used here shows that a good percentage of trades made much higher profits than where the automated exit points occurred.
Fifth, look at the three draw down (DD) numbers. I like to see the largest number at 15% or less of the “Total Net Profit” and the “Max DD” at 5% or less of the “Total Net Profit” (these numbers tell about the draw down risk level during your trades).
Total Profit - $64,440
Peak to Valley DD - $8,960 is 13% of Total Profit
Close to Close DD - $7,120 is 11% of Total Profit
Max DD - $3,420 - 5% of Total Profit
Sixth, Looking at the “Largest Losing Trade” on the report, I like to see 5% or less of the “Total Net Profit.” In my report the “Largest Losing Trade” that occurred was $2,580 which is 4% of “Total Net Profit.”
Seventh, I review the length of time in the average trade. Does the average time in a trade comply with the golden rule of trading; "cut your losses quickly and let your profits run?" You will also want to see if the strategy is built using only profit exits (no real stop loss exits). It might have a nice looking report, but it could show a messed up ratio between average bars per winning trade verses average bars per losing trade if there are no stop loss exits. Here are my average bars:
Average bars per winning trade 7.24
Average bars per losing trade 3.51 bars
This strategy complies with the golden rule of trading. Notice how it cuts losses quickly, at an average of 3.51 bars, and lets the profits run for an average of 7.24 bars.
So what does this all mean? It means this strategy has passed the historical strategy testing phase of strategy testing.

Step 3: Out-of-Sample Data (also called "Walk Forward Testing")
Once you have tested your “in-sample data” and have determined that your strategy is worthy of continued testing, you can now test your strategy against the “out-of-sample data.” If you have not yet tested your “in-sample data, do that before proceeding.
To test the “out-of-sample data” we use the most recent 6 months of data available that you reserved in step 1(a). In step 1(b) of this article, we talked about configuring and covered entering commissions and slippage and using the “look-inside-bar back-testing” option, which must be used to run any performance report used in your strategy testing. Be sure you have configured your trading platform correctly before moving on.
Go into “format symbol” and change the date range to include ONLY the "out-of-sample data" date range that was NOT used during the strategy testing on "in-sample data." This is referred to as testing on the "out-of-sample" data.
Begin with bringing up a trading performance report on the "out-of-sample" data and review all the items that we discussed in step 2 above on this "out-of-sample" performance report. The closer it performs to the Step 2 “in-sample data” performance report, the more robust the strategy is. This suggests that the results were not from curve fitting and you have a good chance of having a viable strategy. This “out-of-sample” date range test is much more important than the strategy testing step on the “in-sample-data” for finding a successful strategy. It is a good idea to test multiple different "out-of-sample" date ranges, which is called “Walk Forward Analysis.”
Robustness: Perry J. Kaufman stated, "Practically speaking, a robust trading strategy is one that produces consistently good results across a broad set of parameter (input) values applied to many different markets tested for many years."
If the strategy fails during this "out-of-sample data" test, do NOT optimize using your reserved “out-of-sample data.” This would defeat this vitally important step in strategy development. You can go back to your strategy and fix it, or else drop it and develop a new strategy idea.
One caveat - if your strategy is capitalizing on a certain market condition, like the current volatility, and then you “out-of-sample data” test a non-volatile date range, it may not perform well, However in our next phase of testing, “ Live Forward Testing,” it could prove to be successful since we are still in a volatile market. You must understand why your strategy works, under what market conditions it performs well, and in what market conditions it does not perform well.
Now that you have tested your “out-of-sample” data and your strategy is promising, you are ready to live forward test your strategy on the simulator account.
At this point you have configured your trading platform so your performance report will be accurate, you have tested your “in-sample data” and your “out-of-sample” data and your strategy still looks great. Now you are ready to live forward test your strategy on the simulator account.
Since step 3 "Walk Forward Testing" is so vital to properly test a trading strategy, I highly recommend reading Chapter 11 of Robert Pardo second edition book titled "The Evaluation and Optimization of Trading Strategies".

Step 4: Live Forward Testing on the Simulator Account
During your live forward testing on the simulator, you want to verify that the live data feed entries and exits are similar to historical entries and exits. After you have made live data trades for a day, save the live trade list. Now reload this same chart so the strategy recalculates based on the historical for this same day. Record the historical trade list and compare the live entries and exits to the historical entries and exits. Are they the same or at least similar? Do you understand the differences and the impact your "Live Data" test says about your strategy?
Only by monitoring the program daily can performance be seen under real "live" market conditions. Continue Live Forward Testing on the simulator until you are totally comfortable that your strategy works on live data. Real time results will often be less profitable than your historic results. The key question is does the real time testing show that you have a profitable strategy that is worth trading?

Step 5: Real Live Trading Execution
Once you have done your due diligence and are comfortable with your strategy results on the simulator, you are ready to trade live. Since you are trading a brand new strategy with real money, start with a significantly reduced position sizing risk of 1/4 of 1% of your account equity at risk per your stop loss point per trade. Continue trading with minimal risk until you have verified everything is working properly within your new strategy during live order executions.
Once your strategy is making money in the live market, slowly over time begin to increase your position sizing risk. Move your risk upward from 1/4 of 1% toward 1%. Continue trading at 1% risk until you have several weeks to months of consistent trading performance. If you want to be aggressive and use more, you can continue to increase SLOWLY toward 2% but I would NEVER recommend going over the maximum of 3% of your account equity at risk per trade.
If you follow the 5 steps as outlined in this article, you will now be able to confidently strategy test any strategy idea that you have. Keep this article for reference so the next time you are inspired with a great idea, you will be able to prove it out, protect your trading account, and have confidence in live trading your strategy.
Click on this link to learn how to quickly and accurately strategy test any trading idea you might have.

Trend Following Wizards in August
September 30th, 2016

A full set of negative returns for the Trend Following Wizards last month (with delays in reporting due to some funds reporting later than usual), and a strong negative performance. The YTD performance is mixed, with a near-neutral average return.
Below are the full results as of end August 2016:

Revisiting Tom Basso: How Important Is Your Entry Really?
By Justin Paolini

Many aspiring traders focus on setups and entries. I would say 90% of their time is actually dedicated to perfecting entries. That is one way to miss the forest for the trees. Back in the 1990’s, Tom Basso and Van Tharp had already issued research conclusions on the relative unimportance of entries in producing good trading results. Their “Coin Flip” study showed that across 10 futures markets, a simple random entry with a trailing stop made money.

Assuming they did not cherry pick the situations for the test, is the relative unimportance of entries still valid now? Does the random entry still work or have the markets changed? We decided to answer that question with a research project using current Forex data.

Tom Basso’s Coin Flip Study

In his book Trade Your Way..., Van Tharp explained how he and Tom Basso came up with their idea to test a trading system using random entries:

"I was doing a seminar with Tom in 1991. Tom was explaining that the most important part of his system was his exits and his position-sizing algorithms. As a result, one member of the audience remarked, 'From what you are saying it sounds like you could make money consistently with a random entry as long as you have good exits and size your positions intelligently'”. – Van Tharp, Trade Your Way...

Here are the very simple rules Tom Basso used to test the viability of a random entry system:

1) Hypothetical $1 Million account → this is required in order to simulate diversification amongst futures contracts, withstanding margin requirements and drawdowns.

2) Select markets that have more of a tendency to trend so at that time, this meant commodities and futures markets. In particular, the markets tested were Gold, Silver, US Bonds, Eurodollars, Crude Oil, Soybeans, Sugar, Deutsche Mark, the Pound and Live Cattle.

3) The exit is 3X Average True Range (10 day period) subtracted from the close. The trailing stop can only get closer to the current market price, not further away.

4) Position sizing strategy: risk 1% of equity per position

5) Selected markets must be liquid (so that trades can be entered and exited immediately with low slippage).

6) Always be in the market (so as soon as one trade closes, another is opened).

We used these same rules to run simulations in MT4 with the help of our resident programmer Craig Drury. We tested the six FX Major pairs along with Gold from January 1st 2014 to June 30th 2016 (except for NZDUSD which because of data errors, was tested only until the end of February 2016). Effectively, we tested the random entries through trending and range bound environments over the entire test period. Unfortunately, MT4 doesn’t have a Monte Carlo generator so we had to do all the runs manually (20 runs) and it was a lengthy process.

http://www.vantharp.com/Tharps-Thoug...803-chart1.jpg
Click here for a larger image.
Random Entry Trades taken as per Basso’s Rules in EUR/USD coded by Craig Consulting on MT4

Our findings? We found no significant deviations from the core concept: Tom Basso’s system’s rules using the Coin Flip entry remain as sturdy today as they were back in the 1980s/1990s.

http://www.vantharp.com/Tharps-Thoug...803-chart2.jpg
An example of the output in Excel —
Tom Basso’s Random Entry is in fact profitable in most cases.

As you can see, the random entry method ends up with a profit in most cases (and this was a robust finding across all runs). While the profit factor is also interesting, there were very few trades overall and the system produced a very low win rate. Van talks about the psychological part of trading and even with robust statistics at your disposal, you can see how traders would find it very hard to stomach this kind of a system in reality.

It was at this point that additional questions and possible complications to the test arose. We asked ourselves: just how random was Basso’s system?

To keep the discussion short & sweet, here are our thoughts:

1. Truly random entries should be random in both direction and timing. Being in the market at all times is not really random. Basso’s “randomness” simply asked the algorithm to be “long or short” randomly at a given starting date and then randomly picked long or short after each trade closed. So this means the starting point and initial conditions were likely very influential in the results.
2. The markets weren’t randomly selected. Forex and commodities exhibit autocorrelation (trendiness) just like the futures contracts used in the original study. This is another bias to Basso’s test as stocks do not exhibit the same degree of autocorrelation in returns.
3. The exits weren’t random at all, are they? The rules for exiting were very clearly defined as to not be random at all. Basso was not testing a purely random system — and neither did we. So we’re not saying that it is possible to obtain decent results simply flipping a coin in the market as to when to get in and flipping a coin as to when to get out.

In fairness, Basso was not testing for profitability of a system with completely random entries and exits. Instead, he simply devised a test to see if exits were much more important than entries. That test was positive but even then, Van said traders can do a lot better than using just a random entry — and still not spend most of their effort on entry refinement. Our initial research results got us wondering about the possibility of putting additional randomness into the test and seeing what came out. When we did, we found both confirming results on some fronts and surprising results on others.
One of the confirming conclusions that will emerge in next week’s part 2 article of this series is that the exit strategy influences returns more than the average trader expects. In particular, we used a trailing stop which is particularly suited to trending markets. When random entries were paired with trailing stops in range-bound environments, positions were “chopped up”. In a trending environment, though, they really “bucked the trend.” The importance of the environment (or as Van labels it — market type) turned out to be one of the more surprising results from the tests.

Join us next week for more about the additional research results and the rest of our conclusions.


Editor’s Note, Van added this comment: "One key to getting the random entry system to work was the 3 times volatility trailing stop (which I think I remember as 20 days). That kind of stop meant that positions could go through a lot of chop before a trend started. If the stop was shortened to 2 times ATR, even with a 200-day period — it really wouldn’t do it (my guess but I’m not sure). Another key to the system — it was ALWAYS in the market with the random part being long or short. And yes, the starting point was important, but I remember the study using 10 years’ worth of data — so then the starting point wasn’t that important in the end. I actually thought the system would have stopped working because the big commodity trends ended so I’m surprised they got it to work."

Revisiting Tom Basso: How Important
is Your Entry Really? Part II
By Justin Paolini

Editors Note: In last weeks Part I article "Revisiting Tom Basso: How Important is Your Entry Really?", Justin described the inspiration for testing Tom Basso and Van Tharps random entry system that Van wrote about in his Trade your Way... book. Using recent Forex and gold price data, the system still tested profitable but those results then led to additional questions. This week, we rejoin Justin as he describes the results of further research into the effects of more entry randomness and several other changes to the rules.

Basso Test Variation Pure Random Random Entry

The first robustness check we performed, was to make the entry method more random than in Bassos original system. We ran the iterations again with the following algorithm: a number from 1 to 20 was randomly extracted but only a 1 (Buy) or 2 (Sell) opened a position. So this system was effectively random in time, direction, and price. This system was not in the market the whole time of the test period. We also lengthened the ATR parameters. Instead of using a 10 Day ATR, which is much closer to the current markets volatility conditions, we adopted a 200 Day ATR which would be less biased by volatility clusters in the data. Also, instead of using a fixed 3xATR initial & trailing stop, we used a 1xATR Initial Stop and a 2xATR Trailing stop. The idea was to give back less profit in trending situations, and get out of the market early in choppy situations.
http://www.vantharp.com/Tharps-Thoug...4-chart1sm.jpg
Click here for a larger image.

Random Random Trades with our new rules, coded by Craig Consulting on MT4

http://www.vantharp.com/Tharps-Thoug...804-chart2.jpg

An example of the output in Excel our own Random-Random Entry

Running various iterations of our Random-Random system, we got much closer to the random results we would expect. Some interesting outcomes

1. Results ended up being much more variable. Equity curves were more of a mix between end profit and end loss compared to Bassos settings. This is to be expected because we were voluntarily adding randomness to the method.
2. The total number of trades was higher. Of course, we were not in the market as long as Bassos system was and we had tighter stops - so we have more trades. This increased our sample size and added robustness to the results we think.
3. Using a smaller trailing stop than Basso, our drawdowns were smaller yet our average profit was still larger (double) than our average loss. So the trailing stop still worked very much in our favor.
4. Strangely, the profit factor was still positive. We wouldnt expect this if the market was purely random. That result wasnt just confined to one iteration; it was consistent over all runs we performed.

At this point in our journey, Craig & I started to see through the data. We were effectively observing Forexs tendency to trend characteristic that commodities also display. So any variation of a random entry and trailing stop should yield similar results. The trailing stop is in fact a simple trade management vehicle to capture trends and we continued to find this tendency through more and more testing.*

The Most Important Thing

After thinking through our observations, we re-ran the tests but this time we gave the random entry generator a trend filter. Of course, one could debate what kind of trend filter to use because there are many variations on the theme but we wanted it to be as robust and non-discretionary as possible. We told the algorithm to look for situations like this:
http://www.vantharp.com/Tharps-Thoug...804-chart3.jpg

The non-discretional trend-filter, as applied on USD/JPY daily chart since 1/1/2015

As with many tools, the methodology is not perfect but it gets the job done. The unshaded areas were considered range periods. Remember that when we applied our random entry to the filtered market states:

 

1. Entries were still random in time so the beginning of a trend state does not imply any trade initiation
2. Entries were still random in direction, so the model also looked at long entries in downward trends and short entries in upward trends.

We were attempting to maintain the random-random nature of the signals to verify further whether filtering trending markets could enhance the performance of the signals. Our logic was as follows: if the market state is truly the most important factor, then by only entering the market in those moments (albeit randomly) we should get better results than entering at random just anywhere, anytime. In other words, were trying to help the trailing stop do its work.

Here is a sample of the results:
http://www.vantharp.com/Tharps-Thoug...804-chart4.jpg

Sample run of Tom Bassos random entry and trailing stop combined with our trend filter

We applied Tom Bassos random entry settings combined with our Trend Filter and these were our main takeaways after multiple runs:

 

1. The profit factor was more stable.
2. Wins were much higher on average which compensated for a lower hit rate.
3. End profit distribution skewed more to the positive side when compared to our runs of Bassos original random entry rule.

The trend filter worked to a certain extent but to find out if what we were seeing was really non-random, we needed to verify the opposite: the results should be bad if we used a random entry with the trailing stop (best for trending markets) inside a range.
http://www.vantharp.com/Tharps-Thoug...804-chart5.jpg

Some recent range situations as identified by our non-discretional trend filter.
A range is defined as not in trend.

Here is a sample of the results:
http://www.vantharp.com/Tharps-Thoug...804-chart6.jpg

Sample run of a random entry in a no trend environment

From this set of runs, the main takeaways are evident:

 

1. The final profit was consistently negative which was interesting considering that we were using random entries.
2. The profit factor was decisively lower than on any other random entry test.
3. The average win was no longer consistently larger than the average loss.

Summary Conclusions

After all of these test runs, its quite easy to forget the original objective of the test runs. We were attempting to find out how important the entry was for trading FX.

Our main conclusions were

 

1. A random entry with a volatility-based trailing stop on average makes money over long enough sample sizes in Forex and commodities because those markets trend for extended periods.
2. Random entry positions get chopped up inside a range.
3. The Market Type (Trend or Range) is the overriding variable affecting system performance in our tests.

So what are some of the implication for traders?

1. Focus more on identifying market types and then deploy an appropriate trading system in line with that market type.
2. While a purely random entry strategy may be viable on paper, its impractical and tough to stomach in real life. You can do much better than random for an entry strategy and still not spend most of your time refining it.
3. Keeping losses small is not enough of a strategy to make money over time since you can easily die from a thousand paper cuts.

Over to You

This is our first evidence-based piece and I do need to thank our resident programmer Craig Drury for his efforts. We would appreciate any comments on our tests and if you have ideas on how to make the results even more robust, or if you have other feedback, it would be very much appreciated.

*(for all results and iterations, please feel free to contact Craig at [email protected]).
About the Author: Justin Paolini has 10 years of experience trading FX. He has studied Van Tharps trading principles and incorporates many of those in the weekly blog posts he writes. He currently works for Forex signal provider FX Renew as a trading coach.

SEPTEMBER TREND FOLLOWING: DOWN STREAK
October 5, 2016 by Jez Liberty

SEPTEMBER 2016 TREND FOLLOWING: DOWN -5.13% / YTD: -12.37%

A second successive strong down month for the State of Trend Following index, bringing the Year-To-Date performance further in the red as we enter the last quarter of the year.
The current drawdown is getting closer to the max Drawdown (24% vs. 32%) which, as we highlighted last month, can be a good time to start trading a trend following strategy.
Below is the full State of Trend Following report as of last month.
Performance is hypothetical.

TREND FOLLOWING AFTER DRAWDOWNS
September 12, 2016 by Jez Liberty

Is this a good time to start trading a trend following strategy.

Cable implodes - flash crash in the pound

GBP/USD hits a 700 pip air pocket

It's madness in the markets as the bottom falls out of the pound.
The bids have been pulled with no buyers on the downside as the Brexit trade capitulates.
A lot of times something like this is a bottom but all those politicians who patted themselves on the back for avoiding disorderly markets after the vote better have another look.
We haven't seen this kind of move a major currency on no news (at least I think there's no news) in a long time.

I really don't know whats the Fuss is all about, flash crash, ok, but the trend was already down and its accelerating now, Brexit oblige.

Sunday, August 19, 2012
How to measure a system's edge

Many traders and authors out there constantly say things like "You need a system with positive edge", "You need to trade with odds on your side". But very few people make a more serious effort at going deeper. What does it mean to have a positive edge? and more importantly how do we measure and mathematically demonstrate that a given strategy has a positive edge?

Well, in today's post I want to address this subject by specifically talking about how to measure the edge of a given entry logic. I will use the Forex markets as I find it easy to program routines and test them on the Metatrader 4 platform. A very popular tool that's also freely available to everyone.

But first, let's define a couple of concepts that will be important:

Maximum Favorable Excursion (MFE): This statistic measures the largest temporary profit that a trade had during the whole time it was open. That is, if you entered a trade based on some logic, and you closed the trade x number of bars later, MFE measures the most favorable balance that the trade had during those x bars.

Maximum Adverse Excursion (MAE): MAE measures the largest temporary loss of trade during all the time it was open. That is, if you entered a trade based on some logic, and you closed the trade x number of bars later, MAE measures the most negative balance that the trade showed during all that time it was open.

Let's say for example that you enter a Long (BUY) order on EURUSD at 1.3000.
Your trade goes down to 1.2980, that is 20 pips against you, but then it reverses and goes up to 1.3050.
Now it is 50 pips in your favor. You decide to stay in the position.
A few minutes later EURUSD drops to 1.3040 and you decide to exit for a profit.

As a result, you had a profitable trade that yielded +40 pips, but your Maximum Favorable Excursion (MFE) was 50 pips. That is the most the trade went in your favor while it was open. On the other hand, the trade presented a Maximum Adverse Excursion (MAE) of 20 pips, that is the worst this position was, got to be 20 pips against you.

In order to analyze and quantify the edge of your entry logic, you need to calculate the MFE and MAE for as many past trades as possible. All the entries of course based on the same entry logic you are trying to evaluate.

Finally you calculate the average of all the MFE values and the average of all the MAE values. If the average MFE is greater than the average MAE, then the entry has a positive edge. The greater the MFE in respect with the MAE, the more pronounced and favorable the edge of your entry logic. This is the basis of what is called a Mathematical Expectancy Analysis.

Important: When you do a Mathematical Expectancy analysis of an entry logic, you are not evaluating position sizing, or exit logic or anything. You just determine what would have been the entry point and direction of a trade, and then you calculate the MFE and MAE a number of bars later. If you want your study to be more thorough you calculate the MFE and MAE 10 bars after the entry, 15 bars after the entry, and 20 bars after the entry. Or any number of bars for that matter, and that will show you whether your edge is more significant right after entering the position, or if it is expressed after a while being in the position.

Let's say I have a system with an entry logic that after 10 bars provides an average MFE of 50 pips, and an average MAE of 40 pips. And let's say the study was carried out for a 10 year period during which the entry logic materialized 1000 times. Well, this is a good candidate, and apparently an entry with a positive edge. Judging by the MFE and MAE values alone, you can design a simple exit. If you put your Stop Loss at 45 pips, that is five pips beyond the Maximum Adverse Excursion, and Target Profit also at 45 pips, that is 5 pips below the Maximum Favorable Excursion, then on average this system is bound to reach Target Profit more often that it reaches Stop Loss, and over time it should result in a growing equity curve.

Now, in terms of system design, you should never define a static, hard number of pips as target or stop loss. The reason is that over time, the volatility of a currency pair might and will change. For example, maybe the EURUSD usually moves 120 pips a day today, but in 10 years, days of 200 pips could become the norm. Nobody knows. That's why it is dangerous to set TP and SL with hard numbers of pips assuming the currency will be as volatile in the future as it is right now, or as it has been in the past. You need to determine you targets and profits based on the current volatility, and your Mathematical Expectancy Analysis, should also be done using a measure of volatility rather than hard numbers of pips.

How to include volatility in your Analysis.

One of the most efficient ways to measure the distances is to use a percentage of the Average True Range (ATR) to calculate your MFE and MAE, and likewise, your system would determine Target Points and Stop Loss levels dynamically based on entry price plus/minus an appropriate percentage of the Average True Range that is beneficial according to the Mathematical Expectancy Analysis.

The Average True Range measures the average size of the most recent bars in history. The usual configuration is the last 14 bars.

Now, if a trade moves 50 pips in your favor, when the Average True Range at the time was 80 pips, you wont report a MFE of 50 pips, instead you will report a MFE of 62.5% of the ATR.

Now, you run your analysis for 10 years again and you obtain that your entry logic has an average MEF of 60% of the ATR, and an average MAE of 40% of the ATR for all the entries after 15 bars.

Well, at this point you can determine that if you put your Targets not 60% but a little below, say 55% of the ATR away from the entry price, and Stop Losses 45% of the ATR away from the entry, your system is bound to hit TP more often than SL and also your winners will be bigger in this case, as the TPs are larger than the SLs. When the entry logic presents itself your system verifies the value of the current ATR and calculates the number of pips necessary for TP and SL. In this case let's suppose we have a signal to go long, and current ATR is 100 pips. Well the Target Point will be 55 pips above the current entry price (55% of ATR) and the Stop Loss should be set 45 pips below entry price (45% of ATR). Obviously, for other trades, the number of pips for TP and SL will be different because the numbers will always be calculated based on the volatility at the moment, reflected by the current ATR.

Well, I hope this explains part of the mystery of the phrase: "you need a system with a positive edge". It took me a while to find information about it and a lot of reading to grasp the concepts and put the different pieces together.

In future articles I will evaluate and discuss whether an entry based on some indicator's behavior has a positive edge or not. I will begin by analyzing the MACD Divergence Indicator built by me a few months ago. I will demonstrate whether it is possible to create a system with a positive edge, based only on MACD Divergences, and I will also evaluate which MACD Divergences are better to trade, Hidden or Classical Divergences, or whether they are useless and do not offer a positive edge over a significant number of trades.

The Lazy Trader

Tuning Up The Turtle

Dissecting the original Turtle strategy illustrates the difficulty of designing a system that can perform consistently over decades.

https://anthonyfjgarner.files.wordpr...garner0210.pdf

My tuning up the Turtle System Version applied to an H4 EURUSD 10 Years IS (In sample) and OOS (Out of sample) time series period.
All commissions, slippage and Interest with 30 % OOS (Out of sample) period included, the WFA (Walk forward analysis) is very encouraging also .

Refining the Strategy to work on a portfolio of : 10 Currencies, 10 Futures (Including ; 2 Metals, 3 Energy, 4 commodities), 4 Indices and 2 Bonds is underway .

Indulge yourself.
http://skilledacademy.com/courses/ho...ectures/683505

everytime i can get a hold of this guy i try to listen in
thou, on this setup trigger order vs trigger setup order it seems its a bit confusing and seems as the same thing. nevertheless


great results b! congradz

Andrea knows his stuff, the course is very informative but a little bit pricey.

I like Andrea's logical approach to developing trading models, he refrains from optimizing metrics but rather eliminate all things that contribute negativily to a strategy performance which produce non curve fitted models and make very robust and long term profitable systems, such a trading model will keep on performing for decades rather than weeks and months.

Hope ur pigs are doing fine and produce healthy profits.

Cheers .

Find trading strategies that work on multiple instruments with Portfoliolizer and StrategyQuant .

http://www.portfoliolizer.com/

TSSB
Trading System Synthesis & Boosting

TSSB is FREE software platform for rapid R&D of statistically sound predictive model based trading systems via machine learning.

1. Development team with over 50 years of experience
2. Statistical soundness: trading systems likely to perform well in the future
3. Rapid prototyping of trading systems without any programming
4. Objective functions based on financial performance rather than statistical fit
5. Systems based on machine learned predictive models, not human proposed rules
6. Walk-forward & cross-validation fully integrated
7. Over-fit suppression for superior out-of-sample generalization
8. Out-of-sample performance stats and p-values free of training or selection bias
9. Ensembles & Oracles and regime-specific systems
10. Portfolios of trading systems systems
11. Statistical significance testing robust to guided search training bias
12. Library of over 100 indicator families including wavelets and ARMA
13. Dimensional compression via PCA, Linear and Quadratic methods
14. Find independent groups of non redundant predictors automatically
15. Unique Graphics including predictive heat-maps