Automated Trading & Backtesting Course - Part 2: System Development

var *Price = series(priceUSD());
var *Trend = series(LowPass(Price,optimize(250,200,2000)));

Would that mean that series is calling priceUSD() for the number of historical Data availlable so that *Price can be used as the synthetic asset?
How is the price data of the underlying assets synchronized in time?
I mean that the timestamp of the bars are matching for all assets used

Yes, your code calls priceUSD() every bar and fills the series. You can then use Price[0], Price[1] etc. for the historic prices of the synthetic asset.

The bars are synchronized. The first asset() call in the code generates the bars; the subsequent asset prices are sampled in a way so that they match the start and end times of the bars created by the first asset call.

That's really looking fantastic
Thanks for your help and explanations

Let's now analyze Alice's new code. The tradeTrend() function uses a similar trade algorithm as in the first script of this course, only without the filter. Instead, this time the LowPass time period and the stop loss value are optimized. Note that the bar period used in this strategy is now 4 times higher - 240 minutes instead of 60 minutes - so the default value for the lowpass time period is now 1/4 of the previous value. You might also have noticed that Alice allows a time period up to 2000, although the lookback period is still only 500 bars. This is possible because the time period for the LowPass function is just a filter parameter. It does not require lookback time, and thus gives us more data for trading.

And why has Alice explicitly set the Trail variable to 0 in this function?

Trail = 0;

Trend trading works best when profitable trades last a long time; trailing would stop them too early. However, the Trail variable is set in the tradeCounterTrend function, which is identical to the previous script. If Alice had not reset Trail to 0 (meaning no trailing), it would keep its last value, tradeTrend would trail too, and its profits would go down! When predefined variables are used anywhere in the script, make sure that they have the right value at any place where they are needed. This is a typical source of mistakes, so always examine the trade log to be sure that trades behave as they should.

The first lines of the run function are just as in the counter trend trading script. However the core of the strategy looks very different:


We have two 'nested' while loops, each with two nested functions. Let's untangle them from the inside out:

loop("EUR/USD","USD/CHF")

The loop function takes a number of arguments, and returns one of them every time it is called. On the first call it returns the first parameter, which is the string "EUR/USD". We have learned in the first programming lesson that a string is a variable that contains text instead of numbers. Strings can be passed to functions and returned from functions just as numerical variables. If the loop function in the above line is called the next time, it returns "USD/CHF". And on all futher calls it returns 0, as there are no further arguments.

asset(loop("EUR/USD","USD/CHF"))

The strings returned by the loop function are now used as arguments to the asset function. This function selects the asset to trade with in the following part of the code - just as if it had been choosen with the [Asset] scrollbox. The string passed to asset must correspond to an asset subscribed with the broker. If asset is called with 0 instead of a string, it does nothing and returns 0. Otherwise it returns a nonzero value. This is taken advantage of in the outer while loop:

while(asset(loop("EUR/USD","USD/CHF")))

This loop is repeated as long as the while argument, which is the return value of the asset function, is nonzero (a nonzero comparison result is equivalent to 'true'). And asset returns nonzero when the loop function returns nonzero. Thus, the while loop is repeated exactly two times, once with "EUR/USD" and once with "USD/CHF" as the selected asset. If Alice had wanted to trade the same strategy with more assets - for instance, also with commodities or stock indices - she only needed to add more arguments to the loop function, like this:

while(asset(loop("EUR/USD","USD/CHF","USD/JPY","GBP/USD","XAU/USD","USOil","SPX500","NAS100",...)))

and the rest of the code would remain the same. However, Alice does not only want to trade with several assets, she also wants to trade different algorithms. For this, nested inside the first while loop is another while loop:

while(algo(loop("TRND","CNTR")))

The algo function basically copies the argument string into the predefined Algo string variable that is used for identifying a trade algorithm in the performance statistics. It also returns 0 when it gets 0, so it can be used to control the second while loop, just as the asset function in the first while loop. Thus, for every repetition of the first loop, the second loop repeats 2 times, first with "TRND" and then with "CNTR". As these strings are now stored in the Algo variable, Alice uses them for calling the trade function in the inner code of the double loop:


As we see, a string variable can be compared to a string constant just as if we would compare a numeric variable to a numeric constant. Due to the two nested loops, this inner code is now run four times per run() call:

* First with "EUR/USD" and "TRND"

* Then with "EUR/USD" and "CNTR"

* Then with "USD/CHF" and "TRND"

* And finally with "USD/CHF" and "CNTR"

If Alice had instead entered 10 assets and 5 algorithm strings in the loops, we had 50 (10*5) strategy components and the inner code would be run fifty times at every bar.

Tomorrow we'll be going to train this strategy. We'll check out what happens when you train a portfolio strategy that consists of many assets and algorithms with completely different parameters.

Hi, JCL.

I've a question.

In the line

"while(algo(loop("TRND","CNTR")))"

algo is written with "a",

but in the line

"if(Algo == "TRND")"

Algo is written with "A".

It don't generates compiler error or execution error? Or algo is the function and Algo is the variable?

Thanks

Algo, like Asset are script variables so the don't have to be declared
(see tutorial/help)

Hi jcl,

printf("Hello world") and I'm struggeling
Simply wanted to plot the price curve of priceUSD() but can't get it running. printing the values with printf() is also only resulting in a single value and that's 0.000
Could you show me how to plot the curve?

edit:
reason found
last selected asset by code was nzd/usd and for the test eur/usd was selected

When I'm using StartDate either year only or full date, Zorro crashes
I got a message that for one of the pairs there is no history data and then Zorro left me alone
NumYears is ok

In the first run before prices are loaded, all prices are at 0.00. For printing in all run cycles, use printf("#... - the "#" prints all cycles.

http://zorro-trader.com/manual/en/printf.htm

Can you let me know which asset and StartDate you used? We only included historic prices back until 2006, but it should not crash when historic data is missing. It should only give an error message.

I'm currently playing around with 14 assets...major xxxUSD and xxxJPY
and the resulting synthetic asset
Took nearly half a day yesterday to download the history for all of them
First time the error occured was with nzd/usd history missing. I commented that pair and then it was another asset...don't remember which one
StartDate was 20120101 and then 2012 only

Yep, you're right - that indeed crashes. Incredible. Obviously no one had tested with a 2012 startdate so far. This will be fixed in the next update - until then please use NumYears instead.

Hi jcl,
I'm running into problems when using more than 1 #include instruction
I wanted to use one inc for basket prices (OHLC) and one for opening long/short trades.
If I include both files in my run script I get a compiler error.
But if the first include is included into the second it's all good

#include <priceBasket.c>
#include <tradeBasket.c>

function run()
{
.........
}
doesn't work

But after including priceBasket into tradeBasket it's ok

You must always include <default.c> before including anything else. Only when you include nothing you need not do that, because then it's included automatically for beginner's sake. default.c contains all the trading stuff.

Also note that #include follows the C convention: <...> is from the include folder, "..." is from the current folder, normally the strategy folder.

ok, thanks....<default.c> did the job

Just found out that if u are using Date instead of StartDate it works although the editor doesn't mark it as script variable...depreciated?

Yes, depreciated. Zorro is not very old, but already underwent some variable-redefinition and renaming cycles.

- We're now going to [Train] the new portfolio strategy. Because we have now four components - asset/algorithm combinations - the training process now takes four times longer than with the last script. As soon as it's finished, let's examine the resulting parameters. Use the SED editor to open the Workshop6_1.par file in the Data folder. This file contains the optimized parameters from the first WFO cycle. It should look like this:


As we can see, parameters for every asset/algo combination are optimized separately. Each lline begins with the identifier for the asset and algorithm; then follows the list of parameters. This allows Zorro to load the correct parameters at every asset() and algo() call. We can see that both the tradeTrend() function and the tradeCounterTrend() function uses 2 parameters, and their optimal values are slightly different for the "EUR/USD" and the "USD/CHF" assets. The last number behind the "=>" is not a parameter, but the result of the objective function for the final optimization; it can be ignored.

Finally, let's have a look at the performance of this portfolio strategy. Click [Test], then click [Result].

http://zorro-trader.com/manual/images/work6_11.png

You can see in the chart below that now two different strategies are traded simulaneously. The long green lines are from trend trading where positions are hold a relatively long time, the shorter lines are from counter-trend trading. The combined strategy does both at the same time.

http://zorro-trader.com/manual/images/work6_21.png

Now look at the end of the performance report that popped up when clicking [Result]. You can see how the single components were performing during the test:


The performance is listed separately for long (":L") and short (":S") trades. We can see that the components performed very different: The best performer is EUR/USD:CNTR many profitable trades and good profit factors. The worst performer is USD/CHF counter trend trading; however with the USD/CHF asset, trend trading performs much better. Different assets apparently sometimes require different trade methods.

The performance report raises an obvious question. Wouldn't it be wise to invest more money in the more profitable components, less money in the less profitable, and no money at all in the components that turned out unprofitable in the test? That's called "money management" and will be the topic of tomorrow's lesson.

Money management is an algorithmic method of distributing your money among a portfolio of assets and trade algorithms in a way that your profit is maximized and your risk is minimized. As you can imagine, there are many money management methods and philosophies. Zorro follows a method developed by Ralph Vince; he uses a computer algorithm that evaluates the equity curve for calculating the optimal percentage of the gained capital to be reinvested. This percentage is called the OptimalF factor (OptF in the above performance sheet). Multiply OptimalF with the capital earned so far, and you'll get the maximum margin to be allocated to a certain trade.

Normally people try to stay well below this maximum margin; OptimalF was calculated from historic performance, after all, and there's no guarantee that this performance will continue in the future. Exceeding the maximum margin is far worse for your account than staying below it, thus normally only half or a third of OptimalF is reinvested.

Alice does not use OptimalF for it's intended purpose as an optimal reinvestment factor. Instead she uses it to calculate the optimal trade margin individually per component. For this she has modified the run function (Workshop6_2):


Alice is now generating OptimalF factors in [Train] mode. How that works will be tomorrow's topic.

At the begin of the script, Alice has now additionally set the FACTORS flag:

set(PARAMETERS+FACTORS);

You can set several flags at once by just adding them in the set function. If FACTORS is set, Zorro now also calculates OptimalF factors in [Train] mode. The factors are then used in the inner loop for determining if Zorro should trade with this component or not:

if(OptimalFLong > 0 and OptimalFShort > 0)
{
...

Zorro calculates the OptimalF factors separately for long and short trades, and sets them to 0 when the component and trade direction was unprofitable in the backtest. Only when both are greater than 0, the if(...) condition is true and the next lines are executed. Otherwise Zorro does not trade that strategy component.

Margin = 100*(OptimalFLong+OptimalFShort);

Margin is the amount invested per trade. Depending on leverage, it is a certain percentage - for instance 1% for 1:100 leverage - of the trade volume. The higher the margin, the higher the profit (or loss) per trade. A hypothetical 100 $ investment is multiplied with the sum of OptimalFLong and OptimalFShort for calculating the Margin variable from the average of both OptimalF factors. Margin determines the number of contracts purchased. This is the reward for Alice's script modifications:

http://zorro-trader.com/manual/images/work6_31.png

Disabling unprofitable components and setting the margin individually per component reduces the drawdown and generates a smoother equity curve:

http://zorro-trader.com/manual/images/work6_41.png

The OptimalF factors are not calculated in every WFO step, but for the whole period. This is because money management factors have to be calculated from the whole equity curve; but strictly speaking this violates the out-of-sample test philosophy. Therefore, the real trading performance might be worse than the profit predicted by the WFO test when you use OptimalF factors. This should be kept in mind.

Almost 300% hypothetical annual return is certainly fine, but can we squeeze more than $3000 out of this strategy over 2 years, from the same ~$400 required initial capital? By re-investing our profits we can increase the overall result - the more the longer we trade. This is the real purpose of the OptimalF factors. How this is done will be our topic for tomorrow.

For reinvesting profits, Alice modifies the script one last time and adds:

var Gain = WinTotal - LossTotal + WinValTotal - LossValTotal;
Margin += Gain/8 * (OptimalFLong+OptimalFShort);

Gain is the sum of all profits so far, including all open trades; is it calculated from the win/loss statistics. 1/8 of it is multiplied with the OptimalF factors and added to the margin for reinvestment. The 100 $ profit-independent margin are still required, otherwise Zorro would never enter a trade because Gain is 0 at the start of the test. But why 1/8? The strategy has 4 components and we add both the long and short OptimalF factors, so we need the division by 8 for dividing the profit among all components. Note that Alice did not reinvest the single component profit, but the profit of the whole strategy. It's more steady and we'll see the returns accumulate faster this way. Let's check how the reinvestment affects the equity curve:

http://zorro-trader.com/manual/images/work6_52.png

We can see that reinvesting profits has two effects. The return after 2.5 years is now about 3 times higher than before. Making $9K out of the $400 initial capital more than 2000% profit... no wonder that some traders have become quite rich. The bad news is that the drawdowns are now much deeper - that's why the displayed annual return is now quite low. The equity curve shows that the strategy is almost all the time in a drawdown, and we've lost most of our money several times in 2011 and 2012. In real trading we would certainly reinvest a smaller part of the profit; Bob is well advised to divide it by 16 or 24 instead of 8. Then, of course, the overall return will also go down. No risk, no fun.

What have we learned?

* The loop function can be used to go through a list of assets or algorithms.

* The asset and algo functions select a certain asset and set up an algorithm identifier.

* The strstr function can be used for comparing strings partially.

* The OptimalF factors give the maximum possible capital allocation to a strategy component.

* Increasing the return by reinvesting profits comes at a cost.

This was the last lesson of part 2 of the automated trading course. It's now up to you to experiment with your own trade ideas and systems...

If anything was unclear or bad explained, please post here and I'll try to help.

Thanks for taking the trouble to post in so much detail (I also appreciate the spirit behind the Zorro project, amid the current rich-get-richer vs poor-get-poorer economic climate we're in)

A few questions:

1. Regarding the calculation of OptimalF factors, Zorro calculates them only across 2 dimensions: long vs short, am I right? I mean we can't specify Zorro to calculate long trend, long counter-trend, short trend, and short counter-trend?

2. Regarding the margin calculation, you mention it is calculated "from the average of both OptimalF factors", so shouldn't the formula be 100 * (OptimalFLong + OptimalFShort) / 2 ?

1. Zorro calculates the OptimalFLong and OptimalFShort factors for every strategy component separately. The factors change after any asset() or algo() call, and thus have different values dependent on the current asset and algo inside the loop.

2. You're right. In fact the average margin per component in the example is $200, so we have Margin = 200 * (OptimalFLong+OptimalFShort)/2; which is equivalent to Margin = 100*(OptimalFLong+OptimalFShort);.