Quantitative Approach to Backtesting | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Japan vs US QE (Quantitative Easing) 2 replies

Is there anyone using behaviour+quantitati ve Analysis ? 0 replies

applications using quantitative data on the NYSE 0 replies

The Shark's Trading Book - Quantitative Trading 2 replies

Quantitative Qualitative Estimation for MT4? 0 replies

Attachments: Quantitative Approach to Backtesting
Exit Attachments

Quantitative Approach to Backtesting

  • #1
  • First Post: Dec 22, 2008 1:16pm Dec 22, 2008 1:16pm
Just curious who the individuals are in this message board that actually TEST the ideas they post with backtests and understand how to properly design a test (avoid curve-fitting, peeking, bias, include trading costs, etc.)

I see 100's of pages of nonsense in the trading systems forum where it changes completely from page 1 to the Last page and you hardly recognize what your looking at anymore!

The purpose is to perhaps collaborate with individuals, I have a number of mean reversion models that work extremely well in backtest and live testing however I am struggling to incorporate a trend-following or long volatility strategy to combine in my portfolio to reduce draw-downs.
  • #2
  • Dec 22, 2008 1:19pm Dec 22, 2008 1:19pm
Quoting knocks420
Just curious who the individuals are in this message board that actually TEST the ideas they post with backtests and understand how to properly design a test (avoid curve-fitting, peeking, bias, include trading costs, etc.)

I see 100's of pages of nonsense in the trading systems forum where it changes completely from page 1 to the Last page and you hardly recognize what your looking at anymore!

The purpose is to perhaps collaborate with individuals, I have a number of mean reversion models that work extremely well in backtest and live testing...
FINALLY,
Someone in FF understands it like me.
Request custom Expert Advisor or Indicator: dostapyuk "at" gmail . com
  • #3
  • Dec 22, 2008 1:28pm Dec 22, 2008 1:28pm
The vast majority of newbies here who add modifications to a system do so because it "looks good".

I usually don't bother with modifications unless whoever is modifying the system can completely explain to me why they want to make the modification.
  • #4
  • Edited 1:38pm Dec 22, 2008 1:29pm | Edited 1:38pm
lol, yes I have a good amount of experience with US equities and futures and looking more and more into FX. Reading some of these threads is liking watching a train-wreck! One or two simple conditional statements become a curve-fitted monster but worst of all its fit to the last month or maybe 2 max! {Let me first to say I'm as guilty to curve-fit as the next guy but at least I test on years of data and walk forward!} Poor newbies....

Back to the original intent, looking for tested trend-following strategies. I've tested Variance Ratio's, Drift Estimators, Vol/Channel Breakouts, even MA crosses! but haven't come across anything worthwhile. I would like to stick to high-frequency (intraday) since that is what my other models trade.

EDIT:
Just as an afterthought, I haven't tested seasonality of data. I would probably have to dump the data in R or other statistical software but will report back if I find anything
  • #5
  • Dec 29, 2008 11:37pm Dec 29, 2008 11:37pm
Quoting knocks420
lol, yes I have a good amount of experience with US equities and futures and looking more and more into FX. Reading some of these threads is liking watching a train-wreck! One or two simple conditional statements become a curve-fitted monster but worst of all its fit to the last month or maybe 2 max! {Let me first to say I'm as guilty to curve-fit as the next guy but at least I test on years of data and walk forward!} Poor newbies....

Back to the original intent, looking for tested trend-following strategies. I've tested Variance Ratio's, Drift...
Well I tested many systems in this forum with a tick precise reply and none of them work consistently or they don't work at all

And yes you can curve fit if you adjust the variables even on two indicators
the whole idea is to generalize ...pick a MA aor any indicator that looks ok and try it. don't mess with their variables
I'd rather work with profit targets, stops and break even levels
Also care must be taken when picking indicators
Introducing new indicator to the system will not always add more interpretation of the price action but always add more noise

I don't believe that a linear system can work
  • #6
  • Dec 30, 2008 4:12am Dec 30, 2008 4:12am
Quoting knocks420
Just curious who the individuals are in this message board that actually TEST the ideas they post with backtests and understand how to properly design a test (avoid curve-fitting, peeking, bias, include trading costs, etc.)
I don't really know if there is somebody doing this...Most forex system have +3 indicators and never produce significant results on the long run.

About seasonality, I did not test it but I think is more related to the market's trending status and ATR.

Ironically, we're talking about curve-fitting and quant stuff, when most system developers don't even know that 90% of their indicators are based on moving averages...
"Abandon all hope, you who enter here" La Divina Commedia, Dante Alighieri
  • #7
  • Dec 30, 2008 4:16am Dec 30, 2008 4:16am
Quoting knocks420
I have a number of mean reversion models
for my curiosity, what platform did you use to backtest the mean reversion models? I'm into them as well )

Mt4 does not allow portfolio trading, Wealth-lab let me a little disappointed and writing my own backtesting software is the least thing I want to do
"Abandon all hope, you who enter here" La Divina Commedia, Dante Alighieri
  • #8
  • Dec 30, 2008 9:41pm Dec 30, 2008 9:41pm
I use Tradestation and MatLab but try as hard as possible to back-test on Tradestation. It is just more user-friendly. I actually like Wealth-Lab quite a bit, its powerful but again, not as user-friendly as TS.
  • #9
  • Jan 2, 2009 5:32pm Jan 2, 2009 5:32pm
I found an FX trend-following program that advertises 11.9% annual return, 6.2% vol, 1.92 Sharpe. It states it utilizes Multiple Moving Averages on 8 pairs. Any insight on multiple MA's on a portfolio level??
  • #10
  • Jan 2, 2009 10:03pm Jan 2, 2009 10:03pm
perhaps you can look for this program in forexpeacearmy and see if its been discussed there.
Programming for a better future.
  • #11
  • Jan 2, 2009 10:14pm Jan 2, 2009 10:14pm
Quoting knocks420
I found an FX trend-following program that advertises 11.9% annual return, 6.2% vol, 1.92 Sharpe. It states it utilizes Multiple Moving Averages on 8 pairs. Any insight on multiple MA's on a portfolio level??

I am currently testing a series of EA's based on this method (well, a similar method) . Attached is the live testing statement, it has been open for about a week and is up roughly 3%. The floating P&L will always be equal to roughly the inverse of the closed profit in this case -80 compared to +109k but the float will earn significant carry rates ~%50 per year which makes it a fundamentally sound system.

My findings so far have been that the using "Baskets" instead of speculating on a single currency pair is a great way to reduce your risk ( you hedge your exposer while capturing the market volatility) . Also with the added revenue from the Carry if managed correctly you can earn significantally higher and consistent returns while maintaining low volatility in the accounts equity.


How about a discussion on managing "baskets" using MQL, im finding this pretty diffucult task to acomplish since you can only select one open order at a time....
Attached File(s)
File Type: pdf Swarm Tester Statement.pdf   573 KB | 893 downloads
Keep it simple stoopid....
  • #12
  • Jan 3, 2009 12:25pm Jan 3, 2009 12:25pm
Quoting Kurka Fund
I am currently testing a series of EA's based on this method (well, a similar method) . Attached is the live testing statement, it has been open for about a week and is up roughly 3%. The floating P&L will always be equal to roughly the inverse of the closed profit in this case -80 compared to +109k but the float will earn significant carry rates ~%50 per year which makes it a fundamentally sound system.

My findings so far have been that the using "Baskets" instead of speculating on a single currency pair is a great way to reduce your...
In regards to baskets, especially with trend-following strategies, correlation can be used to reduce overall volatility. Baskets should be constructed utilizing the most non-correlated securities available. This is why MPT stresses allocation between bond/stocks to produce higher returns then an all stock correlation despite the fact that stocks have a higher average return. Also something to consider is how correlations change over time. Kurka, your recent returns are good but is much too small a sample to make any conclusions. Do you have any backtests? Carry strategies display a short gamma P&L profile, frequent and consistent returns followed by infrequent but especially large losses. Unless you have an approach to risk mgmt through stops, discretion, and proper leverage, I would not include carry in your analysis and just accept it as a nice cherry on top.
  • #13
  • Jan 4, 2009 4:34pm Jan 4, 2009 4:34pm
Quoting knocks420
In regards to baskets, especially with trend-following strategies, correlation can be used to reduce overall volatility. Baskets should be constructed utilizing the most non-correlated securities available.
Please explain... If your following a trend why would you want an inversley correlated basket... the point is to expose your self to the trend not to hedge. (infact the NFA just banned hedging because there was no economic benefit to the traders) Correlation strategies always profit from volatility not trends.


Quoting knocks420
This is why MPT stresses allocation between bond/stocks to produce higher returns then an all stock correlation despite the fact that stocks have a higher average return.
Rule number 3 when trading forex... throw everything your know about other markets (stocks, bonds, commodities ect...) out the door.

Quoting knocks420
Also something to consider is how correlations change over time.
Ok, but how should that effect what is true today? I dont see why it matters when i am considering entering or exiting a position currently. Please explain

Quoting knocks420
Kurka, your recent returns are good but is much too small a sample to make any conclusions.
I agree. thats why im testing...

Quoting knocks420
Do you have any backtests?
Cant produce a back test with a system that trades 32 currency pairs. Unless you have another method ?

Quoting knocks420
Carry strategies display a short gamma P&L profile, frequent and consistent returns followed by infrequent but especially large losses. Unless you have an approach to risk mgmt through stops, discretion, and proper leverage, I would not include carry in your analysis and just accept it as a nice cherry on top.
Of the larger forex hedge, funds most of the profitable ones always have a positive carry. And it is usually clearly stated in the funds prospectus.

Lets discuss managing baskets of positions using MT4...
Keep it simple stoopid....
  • #14
  • Edited 6:40am Jan 5, 2009 6:26am | Edited 6:40am
I could not abstain myself from these questions, even though 1000 question marks came into my head when I read your reply

Quoting Kurka Fund
Rule number 3 when trading forex... throw everything your know about other markets (stocks, bonds, commodities ect...) out the door.
Where did you get this??????????????????

Let me guess, rule number 2 says rule number 3 is crap??? No question you put those questions about correlation when you trade using such ideas (I personally call them myths)

Quoting Kurka Fund

Cant produce a back test with a system that trades 32 currency pairs. Unless you have another method ?
You can use both Wealth Lab and Tradestation for backtesting portfolio systems.
"Abandon all hope, you who enter here" La Divina Commedia, Dante Alighieri
  • #15
  • Jan 5, 2009 8:45am Jan 5, 2009 8:45am
Apologies, I should have clarified that you want UNcorrelated securities (i.e. correlation ~ 0 ) rather then 1 or -1. The idea is to create a combination with weighted volatility that is lower then a single security alone. You are actually already aware of this since you are combining 32 different pairs. Perhaps something you should also be aware of is that the benefits of this diversification actually fall off after combining ~5 securities (I believe, someone please double check). Add on the additional slippage/spreads and you are likely doing a disservice to your portfolio.

If you agree with the above, you should be concerned with how these corr. change over time.

This method has implications far beyond finance, in fact MPT has been applied to IT spending, social psychology, and a number of unique applications so it is quite robust.

As semar stated, Wealth-Lab is an alternative. I have no experience and don't utilize MT4.
  • #16
  • Jan 5, 2009 12:37pm Jan 5, 2009 12:37pm
Quoting knocks420
Apologies, I should have clarified that you want UNcorrelated securities (i.e. correlation ~ 0 ) rather then 1 or -1. The idea is to create a combination with weighted volatility that is lower then a single security alone.
Please explain how this works? If your following a trend wouldent you just expose yourself to that trend to get the max benefit? Why trade any correlation at all? I think that we are speaking apples and oranges here...
My systems use correlation and Baskets to profit from market volitility regardless of market direction. The more volitile the market the more profit.

Quoting knocks420
You are actually already aware of this since you are combining 32 different pairs. Perhaps something you should also be aware of is that the benefits of this diversification actually fall off after combining ~5 securities (I believe, someone please double check).
Please explain ?

Quoting knocks420
Add on the additional slippage/spreads and you are likely doing a disservice to your portfolio.
The transaction costs on any basket system is higher, but thats the cost of trading with greater confidence intervals. Look at the tests ( Live results : http://earesults.com/system/365/1/ticket/asc ) the system is trading within the mathematical constraints.

Quoting knocks420
If you agree with the above, you should be concerned with how these corr. change over time.
Again please explain. I understand that correlations will change over time but how should that change what I do at any given current time ? ( i guess it would be nice to predict the future in anything but I tend to trade what I see)

Quoting knocks420
This method has implications far beyond finance, in fact MPT has been applied to IT spending, social psychology, and a number of unique applications so it is quite robust.
What does this statement have to do with anything related to the fourm ?

Quoting knocks420
As semar stated, Wealth-Lab is an alternative. I have no experience and don't utilize MT4.
Ill give it a try but I think that anyone here will agree that forward testing is the way to go...

Can, we discuss managing baskets now? I would like to hear some constructive imput on how you are managing your baskets of open trades ...
Keep it simple stoopid....
  • #17
  • Jan 5, 2009 12:58pm Jan 5, 2009 12:58pm
I think you are misunderstanding correlation and serial correlation (trends), one describes a single time series whereas the other describes co-movement.

Let me ask you a question and perhaps you can come to your own conclusion; why are you utilizing 32 pairs?? Why not 16, 20, 15, or 10?

Forward testing is essential but with so much uncertainty in financial series and the relative ease of back-testing why wouldn't you?


Do you manage some kind of funds or OPM?
  • #18
  • Jan 5, 2009 1:15pm Jan 5, 2009 1:15pm
Quoting knocks420
I think you are misunderstanding correlation and serial correlation (trends), one describes a single time series whereas the other describes co-movement.

Let me ask you a question and perhaps you can come to your own conclusion; why are you utilizing 32 pairs?? Why not 16, 20, 15, or 10?

Forward testing is essential but with so much uncertainty in financial series and the relative ease of back-testing why wouldn't you?


Do you manage some kind of funds or OPM?

please explain the applications of correlation and serial correlation... within the scope of the Spot FX market ?

32 pairs, because that is what is available on the MT brokers that I am testing with. The point is to create an absolute return using the whole market. Since we are dealing with derrivitives the more the better. Every additional pair adds mathmatical constraints to the neural net which are proportional to number of other available pairs. Although some pairs will not be traded since the transactional costs outweigh the benefit. Simply stated, the more rules you know that confine the market the better you will be able to trade it.

Yes I manage OPM.
Keep it simple stoopid....
  • #19
  • Jan 5, 2009 2:10pm Jan 5, 2009 2:10pm
Quoting Kurka Fund
please explain the applications of correlation and serial correlation... within the scope of the Spot FX market ?

32 pairs, because that is what is available on the MT brokers that I am testing with. The point is to create an absolute return using the whole market. Since we are dealing with derrivitives the more the better. Every additional pair adds mathmatical constraints to the neural net which are proportional to number of other available pairs. Although some pairs will not be traded since the transactional costs outweigh the benefit. Simply...
Ah, you are utilizing the pairs as inputs into your Neural Network. If using an NN I would highly recommend that you walk-forward your strategy especially on out-of-sample data as most NN's are notorious curve fitters. Now I am at a lost of how or why you would trade a NN without this?? In addition you are trading fairly frequently which would leave me to believe you entered high-frequency data as your input, which again is embedded with noise.

Now that that is cleared up - Not letting you off the hook so easy on correlation Why do you trade ALL 32 pairs, what happens if you continue to use all pairs as inputs into the NN but only traded 1, say EURUSD?
  • #20
  • Jan 5, 2009 2:54pm Jan 5, 2009 2:54pm
Quoting knocks420
Ah, you are utilizing the pairs as inputs into your Neural Network. If using an NN I would highly recommend that you walk-forward your strategy especially on out-of-sample data as most NN's are notorious curve fitters. Now I am at a lost of how or why you would trade a NN without this?? In addition you are trading fairly frequently which would leave me to believe you entered high-frequency data as your input, which again is embedded with noise.

Now that that is cleared up - Not letting you off the hook so easy on correlation Why do you trade...
Explain "Walk Forward" ...

The NN solves for the best solution for each pair, hence curve fitting to find that solution... I thought that was the point. Each pair will have its different solution for Entry/Exit and that solution changes based on the nurons used. These trading solutions change every munite so its impossible to know exactly which strategy you are using at any given time. You just know that it is mathematically the healthiest. Thus giving you the maximum number of trading opportunities.... In addition the use of all available pairs allows you to hold on to desirable positions for longer periods because you can control the downside using additional market constraints (triangular,series,correlation,ect..) which controls your risk and is more important than you r entry and exits. In fact I am also testing this using a random trade engine (basically a coin toss) and am getting similar results... why don't you scroll through the live statement I posted a link to. I could easily add a rule that says to only trade EURUSD when signals come up but that would be much less profitable and would often result in orders being stopped out given the position risk is proportional to the square root of the number of orders in the order table / 20% of account balance. Trading without stop losses is a huge benefit here too. (which is a whole new discussion)

The whole point being that speculating on a single pair is much less efficient and more risky than taking calculated positions on the whole market...

For instance my live demo testing account is up over 10% (including the floating draw down) with a maximal draw down of %0.1 in a week. And more importantly the system is scalable unlike scalper systems or other high frequency strategies.
Attached Image (click to enlarge)
Click to Enlarge Name: Live Statement Snapshot.jpg Size: 88 KB
Keep it simple stoopid....
0 traders viewing now