How long should you test a system for? | Forex Factory

Options

Permalinks
Print Thread

Similar Threads

How many years should I test My system, to consider it, profitable? 18 replies

How long should you test a strategy? 22 replies

How long should I test this system? 20 replies

Why EA test is forward test better than back test?? 1 reply

How long do you forward test a discretionary system? 7 replies

How long should you test a system for?

  • #1
  • First Post: Aug 3, 2009 2:41pm Aug 3, 2009 2:41pm
How long should you test a system before you know if it is profitable or needs to be taken back to the drawing bored? Some say one year, others say 90 days...I would like to hear some input from exp profitable traders. How long in your opinion does it take to know if what you are using is making consistent profits or just on some lucky run? I'm sure this is a common question. Any answer would be appreciated thanks.
  • #2
  • Aug 3, 2009 3:32pm Aug 3, 2009 3:32pm
IMHO it needs to be backtested at least for 1 year. That way you will have seen many different types of trading atmospheres. If you have the time and patience I would recommend even further back. If you are anxious to know how a system might perform then do it on a demo acct as a forward test. I recommend always having a demo account to test any idea with no risk involved.
  • #3
  • Aug 3, 2009 3:59pm Aug 3, 2009 3:59pm
continually test it. test it before trading it, test it while trading it, test it after trading it. I'd say test for at least 3-6 months before trading it. just remember, keep testing it. always.
  • #4
  • Aug 3, 2009 5:24pm Aug 3, 2009 5:24pm
depends on the time frame you`re trading in.

for a daily system I would like to go back ten years to see how the system performs in ranging and trending years.

while trading a system you will have to adjust occasionally for changing circumstances ie lower/higher volatility etc.
  • #5
  • Oct 3, 2009 7:36pm Oct 3, 2009 7:36pm
One thing I have been thinking about when considering how long to test a system for is the law of large numbers. 'a theorem in probability that describes the long-term stability of the mean of a random variable. Given a random variable with a finite expected value, if its values are repeatedly sampled, as the number of these observations increases, the sample mean will tend to approach and stay close to the expected value.' If this statement is true then would it apply to a series of trades as well? And this brings up the question does it matter more how many trades you take and not so much how long. http://en.wikipedia.org/wiki/File:LLN_Die_Rolls.gif The graph there shows the law of large numbers in average of die rolls. You can see that by 400 rolls the average is pretty consistent, meaning the results of the random rolls of the die are fairly stable. Does this mean at 400 trades you should be able to see stability in your results? Or by another number? Just some things I have been thinking about. Curious to hear some input.
  • #6
  • Oct 3, 2009 8:04pm Oct 3, 2009 8:04pm
What type of system is it? Unless you are experimenting with great calculations on some niche you discovered, or otherwise have a 100% mechanical system, I doubt testing it will do much of anything for you. So, do you have an absolute, set in stone, criteria for entry and exit? You will trade it by the book 100% of the time? You will not take a single trade off based on either a win or a loss? You will trade the exact same % of the account with every single trade? Unless you answer yes to all of these, plus probably about a dozen I haven't asked, backtesting is going to be, more than likely, a total waste of time.

If it works, keep trading it. Don't worry about if it is "luck" (hate that word) or not. If you are worried about that, cut your trade value by a quarter of what you have it at now. If it is a good system, the same % of the account will grow steadily enough, if it turns out not to be any good, then you didn't lose a whole lot.
The market is my nation. Traders, my family. Hello, brothers and sisters!
  • #7
  • Oct 3, 2009 8:23pm Oct 3, 2009 8:23pm
I agree with Stock Jay, backtesting is not the best kind of testing.

I would add two reasons. First the market changes and it changes consistenly a system that is profitable now probably won't be so in a couple of years, that means that you can probably backtest a trading system with good results but that won't be good on forward trading or testing. Second, a lot of brokers have some dark practices that a backtest won't reflect (like adjusting the spreads on news time), but that can bring any system down easily.

I would recommend you to demo test it at first, just to see that you're trading system is reasonable (that won't blow your account in a week) and then just trade for around 6 months on real money but in a very conservative way (minimum money).
  • #8
  • Oct 3, 2009 9:08pm Oct 3, 2009 9:08pm
Quoting StockKJay
What type of system is it? Unless you are experimenting with great calculations on some niche you discovered, or otherwise have a 100% mechanical system, I doubt testing it will do much of anything for you. So, do you have an absolute, set in stone, criteria for entry and exit? You will trade it by the book 100% of the time? You will not take a single trade off based on either a win or a loss? You will trade the exact same % of the account with every single trade? Unless you answer yes to all of these, plus probably about a dozen I haven't asked,...
I was talking about a 100% robust mechanical system. The median is a robust measure of central tendency. The median absolute deviation and interquartile range are robust measures of statistical dispersion. What is important is to have an adaptive mechanical system when back testing. But whether back testing works or not is a different topic. I'm more concerned with stability. Luck comes and goes.
  • #9
  • Oct 3, 2009 9:34pm Oct 3, 2009 9:34pm
Quoting Kas1
How long should you test a system before you know if it is profitable or needs to be taken back to the drawing bored? Some say one year, others say 90 days...I would like to hear some input from exp profitable traders. How long in your opinion does it take to know if what you are using is making consistent profits or just on some lucky run? I'm sure this is a common question. Any answer would be appreciated thanks.
It's not how long should you test for but how many trades one should have before you assume you have a statistically valid sample, I look for a > 90% t-test rejection of non-zero mean on the last 120 trades, with in minimum of 50. I re-evaluate this statistic after every trade.
The breaking of a wave cannot explain the whole sea.
  • #10
  • Oct 5, 2009 1:01pm Oct 5, 2009 1:01pm
Quoting Craig
It's not how long should you test for but how many trades one should have before you assume you have a statistically valid sample, I look for a > 90% t-test rejection of non-zero mean on the last 120 trades, with in minimum of 50. I re-evaluate this statistic after every trade.
I do not understand completely, can you explain what you mean by >90% t-test rejection of non-zero mean?
  • #11
  • Oct 5, 2009 1:39pm Oct 5, 2009 1:39pm
Quoting Kas1
How long should you test a system before you know if it is profitable or needs to be taken back to the drawing bored? Some say one year, others say 90 days...I would like to hear some input from exp profitable traders. How long in your opinion does it take to know if what you are using is making consistent profits or just on some lucky run? I'm sure this is a common question. Any answer would be appreciated thanks.
Always assume a lucky streak and test it till you find that it is faulty.
The only way to get smarter is by playing a smarter opponent.
  • #12
  • Oct 5, 2009 1:39pm Oct 5, 2009 1:39pm
Quoting Kas1
I do not understand completely, can you explain what you mean by >90% t-test rejection of non-zero mean?
Have a search though some of my recent posts, I explained it to somebody else in detail a couple of months ago.
The breaking of a wave cannot explain the whole sea.
  • #13
  • Oct 5, 2009 2:57pm Oct 5, 2009 2:57pm
Use a moving window of N last trades and calculate the ratio of the average return to the standard deviation of returns (Sharpe ratio). If the number is positive and constant or growing and preferably > 1 then you can safely trade your system. The higher the ratio, the higher leverage you can safely use. Once the ratio starts decreasing it's time to reevaluate the rules of your system (possibly even scrap it).
The number N I used above depends on your timeframe and how frequently you trade and even the entry/exit conditions of your system. Since you didn't give us any details you'll have to experiment a bit to determine the number.
Good luck.
  • #14
  • Oct 5, 2009 3:19pm Oct 5, 2009 3:19pm
Quoting Craig
It's not how long should you test for but how many trades one should have before you assume you have a statistically valid sample, I look for a > 90% t-test rejection of non-zero mean on the last 120 trades, with in minimum of 50. I re-evaluate this statistic after every trade.
Thank you Craig! You took the words right out of my mouth! Backtesting anything is not about "how long" but the "results". Scientists don't test and re-test things based on a time map, they test and re-test variation of the formula until they get it right. That could take anywhere from 1 day to 25 years. If the results of your testing is not to your expectations then you search for another alternative (in this case, strategy) and test the hell out of that, until the results are to your likings.
  • #15
  • Oct 5, 2009 3:36pm Oct 5, 2009 3:36pm
Quoting Craig
It's not how long should you test for but how many trades one should have before you assume you have a statistically valid sample, I look for a > 90% t-test rejection of non-zero mean on the last 120 trades, with in minimum of 50. I re-evaluate this statistic after every trade.
And that my friend is how you adapt to the changes in the market

I am doing the exact same thing, thought I were alone doing it

pay attention to this post guys, all you need to know in bcktstng.
  • #16
  • Oct 5, 2009 8:31pm Oct 5, 2009 8:31pm
Keep testing, keep tweaking, keep learning.
  • #17
  • Oct 5, 2009 9:17pm Oct 5, 2009 9:17pm
A year or two should do it. Depending on the market you will have to modify/tweak from time to time for better profit potential. (Stops, targets, average range,etc.)

It would also depend on what type of trader you are or what timeframe your working with. A system that gives 5-10trades a year would require a longer period to test than a scalper's. Just a thought.
  • #18
  • Last Post: Oct 6, 2009 12:23am Oct 6, 2009 12:23am
For me its all a matter of how many trades that I take. I would like to see a minimum of 40 trades and close to 200 trades on the high side. I have software that will allow me to replay the markets like they are in real time. I use this to my advantage to back test the markets in "real time" when the volume is low or the markets are not active. I focus on my winning %, return ratio, and other money management factors. If I like my results then I go for it. If not, I go back to the drawing board to see what I possibly missed. I dont think you need to go back in the last 10 years and test it...the markets have changed too much since then. Shoot they have changed a great deal since last year. Focus on how many trades and your rules rather than the time you go back. If you are new to trading...know that when you back test a strategy that you will find new factors when you start with a live account. Emotions that you didnt have with back testing now come into play. There are many educators out there to help with that. Hope this helps you out!
0 traders viewing now