Forward Testing! | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Mismatch between live forward testing and back testing 4 replies

Question on forward testing 5 replies

Forward Testing Discussion Thread 241 replies

NEW " SET & FORGET" Forward Testing for VP Day Trading System 16 replies

Forward Testing Results Only 33 replies

Forward Testing!

  • #1
  • First Post: Jan 17, 2016 1:18am Jan 17, 2016 1:18am
I would appreciate it if people that can answer this question and post on this thread are people who have been trading Live accounts profitably and consistently for over a year, im just trying to get as much valuable information from experienced traders as possible and not saturate the content with NOOBS TRYING TO LOOK INTELLIGENT.



1. How Long do you Forward test a strategy before you can label it as profitable?

2. When forward testing a strategy what do you look for?

3. My understanding is that each strategy has its own way of backtesting, because strategies differ,scalp, trend,news,counter trend etc. and because markets are also different trending, range bound etc. so how do you make sure that you perform sufficient tests that are not deceiving and are as accurate as possible?

4. If theres any question that you feel i left out please dont hesitate to either post it here or provide an answer.
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #2
  • Jan 17, 2016 10:42am Jan 17, 2016 10:42am
bump
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #3
  • Jan 17, 2016 10:53am Jan 17, 2016 10:53am
bump
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #4
  • Jan 18, 2016 1:22am Jan 18, 2016 1:22am
Interesting topic..
Quote
1. How Long do you Forward test a strategy before you can label it as profitable?

Totally depends on the time frame. If you're talking H4, (assuming you have sufficient back test data) then prefer to have 3 months+ live data before I commit to full risk. In that time there won't likely be hundreds of signals for data, but enough of a cycle to experience it.
H1 I will usually apply full risk after a month or so, depending on the current market volatility. But, a month is not the same as any other month... for example, I wouldn't consider using December as a realistic reflection of anything.
Lower frames, the picture is less clear. If you are scalping M1 charts you may have 5-10 signals per day. My opinion is that on M1 each day represents it's own cycle, especially as I'm session specific...trading in windows of liquidity. I need to feel that I've experienced a variety of scenarios before applying full risk. This could be in 20 trades or 200 trades, depending on the frequency of signals.

Quote
2. When forward testing a strategy what do you look for?

Large variants between my BT data and the live experience. When back testing, I look for specific data...max consecutive wins/losses, overall win %, average wave formations, ADR, slippage issues...that type of thing.
If things start to behave significantly adversely to my BT expectations, then it's time to re assess.

Quote
3. My understanding is that each strategy has its own way of backtesting, because strategies differ,scalp, trend,news,counter trend etc. and because markets are also different trending, range bound etc. so how do you make sure that you perform sufficient tests that are not deceiving and are as accurate as possible?

As I touched on above, it depends on TF and when your forward testing is taking place. The market is constantly changing, adapting to new information. You can ever really say you are 100% certain of anything. I approach each new method with the knowledge that at some point, the market dynamics will change, and there is a high probability that I will have to drop the system. I don't get too married to any strategy. They work until they stop working. If there was any single method that performed consistently for years and years, there would be no need for forums such as this.
Personally I reject the idea that the longer something is tested, the more secure you can be in it's performance. You can have 15 years of BT history for a system, start trading for a few months with all going well, and then it can blow up. So, was it worth going back 15 years? Did it help prevent anything?

Because by nature, each method is looking for a specific set of conditions, in an ever evolving market it's not a question if the conditions will change it's when.
Our job is to recognize that shift, decide what tolerance we have in the system for the adverse conditions, and if needs be, kill the system and move on.

One thing I'd like to add, is the validity of the BT can be questionable. Some people tend to fool themselves. Observational bias is a real issue.
With a purely mechanical system, it's fairly simple. If the lights are green, go in. Most traders overlook news events, when back testing which can dramatically impact some methods. But that can be fixed with either an indicator, or a thorough BT including labeling news to void impacted trades.

But...if the system uses any kind of discretion, then the BT becomes far more complex. Everything is easy in hindsight, when the outcome is in front of you. But, to trade a discretionary method, such as the famous 00 method, you literally need to go candle by candle and then objectively decide if you would enter that trade!
I use 00 trading as my example, because there have been countless threads, developing EA's and different applications of that concept which have failed miserably. The system on BT is not profitable. But...plenty of people are claiming long term success.

So, I guess my only additional question I think is important is; when a system is demonstrating higher than usual DD, what is the cut off point where you decide the system must be modified or abandoned?
Wealth comes from what you keep, not what you earn
  • #5
  • Jan 18, 2016 5:01am Jan 18, 2016 5:01am
Quoting Soros
1. How Long do you Forward test a strategy before you can label it as profitable?
There isn't a need for forward testing on only recent data. Just save yourself the waiting time and run a backtest.
Quote
so how do you make sure that you perform sufficient tests that are not deceiving and are as accurate as possible?
Depends on what you mean by 'accurate'.

Avoiding deception is quite simple. Just maintain an objective mindset and follow your research where it leads taking care not to over-fit any part of the process. People deceive themselves when they try to prove themselves right regardless of the conclusion. Over-fitting is the typical way that people try to rescue invalid systems.
  • #6
  • Jan 18, 2016 6:47pm Jan 18, 2016 6:47pm
Quoting Soros
I would appreciate it if people that can answer this question and post on this thread are people who have been trading Live accounts profitably and consistently for over a year, im just trying to get as much valuable information from experienced traders as possible and not saturate the content with NOOBS TRYING TO LOOK INTELLIGENT. 1. How Long do you Forward test a strategy before you can label it as profitable? 2. When forward testing a strategy what do you look for? 3. My understanding is that each strategy has its own way of backtesting, because...
at least 300-500 valid profitable setups
  • #7
  • Jan 18, 2016 8:27pm Jan 18, 2016 8:27pm
Quoting trader34
Interesting topic.. {quote} Totally depends on the time frame. If you're talking H4, (assuming you have sufficient back test data) then prefer to have 3 months+ live data before I commit to full risk. In that time there won't likely be hundreds of signals for data, but enough of a cycle to experience it. H1 I will usually apply full risk after a month or so, depending on the current market volatility. But, a month is not the same as any other month... for example, I wouldn't consider using December as a realistic reflection of anything. Lower frames,...

Wow, that was a very informative and beautiful post, basicaly its a 4H system, i have been farward testing it on all pairs and so far it has 15 trades and 1 loss with a descent RR, i dont wana get too excited because i dont have sufficient data but so far it seems very accurate. I wanted to know when will it be safe to say it works.
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #8
  • Jan 18, 2016 8:29pm Jan 18, 2016 8:29pm
Quoting Chrisian
1. How Long do you Forward test a strategy before you can label it as profitable? When you run it live and it is still profitable. 2. When forward testing a strategy what do you look for? Steady profit and no dramatic changes to the equity graph 3. My understanding is that each strategy has its own way of backtesting, because strategies differ,scalp, trend,news,counter trend etc. and because markets are also different trending, range bound etc. so how do you make sure that you perform sufficient tests that are not deceiving and are as accurate as...
Tick data quality changes everything.[/quote]


Hi, its a manual 4H system
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #9
  • Jan 18, 2016 8:30pm Jan 18, 2016 8:30pm
Quoting Bicarus
{quote} at least 300-500 valid profitable setups

why 300-500, its a 4H system trading on all pairs
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #10
  • Jan 18, 2016 9:16pm Jan 18, 2016 9:16pm
Quoting Soros
{quote} why 300-500, its a 4H system trading on all pairs
tts the standard sample size used by industry for statistical analysis
  • #11
  • Jan 18, 2016 9:29pm Jan 18, 2016 9:29pm
Quoting Soros
{quote} Wow, that was a very informative and beautiful post, basicaly its a 4H system, i have been farward testing it on all pairs and so far it has 15 trades and 1 loss with a descent RR, i dont wana get too excited because i dont have sufficient data but so far it seems very accurate. I wanted to know when will it be safe to say it works.

On H4 I just try to capture a couple of cycles. I prefer to see what happens in a sharp trend and a range period if possible, before I commit to full risk. Those are the 2 basic features in a market, so I can usually tell from seeing them both how viable my concept is. Also, if using many pairs, I'm mindful of correlative issues.
There's no fixed number of trades in my head, because at some point it becomes irrelevant. Especially 300-500 on H4.

Think of it this way, each pair is individual. You may find later that some pairs don't work so well. You can't assume what works on EU H4 will perform on GU.

I'm not sure of your system, so as an example I'll just use an arbitrary realistic number of setups for a H4 system, say 10 trades per month on average, per pair.
If you had to wait for 300 valid and profitable setups on a single pair, you would need to wait 30 months. 2 and a half years forward testing.
Even at 1 trade per day, per pair, so 20 per month, waiting for 300 setups (not even profitable ones), you would still need to wait 15 months. What's the point?

As i said earlier, I believe a multi year test sample doesn't make a method any more sure.
Every year we have a new scenario; exuberance or crisis.
2015 is was the SNB fallout and "grexit" worries.
2016 we will have the oil issue, Fed rate rises and the fear of "Brexit."
Each time new data/panic evolves, the markets react differently. I don't think knowing what happened to your method 3 years ago will save you from a disaster now, and nor should it deter you from using it.

14 wins out of 15 trades is excellent, but you are wise not to get too excited.
Start off low risk, see how it goes in a few conditions and increase the risk as you see more stability in the numbers.
Just my opinion
Wealth comes from what you keep, not what you earn
  • #12
  • Jan 20, 2016 1:58am Jan 20, 2016 1:58am
Quoting trader34
{quote} On H4 I just try to capture a couple of cycles. I prefer to see what happens in a sharp trend and a range period if possible, before I commit to full risk. Those are the 2 basic features in a market, so I can usually tell from seeing them both how viable my concept is. Also, if using many pairs, I'm mindful of correlative issues. There's no fixed number of trades in my head, because at some point it becomes irrelevant. Especially 300-500 on H4. Think of it this way, each pair is individual. You may find later that some pairs don't work so...

now its 24 closed trades and only lost 3.
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #13
  • Jan 20, 2016 2:00am Jan 20, 2016 2:00am
ill do 300 trades and if i get a 80%win rate and above with 100 trades i think thats my cue to go
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #14
  • Jan 20, 2016 2:10am Jan 20, 2016 2:10am
But...for it to be a valid statistical analysis, you would have to have 300 trades per pair.
I'm assuming the 24 wins, 3 losses was on all pairs?

In any case I don't think you need 300 per pair.
Look at the conditions, and see what you have encountered.
Most pairs are moving pretty well right now.
You may want to see how it goes with lower ADR's and ranging conditions
Wealth comes from what you keep, not what you earn
  • #15
  • Jan 20, 2016 2:12am Jan 20, 2016 2:12am
ill do 300 trades and if i get a 80%win rate and above with 100 trades i think thats my cue to go
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #16
  • Jan 20, 2016 2:19am Jan 20, 2016 2:19am
Quoting trader34
But...for it to be a valid statistical analysis, you would have to have 300 trades per pair. I'm assuming the 24 wins, 3 losses was on all pairs? In any case I don't think you need 300 per pair. Look at the conditions, and see what you have encountered. Most pairs are moving pretty well right now. You may want to see how it goes with lower ADR's and ranging conditions

yeah youre right, right now its easy money if you know what you are doing, youre also right about the 300 per pair, which will take me months to do, im not prepared to do that, yes 24 wins and 3 losses for all pairs.... this is frastrating me because i dont really wana get my hopes high to early, but if this works it can do some wonders alongside my arsenal of other strategies im using.. i remember back in the days when everybody was following eitan, well, i had like a three week good run with eitans system and then it just burnt me for a week and then i stoped using it, well this system im testing right now seems to be razor sharp acurate, i wish i was smart enough to figure out the answer as to the best way to test this strategy, well the most acurate way
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #17
  • Jan 20, 2016 2:22am Jan 20, 2016 2:22am
Quoting trader34
But...for it to be a valid statistical analysis, you would have to have 300 trades per pair. I'm assuming the 24 wins, 3 losses was on all pairs? In any case I don't think you need 300 per pair. Look at the conditions, and see what you have encountered. Most pairs are moving pretty well right now. You may want to see how it goes with lower ADR's and ranging conditions

started testing this strategy last week wednsday and im up almost 20% risking 3% per trade..
I am what Many Dream to be but only a few can achieve, im a part of the 1%
  • #18
  • Jan 20, 2016 2:33am Jan 20, 2016 2:33am
Eitan was a total fraud.
He released "ideas" that were just ridiculous, claiming +90% win rates, got a bunch of people to participate in his threads to give the appearance that they were optimizing things, then fooled too many poor gullible FF members into giving him money for nothing. he made tens of thousands from people he conned.

He blew every account! http://www.myfxbook.com/members/Zarfati
Sadly, his disciples blew theirs too.
But he took more than he lost from the poor customers. Then after the 1st blow up, he told his clients there was a new method, but of course they would have to pay for it....for all his "hard work." Of course, that blew up again! His membership should have been revoked a long time before it was, as his intentions were clear.
When he was banned, thankfully many of his minions went with him.
His "systems" were worthless. Yet, the minions on his threads promised to reveal the wonderful BT results of his methods. That will be a long wait!

Anyway, enough about scammers!
A hypothetical scenario for you:
You test 300+ signals, and all goes well. Stats hold, and then you decide to go in full risk.

A month goes by, all good...then for whatever reason it breaks down. All gains lost.

For this reason, i don't spend too long on forward testing.
All that time lost is money lost if the system stacks up to logic. At some point, you just need to jump in.
If your other methods are not strongly correlated, then hopefully any issues arising can be soaked up by the others.
Wealth comes from what you keep, not what you earn
  • #19
  • Jan 20, 2016 2:33am Jan 20, 2016 2:33am
I don't know the method...but 3% per trade is a little steep for me
Wealth comes from what you keep, not what you earn
  • #20
  • Jan 20, 2016 2:34am Jan 20, 2016 2:34am
Especially if it's possible to have multiple correlating positions open at once.
Wealth comes from what you keep, not what you earn
0 traders viewing now