Forward Testing Discussion Thread | Page 9 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

MT forward testing and going live 5 replies

forward testing flippy 42 replies

Question on forward testing 5 replies

NEW " SET & FORGET" Forward Testing for VP Day Trading System 16 replies

Forward Testing Results Only 33 replies

Attachments: Forward Testing Discussion Thread
Exit Attachments

Forward Testing Discussion Thread

  • #161
  • Feb 26, 2006 5:54pm Feb 26, 2006 5:54pm
Quote
QUOTE=BeachBum]Strat #3 could be recovered for the missing weeks. However, after trading this one myself for two weeks, I don't think it is worth the effort, this has not been a productive strat.
If recovering it effectively means backtesting it then don't. This is a forward testing environment and I don't want any exceptions. If the data was lost because the tester was unable to provide it (was that our guy who went missing?) then it is lost. My only reason for being interested was to examine why it was a lousy system but I'm not sure we need that lost week to determine that.

Quote
drjoe4x has been testing #4 for us and I know he has a very busy schedule. If he would prefer to have someone else test this one, I would be glad to continue with it.
Let us know DRJOE. We are grateful for your testing work and don't want to take it away if you are enjoying your participation but feel free to pass it on to BB if you are too busy. It will not mean you'll be excluded from future testing because you did complete all your work. Thanks!

Quote
I too would like to see us continue without missing any trading days.
Cool.

FT
  • #162
  • Feb 26, 2006 9:46pm Feb 26, 2006 9:46pm
Quoting fijitrader
Let us know DRJOE. We are grateful for your testing work and don't want to take it away if you are enjoying your participation but feel free to pass it on to BB if you are too busy. It will not mean you'll be excluded from future testing because you did complete all your work. Thanks!
I will be glad to continue testing Strat 4 through this week. I assume after this week we will be focusing on improving the profitable stategies...looking at which stategy worked the best for each pair, what the NI is indicating if anything, and any other relevant variables...does the day of the week matter for a pair or a strategy or a pair and a strategy? Looking forward to continuing and I will continue to try to get quicker at posting testing results.

By the way...I didn't see an attached file in the Testing results section earlier for the 4th week results, just wondering.

Thx to all of you

drjoe
  • #163
  • Feb 26, 2006 9:59pm Feb 26, 2006 9:59pm
Here are the totals so far:

Strat # 1 -351
Strat #2 403
Strat #3 missing one week, poor performance other weeks
Strat #4 573
Strat #5 133
.[/quote]

I have some concerns about our results;
firstly I totaled the pips directly from the strat #1 spreadsheet and arrived at -279 could you double check that pls, even that is a horrible result compared with the other strategys.

I compared start #1 with strat #4 to try and figure out why they are so different ... the strategy is very similar just diff targets so they should be a lot closer.

reviewing just EUR/US for the week of feb 24 I found that there are differences in the data that would have made a big difference.
The strat #1 data consistently was between 4-8 pips difference than strat #4. And comparing the price entry that difference was enough to change a potentialy winning trade into a losing trade or a trade with less profit.
That would mean that for a short trade if I had used an entry 5-10 pips higher, the results would have been much more positive for strat #1.


There where also a couple of missed trades with both strats which would have added profit to each. These appear to be mistakes entering the trades possibly, since in both cases the entry price was within a couple of pips and the opposite strategy did catch the trade.

Also a huge difference in pH/pL on feb 20 in the AUD/USD pair
my VP data had Feb 20 pH .7446 with Pl .7384
strat 4 data pH .7432 with Pl .7363

drjoe could you double check that date, may just be an entry error
It made strat #4 32 pips while strat #1 lost 50 pips.

This difference in data is a definate concern, and may require different data users to shift a strategy ... any thoughts on this?

Rocket
  • #164
  • Feb 26, 2006 10:14pm Feb 26, 2006 10:14pm
Quoting drjoe4x

By the way...I didn't see an attached file in the Testing results section earlier for the 4th week results, just wondering.

Thx to all of you

drjoe
Results are posted for the fourth week............
  • #165
  • Feb 26, 2006 10:16pm Feb 26, 2006 10:16pm
Quoting fijitrader
If recovering it effectively means backtesting it then don't. This is a forward testing environment and I don't want any exceptions. If the data was lost because the tester was unable to provide it (was that our guy who went missing?) then it is lost. My only reason for being interested was to examine why it was a lousy system but I'm not sure we need that lost week to determine that.


FT
It would have to be a backtest for the two missing weeks (a result of spudbox gone missing). I will not do the test.
  • #166
  • Feb 26, 2006 10:23pm Feb 26, 2006 10:23pm
Rocket,

Just checked my VP data and my Feb 20 PH/PL was correct at .7432/.7363 as predicted for Feb 21. Entered Long at .7373 and had a low of .7369 before closing the trade at the close of the day at .7389. I think this is definitely warrants a little further investigation, as possibly strat 4 more productive based on Genesis data being more similar to MT4 IBFX data or something of the sorts.

BB/Fiji?

drjoe
  • #167
  • Feb 26, 2006 10:28pm Feb 26, 2006 10:28pm
Quoting Rocket
Here are the totals so far:

Strat # 1 -351
Strat #2 403
Strat #3 missing one week, poor performance other weeks
Strat #4 573
Strat #5 133
.
I have some concerns about our results;
firstly I totaled the pips directly from the strat #1 spreadsheet and arrived at -279 could you double check that pls, even that is a horrible result compared with the other strategys.

Rocket[/quote]

I went back and looked at all four excel sheets for strat #1 and my total remains the same...................-351.
I do not know where you got -279, but I believe this amount is in correct.
  • #168
  • Feb 26, 2006 10:43pm Feb 26, 2006 10:43pm
ok -351 is for the 3 weeks of testing, I thought it was week of feb 24.

I definately feel that the diff in data influences the results.
for future testing it may be better to run my data as a comparison rather than a stand alone.
  • #169
  • Feb 26, 2006 11:05pm Feb 26, 2006 11:05pm
Rocket & drjoe4x..........

are the two of you talking about the same trading day?

please take a look at this screenshot and see how it compares.......
<?xml:namespace prefix = v ns = "urn:schemas-microsoft-com:vml" /><v:shapetype id=_x0000_t75 stroked="f" filled="f" path="m@4@5l@4@11@9@11@9@5xe" oreferrelative="t" o:spt="75" coordsize="21600,21600"></v:shapetype>
<v:shapetype stroked="f" filled="f" path="m@4@5l@4@11@9@11@9@5xe" oreferrelative="t" o:spt="75" coordsize="21600,21600"></v:shapetype>
Attached Image
  • #170
  • Feb 26, 2006 11:08pm Feb 26, 2006 11:08pm
my data compares with feb 20

pH .7446 Pl .7384
  • #171
  • Feb 26, 2006 11:20pm Feb 26, 2006 11:20pm
drjoe4x............


I would like to see the same screenshot I just posted from your VP data. I went back and looked at your spreadsheet for the day in question and there is a substantial difference here. Can the difference be a result of your using Genesis data????????????? I'd like to get to the bottom of this. Thanks!
  • #172
  • Feb 26, 2006 11:39pm Feb 26, 2006 11:39pm
I've trying to post a screenshot but can't get it small enough to upload...if you would be so kind as to give a few instructions on getting a screenshot...be specific and I will get this there, been working on it for 15 minutes.

thx,
  • #173
  • Feb 26, 2006 11:46pm Feb 26, 2006 11:46pm
I'll try it in a zip file in PAINT, if you can't open it, just LMK.

thx

drjoe
Attached File(s)
File Type: zip vpaudusd2.zip   83 KB | 453 downloads
  • #174
  • Feb 27, 2006 7:43am Feb 27, 2006 7:43am
Quoting drjoe4x
I'll try it in a zip file in PAINT, if you can't open it, just LMK.

thx

drjoe
drjoe4x.........

thanks for posting this. The difference on some days concerns me: Feb 20 p/l from CSI data is .7381, p/l from Genesis data is .7363............that's an 18 pip difference! A difference that can have a serious impact on a trading system.

FT............any thoughts on this???
  • #175
  • Feb 27, 2006 7:54am Feb 27, 2006 7:54am
here are the two VP reports for comparison:
Attached Images
  • #176
  • Feb 27, 2006 12:00pm Feb 27, 2006 12:00pm
Fiji/BB,

I think it might be beneficial to check the historical VP data we have all submitted with MT4 historical data...is it possible that Genesis data more closely resembles that of MT4 IBFX? If we could get a glimpse of how closely actual end of day data compares to MT4 for each of the data providers, this may provide more clues into this phenomenom.

Just a thought...

drjoe
  • #177
  • Feb 27, 2006 7:11pm Feb 27, 2006 7:11pm
Quoting drjoe4x
Fiji/BB,

I think it might be beneficial to check the historical VP data we have all submitted with MT4 historical data...is it possible that Genesis data more closely resembles that of MT4 IBFX? If we could get a glimpse of how closely actual end of day data compares to MT4 for each of the data providers, this may provide more clues into this phenomenom.

Just a thought...

drjoe
What are you getting at about MT4 data, what difference does it matter where the data comes from .. I think matters is that we have 3 different data providers and therfore possibly 3 different outcomes to the same strategy. I think we just need to compare the diff between the 3. And possibly adjust startegy depending on data provider. As I pointed out yesterday Strat 1 useing CRB data isnt profitable, But it may be with Genesis data. I think that we need to run a comparison of the same strategy with the 3 different data providers. Perhaps useing just 2 pairs and comparing 3 strategise against each other.
  • #178
  • Feb 27, 2006 8:19pm Feb 27, 2006 8:19pm
Quoting Rocket
What are you getting at about MT4 data, what difference does it matter where the data comes from .. I think matters is that we have 3 different data providers and therfore possibly 3 different outcomes to the same strategy. I think we just need to compare the diff between the 3. And possibly adjust startegy depending on data provider. As I pointed out yesterday Strat 1 useing CRB data isnt profitable, But it may be with Genesis data. I think that we need to run a comparison of the same strategy with the 3 different data providers. Perhaps useing just 2 pairs and comparing 3 strategise against each other.
What I'm getting at is exactly what you said, just thought it might be easy to do a historical data comparison to see which data provider, if any, more closely resembles the MT4 data since this is the platform we have chosen do our trading with. It may be that Genesis data doesn't relate as closely, which would make no sense if VP Pred Data is as accurate as we hope, but it may be if the only difference b/w the more profitable strategies is the data provider in relation to MT4, then this is of significance and would either warrant a future switch to Genesis if you so choose or a change of trading platform with data more closely relating to your chosen data provider.

I just think it might be important to see how each of the data providers daily high and low relates to the MT4 daily high low. Obviously if you lost 50 pips and I gained 32 (82 PIP DIFFERENCE) on the same trade using MT4 and the only difference is our data provider for VP, is this scenario playing out over and over again, or is it a random and unpredictable event. I just think a little clarification is needed and it is probably something Fiji or BB can put together rather quickly.

To me, knowing my data provider is going to be very accurate to my actual trading platform data is of importance.

drjoe

I also understand your statement about the 3 data providers and testing the strategies with each provider to see if one strat works better with a different provider, although I believe it may be too exhausting for us to continue to trade the same unprofitable strategies, as most likely, I don't foresee there to be major differences...or maybe I'm wrong.
  • #179
  • Feb 28, 2006 9:04am Feb 28, 2006 9:04am
Because of recent comments about data used for VP predictions, I quickly put together a small spread sheet comparison of the aud/usd pair for the days that have been recently discussed in this thread. I am not drawing any conclusions from this other than the fact that there are some major differences in the values between Genesis, CSI and Interbank FX data. Could these differences account for different results from the same trading strategies? Food for thought.
Attached Image
  • #180
  • Edited 4:08pm Feb 28, 2006 2:38pm | Edited 4:08pm
Quoting BeachBum
Because of recent comments about data used for VP predictions, I quickly put together a small spread sheet comparison of the aud/usd pair for the days that have been recently discussed in this thread. I am not drawing any conclusions from this other than the fact that there are some major differences in the values between Genesis, CSI and Interbank FX data. Could these differences account for different results from the same trading strategies? Food for thought.
Indeed this makes some serious differences between system performance with different providers. Did anyone read the comparison spreadsheets and follow up posts from a couple weeks ago where I conveyed VP's suggestion to find the provider that most closely resembles the forex market makers we us? Does anyone recall my suggestion to also contact their data providers for explanations?

I see a couple of possible reasons for the differences in data:

1. different time of day when data is collected.
2. different source.
3. both.

I know that such price disparities do not exist between the retail platforms I have compared. At least not that consistently and not such large differences. Otherwise the arbitrage opportunities between providers would be much more lucrative. In the data comparison spreadsheets there are differences that extend greater than 80 pips, several in the 50 pip range and way too many in the 20 to 35 pip range. I could make a decent amount of money in arbitrage just trading the end of day differential these figures seem to show. The reality is that such differences do not exist as frequently or with the magnitude we see between data provides.

I contacted VP for assistance and the guy I spoke to told me he'd follow it up. So far I've not heard back from him.

FT
0 traders viewing now