Forward Testing Discussion Thread | Page 8 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

MT forward testing and going live 5 replies

forward testing flippy 42 replies

Question on forward testing 5 replies

NEW " SET & FORGET" Forward Testing for VP Day Trading System 16 replies

Forward Testing Results Only 33 replies

Attachments: Forward Testing Discussion Thread
Exit Attachments

Forward Testing Discussion Thread

  • #141
  • Feb 17, 2006 4:01pm Feb 17, 2006 4:01pm
FT.............your last two posts are very enlightening, thank you!

At this point are we to provide EOD data to you for this comparison by VP?

And yes, the forward test continues. I am doing Strats #3 & #5. robrob has been emailing me each night with his daily results using Strat #2 and I believe Rocket and drjoe4x are on schedule as well. As soon as I receive the weeks results from everyone, I will post the combined results from week #3.

(As a side note, my demo account has grown very nicely using Strat #5 and CSI data along with my VP!)



Quoting fijitrader
I advised Len that the average differences in data discrepancies we are experiencing are not the problem but the anomolies over 10 pips are the real issue. I mentioned that such discrepancies do not occur in the real world with such frequency and magnitude or the arbitrage opportunities in forex would be huge -- but in fact they are not.

His response was to ask if we can provide EOD data from the companies we are getting our trading data from as a comparison. He also indicated that he cannot simply go to CRB and say that Genesis data is different because CRB would just say "we get our data from another source and that is why our figures are different".

At this point one thing we can do is check the EOD data from our own brokers or chart providers and determine which of the data sources (crb, CSI, GEN) are most closely in line with our own trading data (dealing station).

Regardless after further discussion I was pleased that Len agreed to compile actual data (as opposed to the predicted data) and look further into this issue on our behalf and I'll keep you posted of new developments.

One thing that did come out in our discussion that we will most likely find very useful is that there is a one to one correlation between input data and ph/pl output. Therefore I'd say this confirms one contributors previous observation in this forum -- that when there is a gap in market data over the weekend it is advisable to add or subtract the gap to the predictions for the day following the gap. For example if the predicted high for the euro is 1.2100 and the gap over the weekend was +80 pips then the predicted high for the euro should be 1.2180 and the predicted low should also have that amount added.

I have not seen any new posts here lately. I hope everyone is still continuing with their forward testing. This has been an enlightening experience so far.

FT
  • #142
  • Feb 17, 2006 4:11pm Feb 17, 2006 4:11pm
Quote
QUOTE=BeachBum]FT.............your last two posts are very enlightening, thank you!
My pleasure. I'm encouraged by VP's responsivness in assisting us. Unfortunately the data providers are unwilling to reveal where their data comes from or do much at all to resolve the issue. This limits what VP can do to help us but at least they have some leverage with these data providers that we do not.

Quote
At this point are we to provide EOD data to you for this comparison by VP? If you have a source of EOD data from either a popular chart company like MT or a dealer like FXCM, OANDA, Forex.com or whoever then yes that would be a nice addition to our arsenal. However we'd also need to know the time of day their EOD data is compiled.

Quote
And yes, the forward test continues. I am doing Strats #3 & #5. robrob has been emailing me each night with his daily results using Strat #2 and I believe Rocket and drjoe4x are on schedule as well. As soon as I receive the weeks results from everyone, I will post the combined results from week #3.
Excellent. Also an additional thanks to those trading strategies that are less than steller for staying the course in spite of discouraging results.

Quote
(As a side note, my demo account has grown very nicely using Strat #5 and CSI data along with my VP!)
Thanks. That is very inspiring. Is your live account still doing better?

FT
  • #143
  • Feb 17, 2006 8:22pm Feb 17, 2006 8:22pm
Data problems ....

I have contacted CRB regarding data gaps and the response is below, with the latest email response first .. so to follow the conversation start from the bottom up;

though to summarise, CRB data has no gaps, any gaps appear to be downloading data from CRB to Vantage Point, thus it may be that Vantage Point did not register all of the data.

Following the instruction from CRB, it was simple enough to download all the files with data to back fill any gaps. I have checked and it appears that all the gaps are now filled. For CAN/USD, I will confirm the other pairs..

I had noticed a few missing days for EUR/US in january 06 when I had downloaded for the day. I did not understand why the missing data at the time, I filed it away as a ? mark. Now I know that the data was downloaded... the VP software didnt register it.!!

Having gone through the exersise suggested by CRB I see the missing data is now complete. As to the last email from CRB support suggesting I had a bad CD, I dont think that was the case, the evidense I have seen, leads me to believe that VP did not pick up the data and thus left gaps. The reason for my conclusion is that;
1. I have seen gaps in data for EUR/US in January directly after dowloading, VP left those dates blank.
2. I confirmed data gaps in my data files useing CRB software and have now backfilled those gaps directed by CRB support.


........ email trail below .. I have deleted header email adresses for privacy.
Also my subscription to CRB started in January 2006.

Also I want to say that CRB Support responded immediaely and and as you can see from the emails, solved my data problem.

VP still has a problem .. not downloading data ... but if it is checked daily as it should be when downloading .. it will be obvious that a gap exists and can be backfilled from the data provider.

............................................

Nigel, it is possible that you just got a bad CD,
but I would have thought you would have noticed
this back in December when you started. It also
doesn't make any sense that you would be missing SO MUCH data.

You can either try to download the individual
files using Powerlink going back to the first
date you are missing, or you can ask for a new CD
and I can send one out to you, but you will need
to uninstall the program and delete the folders containing the corrupt files.

To download the files using Powerlink:

1. Start Powerlink and click "next". From the
menu on the left, click on "Futures [Major Market
Futures Prices]". Then a menu will appear on the right side.

2. From the menu on the right side, click on the
drop down "download" list, and select "Download
Data Since Selected Date". This will bring up a calendar.

3. From the calendar select the month of the
first day of data you are missing
(September). Then select the year of the first
day of data you are missing (2005). Then select
the date by clicking on the number of the trading
day BEFORE the first day of data you are missing. Then click "next".

4. Click "next" again and it will begin
downloading the files from the internet (as
opposed to the CD). Once this is done, click on
"done" and it will launch PowerDist. This will
then distribute the data to the data files that
are missing them. Just click "next" all the way through PowerDist.

If this doesn't work, or you dont' have a
connection that will allow you to download that
much data, then I will have to send you another CD.

Let me know.

Mark



At 08:30 PM 2/15/2006, you wrote:

>ok after following your instructions here is the missing data for the
>USD-CAN pair
>
>jan 16 06
>jan 02 06
>nov 24 05
>nov 25 05
>nov 07 95
>nov 08 95
>nov 09 95
>sept 05 95
>
>as i look back through the data there are lots more empty green slots
>
>what do you suggest.
>

>
>

>Sent: Wednesday, February 15, 2006 6:10 PM
>Subject: Re: CRB - Bug
>
>
>Nigel, November 5th was a Saturday and the 6th
>was a Sunday so there won't be any data on those
>days. There should have been no problem with any active trading days.
>
>Firstly, lets make sure that it is actually the
>data you are missing instead of some problem with
>Vantage Point using the file. Click on "start"
>then "program files", then "CRB Powersystem" and
>then "CRB PowerPro". From the left hand menu you
>will see a list of symbols. The CD/USD symbol
>is CD----Y. Double click on that file. Follow
>the dates and scroll back to 11/07/2005 and see
>if you have data (and for that matter if there is
>data in the next five lines). If so, then we
>will have to try to figure out why your Vantage
>Point program isn't reading the files
>properly. If you DON'T have data there, then it
>is an easy enough fix. Put the CD in the CD ROM
>drive, then start PowerDist. Select "startup CD
>or Infotech CD" from the drop down and click
>"next". Then just say "yes" or "yes to all" for
>any other questions (basically you are just
>redownloading the data from the CD). Then you are done.
>
>Mark
>
>At 04:40 PM 2/15/2006, you wrote:
> >Mark
> >You say the data should be on my Vantage Point CD ... do you mean from the
> >CRB Vantage Point CD or my original Vantage software CD.
> >The CRB software was installed.
> >
> >please noe: my typo below should say Nov 05 to Nov 09 ... forex CAN/USD
> >
> >I am not trading futures so dont think your refernce is applicable.
> >
> >pls advise
> >
> >thks
> >Nigel
> >
> >

> >Sent: Wednesday, February 15, 2006 1:12 PM
> >Subject: Re: CRB - Bug
> >
> >
> >Nigel, the data is on the historical data CD that
> >should have been included with your Vantage Point
> >CD. The symbol for the nearby futures file is CDNFVOI.pro
> >
> >The data for May 9th, 2005 is (using
> >symbol/date/open/high/low/close/vol/openint):
> >
> >CDNFVOI,0.8057, 0.8090, 0.8044, 0.8079, 16739, 88695
> >
> >There is no data missing from the CD, so you
> >should not see any gaps in your files (unless you
> >had problems with the installation of the CD and data).
> >
> >Mark
> >
> >At 09:57 AM 2/15/2006, you wrote:

> > >
> > >Customer Message:
> > >-------------------------------
> > >Hello, I am subscribed to CRB, Vanatge Point
> > >Service, I have discovered several missing dates from the pair USD/CAN
> > >Non 05/09 2005 inclusive.
> > >
> > >Could you explain why this data is missing and why it hasnt been
> > >replaced?
> > >
> > >Are there other pairs with data gaps?
> > >

> >
  • #144
  • Feb 17, 2006 11:43pm Feb 17, 2006 11:43pm
Thanks for your work Rocket. Now that your missing data has been replaced does VP now output history files that are complete or deficient?

Thanks,

FT
  • #145
  • Feb 18, 2006 12:27am Feb 18, 2006 12:27am
Quoting fijitrader
Thanks for your work Rocket. Now that your missing data has been replaced does VP now output history files that are complete or deficient?

Thanks,

FT
Here is the revised CAN-USD file, could you check it against the files you have.

thks
Attached File(s)
File Type: zip CAN-USD revised.zip   2 KB | 336 downloads
  • #146
  • Feb 18, 2006 5:53am Feb 18, 2006 5:53am
Yes, because I use Strat #5 trades to the median price only on my live account.

Quoting fijitrader
Thanks. That is very inspiring. Is your live account still doing better?

FT
  • #147
  • Feb 20, 2006 5:22pm Feb 20, 2006 5:22pm
CSI is not providing data because today was a US holiday.

Are the other providers doing the same, or am I the only one without data to use with VP for tomorrows trades???

If anyone has the needed data from another provider, please let me know.......
  • #148
  • Feb 20, 2006 6:22pm Feb 20, 2006 6:22pm
Quoting BeachBum
CSI is not providing data because today was a US holiday.

Are the other providers doing the same, or am I the only one without data to use with VP for tomorrows trades???

If anyone has the needed data from another provider, please let me know.......
Here is CRB data for Presidents Day.
Attached File(s)
File Type: zip VP numbers for Pres holiday.zip   7 KB | 358 downloads
  • #149
  • Feb 20, 2006 6:34pm Feb 20, 2006 6:34pm
thanks...............



Quoting Rocket
Here is CRB data for Presidents Day.
  • #150
  • Feb 20, 2006 6:59pm Feb 20, 2006 6:59pm
Quoting BeachBum
CSI is not providing data because today was a US holiday.

Are the other providers doing the same, or am I the only one without data to use with VP for tomorrows trades???

If anyone has the needed data from another provider, please let me know.......
I just downloaded my CSI data for today. It looks like I got an update for forex but not the indicies.


Hope that helps!
  • #151
  • Feb 20, 2006 8:08pm Feb 20, 2006 8:08pm
mrbintx.........

I called CSI at 6:30 pm est and was told by the tech department that they did not send new data for today because the US markets were closed. The data I received from them was old and would not update my VP. I used the p h/l from Rocket.


Quoting mrbintx
I just downloaded my CSI data for today. It looks like I got an update for forex but not the indicies.


Hope that helps!
  • #152
  • Feb 21, 2006 9:54am Feb 21, 2006 9:54am
Yesterday I was unable to receive current data from CSI to update my VP (they said it was because it was a US holiday). However, Rocket received data from CRB (which he kindly shared with me!). Hmmmmmmmmm!!!!!!!!!

This and some of the info that FT has posted here moved me to take a little closer look at our limited forward test results for the past two and half weeks.

Rocket and drjoe4x have the same entry points for their trades using Strats #1 & #4, therefore they should each have the same number of trades per day (and they both trade two lots). But they don't!

drjoe (who uses Genesis) has more trades than Rocket (who uses CRB). Why is this? Could it be because of a difference in the data each one of them receives? I would like to know this!

robrob and I have the same entry points with Strats #2 & #5 (but I trade 2 lots). I don't know who his data provider is, but I have exactly twice the number of trades he has (as it should be). I should mention that sometimes robrob has two trades on a pair on the same day; this is a function of his inability to close out the second trade after the first is triggered and I filter this out of the test results.

So here we are back at a point where it appears that who the data provider is directly impacts trading with VP (at least when using a 'set & forget' system as we are in our forward test).

Something to think about.....................
  • #153
  • Feb 21, 2006 10:06am Feb 21, 2006 10:06am
Quoting BeachBum
Rocket and drjoe4x have the same entry points for their trades using Strats #1 & #4, therefore they should each have the same number of trades per day (and they both trade two lots). But they don't!

drjoe (who uses Genesis) has more trades than Rocket (who uses CRB). Why is this? Could it be because of a difference in the data each one of them receives? I would like to know this!

Something to think about.....................
I dont trade 2 lots ... I only trade 2 lots when the diff between the target proce and the median price is >7. If less than 7 pips I target the median.

I have pointed out in previouse posts that I often get up to 4 pairs that I just trade to the median.
  • #154
  • Feb 21, 2006 11:12am Feb 21, 2006 11:12am
Quoting Rocket
I dont trade 2 lots ... I only trade 2 lots when the diff between the target proce and the median price is >7. If less than 7 pips I target the median.

I have pointed out in previouse posts that I often get up to 4 pairs that I just trade to the median.
Evidently, I am confused about your strategy. I went back and read my written notes, previous posts on this thread and the workbooks that you built for our record keeping which also describe the strategy. It was my understanding that you would trade two lots: one with the median price as t/p and the other with a t/p of 50 pips regardless of price. This difference may be neglible, I just needed to clarify my understanding as it relates to the results of this test. Because I am trying to establish what system works best, when and with what pairs, this clarification is important to me. Thanks!
  • #155
  • Feb 21, 2006 2:45pm Feb 21, 2006 2:45pm
Quote
I dont trade 2 lots ... I only trade 2 lots when the diff between the target proce and the median price is >7. If less than 7 pips I target the median.

I have pointed out in previouse posts that I often get up to 4 pairs that I just trade to the median. [img]images/buttons/quote.gif[/img]


Quote
Quote:
<table border="0" cellpadding="6" cellspacing="0" width="100%"> <tbody><tr> <td style="border: 1px inset ;" class="alt2"> my target profit is 50 pips, the medium would have been 99/2 on that day so also 50 pips ... it didnt make either.

I have noticed that with a target of 50 pips the medium is usually close to that as well. </td> </tr> </tbody></table>
Good point. The daily range is frequently around 100 to 120 pips with many of the currencies. I'd probably make a minor modification -- if the median is within around 5 to 7 pips of 50 -- just use whichever target is smaller for both sides of the trade. There is no point in risking a 25 pip loss on the second 1/2 of the position just for a tiny handful of pips. I never like to make changes of any significance during a forward test but this change will make the test more valid and not less because it is the kind of change that does not violate the intent of the original rules. What do you think rocket? bb?
I've enclosed a previous post from way back because I don't think it was read. Did anyone catch the significance that when you put on a position with 2 halves and the first one is closed at the median but the second one is left on to catch another 10 pips or so your risk reward becomes extremely skewed. Essentially your stop is still 25 pips below break even and you are 50 pips into profit. That means you have a trade with a risk of 75 pips and your remaining target is only 10 pips away or even 20 pips away. The risk reward on that sort of trade is badly skewed against you. That is why I thought we'd made a revision to the 50 pip/median pair -- because the odds are stacked against you on the second 1/2 of that trade.

FT
  • #156
  • Feb 21, 2006 3:37pm Feb 21, 2006 3:37pm
Quoting fijitrader

I've enclosed a previous post from way back because I don't think it was read. Did anyone catch the significance that when you put on a position with 2 halves and the first one is closed at the median but the second one is left on to catch another 10 pips or so your risk reward becomes extremely skewed. Essentially your stop is still 25 pips below break even and you are 50 pips into profit. That means you have a trade with a risk of 75 pips and your remaining target is only 10 pips away or even 20 pips away. The risk reward on that sort of trade is badly skewed against you. That is why I thought we'd made a revision to the 50 pip/median pair -- because the odds are stacked against you on the second 1/2 of that trade.

FT
FT............thanks for jogging my memory. This post was read but not absorbed.

The risk is evident.
In a previous post of mine about Strat #5, I observed that profit is made consistently on trade #1 which closes at the median price. Trade #2 which targets 15 pips inside the opposite predicted price is not a money maker, it produces zero gain. When this forward test is complete, I will attempt to determine the optimum t/p point (I believe it lies somewhere between the median [50% price level] and the second t/p [which averages around the 80% level of the atr]). This price level is important in a 'set & forget' trading system.
  • #157
  • Feb 23, 2006 1:09am Feb 23, 2006 1:09am
I have posted results for last week strategy 4...again, sorry for the delay and I apologize for my recent lack of communication within the forum...just busy.
Keep up the good work BB and others.

drjoe
  • #158
  • Feb 26, 2006 8:57am Feb 26, 2006 8:57am
Last week was a tough one for us in the VP forward test, some of the Strats really got hammered. However...........take a look at Strat #4 for last week. While some of us were loosing triple digit pips, strat 4 only had a total loss of 7 pips!!!!

Here are the totals so far:

Strat # 1 -351
Strat #2 403
Strat #3 missing one week, poor performance other weeks
Strat #4 573
Strat #5 133

Strat #4 stands out at this point in time as a real winner, with #2 looking decent in second place after 18 days of trading. We may find that it is possible to establish a productive "set & forget" system using VP data.

Recently deecos posted some information about using the neural index as an indicator for trading, and although he has since written that his findings were flawed, it may be that this number can be used to indicate strength in one direction and therefore trades could be planned in that direction toward the P h/l. This would be in agreement with something FT wrote a long time ago about trades in the trend direction being more successful and productive. I have communicated with deecos and he tells me he is doing a new test with results to be available in the near future.

In the meantime.............thanks to everyone working on this forward test. Keep up the good work.................one more week to test and then we will ask FT to help us analyze our work and plan for the future.
  • #159
  • Edited 5:42pm Feb 26, 2006 4:08pm | Edited 5:42pm
Quoting BeachBum
Strat # 1 -351
Strat #2 403
Strat #3 missing one week, poor performance other weeks
Strat #4 573
Strat #5 133

Strat #4 stands out at this point in time as a real winner, with #2 looking decent in second place after 18 days of trading. We may find that it is possible to establish a productive "set & forget" system using VP data.
I agree that our findings indicate we may find it is possible to develop such a system. The markets were choppy and not particularly kind to trending systems during the last four weeks of trading but with the simplest of set and forget systems we've seen a profit.

Is there any way to recover the Strat3 data or was it permanently lost?

Some preliminary thoughts:

1. Would someone continue trading strategy four after next week. I've got a feeling that the last four weeks were nothing special and that strategy four may continue to produce consistent positive results. The only way to gain history to substantiate or refute this supposition is to continue trading it without missing trading days. Any volunteers?

2. We may also want to consider doing the same with strategy 2 since it may have just had a bad month or it may have just been it's best month. Again we won't know for sure without continuing. Any volunteers?

3. Once the trading week is finished we may want to take some time for evaluation before continuing to trade again but, if we work on our ideas this week and are quick over the weekend, we may be able to have our new strategies in place to continue our forward testing without missing a trading day. However if that puts too much pressure on people with busy schedules who want to participate then we can take a week or two between testing to discuss and design our new strategies. Except of course for the forward testing of strat 4 and possibly 2 as well.

FT
  • #160
  • Feb 26, 2006 5:26pm Feb 26, 2006 5:26pm
Quoting fijitrader
I agree that our findings indicate we may find it is possible to develop such a system. The markets were choppy and not particularly kind to trending systems during the last four weeks of trading but with the simplest of set and forget systems we've seen a profit.

Is there any way to recover the Strat3 data or was it permanently lost?

Some preliminary thoughts:

1. Would someone continue trading strategy four after next week. I've got a feeling that the last four weeks were nothing special and that strategy four may continue to produce consistent positive results. The only way to gain history to substantiate or refute this supposition is to continue trading it without missing trading days. Any volunteers?

2. We may also want to consider doing the same with strategy 2 since it may have just had a bad month or it may have just been it's best month. Again we won't know for sure without continuing. Any volunteers?

3. Once the trading week is finished we may want to take some time for evaluation before continuing to trade again but, if we work on our ideas this week and are quick over the weekend, we can pretty much have our new strategies in place to continue our forward testing without missing a trading day. However if that puts too much pressure on people with busy schedules who want to participate then we'll take a week between testing except for the forward testing of strat 4 and possibly 2 as well.

FT
Strat #3 could be recovered for the missing weeks. However, after trading this one myself for two weeks, I don't think it is worth the effort, this has not been a productive strat.

drjoe4x has been testing #4 for us and I know he has a very busy schedule. If he would prefer to have someone else test this one, I would be glad to continue with it.

I too would like to see us continue without missing any trading days.
0 traders viewing now