Backtesting to write to file question | Forex Factory

Options

Permalinks
Print Thread

Similar Threads

How to write ticks to file? 3 replies

write value to a file? 2 replies

file-write problem 4 replies

Can't seem to write to a file 5 replies

TestGenerator: Write file error 1 reply

Backtesting to write to file question

  • #1
  • First Post: Edited 9:50am Jun 4, 2013 1:41am | Edited 9:50am
I have an EA that I'm working on that writes data to a .csv file (one row of data for each day of backtest). It works fine except for a big gap between 10-1-2012 to 12-26-2012 where no data is written to the file. The backtest is for almost an entire year till now. I have tried deleting the history and re-downloading the chart data with the same result. The history data looks fine. When run on a second MT4, it works better, but still has some days missing. It concerns me because I'm not sure if it is a bug in my programming or MT4 or something else. Any advice or suggestions would be appreciated.

void DoDailyProfitWrite()
{
int handle;
if( CurrDay == DayOfWeek()) return;
if(DeleteDailyFile == 0) {FileDelete("Daily" + EA_Comment + Sym + ".csv"); DeleteDailyFile = 1;}string FileDate = TimeToStr(TimeCurrent(),TIME_DATE);
if (DayOfWeek()== 1 || DayOfWeek()== 2 || DayOfWeek()== 3 || DayOfWeek()== 4 && Hour()== 23 && Minute()== 59 && Seconds() >= 29 )
//Seconds greater than 30 or 40 seem to miss some days in the spreadsheet
{
handle=FileOpen("Daily" + EA_Comment + Sym + Period() + WriteFileComment + ".csv", FILE_CSV|FILE_READ|FILE_WRITE, ',');//use comma for excel
if(handle>0)
{
FileSeek(handle, 0, SEEK_END);
//---- add data to the end of file
FileWrite(handle, DayOfWeek(), FileDate, GetProfitToday(), GetOrdersToday() );
FileClose(handle);
handle=0;
CurrDay = DayOfWeek();
}
}//end Monday thru thursday

if(DayOfWeek() == 5 && Hour() == 19 && Minute() == 59 && Seconds() >= 29 ) //Friday write
//Seconds greater than 30 or 40 seem to miss some days in the spreadsheet
{
handle=FileOpen("Daily" + EA_Comment + Sym + Period() + WriteFileComment + ".csv", FILE_CSV|FILE_READ|FILE_WRITE, ',');//use comma for excel
if(handle>0)
{
FileSeek(handle, 0, SEEK_END);
//---- add data to the end of file
FileWrite(handle, DayOfWeek(), FileDate, GetProfitToday(), GetOrdersToday() );
FileClose(handle);
handle=0;
CurrDay = DayOfWeek();
}
}//end Friday
return(0);
}//DoDailyProfitWrite
  • #2
  • Jun 4, 2013 2:44am Jun 4, 2013 2:44am
Quoting Jeysun
I have an EA that I'm working on that writes data to a .csv file (one row of data for each day of backtest). It works fine except for a big gap between 10-1-2013 to 12-27-2013 where no data is written to the file.
1st issue, you probably don't have data from the future, if you do can you send me a copy ? "big gap between 10-1-2013 to 12-27-2013 "

I suspect your issue is your History data, check that it's timezone hasn't changed part way through, check the close of market time around the date when you are having an issue.
20 pips a day isn't too much to ask . . .
  • #3
  • Jun 4, 2013 9:21am Jun 4, 2013 9:21am
Thanks for the reply.
I'll get you a copy of that data at the end of this year
Sorry, I was half asleep when I posted. It is 10-1-2012 to 12-26-2012

I do not believe it is a timezone issue. My broker(IBFX) uses GMT and I have not changed time zones.
The historical data is there and has all the last 15 minute bars starting at 23:48, it is an odd time which is normal for IBFX - since there is no trading between 21:00 and 21:03 due to high volatility or something.

I have found an error where one platform does not make the same trade as the second platform. I have to investigate this further...
  • #4
  • Jun 4, 2013 10:23am Jun 4, 2013 10:23am
Quoting Jeysun
Thanks for the reply. I'll get you a copy of that data at the end of this year Sorry, I was half asleep when I posted. It is 10-1-2012 to 12-26-2012 I do not believe it is a timezone issue. My broker(IBFX) uses GMT and I have not changed time zones. The historical data is there and has all the last 15 minute bars starting at 23:48, it is an odd time which is normal for IBFX - since there is no trading between 21:00 and 21:03 due to high volatility or something. I have found an error where one platform does not make the same trade as...
Jeysun-
I'm also on IBFX.
My question is are these Demo or Live accounts?
  • #5
  • Jun 4, 2013 2:38pm Jun 4, 2013 2:38pm
Quoting cyber1
{quote} Jeysun- I'm also on IBFX. My question is are these Demo or Live accounts?
I have tried backtesting on two demo accounts and one live account. Each one seems to skip different days, or in the case of the one demo, almost three months.
  • #6
  • Jun 4, 2013 6:33pm Jun 4, 2013 6:33pm
Quoting Jeysun
{quote} I have tried backtesting on two demo accounts and one live account. Each one seems to skip different days, or in the case of the one demo, almost three months.
Jeysun-
Which pairs?
What time frame?
Does this skipped data appear on the chart or is this just in the backtest data?

If you could tell me the pair and the timeframe I'll check my charts as I have not noticed anything in any of the time frames on any pair?
  • #7
  • Jun 4, 2013 7:28pm Jun 4, 2013 7:28pm
Quoting cyber1
{quote} Jeysun- Which pairs? What time frame? Does this skipped data appear on the chart or is this just in the backtest data? If you could tell me the pair and the timeframe I'll check my charts as I have not noticed anything in any of the time frames on any pair?
Thank you for you questions.
I have been backtesting EURJPYm at a 15 minute timeframe. All the data is in the history and on the chart. I should note that the backtesting results show trades were made during the dates that are missing from the .csv file. I just backtested EURUSDm at 15minutes out of curiosity and here is a portion of the result in the .csv file. First column is the day of the week (Monday being 1). Second column is the date. Third column is the profit for the day. Fourth column is the number of opened and closed trades for that day.

You'll notice some weekdays missing.

1 2012.09.17 -66 7
2 2012.09.18 -32 6
3 2012.09.19 62 5
4 2012.09.20 87 3
5 2012.09.21 -261 7
1 2012.09.24 -42 5
3 2012.09.26 -15 6
5 2012.09.28 0 7
1 2012.10.01 -1 4
2 2012.10.02 1 5
4 2012.10.04 79 2
5 2012.10.05 -51 5
1 2012.10.08 -40 7
2 2012.10.09 42 5
3 2012.10.10 10 4
4 2012.10.11 35 5
5 2012.10.12 23 4

I suspect the error is in my code as I am not the best coder and writing to a .csv file is new to me.
  • #8
  • Edited 7:46pm Jun 4, 2013 7:34pm | Edited 7:46pm
Quoting Jeysun
{quote} Thank you for you questions. I have been backtesting EURJPYm at a 15 minute timeframe. All the data is in the history and on the chart. I should note that the backtesting results show trades were made during the dates that are missing from the .csv file. I just backtested EURUSDm at 15minutes out of curiosity and here is a portion of the result in the .csv file. First column is the day of the week (Monday being 1). Second column is the date. Third column is the profit for the day. Fourth column is the number of opened and closed trades for...
Jeysun-
Are you you sure there were valid trade setups for the missing weekdays?

I would also question this.
Inserted Code
if(DayOfWeek() == 5 && Hour() == 19 && Minute() == 59 && Seconds() >= 29 ) //Friday write

You are really trying to force it to write to the file in that less than 30 seconds which sometimes just fails.

I would try and use a once per bar approach.


also this:
Inserted Code
void DoDailyProfitWrite(){

int handle;
if( CurrDay == DayOfWeek()) return;
With the small snippet of code I don't know what the value of CurrDay is being its at the beginning of the function it must be a global variable.
  • #9
  • Jun 4, 2013 9:45pm Jun 4, 2013 9:45pm
Quoting cyber1
{quote} Jeysun- Are you you sure there were valid trade setups for the missing weekdays? I would also question this. if(DayOfWeek() == 5 && Hour() == 19 && Minute() == 59 && Seconds() >= 29 ) //Friday write You are really trying to force it to write to the file in that less than 30 seconds which sometimes just fails. I would try and use a once per bar approach. also this: void DoDailyProfitWrite(){ int handle; if( CurrDay == DayOfWeek()) return; With the small snippet of code I don't know what the value of CurrDay is being its at the beginning of...
Cyber1-
Yes, there were valid trades.
I have tried changing the hour to 9 instead of 23 and 19. There was no change to the result in the .csv file.
Your right, it is a global variable initially set to 0
Inserted Code
  datetime     CurrDay=0;
I use this to make sure it only writes to the file once per day.

I'm thinking the problem could also lie in memory or some other technical variable in the platform itself.
My EA has the option to close trades at the end of the day so as to avoid negative gaps especially over the weekend. I suppose this is not necessary when writing historical data to a file. The once per bar approach is worth a try.
I think I will try to re-write the code to write on a new day with yesterdays results.
  • #10
  • Last Post: Jun 8, 2013 6:13pm Jun 8, 2013 6:13pm
Quoting Jeysun
I have an EA that I'm working on that writes data to a .csv file (one row of data for each day of backtest). It works fine except for a big gap between 10-1-2012 to 12-26-2012 where no data is written to the file. The backtest is for almost an entire year till now. I have tried deleting the history and re-downloading the chart data with the same result. The history data looks fine.
for history data you may want to cross check with some others: see info here: MT4 Historical Rates - Backtest
Money is important, Love is essential.
0 traders viewing now