Plot News indicator (version 3.0)

Sept 13, 2015: Please ignore the rest of this thread. Version 3 no longer works correctly, due to changes in the format of the FF calendar. I finally decided to rewrite the news downloader completely (using Autohotkey instead of my antiquated version of Clarion). The latest version of Plot News (currently 4.07) can be found here.
_______________________

July 19, 2012: It appears that this indicator no longer works correctly, as the format of the FF calendar has been changed.
Here are three other news indicators that you might like to look at: 1 2 3
_______________________

Please note: Running this indicator on a Vista or Win7 configuration may cause unpredictable results. Use at your own risk. I do not have the facilities to troubleshoot problems on either of these platforms. Please read posts #198 thru #208 here, and/or contact either Microsoft or your Windows supplier for more information.

This is the final version of the indy, and I am no longer answering questions in this thread. Reasons:
1. Increased work commitments elsewhere.
2. The indy has been running correctly for several months now. Most questions seem to be due to Vista or Win7 vagaries.
3. In many cases, I seem to be typing the same tired old answers repeatedly. If your questions aren't answered in the instructions provided, please feel welcome to read the posts in the thread, and experiment with the settings yourself.
4. Please note that the indy is FREE, and is therefore supplied on an 'as is, take-it-or-leave-it, use-at-your-own-risk' basis.
_______________________

Latest release: v 3.03, Aug 30, 2010.

INSTALLATION

1. Download and unzip the News v303.ZIP file into your MT4 folder, unzipping the files within to the relevant sub-folders, and overwriting any existing files of the same name.
2. Restart MT4.
3. Run the Calc Timeshifts.EX4 script if you wish to have the time offsets calculated automatically for you.

For further instructions, additional information, screenshots, release history, etc, please read the 'Operating instructions' document (it's in the ZIP file).
____________________________________

Release 3.10 --- May 16, 2011

Version 3.10 includes:
--- all of the functions available in v3.03 (please read the instruction manual in the News v303.ZIP file, before posting questions)
--- ability to plot both the circular symbols and rolling window in a separate sub-window
--- ability to filter events by their description
--- a script that converts news events in @[email protected] to GMT+X timestamp, auto-adjusting for daylight saving
--- a weekly reminder to run FFcal.EXE (credit to pips4life for this; however, I prefer to run it at least twice daily )
--- zip file of news events from 1 Jan 2007 thru 22 Oct 2011 (unzip into .../experts/files folder, overwriting any existing files)


Recent News v310 indy (download into ...../experts/indicators folder) --- new parameters:

MainChart:

1. TRUE = plots both the circular symbols and rolling window in the main chart window, exactly as before. Note that v3.10 must create a new sub-window regardless, due to MT4 idiosyncrasies. Drag the separator to make this empty sub-window as small as possible (or use v 3.03 instead)
2. FALSE = plots both the circular symbols and rolling window in a separate chart sub-window. Use VertSpacingPips, SymbolSize, etc to adjust symbol spacing (and/or drag MT4's between-widow separator), and ShowUpcomingXEvents, EventCorner, EventFontSize, etc to likewise adjust the rolling window of impending events.

DescriptionContains1
DescriptionContains2
DescriptionContains3
DescriptionContains4:
These parameters are 4 independent tests that allow filtering of news events by description.
All fields are case-insensitive and, if left blank, perform no filtering (i.e. are ignored).
Note that all other filters (e.g. PlotTheseCurrencies, PlotTheseImpacts, etc) must also be met, in order for an event to be plotted, displayed, exported, etc (depending on other parameter settings)

1. If the text in the news event description (e.g. "GDP") matches ANY one of the 4 filter parameters, then the event passes the test (i.e. think "OR" condition), and will be plotted, displayed, exported, etc
2. You can create multiple (up to 20) conditions in each parameter, separated by vertical bars (|), ALL of which must be met in order for the event to pass this part of the test (i.e. think "AND" condition).
3. You can precede a component condition by a tilde (~), which means that the following text must NOT be present in the event description

Example: If you set the parameters as follows:
DescriptionContains1: FARM|PAYR|~ADP
DescriptionContains2: GDP
then the only news events that will be output will be:
(1) any event that contains the text "FARM" (any mix of upper/lowercase) and also the text "PAYR", anywhere in its description, but NOT the text "ADP";
(2) any event that contains the text "GDP"


Presets---Recent News.TXT file (download into ...../experts/files folder) --- needed for compatibility with Recent News v310 indicator.


Weekly Plot_News reminder.mq4 indy (download into ...../experts/indicators folder) --- if required


Convert to GMT.mq4 script (download into ...../experts/scripts folder) --- if required

This will read all of the news events in the file ....../experts/files/@[email protected] (or whatever your InputFile entry is)
and re-timestamp them into the output file ....../experts/files/@[email protected] (or whatever your OutputFile entry is)
according to your ShiftToGMTPlus parameter entry.

The script assumes that all events in FF calendar are New York time, and that New York is GMT-4 when observing daylight saving time, and GMT-5 when it's not. All of this is correct, as far as I know.

It also assumes that the InputFile already exists in ....../experts/files. [Note: FFcal.EXE re-creates the ....../experts/files/@[email protected] file of all events in its database (the CALENDAR.* files) each time it is run].

It then re-timestamps the events to GMT+X hours, where X is the value that you enter in ShiftToGMTPlus. This value may be positive, negative (for GMT-X), decimal (e.g. 9.5 = 9 hours 30 mins), and/or greater than 24 if you want to shift by more than a whole day.

The process should take a few seconds only, and a message pop-up will appear when it's complete.
____________________________________

Release 3.11 --- November 20, 2011

Version 3.11 includes:
--- Everything in v3.10
--- Popup calendars now run Sun-->Sat (in keeping with the FF calendar) instead of Mon-->Sun
--- Calendar files now include (FF calendar-based) news events from 1 Jan 2007 thru 19 Nov 2011


INSTALLATION

Unzip the News Setup v311.ZIP file to obtain the installer program NewsSetup.EXE. Run this program and follow the on-screen instructions.

See the user manual (News Indicator v 3 - operating instructions.doc) for more information. It will be located in your MT4 folder.

___________________________

MT4 Indicators/EAs by Hanover:

Recent Strength — display line-based plot of weighted average of currency (as opposed to pair) strength
Recent Prices — display line-based plots of any combo of pairs/timeframes on your current chart, to compare their relative strength
Recent Candles — display candles of any combo of pairs/timeframes on your current chart
Recent SR — auto-plot horizontal support/resistance lines based on a wide variety of settings
Recent News — display upcoming and/or historical news announcements from FF calendar; countdown/alert upcoming announcements
Daily Lines/Boxes — auto-plot horizontal or vertical lines, boxes, symbols on user-defined times, days of the week, etc
Display Info all pairs — display spread, daily range, dollars/pip, swap rates etc for all pairs
Spaced Lines — auto-plot horizontal lines on your charts
Stealth Master EA — hide your SL and TP from unscrupulous br0kers!
Do-it-yourself alerts builder kit — code template that you can copy/paste that will add pop-up and/or e-mail alerts to most 'standard' indicators

hanover,

I'm grateful for this update as the last version is no longer working anymore. However, why is it so much more complicated then the last version? The version I have was one simple mq4 indicator that you throw on to the chart and wha-la all the news reports are there. But this one has exe files executing and the MOST annoying thing is setting the time relative to New York. I would prefer relative to GMT as this is what is in my previous version and works well..

anyway, thanks for this.. still playing around with it...

Digital Soul.............How can it be more complicated ? it has one additional step which u could have a short cut key added to execute it. Try to understand the tool b4 u make ungrateful comments !!! The core functionality is the same, Hanover has added a download script function so we do not hit FF site often.

Hanover,

Sorry I missed the file I had only just downloaded the Word doc and was reading so I could understand the changes.

The doc was str8 forward and the process shows me u only added one more step. I really believe DS did not read the document as it is clear that the script now executes the download.

I am personally interested in rec'ing this version 3.0. Not sure what ur plans are but I hope u will repost, it thks for your work.

The previous news source had become flaky. I'd received a large number of requests to return to using FF calendar as the news source, which is based on New York time, not GMT. If you want this changed, you'll need to contact the FF webmaster, as it's outside of my control.

I have released v 3.01, which fixes a couple of minor bugs, and reverts to a GMT-based offset system. In any case, changing the time settings needs to be done only twice per year, when daylight saving shifts in and out.

The EXE runs seamlessly and automatically; there's no user intervention required. Using MQL4 to read the FF calendar HTML is way too slow – it takes anything up to 10 minutes to parse a week's worth of news. Hence the EXE.

The download is now a separate process in an attempt to reduce overhead on both the user's computer, and the news provider's servers. It's inefficient to download and parse the calendar every time a user refreshes his MT4 chart, for whatever reason; especially since the download needs performing only if announcement data on the calendar has changed (maybe a couple of times per day?). So now you can switch between pairs or timeframes on a chart without unnecessary delays. In my view that's an improvement.

It's possibly fair to say that, when forced to compromise, I lean toward writing feature-rich, rather than user-friendly, software. However, the above changes were largely forced upon me, for the reasons that I described.

Please re-read the disclaimer. The indicator is offered free of charge, on a "take it or leave it" basis. If you find it "MOST annoying", you're welcome to use a different indicator. Derk Wehler's is pretty good.

Thanks for your feedback. I've reposted the indy, now that I've got v 3.01 working correctly.

David,

I am running multiple MT4s installed in diff directories, one per traded currency pair.

I would like to run the script once and then run a BAT file to copy the data to other MT4 installations, so I do no hit the FF site....Which files should I copy in my BAT file ?

ALARMS.....I did a parameter file work around bcoz I wanted all events H, M and L plotted but I only wanted an audio alarm for H and M, so I added two Recent News one for plotting events only and one for alarms only, but just the H and M importance levels. No major performance issues, works great.

Again THK YOU.

The only file that RecentNews.EX4 indy needs is ....../experts/files/news.csv

If you have two instances of RecentNews.EX4 on the same chart, there will likely be issues with the object names, i.e. if both indys are plotting symbols underneath the candles, for example, then not only would the respective symbols overplot, but their names would also be identical, probably causing unpredicatbale results. If you want to run multiple instances of the indy (for alerting etc) then it's probably safer to attach each instance to a separate chart.

Hi Hanover,

I've used this a bit more and now that I understand it it's looking good. Yes I have noticed that the previous file used to freeze and almost crash my MT4 when downloading from FF but now it doesn't do that which is excellent.

Also, the GMT offset feature is fantastic, I withdraw my previous comments and thank you for an outstanding piece of software for which I am grateful.

Glad to hear that you've got it working to your satisfaction. Once you've got the indy attached to your chart with your desired parameter settings, it's simply a case of running the DownloadNews script each time you want the news data updated, and letting it do its job (and then switching between timeframes to have the indy plot the updated data). I run it a couple of times each day.

I hope you'll find the FF calendar a much more reliable and accurate source of news data, and one that updates itself in a more timely fashion. The news announcement descriptions tend to be more concise, and the impact values (high, medium, low) more sensible. And as a further bonus its history dates back as far as the beginning of 2007 (if you want to plot bygone announcements to study their effect on price). However, every news indy is vulnerable to changes that the provider's web programmers might make. If they change the location or format of the data, then the news download software needs to be modified accordingly. Hopefully the FF calendar will continue to remain fairly stable.

I've finally succeeded in (programatically) getting the latest New York time from a web site, which means that I could now potentially automate the timeshift calculation completely (no parameters necessary). But - on second thoughts - I prefer to leave it open-ended, so that users can manually adjust the symbol plot locations, if necessary, as a last resort. Using GMT based offsets necessitates the use of 3 timezone parameters instead of 2, but I agree that it's conceptually simpler.

Well u could give the traders the option, auto set time or manually set time.

I've included a script that performs the calculation, as a one-time setup task (re-visit post #1).
I'm trying to keep away from reading websites in the Recent News indy itself, to avoid unnecessary overhead.

Hi Hanover,

thanks so much for this wonderful indicator.

however in your thread starting post i see 1 mistake.

2. Download the Download News.EX4 script into your ……/experts/scripts folder.
3. Download the Download News.EX4 script into your ……/experts/scripts folder.

in your word document it has

3. Download the Calc Timeshifts.EX4 script into your ……/experts/scripts folder.

anyway your a marvel at programming.

and this news indicator is incredible usefull to see if we will have big market movers yes or no.

now if you could only program the indicator to give the advise to buy or to sell now that would be amazing

just kidding. amazing work and thanks again!

1 more thing.

in the netherlands were i come from. we use summertime, wintertime.

so your automatic program calculate timeshift.

it calculated my time to +1GMT while it should have been +2GMT.

i believe this happens because of the summertime. now we are +2 GMT.

Thanks for pointing this out; I have fixed it now.

You are correct. Time in Netherlands is currently GMT+2.

Hi,
I just tried to download Calc Timeshifts and it came up as attachment.php...is this not currently available?

Nevermind: I can adjust the timing in the inputs box for Recent News.

Thanks...nice work!

Three possible solutions:

1. I've just uploaded a new version (see post #1) - you could try downloading that one; OR

2. Try downloading via the 'paper clip' symbol at the top right of the FF window - more info here; OR

3. Simply rename the file after you've downloaded it.

OK, now I got it...thanks again.

This is a really neat indicator! Thanks for sharing.

hanover,

I want to be able to extract the data fom this and apply it to my EA so it can decide to exit or not after a news release. I'm not looking for a spike trader or anything, just some general logic so if the news isn't looking good it can close some open positions. Now i haven't had a deep look into it but i assume i would be extracting data from the .csv file thats here. is there anything else i need to know?

You could use either:

(1) the news.csv file that's created by the Download News script. The date/time in this file is New York time, so you'd likely need to have your EA apply a suitable timeshift to convert to MT4 time (sorry!). The format of this file is
date time; spacing; wingdings#; apply-to-currency-charts; apply-to-timeframe-flags; description line #1; description line #2

where
description line 1 = NY-time currency impact event-descr FF-id#
description line 2 = A: actual | F: forecast | P: previous | R: revised from

OR

(2) the @[email protected] file that's created by the same process (also NY time). This file contains all news events back to Jan 1,2007 (probably no advantage to an EA that's trading in real time, though). Format is
"date", "time", "currency", "description", "impact", "actual","forecast", "previous", "revised from", "wingdings symbol#", "level", "spacing", "FF id#"

OR

(3) create your own export file by setting OutputFile = <filename>
In this case, fields are output in the following order:
date/time, currency, description, impact, actual, forecast, previous, revised from
This has the advantage that you can customize the format of each output field very flexibly, and also set the Date_Source_MT4_Local parameter to output any of NY date/time, GMT, MT4 time, or local time.
How frequently your output file gets (over)written is determined by the RefreshEveryXMins parameter setting.
See the Word doc attached to post #1 for detailed info.

However........
All of the options depend on the premise that you're running the Download News script as frequently as you need up-to-date data (there's no way of knowing exactly when the calendar data has been changed, i.e. when the "Actual" announced value has been posted, without downloading it). Unless you can find a way of calling this script from within your EA, and passing it the appropriate parameters, this would need to be a manual process (which, given your trade exit-based requirement, I guess is a significant problem ). If you do manage to have your EA call it continuously, then this could potentially overload the FF servers (possibly incurring the wrath of the FF webmasters!!). And in any case, with some of the high impact announcements, the price hikes occur almost immediately, hence the effect on your trade P/L would occur long before the FF calendar gets updated with the "Actual" value. I assume that's why many EAs are programmed to exit trades before certain scheduled announcements.