Plot News / Downloader version 4.00

Thank You and a Happy New Year.

Hello Hanover.
Will you be posting the calendar for 2024?

I wasn't planning to, but please see the attached document, which is hopefully a better long term solution.


[EDIT]
Sorry, there was an error in the formula in the cell B1 (the actual, forecast, previous, revised_from columns were in the wrong order), I have fixed this and uploaded both the instructions.docx and the result.xlsx files. Please download these again. Sorry for any inconvenience.

[EDIT]
I've uploaded an enhanced instructions document. It now includes some screenshots (a picture is better than a thousand words, or so 'they' say).
Also corrected an error in an Excel formula in the instructions (there was an unwanted space in the time field of the output line: " hh:mm" instead of "hh:mm").
To the 4 members who have already downloaded the files, please repeat to obtain the latest versions. Sorry for any inconvenience. Hopefully it's 100% this time.

Thanks @hanover! Worked like a charm using the instructions you provided plus using the result.xlsx to update database to 2024.01 and 2024.02. You're the MVP of news here!

That's excellent.
Many thanks for the positive feedback. I'm very glad that somebody is getting some use out of it.

For anybody who wants them:

Here is a zip file of all news events from 2007-01-01 thru 2023-12-31 timestamped UTC time:
Attached File(s)

https://assets.faireconomy.media/images/attach/zip.gif FF calendar news events 2007-01-01 thru 2023-12-31 timestamped UTC time.zip 4.5 MB | 111 downloads | Uploaded Jan 8, 2024 2:00am

Here is a zip file of all news events from 2007-01-01 thru 2023-12-31 timestamped New York time:

Attached File(s)

https://assets.faireconomy.media/images/attach/zip.gif FF calendar news events 2007-01-01 thru 2023-12-31 timestamped NY time.zip 4.5 MB | 81 downloads | Uploaded Jan 8, 2024 2:00am

hello - thanks for this, really really great. how can I convert this information into xml to get it in xml as per the attached file.


thanks very much

I've moved all of the text (and image) into the attached instructions document.
See the attachments below.

Blessings Sir Hanover. Thanks for your : Even Still being Here. Who Knows.. Knows.

Good day sir,
please help by enlighten on how to use this excel and cvs please.

To further practise my growing python programming skills, I've written a small app to convert a CSV (FF calendar news) file to an XML file in same format as those in this link: https://nfs.faireconomy.media/ff_calendar_thisweek.xml

This makes the spreadsheet in post #407 redundant, because you should now be able to run this app to convert CSV files created by (for example) the spreadsheet attached to post #403, or simply sections of the CSV files attached to post #1, to XML files.

The app file is 40MB and exceeds the size limitation for uploads to FF forums. I haven't used Google Drive before, but if all works the way I expect that it should, then you should be able to download all of the necessary files from this link. Please let me know if you have problems, although if they are permissions type issues, I doubt that I'll be able to help much.

Note: I've included the python source code (file: FFcalendar_CSVtoXML.py), in case anybody's interested.

Please also note that this app is offered free of charge and in good faith, and while I've made every effort to make it fit for its designed purpose, I can't guarantee that it will work as intended on every possible PC configuration. I'm running Python 3.8.10 64-bit | Qt 5.15.2 | PyQt5 5.15.9 on Windows 10 Pro ver 22H2 build 19045.4046 on an Intel(R) Core(TM) i7-4771 CPU @ 3.50GHz with 16 GB RAM. Further, you agree to use the app at your own risk, I can't accept responsibility for any damages, financial or otherwise, that it might unexpectedly cause.


The name of the app is:
FFcalendar_CSVtoXML.exe

You can manage some of the settings for the app, by modifying the content of the file:
CSV_to_XML_settings.json
but I suggest using copy/paste for the input/output paths as their spelling/punctuation etc must be 100%, or the app will crash.
This settings file MUST reside in the same folder as the app.

root_name:
The root_name isthe name of the header (and footer) that will appear in the second and final lines of your XML. There's no need to change it from the default value.
Don't forget the matching double quotes !!

input_path:
The app expects to find the input (CSV) file in the path specified. If you simply enter a filename without prefixing it with a path, then the file MUST already exist in the same folder as the app. The filename you type here should end with .csv
Don't forget the matching double quotes !!
Also, please note that the app expects to find 12 columns in the input file in the following order:
"Date", "Time", "Currency", "Impact", "Description", "Actual", "Forecast", "Previous", "Revised from", "FF event ID", "Group ID", "Actual better/worse", "Revised from better/worse"

output_path:
If the specified output (XML) file doesn't already exist, it will be created in the specified folder; if it already exists, it will be completely overwritten. The filename you type here should end with .xml
Don't forget the matching double quotes !!

All of the other settings determine what data will be output. You must type either a 1 (if you want the item to be included), or a 0 (if you want it to be omitted).
(The date, time, event title, country and impact columns will always be output; you have no choice).
Any timezone shift in the date is assumed to have already been built into the CSV data.


Run the app and, providing all of your parameter settings are correct, and the required files exist in the correct folders, then the process should end with a message stating that it has run successfully, and you can view the content of the output file to confirm this.

I have included sample input and output files ("FFcalendar_news 2024-01-01 thru 2024-02-24.csv" and "FFcalendar_news 2024-01-01 thru 2024-02-24.xml") in the above link, so that you can check their content against the FF calendar for correctness. The heading row in the input file is optional; the app knows what the headings should be.

If the instructions in the file that I attached to the post are not clear enough, then I'm sorry but it's the best that I can do.

Continuing from post #410:

I've uploaded new versions of both the python source code (file: FFcalendar_CSVtoXML.py) and the compiled, executable app (file: FFcalendar_CSVtoXML.exe) to the Google Drive link. This fixes an error which occurs if there are less than the 13 columns in the csv file
"Date", "Time", "Currency", "Impact", "Description", "Actual", "Forecast", "Previous", "Revised from", "FF event ID", "Group ID", "Actual better/worse", "Revised from better/worse"
by performing a preliminary pass of your input_path file and adding extra ,"" empty columns (fields) to the right hand edge of each row, to bring the total number of columns to 13. So if you've been getting an error that looks something like this:

Traceback (most recent call last):
File "C:\.............\FFcalendar_CSVtoXML.py", line 176, in <module>
file.write(output_str)
File "C:\..............\FFcalendar_CSVtoXML.py", line 105, in csv_to_xml_with_pandas
File "pandas\core\generic.py", line 6307, in __setattr__
File "properties.pyx", line 69, in pandas._libs.properties.AxisProperty.__set__
File "pandas\core\generic.py", line 813, in _set_axis
File "pandas\core\internals\managers.py", line 239, in set_axis
File "pandas\core\internals\base.py", line 98, in _validate_set_axis
ValueError: Length mismatch: Expected axis has 12 elements, new values have 13 elements
[31384] Failed to execute script 'FFcalendar_CSVtoXML' due to unhandled exception!

then the new version of the app should prevent this from occurring.

Download these files as before, overwriting the previous versions, and you should be good to go.

wow thank you Han, very kind of you. sorry for the late reply I didn't see this. if I can ask u something else, but it isnt about programming. u see I have a MacBook and im using mt4 on it. suddenly the terminal and other things that usually show up on the side of the chart have shifted so that it opens as a separate window , the problem is I can't drag indicators onto the chart or see the market to get new indicators, any idea what I can do please to get it to the original way it was downloaded. is there a restore to default settings on mt4 or something about the format to original format? thanks a million

first, thanks for all your efforts and hard work and support!!

question:
i downloaded the exe to convert csv to xml , run it , but im getting this error:
C:\news>FFcalendar_CSVtoXML.exe FFcalendar_CSVtoXML.py
Working -- please wait.... this might take a few minutes if the file is large....
Traceback (most recent call last):
File "C:\Users\David\.spyder-py3\FFcalendar_CSVtoXML.py", line 203, in <module>
File "C:\Users\David\.spyder-py3\FFcalendar_CSVtoXML.py", line 70, in fix_settings_file
FileNotFoundError: [Errno 2] No such file or directory: 'CSV_to_XML_settings.json'
[11260] Failed to execute script 'FFcalendar_CSVtoXML' due to unhandled exception!

some path seem to be hardcoded in the compiled exe? i dont have /users/david

also, where to download the latest version of the news downloader? (which works with the latest url , the old version gives me 301 Moved Permanently ERROR)

thank you!!

i dragged the script on the chart, nothing happens, i opened the News Downloader.exe manually in windows explorer, but getting error message when i run it "unable to process calender"

Hello, I'm sorry, I've never owned an Apple computer, don't know much about them. Somebody on Platform Tech forum might be able to help you.

Error message indicates that 'CSV_to_XML_settings.json' file is not found. Please download this file from here into the same folder on your computer where you downloaded the 'FFcalendar_CSVtoXML.exe' file, and try again. Hopefully will get you one step nearer.

thanks for your answer, i downloaded the script and added to the folder, run the exe again, now that part of the error is gone, the error now is:

c:\news>FFcalendar_CSVtoXML.exe
Working -- please wait.... this might take a few minutes if the file is large....
Traceback (most recent call last):
File "C:\Users\David\.spyder-py3\FFcalendar_CSVtoXML.py", line 205, in <module>
File "C:\Users\David\.spyder-py3\FFcalendar_CSVtoXML.py", line 50, in fix_input_file
FileNotFoundError: [Errno 2] No such file or directory: 'FFcalendar_news.csv'
[2316] Failed to execute script 'FFcalendar_CSVtoXML' due to unhandled exception!

oh wait, the solution is in front of me
FFcalendar_news.csv is missing ))

i renamed the csv file to FFcalendar_news.csv , it worked!! thanks mate!!!!

hi, im back )

do you know what i have to change in the py file to have the conversion result in xml a slightly different?

now it comes as:
<event>
<title>Core CPI m/m</title>
<country>USD</country>
<date>
<![CDATA[ 2023-01-12 ]]>
</date>
<time>
<![CDATA[ 01:30pm ]]>
</time>
<impact>
<![CDATA[ High ]]>
</impact>
<actual>
<![CDATA[ 0.30% ]]>
</actual>
(...............)
</event>

i need:
<event>
<title>Core CPI m/m</title>
<country>USD</country>
<date><![CDATA[2023-01-12]]></date>
<time><![CDATA[01:30pm]]></time>
<impact><![CDATA[ High ]]></impact>
(.............)

</event>

basically i need <date> and </date> in the same line, without line ends, and without spaces in between, (and time and impact)
title and country it was already in the correct way

thank you

I have made the change (assuming that I understand you correctly), and uploaded the new files

CSV_to_XML_settings.json

FFcalendar_CSVtoXML.exe

FFcalendar_CSVtoXML.py

to this link, overwriting the previous versions.

There is a new setting at the end of the settings file:
flag_cdata_next_line

1. if set to 0, the <![CDATA[....]]> code block is printed on the same line as the relevant column name (which is what I think you are asking for).
2. If set to 1, the code block is printed on the following line (which is the way the app previously worked).

Please ensure that all of the files are downloaded to the same folder.

I haven't had much time to test this, but please feel welcome to give it a try.

Hello David, Thanks for your help so far.

Can we get the updated news history up to end of February this year 2024? I used the plot news but it keeps downloading only for up to 2011.