MYSQL or PostgreSQL

For a new project I'm planing to use one of the the db (MYSQL or PostgreSQL).

Mainly: real time Quote data and Order data storage.

As I do not have sufficient experience with either of them

- do you have any reason: Why you would choose one over the other?

MySQL is faster when querying data.

Postgre is faster when inserting data.

Which is more important to you?

I would think the rational for using one over the other is slightly more complex than that, both are fully functional database products in wide use and as such the decision is more about how each meets your own particular usage cases, as opposed to what other people think.

You're very right Craig.

Both are established database systems. I don't think you would notice any major differences between either system until you reach the enterprise level.

Hello!

I created a real-time db system with master and slave account and full web-based order management. I can help you, if you want.
I'm using mysql, I like it.
You can import libmysql.dll, and you can access to database directly. You can insert and select.
You wrote it: you would like to do opposit of other man's think.
It's a very interesting thing. I would like to use this method also. Unfortunatelly, I don't know good strategy on forex. I've got 2 years experience, but I don't know the winner systematic. But I know I thing: 80% of people lose their money on the first 66 day. We should always the exactly opposite trade, and we will win. But unfortunatelly, it's not so easy.
Tell me, what is your strategy.
Sorry, my poor english.
If you would like to write directly, it's my mail address: [email protected]


Bye

Thanks to all so far.

Thanks.

For the time being I decided to go with HDF5 http://www.hdfgroup.org/HDF5/

I see this thread is very old, but this might be a good place to ask - for real-time quote data, has anyone considered "NoSQL" databases (as opposed to MySQL/Posgres which were mentioned above)? I'm looking into Redis and MongoDB and I'm wondering if anyone has real-world experience and can tell me if I can really expect a big improvement in performance for this use case. I am already familiar with MySQL so there is a learning curve involved in adopting these solutions, is it worth the trouble? Also, to make things easier I'm considering using a managed service, there is this one for redis - http://www.redis-cloud.com and this one for Mongo - http://www.mongolab.com. Will this be a problem for real-time quotes, if my application is on a local server? I'm wondering if the latency to the remote service will outweigh the performance / scalability benefits.

Hi Kobi,

Need to note that I'm currently not using real-time market data but I have worked with real time data in other areas. That said I have dabbled with SQL databases for storing FX and stock market tick and bar data - essentially I used which ever database software was convenient to the language I was working in - for trading related projects I've mainly used Python, R and C++ so SQLite, Postgres, MySQL etc. I've used many databases in professional life, including the big name commercial database products.

For "real time" data logging applications (e.g. similar to recording ticks from your broker etc.) I've found that using flat binary files (when speed really matters) are the way to go e.g. you just append data to a file as it arrives. For longer term storage anything goes - my favourite for time series data is netCDF4 (it uses a HDF5 back end). Why use netCDF over SQL databases? It gives you:

1. portability write the files on a Windows, Mac or Linux machine you can open them on any of the others regardless (go for collaborating with others or if you have multiple environments).
2. no server required e.g. less to maintain and backups are just a case of copying the files to backup media (yeah for rsync and crond!)
3. simple well defined API for building the files and accessing the data.
4. netCDF4 also has interfaces for C,C++, Java, Python, R, Ruby, Fortran, C# (essentially all .NET languages) and I even think Javascript (via NodeJS) is available!
5. a key limitation on performance is I/O e.g. if you have bad I/O then you will see poorer than expected performance.

In short all those different programming languages allow me to use the same data with no fuss in a language that best suites the problem at hand e.g. test out a new idea or measure some stats about a market then I'll use R or Python. Serious number crunching to populate and indicator or for real time trade decision then use C/C++. If I lose my biscuit completely I can even use Fortran - but nobody should be using Fortran any more

To answer your question about using Redis, CouchDB or MongoDB I'd have to say not really all that good - performance would be terrible. Not due to retrieval (unless your using a remote server), read performance for these tools is generally really good, but due to the need to process JSON (raw text documents) into to the relevant runtime objects. Throw anything near a couple of megabytes of JSON at Google Chrome (e.g. the fastest javascript engine around) and it even will lag as it parses it. I think noSQL or document databases are good for applications that consume and create objects that can sensibly be represented in JSON - I just don't think regular and irregular time series (e.g price data) is the best option for this.

Hope this helps.

ninety47

Just a quick update...

I've been beavering away on a more generalised data store for "relevant" market data e.g. price, time and sales, market depth, news, social media. I did find this great talk up on slide share produced by the MongoDB folks: http://www.slideshare.net/mongodb/us...atabaseaug2013

A worthy read.

Cheers,

ninety47.

I am totally new to the forex domain and i am a web developer, with more then 8 years of experience in mySQL, PHP, MongoDB, Python, Node.js, etc..

I have the MT4 server installed online, and also i have another server with MT4 report installed on it that get all the data from MT4 and push it to mysql library each few seconds..

I am looking for a good solution to get real time data (maybe mysql enterprise, or Redis, or MongoDB, etc...) and not something that get data every few minutes..

Should i develop a bridge for MT4 (using FIX API)? that get the data from MT4 server and push it to my database directly? Or should i develop a custom script for that using MQL4? or is there any other solution?

really your hints are so appreciated so i can know from where to start and how to go

Thank you