A smart trading system?

Hi everybody, first a word of introduction. I am new to this forum and to the forex trading as a whole. After reading several books, playing a bit with a demo account on forex.com and trying several other platforms I started my first mini account with them.
Two days ago I found this forum and liked the contents and the discussions very beneficial.


I am currently studying bioinformatics and I thought of several systems that I might program to analyze the markets, as the field of bioinformatics is in fact dealing with the same problems as much of the technical analysis - discerning signal out of noise using statistical methods. I wanted to know your opinion whether it actually is worth my while (I mean not necessarily in terms of lots of gained gained money, but more in terms of scientific interest (although if I found a nice scalping system that would provide me with just 10 pips a day I would be a millionaire in <5 years)).

The systems I thought of trying to build should be first of all robust, i.e. they should work among as diverse market conditions as possible - and I also want to ensure their evolvability. What I mean by this that those systems should change their strategies as we go along. The implementation I think should be very computation-intensive and I am not sure that the problem is not NP-complete. The key I thought of would be implementing some kind of a genetic algorithm that the selection function would be measured most of all through robustness.

Now another idea I thought of that might contribute a lot to developing my system is simulating pseudo-market conditions. Tell me what you think of this:
A) Measuring the different parameters of a market and building pseudo-data with those same conditions using random variations on the same theme.
B) Maybe looking for HMMs in the market data and trying to rebuild the data with those same HMMs.
C) some kind of smart shuffling of the data.

Do you think that using those parameters would still overfit the best strategy? Just because it would find my parameters by which I constructed the market data?

I would be very interested in hearing (reading) your comments to the ideas above.

Oh btw - I forgot that I thought about another modification of the system. How about incorporating some news feed into the system which will be oriented on certain keywords and combinations to "understand" the meaning of it. Then modifying the strategy accordingly.

this would surely be a new spin on "trading the news."

welcome to the forum! interesting to hear there are trading parallels in and bioinformatics. man, if you understand that stuff, trading should be a breeze

can you explain what you mean by "HMMs." what parameters are you thinking of measuring? it sounds to me that you want to unleash an algorithm on the market to find some patterns. is that what you are looking to do?

What I am looking mostly is to increase the available volume of data. I am very much afraid that overfitting might occur if I use just the existing data. (Remember that Stocks&Commodities article that random prices can also be very well back-tested and strategies fitted to trade those curves).

What I though t of by looking for patterns is for example:

1) Define certain parameters manually that would describe the market. e.g. try to look for the streaks of time the market is trending (i.e. first derivative averaged on X time points is higher/lower than Y threshold) of course define the variance of this tendency. Also check the second derivative of the price data (the acceleration of the price) and recieve some parameters for that too.
Also would be interesting to see the amplitude of the market movements and the frequency of certain waves or changes.

2) The other approach which I meant by running Hidden Markov Models is learning the system parameters from the (almost) continuous data (the price).


After defining this set of parameters of viable data and avraging over many time points and throwing in a random factor. I may be able to construct a data set which will have some similarity to the existing markets, but not entirely like them and thus test my systems (or create them if I want to use the GA) on a much more significant amount of data.

p.s. another possible approach would be shuffling the markets in order to create the random data (maybe only time points within the markets, and maybe even the markets as a whole after scaling (such as mixing different currencies, but normalizing them to the dollar-adjusted pip-value).

Do any of you know programs that will allow you manipulate data tick-by-tick? It seems like the rate of change in price and the amount of time that a price is consolidating would give very valuable insights on the significance of support and resistance. I don't have much programming experience but I'd like to build my own indicators on an intraday basis......plus typing prices into excel everyday is getting old. It would be nice to have in automatic. Any suggestions?

You can download tick by tick data from:

http://www.fin-rus.com/analysis/export/default.asp

However I don't really know whether I trust their quotes entirely - they seem a bit too "jumpy".

Now about a platform - I don't really know, but I was thinking of doing my work in MATLAB, it has large computing possibilities.

About excel, there is a plugin, although I never worked with it that would let you download automatically data and a free live feed to excel through DDE.

You might check it out at:
http://www.forexite.com/default.html?32

The program is: QuoteRoom 1.48
(Be advised,however, that the site is in Russian, search the web, maybe you will find other programs that might do the same thing).

Hope it helps a bit.

Thanks ivenger. The Russian is a little much for me to work through on this one. But its great to know that this exists. Does anyone know of any other excel feeds like ivenger mentioned?

this is a worthy cause, and youve shown some real creativity in finding ways to get some random data.

but why not just buy some data? you can get tick data for the last 10 years for under $100. i get mine from Disk Trading and it imports into TradeStation using a program called OwnData by TS Support. there are a bunch of different ways to do it.

Yes I could I also downloaded the data for the last 4 years for free from the site above (http://www.fin-rus.com/analysis/export/default.asp). The problem is that I am not sure that even 10 years is enough.

I am now devising the needed restrictions and the goals for the system I want to design. Please tell me if you think they are plausible.
What I want from the system.

A) I want to trade intraday. Thus the average trade should last anywhere from 15 minutes to five hours.
B) The amount of successful trades should be ~40% on average (try to catch weak moves).
C) The losses should be accordingly - the expectancy is a system that will make ~20$ a day on average on a mini lot(20 pips for USD/... pairs).
D) The increase in position should be kind of a pyramid (alhough I'm not sure I wish to restrict the system in such a way. Maybe let it decide by itself the increase/decrease in position during a trade.
E) Maximal drawdown should be ~20% of the equity. I would also very much like something that won't have more than three consecutive losing days.
F) The starting capital should be very low.
G) Time scale: The entry as I see done by the system should be based on a 5 minute chart - without considering the state of the one minutes for the buy (although do consider them for the decision-making process). The 5-min chart is used only in order to make the system more robust to my mistakes in implementing it).
H) The system will by itself generate the exit signals that will adhere to the rules.
I) The equity management also should be generated by the system itself

What do you think?
What I left out?

Remember the system should be automatic - or even a combination of several uncorrelated systems.

I downloaded some of that data, and the accuracy of it looks way off.

It was GBPUSD 15-min data from Feb 2001 until Jan 20 2005.

I think you are right about the data being off the real values.
I too downloaded the data, and it looks very very "jumpy" I'd call it.
However I also downloaded from the other site I mentioned (the forexite) and it has the same data but more smooth.

The trends of both types of the data are the same. Besides, noisy data is even better than a perfect un-noisy one (explanation follows).

I read a solution to overfitting problem in some journal. When testing a system you can deliberately introduce some noise into the data, and if your system is robust enough you would still get pretty good results (although the data is bad). So no real problem with overfitting this noisy data...
This is why I don't really care about the quality of the data. I don't want it to be perfect.

Besides, when I earn enough money to buy the data I will do it, but first I want to try and work with what I have...

For meanwhile this data is good enough.

Another question I had.

Does anybody have an idea after I design and test my mechanical system, how I make it purely mechanical. My broker right now is forex.com - how do I make an interface to let the computer do my trades?

I have yet no idea as to how to do it. Any suggestions are appreciated.
The problem is also that my platform is MATLAB 7.0 and not some especially designed trading platform. Can you think of a way to wire the commands from matlab to another platform which would also be compatible with forex.com?

Have you tried FXEngines ? They work with FXCM and will be going to live trading in a few weeks. You would have to transpose your signals into theirs and may not be completely compatible.

Another idea.. is Global Forex Trading or GFT. I am not sure where they are with live mechanized trading but I think you can do it on a demo right now and a real account later. You might even be able to program your signals into their charts... plus you can trade right from the chart. One caveat I have with them is that their platform (for me) is the least intuitive and most cumbersome I have encountered. It does have a few features I like, so I use it as a back up.

LOU

Yes, I know I could use FXCMs platform (there is a plugin that would let me trade through any platform of their type), however, first of all I don't really want to change a broker, and second there must be an option to do it also thorugh forex.com

I thought about trying to connect through their wap (for wireless) interface. Has anyone tried using some scripting language with ssl secured sites?
Do you thin kit would work?

Forex.com has an API that you can use to send trades. Read about it at http://api.efxnow.com/docs/

How about using the metatrader platform? Interbankfx (www.interbankfx.com) is my broker and they use metatrader. I taught myself MQL (which is the programming language for MT). Pretty easy, like VB. The tought part is finding a strategy that works

fxengines

Anyone have something to say about this plataform for trade the news?
Anyone is going with a live account using this engine?

Post some replies.

Thanks, regards POCOSO

Hello ivenger,

Regarding programming, there are many APIs are available from different brokers: MetaTrader 3 from FXDD, VTAPI from CMS, OandA from OandA and many more for people with bigger pockets. Your algorithms might be a little too complex for writing Expert-Advisors for MetaTrader4, but it might be a possibility too.. Finally, Gain Capital has an API as someone mentioned earlier and I think it can be used with forex.com accounts -- you should check that out.

I've been working on developing 'similar sounding' systems myself. Though I don't have the necessary math background to talk at the same level as you -- my ideas have been focused on writing 'evolutionary' systems. This essentially means, that the system has a 'built-in' backtester.

For example, say I'm using a simple system with two parameters 5 and 13 (eg a 5/13 MA cross system). As prices come in, 5 and 13 may not necessarily be the most optimal parameters for the system. Maybe in the recent market history, they are 4 and 20. A system that can recognize how it can improve its performance and change its parameters to optimal values is my current focus of development.

What this essentially amounts to is writing a good, fast backtester which can be used by the trading system itself.

Regarding free historic data, you can use GAIN Capitals databank as a starter http://ratedata.gaincapital.com/. It goes back good number of years for the major pairs. Of course sifting though the data and incorporating it into a RDMS is a exercise in itself. OandA has also started offering tick-by-tick data but there is delay in getting it and you have to have an account with them.

Hope this helps,