How do you do your market research?

I am an EA head and addict, where I write EA and simply backtest to see the results, and for the most part of my trading career that was how my trading has been done. Simple enough. However lately, what started out of pure curiosity made me a believer in descretional short term trading. This type of trading cannot easily be computerized because I am relying heavily on price action and making trades on the lower TF based on my knowledge of the higher TF.

I can't simply write an EA to backtest so after some thoughts into this I ended up using Microsoft one note 2010 to record down my market backtest analysis which is done manually. In my case for every single day I backtest and research I capture a screen image of the daily, H1 and 15M. I then comment at the bottom what the market is doing on the higher TF and lower TF, what S n R was involved, what signals would have been possible. I then record the trade results categorically by strategy on an excel spreadsheet. eg. A pin bar rejection of the daily pivot and round number with the trend, the 1st take profit at the nearest price action will yield an RR of 1.5. But I tabulate it in excel nicely with less words like a database which calculate the reliability overall RR etc.... So far I am pretty pleased because I am using the same technique to record my live trades results and market analysis day by day.

How do you traders do your market research on strategies which cannot be easily automated with an EA?

Nice post Bin. Here's what I have been trying to do:

1) My tool to do back tests is R (http://www.r-project.org/), a free statistical computing package.

2) Backtests are not completely automatic: I need to override signals from time to time. I provide a separate csv file to note the time of the candle and the reason why I ignore the trading signal. Over time some of these will disappear, ie., they'll become part of the code that generates the signals, but others may stay.

3) Once I review all past signals and add the above mentioned csv file I'll rerun the tests and begin tweaking the parameters (SL, TP, etc.). I organize all data and files in such a way that it's easy to rerun the tests on different currency pairs. This way I'll keep the core part of the back test code intact and only make as few change as possible to run similar tests on a different pair, or during different period.

One thing I hate MT4 is that if you import too much data then it becomes extremely slow. For example after I imported about 20 months of M1/M5/M15/H1/H4/D1 candles of GBPUSD into MT4, it got slow from time to time but still manageable. But when I added EURUSD candles, just M5/M15/H1/H4/D1 without M1, also 20 months, then it became a nightmare. I wonder if you have similar experience.

For R I store data in .csv files and only load them into R when I need to. I use tick data to do back tests instead of M1 candles.

How's your experience with NinjaTrader so far?

Merry Christmas,

screen

wow.

R seems like a powerful tool, i am ignorant on how to use it, so i am going to have to get proficient at it.

I believe statistical analysis is of great importance in trading.
not only to measure realistic objective data, but also on the trading strategy.

Screen, you are really using R? wow. I never knew it will become so popular. I graduated in math and engineering double degree from the university that gave birth to R, Auckland University.(The founder of R Ross Ihaka was my math lecturer for a few papers) So naturally I was required to use R from 1st year till my 5th and final year. I thought at that time it was an inferior language to matlab.

I guess R must have grown up since. Year 2006 I graduated. Anyway what I found effective at that time was to write VBA excel code and call R functions because in Excel it is easier to manipulate data and it have a nicer user interface. I did that for my final year projects.

Ninja trader is good, nice language editor and it have a debugger, but the number of brokers it can use is limited. Hopefully one day all the brokers will switch to ninjatrader.

Anyway I am working on a program in VBA excel to statistically determine if trade entries have an edge. Input is price timeseries for a pair and trader's entry time for the same pair. With it, backtesting and research on descretionary strategy will be a breeze too. =)

PS: I will find time to take a look at the latest R program. I like the way you research with R, screen.

In trading statistics is the only way to determine the truth IMO.

Yes R has improved a lot over time. I didn't know about this until my last job. 7bit even created a bridge so that you can call R functions in MQL4: http://www.forexfactory.com/showthread.php?t=260422 though I wouldn't go that far.

In my case it's for backtesting only. With about 10G tick data from Dukascopy, there's no way I can load them into MT4 and do backtest. VBA is nice but Excel has memory problem. BTW I did the VBA calling R thing as well lol.

It's not for everyone. Debugging isn't as easy, and documentation is bad. I use it because I had exposure to Splus (commercial), its predecessor. R was very similar to Splus at the beginning of its life then they have diverged ever since. This will be a mid to long term project even for me, but once I have all the pieces I need it'll be very useful.

Short of having a crystal ball, you are correct. Good luck to you.