MT4 Strategy Tester Computer Hardware Bottleneck | Page 2 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Trading computer set up (hardware) 3 replies

strategy builder/tester for new scalping strategy 1 reply

2009.03.11 15:41:15Tester memory handler: tester stopped because not enough memory 19 replies

mt4 strategy tester 0 replies

MT4 Strategy Tester Computer Hardware Bottleneck

  • #21
  • Sep 17, 2011 9:03pm Sep 17, 2011 9:03pm
I completely agree with both Paul and RR. Metaquotes is not trader oriented which is probably why Strategy Tester is so weak and it is time to upgrade their platform to bring it out of the nineties. Tying yourself to old technology is a company killer like DEC, Sun Microsystems and others have found.
  • #22
  • Sep 18, 2011 1:27am Sep 18, 2011 1:27am
Okay, I understand where you guys are coming from on a programming perspective. But, what good is it to provide a program like Metatrader to the brokers if only a limited group of their retail clients are capable of using it? You can't tell me MT5 has been developed for the banks to trade with...I don't know much about programming, but I know they would not be relying on Metatrader to trade their money. They will have their own system to generate automatic trades.

For all you programmers, you must know the underlying process required to run MT4 strategy tester. So, what is the most critical hardware component to improve the speed of the test: RAM, CPU, dual-CPU, drives? Cory
  • #23
  • Sep 18, 2011 2:05am Sep 18, 2011 2:05am
Quoting bigsteely101
Okay, I understand where you guys are coming from on a programming perspective. But, what good is it to provide a program like Metatrader to the brokers if only a limited group of their retail clients are capable of using it? You can't tell me MT5 has been developed for the banks to trade with...I don't know much about programming, but I know they would not be relying on Metatrader to trade their money. They will have their own system to generate automatic trades.

For all you programmers, you must know the underlying process required to run...
If you have to use MT4 and since you have a reasonably quick processor already (although not all of it can be utilized) the best way for you to reduce optimization time is to reduce the amount of parameters being optimized in one shot.

Without knowing how your EA works, it's hard to give specifics but generally, start with bigger step values in the optimizer and narrow down the range of start & stop values that provide the best results. Sometimes, using control points is sufficient for this step.

Once you have a rough idea where the sweet spot might lie for your settings, you can then fine tune this smaller range using the every tick mode.

Be aware that over optimizing can lead to curve fitting which means your settings are perfect for your test data but pretty useless on data which it's never seen before.

Honestly, you would be better investing your time in forward testing your EA on a demo a/c or a live 'micro' account. Watch how it behaves with real market data and tweak the settings as you go along. Watch where it opens/closes trades, see if it's obeying your strategy entry/exit rules, etc.

I think you will be disappointed having spent days/weeks/months optimizing the settings only to find it runs like a dog on a live account. Perhaps you'll have to discover this for yourself before you accept it's the best way.

The tester is OK for testing EA logic, ensuring your EA can actually open/close trades, etc. but once you know that bit works OK, you need to get it on an account and watch how it performs on a daily basis with new (different) live data coming in.

My advice is based on personal experience but I hope others will chip in to confirm what I'm saying, I'm not trying to rain on your parade, just save you wasting a lot of time.

Regards, Paul.
  • #24
  • Sep 18, 2011 3:11am Sep 18, 2011 3:11am
Paul, I appreciate all your time and great comments. I do see many similarities from what you have to say with the way I am trading or I see the markets. Many of the ideas, have come from hard times, if you know what I mean!! I like your comment about me having to discover hard times for myself...haha, yes I did have to find those out for myself, and lose money in the process, but I am comfortable with that. I needed to clear my mind of any uncertainties so was willing to try!

I currently have 28 variables over a period of 30 days that are involved in my EA optimization and it is registered to take approximately 128 hrs. I am set at "control Points". Somewhere in this thread I made a comment about purchasing a villa property in Bali once I saw great results from my backtest but I had to quickly put that purchase on hold once I saw real world results on forward testing. (haha, JK).

With that being said, I believe there is a need for both back-testing and forward testing. With potential profits seen in back-testing you at least have somewhere to start your forward testing. Even if it is curve-fitted, at least you have somewhere to start. Otherwise you are just throwing darts hoping for the best to happen in forward testing. From there, you can start manually tweaking as you say by watching how trades performed.
  • #25
  • Sep 18, 2011 11:23am Sep 18, 2011 11:23am
Quoting bigsteely101
Sorry, but how is it that I "get to you"?
Translates to 'makes me cynical'.

That was in response to stevegee58

Quoting bigsteely101
Okay, I understand where you guys are coming from on a programming perspective. But, what good is it to provide a program like Metatrader to the brokers if only a limited group of their retail clients are capable of using it? You can't tell me MT5 has been developed for the banks to trade with...I don't know much about programming, but I know they would not be relying on Metatrader to trade their money. They will have their own system to generate automatic trades.

For all you programmers, you must know the underlying process required...
MT5 was built for brokers. Simple as that. Traders are secondary and understandably enough--they're not paying for the platform. If you don't like it, build your own. I did. Took awhile to learn, but I've built my own little trading platform that just needs a data feed, and hooks into the broker's order entry system. Building one from scratch however is mighty expensive. You seem to forget that you're using the MT platforms for free.

I wouldn't say a 'limited' group of traders can use MT5. The order entry functions are perfectly usable for manual trading, and I'm very sure that those who trade live with EAs for a living are in a very vast minority, and it will stay that way until programming is reintroduced into school curricula.

Your sense of entitlement from Metaquotes is clearly misplaced. If you don't like the way they do things, go and pay them to do what you want them to do. In the past they were willing to do certain things for traders who paid the extra fee(s). I don't see why they aren't offering this service now.

As for your comments on banks having their own custom platforms for trades--yes and no. When I worked at RBS, maybe 20% of the trades by volume were automated. The rest were manually entered on platforms which were built to interact with other banks' platforms.

As for the things to make strategy tester run really fast, here's what you need:

 

  1. Super fast CPU--see if you can get something with turbo boost or the AMD equivalent. I know someone who uses a 5.2GHz super overclocked computer.
  2. 4GB of free ram (Terminal.exe is only 32-bit), any more is excessive.
  3. Moderately fast hard drive. I didn't notice a difference when I used SSDs over regular spindle disks. But I did notice a difference when I use a dedicated RAM disk. Not sure you can justify that extra cost though.


Probably better off renting an Amazon S3 instance. That's what I used before I bought a Dell T110 server.

That said, if you still don't like the performance, start breaking up the optimizations into chunks and run those chunks on separate instances of metatrader.

Or, if none of these are acceptable to you, write your own strategy tester. I did. It's not terribly difficult, just a matter of reading a data file and running your equations for each row. I'm currently trying to implement KDB--so my self optimizing EAs can reoptimize hourly instead of weekly.

  • #26
  • Sep 20, 2011 6:02pm Sep 20, 2011 6:02pm
Thanks for all the comments RR. I took your suggestion and subscribed to the AWS to test the system for greater speed. I am currently testing their "cluster computing" which is dual Intel Xeon processors (5600 series). The results are identical, if not worse than what I get with my Core i5-760 home computer.

I also compared instances placed on my SSD drive and spindle drive, and the results are exactly the same. I was blown away by this, because I run another scanning software on the SSD and I literally get a 80% speed improvement, so I was so shocked to see the same results.

What would be causing the cap on testing speeds?


On a side note, does every Mt4 instance get a single "core" assigned to it, or do all instances share 1 core? And does the same apply for RAM?

Cheers, CH
  • #27
  • Sep 21, 2011 4:10am Sep 21, 2011 4:10am
Quoting bigsteely101
What would be causing the cap on testing speeds?


On a side note, does every Mt4 instance get a single "core" assigned to it, or do all instances share 1 core? And does the same apply for RAM?

Cheers, CH
It's CPU bound . . . each MT4 Terminal can effectively utilise the capabilities of a single core . . it's not as simple as saying "it uses one core" . . because it may actually be using different cores just not at the same time. It may also be just using a single hyper thread . . this I'm not 100% certain of . . my data in Task Manager seem to indicate this.

Each MT4 Terminal should have access to at least 2 GB or memory address space . . .
20 pips a day isn't too much to ask . . .
  • #28
  • Last Post: Sep 21, 2011 6:42am Sep 21, 2011 6:42am
Quoting bigsteely101
Thanks for all the comments RR. I took your suggestion and subscribed to the AWS to test the system for greater speed. I am currently testing their "cluster computing" which is dual Intel Xeon processors (5600 series). The results are identical, if not worse than what I get with my Core i5-760 home computer.

I also compared instances placed on my SSD drive and spindle drive, and the results are exactly the same. I was blown away by this, because I run another scanning software on the SSD and I literally get a 80% speed improvement, so I...
Perhaps it's time to start looking at your EA itself. Because that's about as fast--hardware wise--that you can go. Frankly, I'm just fine with my Dell T110 server (Intel Xeon x3440 W/16GB RAM).

Any iCustom indicator calls that are made? Any looping that might be reduced? That would be the next place to look for a bottleneck.

Also, seeing that you're optimizing, how many permutations are you running? Surely there's a way to reduce that.
0 traders viewing now