quad core multiple CPU's, multiple MT4 sessions, Virtual Machine | Page 3 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

4 sessions for i-Sessions Indicator 7 replies

MT4 EA used 100% of CPU Power? 1 reply

MT4-Best CPU performance with Multiple indicators and EAs 2 replies

MT4 check CPU resources 1 reply

Backtesting with CPU-intensive indicators 3 replies

Attachments: quad core multiple CPU's, multiple MT4 sessions, Virtual Machine
Exit Attachments

quad core multiple CPU's, multiple MT4 sessions, Virtual Machine

  • #41
  • Edited 5:29am Nov 5, 2011 3:51am | Edited 5:29am
If I understand correctly what you want to achieve.

1. Matlab filters the data (from MT4 or non retail broker).
http://www.hpc.maths.unsw.edu.au/tensor/matlab
http://www.mathworks.se/discovery/matlab-gpu.html
2. If BUY/SELL signal was received, Matlab creates a file "signal.dat" BUY=1 SELL=-1
3. MT4 expert reads the file, executes an order, and deletes the file.

I would be glad to help you with MT4/Matlab/c++/c/java/...
Attached Image(s) (click to enlarge)
Click to Enlarge Name: c1.jpg Size: 37 KB
Click to Enlarge Name: c2.jpg Size: 92 KB
  • #42
  • Nov 6, 2011 4:06pm Nov 6, 2011 4:06pm
Try this: Prime95 is a program which searches for very large prime numbers. It has also gained a reputation for getting errors on computers which are even slightly unstable. If there is any problem with your CPU, your RAM, or the parts of your motherboard which are the interface between your CPU and RAM, then Prime95 has a tendency to fail. This is a very useful characteristic if your computer is flakey and you're not sure which component is causing the problem. If your computer can run Prime95 for 24 hours without any errors, then you can usually assume that your CPU and the parts of the motherboard which interface with the CPU and RAM, are stable and working well. The program is free and you can download it from here. http://www.playtool.com/pages/prime95/prime95.html
Additional info: http://www.dragonblogger.com/stress-test-cpu-prime95/
and http://www.overclockingwiki.org/index.php/Prime95
Price = the amount a greater fool is willing to pay!
  • #43
  • Nov 6, 2011 4:16pm Nov 6, 2011 4:16pm
This test that I talk about artificially loads your CPU to the max to test it using some codes or whatever! I think that there is problem in the code and that has the same effect as the testing program! As mem said, for sure it is a problem caused by the code. In my opinion it is impossible to load that heavily those i7 Cpu-s this is the latest and the best CPUs on the market, no way any properly working application on earth to load them so much

Quoting mem
Optimization:
- place counters all over your code and log running time onto them which will help you find part of code that takes most of processor time (probably your IT guy will help you with that and show how this should be done - that's better idea then learning C++ nearly from scratch)
- try to not make same computations every tick/bar, just place them in array and use when needed

With few hours of work you can make it running much smoother over weekend, learing C++ is not something you can master in 2 days.

I had similar problem with indicator...
Price = the amount a greater fool is willing to pay!
  • #44
  • Nov 6, 2011 4:22pm Nov 6, 2011 4:22pm
As much as I know from reviews and reading over the I-net, the new Metaeditor 5 for MT5 and MQL5 is much more optimized, very user-friendly and lot easier to use! I haven't tried it though check it out, it may be helpful!
Price = the amount a greater fool is willing to pay!
  • #45
  • Nov 7, 2011 1:49am Nov 7, 2011 1:49am
I spend the weekend simplifying my code a bit and used some of the other suggestion given here on the thread and it has become manageable now as long as I keep just 2 sessions running per machine.

I am still going to look into a 4.5Ghz machine this week, they are cheap enough to try one out and see how I get on.

Thanks a bundle for all suggestions and pointers here and via PM, there is some good tech knowledge on FF.
  • #46
  • Edited 2:07am Nov 7, 2011 1:53am | Edited 2:07am
Quoting artart
If I understand correctly what you want to achieve.

1. Matlab filters the data (from MT4 or non retail broker).
http://www.hpc.maths.unsw.edu.au/tensor/matlab
http://www.mathworks.se/discovery/matlab-gpu.html
2. If BUY/SELL signal was received, Matlab creates a file "signal.dat" BUY=1 SELL=-1
3. MT4 expert reads the file,...

That will be the ultimate goal, haven't gotten there yet. That is probably a year or so away. As is learning C++
  • #47
  • Nov 7, 2011 2:02am Nov 7, 2011 2:02am
I will spend the rest of the week on the code, I found one unnecessary loop over the weekend and have made the "front end" a bit lighter. I am now convinced that it is my own shortcomings in coding that is causing this. Knowing what the problem is though, does not give you the solution. The math is till heavy for MT4, but I can't really change that without diluting the solution I am aiming for. So still a bit stuck between a rock and a heartbeat, but I am getting there.
  • #48
  • Nov 7, 2011 2:06am Nov 7, 2011 2:06am
Quoting Pip-hunter
Try this: Prime95 is a program which searches for very large prime numbers. It has also gained a reputation for getting errors on computers which are even slightly unstable. If there is any problem with your CPU, your RAM, or the parts of your motherboard which are the interface between your CPU and RAM, then Prime95 has a tendency...

It is not a machine issue, I have 3 machines that run just the MT4 sessions, seperate from the internet and order processing machines and it runs slow on all 3.

Good suggestion though for others to try their machines that don't have backups available.
  • #49
  • Nov 7, 2011 5:33am Nov 7, 2011 5:33am
You are the trader, money is not an issue for you, then forget that your code is slow.

In previous post you mentioned Matlab. Convert your existing(slow) c code into matlab code(matlab has this functionality), I will show how to this, and I don't want to see your secret code, I will show on function like this
double some_function( double p1, double p2 )
{
return 1.0;
}

Believe me, to do parallel computing on Matlab is easier than c++, c, or java.
You have to say matlab "matlabpool open"(see my previous post) and then
parfor i=2:L
m(i)=w*(x(i)-m(i-1))+m(i-1);
end
and it goes parallel.

Matlab is powerful and friendly.

Matlab runs on separate powerful machine and sends data(sell/buy signals, arrays for MT4) over the network and saves into MT4 directory as file. MT4 reads from that file executes orders, and visualize the data.

And don't hesitate to ask, I have a lot of fun reading your posts and your levels are also correct and useful.
Attached Images
  • #50
  • Nov 7, 2011 5:37am Nov 7, 2011 5:37am
Quoting cloggie
That will be the ultimate goal, haven't gotten there yet. That is probably a year or so away. As is learning C++
Don't waist your time on C++, learn better C.
C++ has many features your don't need them at all, and nobody needs them.
You need to implement math functions, c for this is more than enough.
C is the simpliest, friendliest language I ever seen.


The excerpt from good programing book.
"If C is good enough for Doom and Quake, there’s no Earth-shattering reason to use C++."
  • #51
  • Nov 7, 2011 8:29pm Nov 7, 2011 8:29pm
You can only parallel as much as your algorithm allows you to. If there's a lock somewhere, you still need to process that first before you can continue.

It's hard writing good code, it's impossible writing good parallelized code. I know that, I've been there and it's painful.

Cloggie, I recommend taking a go over your algo first then your programming language of choice then your comp specs.

I recommend plain C for the language of choice and parallelize the processing by breaking the dataset down (process multiple dataset at the same time).

Cheers.

Quoting artart
You are the trader, money is not an issue for you, then forget that your code is slow.

In previous post you mentioned Matlab. Convert your existing(slow) c code into matlab code(matlab has this functionality), I will show how to this, and I don't want to see your secret code, I will show on function like this
double some_function( double p1, double p2 )
{
return 1.0;
}

Believe me, to do parallel computing on Matlab is easier than c++, c, or java.
You have to say matlab "matlabpool open"(see my previous post) and then
parfor i=2:L...
  • #52
  • Nov 7, 2011 9:54pm Nov 7, 2011 9:54pm
Cloggie,

Just another 2 bits from a hardware/software geek. It ain't your hardware. Find a true coder you can trust. One that can work at a low level with something like OPENCL. Even just straight up C is likely more than enough with the right guy. The difference between ok and and excellence in programming is often exponential in nature. It's like trading; you don't learn it overnight. And the true greats are a cut above.

You can compartmentalize your logic, and or keep some of the less performance critical parts aside to keep your intellectual property secure. Shouldn't be that hard to mitigate the risks, but the benefits may cut years off your development time, not to mention frustration levels.

Find a hardcore, low-level, number crunching geek. Sign up for the C course and watch that kid who gets his work done in 1/10th of the time the rest of the class takes. His code is cleaner, shorter, and runs 100x as fast. He don't know or care about the 'trading grail' but he likes money, and a good problem to solve. Or just ask the professor who that guy is.

Someone can make your math work efficiently on your hardware, or at the very least pre-compute or parrallelize enough of it to make it manageable. There are a LOT of tricks and frankly your time is worth a whole lot more than the guy that knows them. Speed costs... How fast you wanna go?

Flipside... you could probably trade time straight across, trading knowledge for coding optimization knowledge with the right guy. You share what you want... you both benefit. You have enough pull to have geeks lining up to help you. Don't know if the right guy is around here, but you never know.

Good luck, and thanks for the knowledge and entertainment.

Mike
  • #53
  • Nov 7, 2011 11:40pm Nov 7, 2011 11:40pm
Quoting jeff_b
You can only parallel as much as your algorithm allows you to. If there's a lock somewhere, you still need to process that first before you can continue.
very good notice.

couple more ideas,

1. You have an array lets say of 1000000 closing prices(ticks or 1min or 5min....)(on every tick or 1min... the array is growing), on every tick your code makes decision BUY/SELL and prepares the data for visualisation on MetaTrader from this 1000000++ array. Here could be a performance killer.
But lets say 1000 or 200 or 100 closing prices are enough to make a decision and for visualisation. I added a screenshot/code of two ema crosses where array of 200 elements enough for BUY/SELL signal and visualisation.


2. Avoid function calls. Sometimes code in MT4 int start() function runs faster then native MT4 function calls, or functions from dll.
Somebody can say "dll are fastest", don't believe try yourself.
Sometimes I experiment(if I need speed) native MT4 functions, functions from dll, or inside start() function.
int start()
{
// no dll function calls
// no MT4 native function calls
// all math here
return (0);
}
Attached Image (click to enlarge)
Click to Enlarge Name: c1.jpg Size: 52 KB
Attached File(s)
File Type: mq4 ART_MA_COMB.mq4   4 KB | 344 downloads
  • #54
  • Last Post: Nov 10, 2011 9:04am Nov 10, 2011 9:04am
A little update here is warranted as so many of you have made great suggestions.

I managed to lighten the code further, some to a bit of detriment to the result but I can now run a single session of MT4 without it slowing down significantly. I have also ordered an overclocked machine as I feel that is the simplest solution at this stage.

Thank you all for your invaluable input and I now know what my further strategy is in dealing with this in the future and what steps I need to take from here on.
0 traders viewing now