General tech-talk | Page 2 | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

You talk the talk I walk the talk 8 replies

Much of Forex talk is illusional esoteric talk 141 replies

General Motor's Influence 2 replies

VTTrader: General Discussion 24 replies

Beginner question about MT4 and trading in General. 4 replies

Attachments: General tech-talk
Exit Attachments

General tech-talk

  • #21
  • Aug 29, 2018 4:10am Aug 29, 2018 4:10am
Quoting VEEFX
Thanks emmzett and masterx for the responses. I really appreciate it. Level 9 deep is only on one pair..... the depth also exponentially increases depending on when my signal confluence happens which triggers a recursive crawl if you know what I mean but I have capped it at 32 levels. It's is designed as a sequential crawl and cannot be done via parallel processing so to mimic multi-thread, I run it across several terminals plus one terminal to handle all portfolio/trade mgmt functions but still my CPU clocks continuously non stop at 50%-60%....
Most probably the ex4 file is like a Java source code compiled into bytecode which is known to be slower than C++ code.
Quote
Why Java is a factor 2-3 slower than equivalent C++ program?
I know there is an opinion that programs written in Java and running under JVM as as fast as C++ programs, after introducing just-in-time (JIT). I see many cases when Java is terribly slow for simple programs compared to C++, i.e. about factor 2-3 slower than equivalent C++ programs.
Just give you a small example: This is simple benchmark code to merge particles...
It's show time
  • #22
  • Aug 29, 2018 6:30am Aug 29, 2018 6:30am
Quoting masterx584
{quote} Most probably the ex4 file is like a Java source code compiled into bytecode which is known to be slower than C++ code. {quote}
Hi masterx584,
When you said that it is slower than C++ code, just wondering how much it slower, for some average file? Are we talking about microseconds, milliseconds or... I mean approximately, to get an idea about speed.
Thanks
  • #23
  • Edited 6:48am Aug 29, 2018 6:37am | Edited 6:48am
Quoting zosoforever
{quote} Hi masterx584, When you said that it is slower than C++ code, just wondering how much it slower, for some average file? Are we talking about microseconds, milliseconds or... I mean approximately, to get an idea about speed. Thanks
In some tests for the same task done in C++ 50 ms , JAVA did 170 ms ,both executed on the same machine Powered by Intel i7 processor.EX4 could be even slower.
My app in C++ runs CPU at 0% on 8 symbols and 123 threads.The same algorithm was running 1-2% on 20 symbols and 330 threads.As an expert ex4 the same algorithm on 24 symbols at news time was freezing the entire MT4 which needed a restart
It's show time
  • #24
  • Edited 11:08am Aug 29, 2018 6:47am | Edited 11:08am
Quoting masterx584
{quote} In some tests for the same task done in C++ 50 ms , JAVA did 170 ms ,both executed on the same machine Powered by Intel i7 processor.EX4 could be even slower.
Wow, so much of difference!? Is 50 ms - 50 microseconds or milliseconds?
  • #25
  • Aug 29, 2018 6:49am Aug 29, 2018 6:49am
Quoting zosoforever
{quote} Wow, so much of difference!? Is 50 ms - 50 microseconds or milliseconds. Also how big tasks were (that's important)? Are those tasks comparable with average Expert Adviser or much bigger?
ms = milliseconds.
https://softwareengineering.stackexc...lent-c-program
C++ is close to assembly.
It's show time
  • #26
  • Aug 29, 2018 10:56am Aug 29, 2018 10:56am
Quoting masterx584
{quote} In some tests for the same task done in C++ 50 ms , JAVA did 170 ms ,both executed on the same machine Powered by Intel i7 processor.EX4 could be even slower. My app in C++ runs CPU at 0% on 8 symbols and 123 threads.The same algorithm was running 1-2% on 20 symbols and 330 threads.As an expert ex4 the same algorithm on 24 symbols at news time was freezing the entire MT4 which needed a restart
This doesn't make sense. The i7 has 4 cores, and considering you're only updating 8 currencies, your IO bound operations should be limited to 8 threads. Given the rest of each pair's calculations are CPU bound, what's the point of launching another 115 threads? There's no performance gain to be had since each thread would tie up 100% of each core once IO is finished and calculations begin.
  • #27
  • Aug 29, 2018 11:08am Aug 29, 2018 11:08am
Quoting masterx584
{quote} Most probably the ex4 file is like a Java source code compiled into bytecode which is known to be slower than C++ code. {quote}
No, not anymore. At some point it has been like that (I think <600 builds), but now compiles to native code and runs as fast as C/C++. However, the code is likely not as optimised as you can achieve with the popular C/C++ compilers on max optim. settings.
Return To Disorder
  • #28
  • Edited 1:50pm Aug 29, 2018 11:38am | Edited 1:50pm
Quoting Nicholishen
{quote} This doesn't make sense. The i7 has 4 cores, and considering you're only updating 8 currencies, your IO bound operations should be limited to 8 threads. Given the rest of each pair's calculations are CPU bound, what's the point of launching another 115 threads? There's no performance gain to be had since each thread would tie up 100% of each core once IO is finished and calculations begin.
Because is practice vs theory and because my app was projected multithreaded.It's a native multithreaded.I run 16 asynch tasks * symbol that never join,with 1ms sleep time....some are alot faster than others.I was suspicious about multithreading speed gain initially so i made speed tests against same algorithm single threaded...there is a nice difference.Think only that 8 symbols never have the tick at the same time.
The same algorithms on the "single threaded slow and fancy 32 bit MT4" were freezing not only the terminal.exe but the entire PC ,at news peak time.Under C++ i am running on 64 bit ...under MT4 u are forced ti stay inside 32 bit...huge difference.Can't even compare the resulted machine code : 32 bit vs 64 bit...
Now i'm running with CPU 0% the same algorithm where MT4 was constantly above 60% and freezing on news... the CPU speaks alone if i did a good job or not.Everything is optimized for minimum CPU work in my code.It's mostly about how u structure the entire application and how u organize things and the most important beeing outside MT4 bull$hit.
Quote
This doesn't make sense. The i7 has 4 cores, and considering you're only updating 8 currencies, your IO bound operations should be limited to 8 threads
The practice won against theory.DIVIDE AND CONQUER! Consider also release version vs debug version... mine is release
Attached Image (click to enlarge)
Click to Enlarge Name: Capture.PNG Size: 70 KB

The i7 is hyperthreading...4 physical cores --> 8 logical cores.
It's show time
  • #29
  • Edited 12:52pm Aug 29, 2018 12:20pm | Edited 12:52pm
Quoting rockit
{quote} No, not anymore. At some point it has been like that (I think <600 builds), but now compiles to native code and runs as fast as C/C++. However, the code is likely not as optimised as you can achieve with the popular C/C++ compilers on max optim. settings.
It compiles a really slower version ...the debug version.In C++ i'm compiling the release version..big difference..check my thread Your EA is being HACKED! Time to migrate! about ex4 having the entire source code inside....who knows if is compiled maybe is JAVA style code.
It's show time
  • #30
  • Aug 29, 2018 1:02pm Aug 29, 2018 1:02pm
Quoting masterx584
{quote} It compiles a really slower version ...the debug version.In C++ i'm compiling the release version..big difference..check my thread Your EA is being HACKED! Time to migrate! about ex4 having the entire source code inside....who knows if is compiled maybe is JAVA style code.
This is a very good point and I was able to duplicate your findings. The ex4 file (compiled in release mode and missing the mq4) not only knows the line number of the error but also the exact char count from the left hand margin.
  • #31
  • Edited 1:21pm Aug 29, 2018 1:07pm | Edited 1:21pm
Quoting Nicholishen
{quote} This is a very good point and I was able to duplicate your findings. The ex4 file (compiled in release mode and missing the mq4) not only knows the line number of the error but also the exact char count from the left hand margin.
The ex4 is a debug mode one.On the release mode the entire code,comments etc. is destroyed..
The C++ release version in my case is also 4 times lighter then the debug one.There are lots of factors that helped me to speed up my application.I enumerated only a few in the comments above....even cutting the attached spy dll of Avast antivirus gave me a tiny speed gain.Small details make the difference.
It's show time
  • #32
  • Aug 29, 2018 1:14pm Aug 29, 2018 1:14pm
Quoting masterx584
{quote} The ex4 is a debug mode one.On the release mode the entire code,comments etc. is destroyed.. The C++ release version in my case is also 4 times lighter then the debug one.There are lots of factors that helped me to speed up my application.I enumerated only a few.
That's what I was saying. I was agreeing with your sentiment that all the original source code is in the compiled MQL file. It has to be otherwise how else would it account for the precise location (X, Y) of index errors?
  • #33
  • Edited 1:45pm Aug 29, 2018 1:30pm | Edited 1:45pm
Quoting Nicholishen
{quote} That's what I was saying. I was agreeing with your sentiment that all the original source code is in the compiled MQL file. It has to be otherwise how else would it account for the precise location (X, Y) of index errors?
I even doubt they compiling something.They certainly are encrypting the file in order to give a fake security sentiment to the coder.This can justify the lack of speed.
Nicholishen In any case ,i know u want to go outside MT4 and i can only confirm you that is worthing...security of your code ,better speed with less CPU, customizing your application at your willing,the chance to use an extern and faster data feed server...or making them to believe u are trading manually even if u are running automated...only your imagination can limit you.
It's show time
  • #34
  • Aug 29, 2018 1:50pm Aug 29, 2018 1:50pm
Quoting masterx584
{quote} I even doubt they compiling something.They certainly are encrypting the file in order to give a fake security sentiment to the coder.This can justify the lack of speed. Nicholishen In any case ,i know u want to go outside MT4 and i can only confirm you that is worthing...security of your code ,better speed with less CPU, customizing your application at your willing,the chance to use an extern and faster data feed server...or making them to believe u are trading manually even if u are...
That's why I'm interested in creating an EA to run an api (JSON-layer) over zeromq. You could then develop algos in any language and still take advantage of brokers who offer lower spreads and commissions only thru the MT platform.
  • #35
  • Aug 29, 2018 2:49pm Aug 29, 2018 2:49pm
Quoting Nicholishen
{quote} That's why I'm interested in creating an EA to run an api (JSON-layer) over zeromq. You could then develop algos in any language and still take advantage of brokers who offer lower spreads and commissions only thru the MT platform.
I have made my own trading "terminal" long time ago; mt4 I use only to connect through to the broker. However, I use only low-level protocol (windows api - interprocess communication - either direct invocation of a window's procedure, or shared memory).
Return To Disorder
  • #36
  • Aug 31, 2018 7:48pm Aug 31, 2018 7:48pm
Quoting Nicholishen
{quote} That's why I'm interested in creating an EA to run an api (JSON-layer) over zeromq. You could then develop algos in any language and still take advantage of brokers who offer lower spreads and commissions only thru the MT platform.
Let me ask you guys...I want to see if my current spot fx automation has any merit against stocks and corresponding indices. Example- trading the dow30 by using signals via price feed from the corresponding 30 individual stocks using a "weighted" signal approach as an early indicator. How would I do this within mt4? I only need M1 ohlc data for one year (or two months or two quarters) to generate signals. I am interested in creating a time-weighted index using my algo
Staying in my lane...
  • #37
  • Sep 1, 2018 8:20am Sep 1, 2018 8:20am
Quoting VEEFX
{quote} Let me ask you guys...I want to see if my current spot fx automation has any merit against stocks and corresponding indices. Example- trading the dow30 by using signals via price feed from the corresponding 30 individual stocks using a "weighted" signal approach as an early indicator. How would I do this within mt4? I only need M1 ohlc data for one year (or two months or two quarters) to generate signals. I am interested in creating a time-weighted index using my algo
Sounds like a real PITA. Better to use Python for something like that.
  • #38
  • Sep 1, 2018 12:38pm Sep 1, 2018 12:38pm
Quoting Nicholishen
{quote} Sounds like a real PITA. Better to use Python for something like that.
Sorry, what do you mean by PITA. Google just made me super hungry lol
Staying in my lane...
  • #39
  • Last Post: Sep 1, 2018 1:01pm Sep 1, 2018 1:01pm
Quoting VEEFX
{quote} Sorry, what do you mean by PITA. Google just made me super hungry lol
Pain In The Ass
0 traders viewing now