IndicatorCounted() Twice all Bars?

Anyone knows why we get in the begin nearly all the time twice the whole amount of bars?




http://docs.mql4.com/customind/IndicatorCounted
I just copied part of the example code in the attached test indicator. (test.mq4)



The result is this.

EXAMPLE1 M1:
12:15:58 test EURUSD,M1: loaded successfully
12:15:58 test EURUSD,M1: initialized
12:15:58 test EURUSD,M1: Alert: limit: 2154
12:16:01 test EURUSD,M1: Alert: limit: 2
12:16:01 test EURUSD,M1: Alert: limit: 3
12:16:01 test EURUSD,M1: Alert: limit: 2
12:16:01 test EURUSD,M1: Alert: limit: 2155
12:16:01 test EURUSD,M1: Alert: limit: 2
12:16:01 test EURUSD,M1: Alert: limit: 2
12:16:02 test EURUSD,M1: Alert: limit: 2
12:16:03 test EURUSD,M1: deinitialized
12:16:03 test EURUSD,M1: uninit reason 1
12:16:03 test EURUSD,M1: removed



EXAMPLE2 M5:
12:21:35 test EURUSD,M5: loaded successfully
12:21:35 test EURUSD,M5: initialized
12:21:35 test EURUSD,M5: Alert: limit: 2070
12:21:36 test EURUSD,M5: Alert: limit: 2
12:21:38 test EURUSD,M5: Alert: limit: 2
12:21:38 test EURUSD,M5: Alert: limit: 2
12:21:38 test EURUSD,M5: Alert: limit: 2070
12:21:38 test EURUSD,M5: Alert: limit: 2
12:21:39 test EURUSD,M5: Alert: limit: 2
12:21:39 test EURUSD,M5: Alert: limit: 2
12:21:39 test EURUSD,M5: Alert: limit: 2
12:21:40 test EURUSD,M5: Alert: limit: 2
12:21:40 test EURUSD,M5: Alert: limit: 2
12:21:41 test EURUSD,M5: deinitialized
12:21:41 test EURUSD,M5: uninit reason 1
12:21:41 test EURUSD,M5: removed

EXAMPLE3 H1:
12:22:24 test EURUSD,H1: loaded successfully
12:22:24 test EURUSD,H1: initialized
12:22:24 test EURUSD,H1: Alert: limit: 2050
12:22:25 test EURUSD,H1: Alert: limit: 2
12:22:25 test EURUSD,H1: Alert: limit: 2
12:22:27 test EURUSD,H1: Alert: limit: 2050
12:22:27 test EURUSD,H1: Alert: limit: 2
12:22:28 test EURUSD,H1: Alert: limit: 2
12:22:28 test EURUSD,H1: Alert: limit: 2
12:22:28 test EURUSD,H1: Alert: limit: 2
12:22:35 test EURUSD,H1: Alert: limit: 2
12:22:35 test EURUSD,H1: Alert: limit: 2
12:22:36 test EURUSD,H1: Alert: limit: 2
12:22:37 test EURUSD,H1: Alert: limit: 2
12:22:37 test EURUSD,H1: Alert: limit: 2
12:22:38 test EURUSD,H1: Alert: limit: 2
12:22:39 test EURUSD,H1: Alert: limit: 2
12:22:40 test EURUSD,H1: Alert: limit: 2
12:22:40 test EURUSD,H1: deinitialized
12:22:40 test EURUSD,H1: uninit reason 1
12:22:40 test EURUSD,H1: removed



Anyone knows why we get that always twice?

if one uses a for loop like in the example http://docs.mql4.com/customind/IndicatorCounted

for(int i=0; i<limit; i++) we calculate twice the whole bars?


Any thoughts are appreciated

The situation is explained in this link:
http://docs.mql4.com/customind/IndicatorCounted

Note: The latest bar is not considered to be calculated and, in the most cases, it is necessary to recalculate only this bar. However, there occur some boundary cases where custom indicator is called from the expert at the first tick of the new bar. It is possible that the last tick of the previous bar had not been processed (because the last-but-one tick was being processed when this last tick came), the custom indicator was not called and it was not calculated because of this. To avoid indicator calculation errors in such situations, the IndicatorCounted() function returns the count of bars minus one.

The way to think of it is that MT4 needs to work on the most recent open bar most of the time. In some cases as in the example, it also has to consider the most recent closed bar.

It is up to you to decide what is most appropriate for the indicator you are working with.

Thanks CodeMeister.

My point was a bit different though.

It's not about if the last 1 or 2 or even 5 bars have to be recalculated but that it seems:
That at start of an indicator MT4 IndicatorCounted() does calculate all bars twice.

Which seems to me to defeat the very purpose of it.

You are right, but now I understand. I see that you made the relevant lines a different colour.

I haven't seen this before. I am wondering if the indicator is being Uninit or Reinit? I have seen this happen and never been able to identify exactly why it happens all the time. This would show in the Experts or Journal tab.

in the logfiles I posted above there is nothing to see of any between Uninit or Reinit.

Anyway: personally I think it defeats a bit the purpose of IndicatorCounted().

All the best to you.

to put it bluntly everything you have said is completely wrong!

indicatorcounted is doing EXACTLY what it is supposed to do. it is not 'counting twice' as you suggest. this is not even programmatically possible. its purpose, and what it is correctly doing, is to report all of the bars that would need recalculating for whatever your indicator might do.

what IS happening is that mt4 is deciding that something drastic has changed that would require an indi to recalculate all of the data. indicatorcounted is NOT doing anything other than reporting how many bars you need to recalc so that your indi can redraw correctly and not with bogus data.

what IS ALSO happening is that the start function is being called repeatedly.

let's break down your log file with some commentary :

EXAMPLE1 M1:
12:15:58 test EURUSD,M1: loaded successfully
you attached the indicator to the m1 chart
12:15:58 test EURUSD,M1: initialized
the init function was run (i believe this is all that this message means, but i could be wrong)
12:15:58 test EURUSD,M1: Alert: limit: 2154
the start function was run for the first time and indicatorcounted correctly reports that all 2154 available bars require calculations to correctly draw your indicator buffer data and have everything be up-to-date
12:16:01 test EURUSD,M1: Alert: limit: 2
start function is run for the second time and now indicatorcounted reports that the latest 2 bars require calculations to draw the buffer correctly... it's probably actually just 1 bar, but to get around a potential mt4 issue it reports the number as 2 so that you recalc both the latest bar and the one before it
12:16:01 test EURUSD,M1: Alert: limit: 3
start function runs and indicates that the 3 latest bars require recalculation. in truth it's probably 2 bars, but again, indicatorcounted always reports 1 extra bar to get around a possible mt4 issue that sometimes arises when there are collisions with ticks processed versus ticks not processed.
12:16:01 test EURUSD,M1: Alert: limit: 2
start is called again, and the latest bar plus 1 requires recalcing
12:16:01 test EURUSD,M1: Alert: limit: 2155
start is called again, something drastic has occurred within mt4 and now indicatorcounted reports that all bar buffer values should be recalculated to ensure everything is up to date
12:16:01 test EURUSD,M1: Alert: limit: 2
start is called, 1 bar to update
12:16:01 test EURUSD,M1: Alert: limit: 2
start is called, 1 bar to update
12:16:02 test EURUSD,M1: Alert: limit: 2
start is called, 1 bar to update
12:16:03 test EURUSD,M1: deinitialized
you are closing the chart or removing the indi. deinit is called.
12:16:03 test EURUSD,M1: uninit reason 1
deinit func has finished and the reason it was called was due to the indi being removed from the chart.
12:16:03 test EURUSD,M1: removed
the indi is gone.

at no point does indicatorcounted do anything except its job, and do it correctly. what IS happening is that mt4 decides that all buffers should be recalculated for some reason and so resets its internal counter which indicatorcounted then bases it's return value on.

what you need to appreciate is that when attaching an indicator or expert to a chart mt4 does some funky monkey business, and in the example above it called the start function a bunch of times for no obvious reason and probably not due to new incoming ticks, and at one point mt4 realised that the entire indicator buffer values should be recalc'ed due to mt4 doing it's funky monkey business.

it's likely that each indi stores it's own copy of chart details such as currency cross name, bid/ask price, decimal places reported to, etc etc. my guess is that mt4 pushes these details to the inidicator when it first loads and some of these trigger the indicator start function. mt4 knows this happens and so once all of the data is done being sent it resets the counter for how many indicator buffer bars are correct and then triggers the start function one last time to have everything finally correct and ready to go.

everything is perfectly A OK, and as it is supposed to be. MT4 is is an imperfect program that does many imperfect things. indicators are not meant to do anything tricky and are simply meant to draw information. if all you do is run some math or whatever on the available bar data then it won't matter what mt4 does with it's initial stuff-around when an indicator is first attached as it eventually gets everything to where it should be.

what is interesting is that you seem to somehow attach and then remove an indicator within just a few seconds. you must be quick with the mouse...

It's just MT4s way of saying that something may have changed in the history and you should re-calculate. They did introduce unnecessary recalculations in Build 387, but I think they have fixed that in later builds.

MetaTrader 4 Client Terminal Build 387
Terminal: For custom indicators, added initialization of buffers during the repeated reading of the historical data.

MetaTrader 4 Client Terminal Build 392
Removed unnecessary recalculations of indicators when displaying them on a chart.

MetaTrader 4 Client Terminal Build 399
Fixed excess initialization of calculation buffers of custom indicators.

Thanks guys. BTW: I use Build 399.

That is the point where I have my issues with. Does IndicatorCounted() correctly fulfil it's purpose to report only what 'needs' to be recalculated or does it do as you said so nicely: mt4 doing it's funky monkey business?

Quite simply: I attached the indi - closed the terminal - deleted the log file and just open MT4 terminal and afterwards closed it again. But similar things happens if one switches for e.g. the period of the chart.

That is really where I have the problem with. I understand that if 'something drastic has changed' e.g if for instance one scrolls backwards in the chart enforcing an increase of the Bars within the chart history. But such is not the case here.

As a good example I take my second part from the first post:


EXAMPLE2 M5:
12:21:35 test EURUSD,M5: loaded successfully
12:21:35 test EURUSD,M5: initialized
12:21:35 test EURUSD,M5: Alert: limit: 2070
12:21:36 test EURUSD,M5: Alert: limit: 2
12:21:38 test EURUSD,M5: Alert: limit: 2
12:21:38 test EURUSD,M5: Alert: limit: 2
12:21:38 test EURUSD,M5: Alert: limit: 2070
12:21:38 test EURUSD,M5: Alert: limit: 2
....

There was no increase in the bars whatsoever within 3 seconds. (just a couple of normal ticks.
Whatever drastic stuff has happened it is not very clear to me as someone not so familiar with mt4.

Probably that's what I had in the back of my mind but could not express as well as you did


I'm not saying I understand all of this and I'm aware that MT4 might have a reason for such behaviour - that is what I wanted to find out:

* is it a kind of a bug of mt4 (maybe in line with what 'euclid' posted. Build 399 Fixed excess initialization of calculation buffers of custom indicators. maybe they did not fix all of it ??)
* or what is the reason for such behaviour.

BTW: I asked on the official forum in case someone is interested: http://forum.mql4.com/40308

ahah that explains the speedy init deinit.

whatever the reason you can simply ignore it and it's due to mt4 setting up the indicator. it's simply mt4 inner black magic workings and getting itself set up to be ready to proceed in a more normal fashion.

things that would seem simple at face value are not always so simple. the inner workings of operating systems have heaps of similar head-scratching anomalies that can't be worked-around.

end result : nothing at all is wrong or working incorrectly.

Hi,

I would be interested if anyone else can replicate the issues I pointed out in the first post ( I attached the test indi).

The reason I ask is - that someone told me they had not such an issue).

Basically: I attached the indi - shut-down the terminal and restarted it.

used: Demo account, broker MIG, Terminal Build 399

why are you so concerned about this? i'm on build 225

2011.04.28 18:35:21 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:20 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:19 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:19 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:19 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:18 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:18 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:18 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:18 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:18 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:18 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:17 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:17 nbTestingOthersIndi EURUSD,M1: Alert: limit: 3
2011.04.28 18:35:11 nbTestingOthersIndi EURUSD,M1: Alert: limit: 2
2011.04.28 18:35:04 nbTestingOthersIndi EURUSD,M1: Alert: limit: 1161672

Disclaimer. I do not have inner knowledge of how MT4 is coded. This is an educated guess.

When you start up MT4, it will load the indicators on a chart and plot the history it has. That is your first bunch of ticks.

Then it probably repeats the above for any other charts you have open and does any other initialization and monkey business it needs to. During this time it may receive current ticks. This accounts for the batches of 2/3 ticks.

Then it will check with the broker to see if there are ticks it has not received or if it needs to update the ticks it has. For example, if MT4 has been off for 24hrs, then there is a gap of 24hrs in the local history so this data is absent and needs to be downloaded. That is your next large batch of ticks.

So from your printouts, it appears your brokers server/MT4 instance decides the history data it has is not up to date and/or valid and proceeds to download it again.

Apparently, according to previous posts, this behavior may vary depending on MT4 version/broker.

As stated previously, correctly coded indicators/EA should be able to handle this behavior.

Worth noting too is that the chart may not have valid MarketInfo() data on that first run so if ones indicator/EA needs this, one should abort until valid data is available.

Ex: At the beginning of start(), check if Bid and Ask are 0 and exit if they are.

Thanks guys.

Just some addition: I still guess it is a mistake of MT4.

I had to reinstall the client which was version build 229. It had not the problem.