MT4 Memory Use and Indicators
Every indicator is adding 4* 200000 per buffer. That's ~1 MB * how many buffers * how many indicators. Many indicators use multiple buffers; not all are visible.
Does each indicator call other indicators? Add n * 1MB for each of those.
If you want a specific answer you must post specifics.
I have 12 charts open with 6 indicators (including hidden buffers) and I'm running 80 MB with Max Bars on Chart at 65K.
Thanks for the response.
My indicators are not of the type that use buffers - they create UI elements and manipulate chart objects to produce menus/buttons/line drawings etc.
Internally, the code in various places will perform iterative loops through iOpen/iHigh/iLow/iClose etc, or use CopyRates. But none of them actually hold a single buffer which represents the entire time series on the screen - unless of course it is the case they have these 4x default buffers, regardless of whether they use them, as you say.
No one indi calls another indi - if they need to communicate with one another, it is by sending custom chart events which are then handled.
When I've finished trading for the day today, I'll start to dig into it more deeply, and just brute force trial-and-error my way through adding them back in, combining them all into 1, commenting out chunks of code etc.
...
It seems to me, that each indi internally is allocating its own chunk of memory for all the rates it scans through when performing calculations on historical, and then never releasing it again.
...
True. whenever the event triggers OnCalculate(), a snapshot of the current chart history gets created.
In addition, once you access another chart (even by an innocent iBars command), a snapshot of this chart gets created as well.
You can rebuild the snapshots by RefreshRates() any time.
Regarding the "never releasing", some decrease is noticeable during inactivity period, like the weekends, probably those created by the iCommand.
Regarding the fact the amount of data does not fit size of MqlRates - my blind guess is they create MqlRates snapshot alongside single-value arrays (Time, Open...).
I ran some checks - I actually had 15 indis loaded per chart.
By combining some of the simpler ones into one file, and optimising some code I had which was using the CCanvas (and subsequently creating large bitmap resources in memory), I've managed to halve the memory usage.
At present, 8 charts with 100k bars with 11 indis each is using 750MB - this still isn't great by any means, but at least now I'm not in danger of running out of memory. I was getting into situations where the CCanvas.Create command was failing to create a new bitmap resource, because it had run out of memory.
As I removed each indi one by one, I could see the overall memory usage going down by about 10MB per chart (even all with INDICATOR_BUFFERS set to 0) - so it definitely seems as you guys say, that there is a default amount of duplicated information that is held by default for each indi, regardless of whether it's even doing anything or not.
Thanks for the pointers - I've been coding MQL4 for about a year now, and I love teasing out complex functions and pretty UI elements using the built in functions, but coming from a C# background, was really blissfully unaware of memory considerations. Also the fact that many machines these days have 8GB RAM or more, you don't even really notice these sorts of things becoming an issue until it's got well out of hand.
At least I now know what to do in future if I want to get the memory usage down lower. The remaining indis have a ton of code in them though, and trying to combine them all into a single file is not a trivial job unfortunately. And it's horrendous in code manageability terms.
One for a rainy day...
Just for record, I tried to obtain some explanation from MQ earlier.
That's interesting - it seems like it was a bad design decision from the outset, which has persisted.
Holding the history per chart would make a lot more sense than per indicator - and even that doesn't make much sense compared to just holding a global pool of history for the entire MT4 instance.
10 indis on 10 charts - you're effectively using 100x more memory than is actually required.
My guess would be after 10 minutes. From here
"This is due to the fact that the terminal stores symbol data for 10 minutes since the last access to them; after that unused data are automatically deleted from the terminal memory."
I doubt that.
1) I once asked the service desk if indicators called only at init() are removed (no!) or how they can be removed (again no!!). (before b600, but why should mt change this it is not mt5)
2) I wrote an optimizing script that calls an indicator again and again each time with a different setup and immediately I crash the whole pc - out of memory! (after b600+).
That's interesting - it seems like it was a bad design decision from the outset, which has persisted.
Holding the history per chart would make a lot more sense than per indicator - and even that doesn't make much sense compared to just holding a global pool of history for the entire MT4 instance.
10 indis on 10 charts - you're effectively using 100x more memory than is actually required.
I would not call it a bad design, just a design. Actually they avoided a lot of unstable situations by that trick. Mind non-synchronized iCustom calls from EA threads. After all, any optimization they did recently (like speeding up arrays copy) was rather painful, so let them leave it as it is.
I would not call it a bad design, just a design. Actually they avoided a lot of unstable situations by that trick. Mind non-synchronized iCustom calls from EA threads. After all, any optimization they did recently (like speeding up arrays copy) was rather painful, so let them leave it as it is.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
I've got 5 charts open, 200000 bars in chart.
MqlRates structure is (7 * 8) + 4 bytes = 60 bytes.
I would expect memory usage to be in the region of 5 * 200000 * 60 bytes ~= 60MB. It is actually using 180MB - ok not great, but no disaster.
I have about 10 standard utility indis I've written which add a whole load of functionality. They range from simple hotkeys, to bid/ask line toggles, to very much more complex indis.
For the purposes of this test, I removed them all.
As I add them back in one by one, the memory usage jumps up in chunks of 50-100MB in total for each one I've added back, until when they're all back on all the charts, it is using over 1GB of memory(!)
Now, a large number of these indis are performing calculations by scanning back through historical data. I must stress though, that in terms of local variables, there are very few in each one - just a bunch of primitives, which should be no more than a few 100 bytes. There is 100% definitely not any huge arrays, holding 1000000s of rates in memory.
None of these are "buffer" based indis, so there's absolutely no reason for any of them to be maintaining its own copy of the rates in memory.
It seems to me, that each indi internally is allocating its own chunk of memory for all the rates it scans through when performing calculations on historical, and then never releasing it again. This happens separately for each indi, so multiply the effect by 10 indis across 5 charts, and the memory usage starts to become astronomical.
What I'd like to know is - is this by design? Are there any details anywhere on how MT4 allocates memory to indis? Would I be better served to just combine them all into one single mega-indi, and then only take the memory allocation hit once?
It seems pretty incredible to me that a program so simple can be using up as much memory as it is, but there you go.
As an experiment, tomorrow, I will combine them all into one file, see if it makes any difference, and report back.