why does small malloc fail when freeheap =76k+? #5346
Comments
|
Just to make sure the heap is not too fragmented, can you try calling heap_caps_get_largest_free_block(MALLOC_CAP_DEFAULT) and printing its output as well? |
|
I had tried that before, tried that it: gave exactly the same value as ESP.maxHeapBlock. but I guess I didn't get as far with the code at that time. In the early stages, I see it is indeed the same for a long time, but starts to frgament a little around FH=~185k. AT that point it gets worse and worse fast, but interestingly, while ESP.maxHeapBlock also starts fragmenting at the same point, after a while the two values start to diverge, with ESP.maxHeapBlock showing a much higher figure: ... and a few seconds later when the limit of 78k FH is passed, there is > 10-fold discrepancy: SO I guess the new questions are:
Next I tried to force the crash again: by dropping the limit to 50000. And now I see why the crash is occuring: I won't even bother to do the dump because we both know where it will be! Ok so this explains the FH ~78k issue: its actually the fragmentation reaching crazy levels and other tasks needing that memory. This more than ever convinces me that ESP.maxHeapBlock is broken. I will run it again and see if ESP.getFreeHeap differs from heap_caps_get_free_size |
|
heap_caps_get_free_size(MALLOC_CAP_DEFAULT is showing a huge difference - is that the right call? If not what is the heap_caps_* equivalent of ESP.getFreeHeap? |
|
Trusting ESP.getFreeHeap seems to be the problem: again there is this 10-fold discrepancy at the point of crash: |
|
arduino-esp32/cores/esp32/Esp.cpp Line 137 in c37557c |
So whats the difference between MALLOC_CAP_DEFAULT vs MALLOC_CAP_INTERNAL ? The real question is: what is the correct call to ACCURATELY predict when the heap is dangerously low? Obvioulsy if I'm making a big allocation I would check heap_caps_get_largest_free_block(MALLOC_CAP_DEFAULT) first but in a multi-tasking environment, its the other background tasks I have to think of too in the general picture and not starve them - I need a figure for the overall free heap. Obviously I'm going to add e.g. 10% safety margin to what that figure is at run-time, but I need to know a reliable and accurate way to get it! Already obvious that ESP.* calls aren't the answer! |
|
MALLOC_CAP_INTERNAL describes any internal memory — for example, 32-bit aligned IRAM region is also included. Since you never allocate from that region, it is mostly "free" and this explains why the value gets "stuck". MALLOC_CAP_DEFAULT is the same memory type as will be allocated by malloc. When PSRAM is not enabled, it will be MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT — i.e. internal memory accessible at 1-byte granularity with no alignment restrictions. If PSRAM is enabled, and So MALLOC_CAP_DEFAULT can return either smaller or larger value than MALLOC_CAP_INTERNAL. If you want to see the value that is related to the heap that |
|
@igrr thanks for that clear explanatin so are you saying I should replace my calls to ESP.getFreeHeap() with heap_caps_free_size(MALLOC_CAP_DEFAULT) ? Does this also explaing the max heap block discrepancy between ESP. and heap_caps ? Is this just a documentation isuue? My ultimate goal is to get a single figure figure below which I should back off any more of my own heap allocation to let the other tasks "breathe", and prevent a crash. I can live with "if MBLIMIT < maxblocks OR FSLIMIT < freespace then backoff"... I just need to know the "best" way if the ESP. functions can't give me a good answer, whihc at the moment it seems they can't |
|
@me-no-dev do you recall why ESP.getFreeHeap and ESP.maxHeapBlock used MALLOC_CAP_INTERNAL instead of MALLOC_CAP_DEFAULT? It seems that showing the amount of heap accessible by |
|
Well, it seems like MALLOC_CAP_DEFAULT show all the available memory, including PSRAM on Wrover modules - ESP.getFreeHeap shows 172696 bytes, heap_caps_free_size(MALLOC_CAP_DEFAULT) shows 4356824 bytes. |
|
@igrr what I recall is that prior to 2.0.0, PSRAM was not mapped to malloc and that we wanted FreeHeap to show the internal memory, while FreePSRAM to show the external. It probably makes sense to be changed now that all is available through malloc |
|
Okay, if you decide to keep MALLOC_CAP_INTERNAL, please add |
ESP.getFreeHeap() includes memory that can only be accessed with 32 bit alignment. However malloc etc. will never allocate from this memory. espressif/arduino-esp32#5346 (comment)
|
[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions. |
igrr
added
Type: Bug 🐛
|
Just to close this issue:
In case of further questions, please reopen this issue. |
Hardware:
Core Installation version: 1.6.0
Computer OS: ?Windows 10? ?Mac OSX? ?Ubuntu?
Description:
I have an app specifically designed to find the "safe" lower heap limit which will allow any app to continue to run. Above 80k , it works exactly as designed, forcing the "heap grabbing" routine to back off once the lower limit is exceeded, and allowing it to continue as the heap is freed and returns back above a (higher) safe value.
The UI
The TL;DR is that Q is std::queue of asynchronous SSE writes which update the UI in "real time" (or at least as fast as the TCP TX rate will allow - currently can't seem to get past 20/s) once the TX rate is exceeded they get Q'd and the matching TCP ACK then pops them off the Q, freeing some heap, allowing more sends...rinse and repeat. The "Visualiser rate" is the rate per second that the Visualiser "LED" is flashed. Each "flash" is an SSE event and there is a rotary encoder hooked up to the Visualiser rate so I can just turn up / down the rate at which the Q fills / drains, and so by adjusting that and the cut-in and cut-out values I can force or avoid a low heap crash. The "Go" button (and the center button of the encoder on GPIO32) stop and start the visualiser so I can gauge the rate of Q fill/drain. The aim of the app is to find the exact lowest-safe-heap breaking point so that "production" code can be tuned to avoid it by judicious choice of of cut-out and cut-in values.
It all works wonderfully until it gets below about 80k, then I'm getting some strange results I don't understand which seems "plain wrong"
Crash Log
On a run with a lower bound "Heap Cut-out" of 50000 the following crash occured:
H= free heap, MB=Max free heap block
The failing line is a malloc(1436) call in WiFiUDP, but previous examples have been in my own code, ArduionOTA, MDNS...however in every instance the abort is called on a malloc requesting some small value well below what appears to be available.
Stack Trace
Questions
The text was updated successfully, but these errors were encountered: