code changes: 39644 bytes lost and 57340 bytes lost after WiFi.softAP since 2019/05/01

[marcmerlin]

When using Wifi in my code that is 100% unchanged, I lose 57KB of memory between
50d1429 2019/05/01
49b7664 today

This is bad, because my code does not work anymore with 57KB missing.
I did a bisect and saw there were several things that took RAM before my code gets to do anything. I bisected to the first one, but is at least one more bad CL. The one shown here takes 10KB despite using PSRAM (which the original tree isn't even using, so that's 14KB lost, but I don't care about losing PSRAM given how much I have)
The 2nd CL (or there may be more) loses a whopping 47KB (!). I'm not just being pendantic, I did a git pull on my esp32 hardware tree, and my code stopped working. 2H and a git bisect later, found that it was this code's fault.
I understand that improvements can take RAM, but 10KB is a lot, 57KB is beyond a lot given how little contiguous RAM is actually usable on the chip.
Git bisect result: 390da0d from @me-no-dev

Changes in 1 year's worth of esp32-arduino changes:
before wifi init: 273016 bytes total to 233372 bytes total => 39644 lost
after wifi init: 217700 bytes total to 160360 bytes total => 57340 lost

Code if it matters:
https://github.com/marcmerlin/NeoMatrix-FastLED-IR

Here is the bisect log showing that things get worse later, but I only bisected to the first hit:

---------------------------------------------------------------------------------------------
Original  50d142950d229b8fabca9b749dc4a5f2533bc426
Heap/32-bit Memory Available     : 273016 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 231672 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free
Configuring access point...
[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 13 - AP_START
WIFI AP Started. IP Address: 192.168.4.1
After Wifi/Before SPIFFS/FFat
Heap/32-bit Memory Available     : 217700 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 176356 bytes total, 113792 bytes largest free block
Total PSRAM used: 3676 bytes total, 4190360 PSRAM bytes free


master tree: 49b76649f13acdb8abaf795015d2740bc5f6b340
Heap/32-bit Memory Available     : 233372 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 192812 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free
Configuring access point...
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 14 - AP_START
WIFI AP Started. IP Address: 192.168.4.1
After Wifi/Before SPIFFS/FFat
Heap/32-bit Memory Available     : 160360 bytes total, 97392 bytes largest free block
8-bit/malloc/DMA Memory Available: 119800 bytes total, 97392 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free

273016 bytes total => 233372 bytes total => 39644 lost
217700 bytes total => 160360 bytes total => 57340 lost
----------------------------------------------------------------------------------------

[b30e55efff02becf1a29faadc6009a841dea659e] Added partition size option for Pico Kit (#3258)
Heap/32-bit Memory Available     : 274244 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 231072 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free
Configuring access point...
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 14 - AP_STOP
WIFI AP Started. IP Address: 192.168.4.1
After Wifi/Before SPIFFS/FFat
Heap/32-bit Memory Available     : 201556 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 158384 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes fre
=> bad

[fa55a2c91d31eab673c0115bc57e9437350488cc] Update stale.yml
Heap/32-bit Memory Available     : 273580 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 231000 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free
Configuring access point...
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 13 - AP_START
WIFI AP Started. IP Address: 192.168.4.1
After Wifi/Before SPIFFS/FFat
Heap/32-bit Memory Available     : 214676 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 172096 bytes total, 113792 bytes largest free block
Total PSRAM used: 3760 bytes total, 4190276 PSRAM bytes free
=> good


[0d163a1ce274cf389f8a263d66daaaa36b757d6f] Update IDF 3.2 to 7dd492319 + WiFi fix (#3177)
Before Wifi
Heap/32-bit Memory Available     : 274184 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 230992 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free
Configuring access point...
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 14 - AP_STOP
WIFI AP Started. IP Address: 192.168.4.1
After Wifi/Before SPIFFS/FFat
Heap/32-bit Memory Available     : 206256 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 163064 bytes total, 113792 bytes largest free block
Total PSRAM used: 4264 bytes total, 4189488 PSRAM bytes free
=> bad

[cd5257ad78f0d62d28afaa579257904a767842d9] ESP.getCpuFreqMHz fix (#3007)
Heap/32-bit Memory Available     : 273724 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 231144 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free
Configuring access point...
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 13 - AP_START
WIFI AP Started. IP Address: 192.168.4.1
After Wifi/Before SPIFFS/FFat
Heap/32-bit Memory Available     : 214820 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 172240 bytes total, 113792 bytes largest free block
Total PSRAM used: 3760 bytes total, 4190276 PSRAM bytes free
=> good

[70a896481d978f02567c32350768476ad2d6bb9b] Update Both CIs to run parallel jobs (#3120)
Heap/32-bit Memory Available     : 274352 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 231136 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free
Configuring access point...
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 14 - AP_STOP
WIFI AP Started. IP Address: 192.168.4.1
After Wifi/Before SPIFFS/FFat
Heap/32-bit Memory Available     : 206284 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 163068 bytes total, 113792 bytes largest free block
Total PSRAM used: 4264 bytes total, 4189488 PSRAM bytes fre
=> bad

[f4acac4c2bf83d76f49241489e24fc1d6bbb64e7] Bugfix/http client (#2867)
Heap/32-bit Memory Available     : 273580 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 231000 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free
Configuring access point...
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 13 - AP_START
WIFI AP Started. IP Address: 192.168.4.1
After Wifi/Before SPIFFS/FFat
Heap/32-bit Memory Available     : 214672 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 172092 bytes total, 113792 bytes largest free block
Total PSRAM used: 3760 bytes total, 4190276 PSRAM bytes fre
=> good

[390da0d09080ac6689a6337f0085f4cd2dc3ffb9] bump CONFIG_ESP32_WIFI_STATIC_RX_BUFFER_NUM to improve RX performance (#3119)
Heap/32-bit Memory Available     : 274208 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 230992 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free
Configuring access point...
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 14 - AP_STOP
WIFI AP Started. IP Address: 192.168.4.1
After Wifi/Before SPIFFS/FFat
Heap/32-bit Memory Available     : 206412 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 163196 bytes total, 113792 bytes largest free block
Total PSRAM used: 4264 bytes total, 4189488 PSRAM bytes fre
=> bad

[d5e2bb12ca02ae9066e9dad84d9dbf268aca6fa3] Update IDF to 90747cc8b (#3118)
Heap/32-bit Memory Available     : 274352 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 231136 bytes total, 113792 bytes largest free block
Total PSRAM used: 0 bytes total, 4194252 PSRAM bytes free
Configuring access point...
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 14 - AP_STOP
WIFI AP Started. IP Address: 192.168.4.1
After Wifi/Before SPIFFS/FFat
Heap/32-bit Memory Available     : 216224 bytes total, 113792 bytes largest free block
8-bit/malloc/DMA Memory Available: 173008 bytes total, 113792 bytes largest free block
Total PSRAM used: 4260 bytes total, 4189488 PSRAM bytes fre
=> good

sauron [mc]$ git bisect good
390da0d09080ac6689a6337f0085f4cd2dc3ffb9 is the first bad commit
commit 390da0d09080ac6689a6337f0085f4cd2dc3ffb9
Author: Me No Dev <[email protected]>
Date:   Tue Aug 20 22:07:56 2019 +0300

    bump CONFIG_ESP32_WIFI_STATIC_RX_BUFFER_NUM to improve RX performance (#3119)

 tools/sdk/include/config/sdkconfig.h   |   2 +-

[atanisoft]

If you want control over this you will need to move to Arduino as a component and configure IDF however you want.

[marcmerlin]

@atanisoft yeah, that's what @me-no-dev told me on gitter.
Still, a 55KB growth (almost 1/3rd of the mere 160KB that are available for static arrays) is a pretty terrible growth for arduino-esp32 if you use wifi, but he said it wasn't practical to make ESP32 support more than one wifi stack in the arduino environment (like is available for esp8266), and indeed to switch to esp32-idf to be able to turn off the bloat I don't need.
In the short time I did a git checkout 595c95f231ad746155a9b4a99ed4f1396cd481a5 in the arduino-esp32 checkout, and I'm back to pre-bloat and got my memory back.

Given what he said, this is not going to get fixed, so I'll close it.