Sporadically slow MQTT reaction when using ESP32 (ESP8266 works OK)

[guydvir2]

Few days ago, I posted this question in Arduino stackExchange- but no one answered.
I'll be happy to help and get help
//////////////////////////////////////////////////////////////////////////////////////

Problem description: Every ~20+ published messages (using terminal), MCU receives that message in a very noticeable delay (mostly ~10 sec, few time it got up to 1 min). Upon receive (printed in Serial monitor), MCU is working as expected.

Hardware: ESP32 DEVKIT MCU, PubSubClient.h for MQTT and WiFi.h for WiFi.

Important remark: Code is same for ESP8266 and ESP32.This behavior happens only on ESP32.

Question: Does ESP32 has a know issue regarding receiving MQTT messages? is there a workaround? or perhaps- bug in code?

What did I do to try isolating the problem:

1. Tried on 3 different ESP32s (2 DEVkit MCU, and 1 with relays on board) - same behavior - it is not an specific MCU hardware problem.

2. Used another PC to publish MQTT messages (via terminal. MAC and Linux, and a 3rd party MQTT App)- no change. It has nothing to do with publishing platform.

3. To rule out it may be a MQTT broker's fault (Local RPI3 connected using LAN cable)- Subscribing to that topic in order to see if the delay repeats . MQTT server receive pubs on time. Broker responds on time (while waiting to see when publish is received in Serial monitor).

4. adding a flashing LED every 200ms in loop(), to verify that MCU is not get stuck in a process. loop() is looping as expected (see in code below).

5. Spitting heap free memory to Serial monitor, in order to see if there is some memory issue, degradation. Nothing suspicious.

6. Code simplification- used code segments from ESP32 wifi and MQTT pubsub's example, while disabling every functionality of original code(with and without). Delay still happens sporadically.

OUTPUT1- sending "0" and "1" almost simultaneously, almost 15 sec delay

13:31:21.516 -> Message arrived [myHome/test] 0 <--- right after this, "1" was sent
13:31:22.510 -> 278616 <---- free heap size (send every 1000 ms. non-blocking)
13:31:23.570 -> 278616
13:31:24.630 -> 278616
13:31:25.657 -> 278616
13:31:26.718 -> 278616
13:31:27.778 -> 278616
13:31:28.805 -> 278616
13:31:29.866 -> 278616
13:31:30.926 -> 278616
13:31:31.954 -> 278616
13:31:33.014 -> 278616
13:31:34.075 -> 278616
13:31:35.102 -> 278616
13:31:35.964 -> Message arrived [myHome/test] 1 <--- received 15 sec later

OUTPUT-2: Code

#include <WiFi.h>
#include <PubSubClient.h>

WiFiClient espClient;
PubSubClient client(espClient);


const char *mqtt_server = "192.168.2.100";


void setup_wifi()
{

        delay(10);
        // We start by connecting to a WiFi network
        Serial.println();
        Serial.print("Connecting to ");
        // Serial.println(ssid);

        WiFi.mode(WIFI_STA);
        WiFi.begin("iot", "GdS");

        while (WiFi.status() != WL_CONNECTED)
        {
                delay(500);
                Serial.print(".");
        }

        Serial.println("");
        Serial.println("WiFi connected");
        Serial.println("IP address: ");
        Serial.println(WiFi.localIP());
}

void callback(char *topic, byte *payload, unsigned int length)
{
        Serial.print("Message arrived [");
        Serial.print(topic);
        Serial.print("] ");
        for (int i = 0; i < length; i++)
        {
                Serial.print((char)payload[i]);
        }
        Serial.println();

        // Switch on the LED if an 1 was received as first character
}

void reconnect()
{
        // Loop until we're reconnected
        while (!client.connected())
        {
                Serial.print("Attempting MQTT connection...");
                // Create a random client ID
                String clientId = "ESP8266Client-";
                clientId += String(random(0xffff), HEX);
                // Attempt to connect
                if (client.connect(clientId.c_str()),"guy","kupelu9e")
                {
                        Serial.println("connected");
                        // Once connected, publish an announcement...
                        client.publish("myHome/log", "hello world");
                        // ... and resubscribe
                        client.subscribe("myHome/test");
                }
                else
                {
                        Serial.print("failed, rc=");
                        Serial.print(client.state());
                        Serial.println(" try again in 5 seconds");
                        // Wait 5 seconds before retrying
                        delay(5000);
                }
        }
}

void setup()
{
        Serial.begin(115200);
        setup_wifi();
        client.setServer(mqtt_server, 1883);
        client.setCallback(callback);
        reconnect();
}
void loop()
{
        loop_buttons();
        if (!client.connected())
        {
                reconnect();
        }
        client.loop();
        static unsigned long lastentry = 0;
        static unsigned long lastentry2 = 0;
        if (millis() - lastentry > 200)
        {
                pinMode(2, OUTPUT);
                digitalWrite(2, !digitalRead(2));
                lastentry = millis();
        }
        delay(50);
        if (millis() - lastentry2 > 1000)
        {
                Serial.println(ESP.getFreeHeap());
                lastentry2 = millis();
        }
}

Originally posted by @guydvir2 in #6543 (comment)

[SuGlider]

@guydvir2 - Could you please test your project with this WiFi setup code, and then let mw know the results:

void setup_wifi()
{

        delay(10);
        // We start by connecting to a WiFi network
        Serial.println();
        Serial.print("Connecting to ");
        // Serial.println(ssid);

       WiFi.useStaticBuffers(true);   // <<=== This line ay change the may how WiFi perfroms...

        WiFi.mode(WIFI_STA);
        WiFi.begin("iot", "GdS");

        while (WiFi.status() != WL_CONNECTED)
        {
                delay(500);
                Serial.print(".");
        }

        Serial.println("");
        Serial.println("WiFi connected");
        Serial.println("IP address: ");
        Serial.println(WiFi.localIP());
}

[SuGlider]

WiFi.useStaticBuffers(true); must be called before WiFi.mode()
It will use WiFi Static Buffers and also increase the number of simultaneous connections to 8.
This may improve the MQTT performance. Please try it and let me know.

[SuGlider]

The drawback about using WiFi.useStaticBuffers(true); is that it will "eat" about 60K of HEAP on the application startup.

[guydvir2]

WiFi.useStaticBuffers(true); must be called before WiFi.mode() It will use WiFi Static Buffers and also increase the number of simultaneous connections to 8. This may improve the MQTT performance. Please try it and let me know.

thank you. I'm going to add it to code.

[guydvir2]

error: 'class WiFiClass' has no member named 'useStaticBuffers'
  WiFi.useStaticBuffers(true);
       ^

[SuGlider]

error: 'class WiFiClass' has no member named 'useStaticBuffers'
WiFi.useStaticBuffers(true);
^

Please use Arduino Core 2.0.3+ in order to have access to this feature.
What is the Arduino Core you are using?

[guydvir2]

@SuGlider
Where I can find this ? And how do I update core?

[SuGlider]

Do you use Arduino IDE? For that, just go to Menu->tools->board manager

[guydvir2]

@SuGlider
Funny thing: when I saw 1.0.4 was installed- I check path was https://dl.espressif.com/dl/package_esp32_index.json in my Arduino IDE.

In other tutorial, this path was to use https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json. This one has 2.0.4

Why is that ?

Anyway - Now I'll check

[guydvir2]

@SuGlider after few days of testing, adding WiFi.useStaticBuffers(true); as noted above - still from time a latency of ~10 sec may occur from time to time, but a longer latency as was before (could reach 60 sec), has not been witnessed again.

Appreciate any further help,
Guy

[SuGlider]

@guydvir2
OK, it is interesting. An improvement, but maybe not perfect.
The latency is not due to the MQTT server?
Or maybe some MQTT QoS setup... or Network issues from time to time...

I'd suggest to run the ESP32 in parallel with the ESP8266 to the same MQTT server and check if they behave the same.

[guydvir2]

@SuGlider

My MQTT server is a local RaspberryPi, connected using a LAN cable (reduce Wifi effects).
Some of my tests were during pinging constantly MQTT server to see if there any issues as you mentioned (using a terminal on a Linux machine). Smoking gun was not found, while for some time I tought there is.

To be on the safe side- I'm using now a designated MQTT server on a seperated network (same spec and connectivity), and my UUT is soley sending messages to that new MQTT server. I think this is the best I can do for now in that area.

QoS is 0.

Your idea is refreshing and very interesting, I'll add it to my test (ESP32 in parallel to ESP8266).

BTW - In my trials, I used MQTT explorer (a 3rd party MQTT application), which shows that message is reached and recieved as expected ( without any significant latency ), but not received in ESP32 ( monitoring Serial terminal which prints when msg is received).

I'll be happy to here some more ideas/ suggestions.

Guy

[guydvir2]

@SuGlider
As far as I can tell, after adding WiFi.useStaticBuffers(true) as noted, I didn't witness such behaviour as mentioned.
Thank you.

[hoeken]

I was having a somewhat similar issue to this, and this was one of the top results when searching. I have a WebSocket server that is receiving commands and they would sometimes lag, slowdown, or stutter. It was really frustrating.

This advice helped a little bit, but what really solved it for me was turning the sleep mode off. Here's the code I ended up with:

WiFi.setSleep(false); WiFi.useStaticBuffers(true); WiFi.mode(WIFI_STA);

[guydvir2]

Please explain what behaviour did you encounter.
I had some issues with bursts messages ( among other).

[hoeken]

I am making a board that controls some power mosfets and talks through a websocket server using the EspAsyncWebServer. When I was load testing it with lots of traffic it would bog down and die. I really don't know what the underlying issue is, but turning off the wifi sleep mode fixed it for me.