MQTT transmission time

[savagerex]

Board

ESP32 S3

Device Description

ESP32-S3-DevKitc-1

Hardware Configuration

NONE

Version

v2.0.5

IDE Name

Arduino IDE

Operating System

Windows 10

Flash frequency

80Mhz

PSRAM enabled

yes

Upload speed

921600

Description

I use MQTT lib to transfer data ( '0').

it spend about 5 sec. this is too long.

i test on PC , it need 142ms.

why ESP32 need spend 5 sec???

Sketch

#include <WiFi.h>
#include <MQTT.h>
#include "mqtt_client.h"


const char* ssid = "vivo Y21";  //SW1-WIFI6
const char* pass = "1234567890";   //wireless

const char* mqtt_server = "broker.emqx.io";   //broker.emqx.io  192.168.99.124   192.168.76.176
const char *topic = "DataTopic";
const char *mqtt_username = "emqx";
const char *mqtt_password = "public";

WiFiClient net;
MQTTClient client(75000);

uint32_t i, j, k, m;
uint8_t ui8txLogBuf[75000];

#define PACKAGE 72000

unsigned long StartTime;
unsigned long EndTime;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);

  delay(10);
    WiFi.begin(ssid,pass);

    Serial.print("Connecting.");
    while(WiFi.status() != WL_CONNECTED) {
        Serial.print(".");
        delay(500);
    }

    Serial.print("WiFi connected - IP address: ");
    Serial.println(WiFi.localIP());

    client.begin(mqtt_server, net);

    Serial.print("\nconnecting...");
  while (!client.connect("arduino", "public", "public")) {
    Serial.print(".");
    delay(1000);
  }

  Serial.println("\nconnected!");
    
}

void loop() {
  // put your main code here, to run repeatedly:


  for (i = 0; i < 9; i++) {
      ui8txLogBuf[j] = '0';
      ui8txLogBuf[j + 1] = '0';
      j += 2;
    }

    if(j == PACKAGE) {

               if(m == 0) {
               StartTime = millis();
               }
      
               ui8txLogBuf[j] = '\n';
               ui8txLogBuf[j+1] = '\r';
               //client.write(ui8txLogBuf, PACKAGE+2);
               char *DataBuf = (char*)ui8txLogBuf;
               //client.publish(topic, ui8txLogBuf);
               
               //client.publish(topic, ui8txLogBuf, PACKAGE+2);
               client.publish(topic, DataBuf, PACKAGE+2, false, 0);

               //client.subscribe(topic);
               m += 1;
               j=0;
            }

        if (m==1){     //400
           EndTime = millis();
           Serial.print("**************wifi Transfer Time*************");
           Serial.println(EndTime-StartTime);
           m=0;
    }

}

Debug Message

NONE

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[VojtechBartoska]

Before @SuGlider will start investigating this, can you share your 2 cents on this @david-cermak? Thank you in advance

[SuGlider]

@savagerex - it may take some time to be investigated.
At a first glance, it would be necessary to check the response time from the MQTT server as well.

[SuGlider]

@savagerex - Could you please provide a link to the MQTT libraries used in the sketch of this issue? Thanks.

[savagerex]

link as below

https://github.com/256dpi/arduino-mqtt

[vkjambit]

i just stumbled here, maybe not a solution but i have some projects using mqtt without problems and i always used this library:
https://github.com/marvinroger/async-mqtt-client.git

maybe worth a try.

[SuGlider]

@savagerex - I've researched about a similar issue with MQTT that I had seen in the past:
#7183

Could you please try this new code for the setup() in your testing sketch?

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);

  delay(10);

  // this must be called before WiFi.begin()
  WiFi.useStaticBuffers(true);   // <<=== This line may change WiFi performance...
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, pass);

  Serial.print("Connecting.");
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print(".");
    delay(500);
  }

  Serial.print("WiFi connected - IP address: ");
  Serial.println(WiFi.localIP());

  client.begin(mqtt_server, net);

  Serial.print("\nconnecting...");
  while (!client.connect("arduino", "public", "public")) {
    Serial.print(".");
    delay(1000);
  }

  Serial.println("\nconnected!");

}

[yourchanges]

we face the same problem on esp32-s3-wroom-1 n8r8, we use the https://github.com/marvinroger/async-mqtt-client.git and we have aleady add the "WiFi.useStaticBuffers(true); WiFi.mode(WIFI_STA);" to receive the 156KB data from mqtt server, it uses about 2 seconds.

[bertmelis]

I worked on the async MQTT library and created my own MQTT library but never experienced such a lag. Not saying the bug isn't there but I cannot believe at this point the fault lies within the library.

In cases like this, Wireshark can be useful (dropped packets, retransmission...).

In the given example, you try to send 75k data. with a TCP packet size of 1500 this is 50 packets. This takes some time on an ESP32. There's no point in comparing this mcu with a PC.

[yourchanges]

when we use the esp-idf directly， we config the wifi parameters follow by https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/wifi.html#using-psram
···
WIFI_STATIC_RX_BUFFER_NUM 24
....
···
we can receive the 156KB data from mqtt server in 700ms in the same ENV.

[yourchanges]

Now we use the esp32-arduino-lib-builder with following config modified(in file configs/defconfig.esp32s3)

CONFIG_ESP32_WIFI_STATIC_RX_BUFFER_NUM=24
CONFIG_ESP32_WIFI_DYNAMIC_RX_BUFFER_NUM=85
CONFIG_ESP32_WIFI_STATIC_TX_BUFFER_NUM=32
CONFIG_ESP32_WIFI_RX_BA_WIN=32
CONFIG_ESP32_WIFI_IRAM_OPT=y
CONFIG_ESP32_WIFI_RX_IRAM_OPT=y
CONFIG_LWIP_TCP_SND_BUF_DEFAULT=32768
CONFIG_LWIP_TCP_WND_DEFAULT=32768
CONFIG_TCP_SND_BUF_DEFAULT=32768
CONFIG_TCP_WND_DEFAULT=32768

and then re-generate the esp32-arduino static libs and replace the precompiled libs in "~/.platformio/packages/framework-arduinoespressif32/tools"

At last, we can receive the 156KB data from mqtt server in 400ms in the same ENV in the arduino-esp32 project

Only one question for us, we should run the board as long as possible to check it stability or not?

[SuGlider]

Yes, this is a way to solve this same issue.

When using WiFi.useStaticBuffers(true);, that makes Arduino to use this CONFIG:

CONFIG_ESP32_WIFI_STATIC_RX_BUFFER_NUM=8
CONFIG_ESP32_WIFI_DYNAMIC_RX_BUFFER_NUM=32
CONFIG_ESP32_WIFI_STATIC_TX_BUFFER=y
CONFIG_ESP32_WIFI_TX_BUFFER_TYPE=0
CONFIG_ESP32_WIFI_STATIC_TX_BUFFER_NUM=8
CONFIG_ESP32_WIFI_CACHE_TX_BUFFER_NUM=16

Which is smaller that the one you suggested with Lib Builder.

Unfortunately, Arduino can't define its own WiFi low level configuration.

Another way to move forward with its own WiFi setup, would be going with Arduino as IDF Component and changing the sdkconfig using idf.py menuconfig (or even directly editing it)

[SuGlider]

@savagerex - Let me know if using WiFi.useStaticBuffers(true); had helped you.

If not, I think that the way to solve it, would be by moving to Arduino as Component Project and then setting high performance WiFi in the sdkconfig as suggested above.