LITTLEFS 10 times slower than SPIFFS on ESP32

[fred9999]

Board

ESP32 Dev Module 4MB

Device Description

The ESP32 board info:
Board: "ESP32 Dev Module"
Upload Speed: "921600"
CPU Frequency: "240MHz (WiFi/BT"
Flash Frequency: "80MHz"
Flash Mode: "QIO"
Flash Size: "4MB (32Mb)"
Partition Scheme: "No OTA (2MB APP 2MB SPIFFS)"
Core Debug Level: "None"
PSRAM: "Disabled"
Arduino Runs On: Core1
Events Run On: Core1
Port: "COM9"

Hardware Configuration

no external devices

Version

v2.0.2

IDE Name

Arduino IDE 1.8.19

Operating System

Win10 64bit

Flash frequency

80MHz

PSRAM enabled

no

Upload speed

115200

Description

Hi,

I initially posted it on forum.arduino.cc but got no replies there. Maybe that was the wrong place to post it.

I have a project in mind where I have to write sensor data into file in ESP32 flash memory (yes, I am aware of 10000 writes limit). As I understood SPIFFS is deprecated and LittleFS would be the way to go.

I decided to do a little speed test first. I uploaded the Arduino IDE ESP32 LITTLEFS_test.ino sketch onto the regular ESP32 Dev Module and added a couple of lines of code into loop to measure the append time of text "qwerty" to the file data.txt on ESP32 file system. I uploaded the file via ESP32 Sketch Data Upload tool with LittleFS, that seems to format the flash for using LittleFS.

To my great surprise the append function run time was 20-30ms in the beginning and when the data.txt file grew to a few hundred bytes it became 50-70ms. And there is very odd behavior - every 15th time the append operation takes 250-350ms. That does not depend on the append function call period. Extra long append time occurs every 15th time would the call period be 1, 2 or 3 seconds.

When I run the same script modified to use SPIFFS the append function run time is steady 5ms.

I tested the sketch on 2 different boards with the same results. I also uninstalled ESP32 boards from Arduino IDE and reinstalled. No difference.

Thank you in advance to get the LittleFS working.

Sketch

#include "FS.h"
#include "LittleFS.h"

#define FORMAT_LITTLEFS_IF_FAILED true

unsigned long lastAppendMillis = 0; // append function call period, ms
unsigned long beginAppendMillis = 0; // append function start time, ms
unsigned long appendMillis = 0; // append function end time, ms

void setup(){
    Serial.begin(115200);

    if(!LittleFS.begin(FORMAT_LITTLEFS_IF_FAILED)){
        Serial.println("LittleFS Mount Failed");
        return;
    }

    Serial.println("File system info:"); 
    Serial.print("  Total space:      ");
    Serial.print(LittleFS.totalBytes());
    Serial.println("byte");
 
    Serial.print("  Total space used: ");
    Serial.print(LittleFS.usedBytes());
    Serial.println("byte"); 
    Serial.println();
 
    listDir(LittleFS, "/", 3);
}

void loop(){

	if (millis() - lastAppendMillis > 1000) { 
		lastAppendMillis = millis();

		Serial.println("###########################");
		Serial.println("Append qwerty");

		beginAppendMillis = millis(); 

		appendFile(LittleFS, "/data.txt", "qwerty\r\n");	

		appendMillis = millis() - beginAppendMillis;
		Serial.print("appendMillis = ");
		Serial.print(appendMillis);	
		Serial.println(" ms");
	}
}

// LittleFS functions
void listDir(fs::FS &fs, const char * dirname, uint8_t levels){
    Serial.printf("Listing directory: %s\r\n", dirname);

    File root = fs.open(dirname);
    if(!root){
        Serial.println("- failed to open directory");
        return;
    }
    if(!root.isDirectory()){
        Serial.println(" - not a directory");
        return;
    }

    File file = root.openNextFile();
    while(file){
        if(file.isDirectory()){
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if(levels){
                listDir(fs, file.path(), levels -1);
            }
        } else {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("\tSIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void appendFile(fs::FS &fs, const char * path, const char * message){ 
    Serial.printf("Appending to file: %s\r\n", path);
    File file = fs.open(path, FILE_APPEND);
    if(!file){
        Serial.println("- failed to open file for appending");
        return;
    }
    if(file.print(message)){
        Serial.println("- message appended");
    } else {
        Serial.println("- append failed");
    }
    file.close();
}

Debug Message

no special debug messages

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[atanisoft]

@lorol Any ideas?

[bxparks]

I can confirm that my application sees a 10X-15X slower performance of LittleFS on the ESP32 compared to the ESP8266.

I created a minimal reproducible example (see below) which shows roughly:

• 2X slower File::read() performance on ESP32 compared to ESP8266
• 6-10X slower File::seek() performance on the ESP32 compared to ESP8266

Minimal Reproducible Example

#include <Arduino.h>

#include <LittleFS.h>
#define FILE_SYSTEM LittleFS

//-----------------------------------------------------------------------------

void createFile(const char* fileName, uint32_t fileSize) {
  unsigned long startMillis = millis();

  File f = FILE_SYSTEM.open(fileName, "w");
  for (uint32_t i = 0; i < fileSize; i++) {
    f.write((uint8_t) i);
  }
  f.close();

  unsigned long elapsedMillis = millis() - startMillis;
  Serial.printf("%-10s(%-11s, %6lu): %6lu millis\n",
      "createFile", fileName, (unsigned long) fileSize, elapsedMillis);
}

void readFile(const char* fileName, uint32_t fileSize) {
  unsigned long startMillis = millis();

  File f = FILE_SYSTEM.open(fileName, "r");
  for (uint32_t i = 0; i < fileSize; i++) {
    uint8_t c = f.read();
    if (c != (uint8_t) i) {
      Serial.printf("Unexpected byte at index %lu\n", (unsigned long) i);
    }
  }
  f.close();

  unsigned long elapsedMillis = millis() - startMillis;
  Serial.printf("%-10s(%-11s, %6lu): %6lu millis\n",
      "readFile", fileName, (unsigned long) fileSize, elapsedMillis);
}

void seekFile(
    const char* fileName,
    uint32_t fileSize,
    uint16_t numSeeks,
    uint32_t readSize
) {
  unsigned long startMillis = millis();

  File f = FILE_SYSTEM.open(fileName, "r");
  for (uint16_t i = 0; i < numSeeks; i++) {
    uint32_t offset = random(fileSize);
    f.seek(offset);
    for (uint32_t i = offset; i < offset + readSize && i < fileSize; i++) {
      uint8_t c = f.read();
      if (c != (uint8_t) i) {
        Serial.printf("Unexpected byte at index %lu\n", (unsigned long) i);
      }
    }
  }
  f.close();

  unsigned long elapsedMillis = millis() - startMillis;
  Serial.printf("%-10s(%-11s, %4u, %4lu): %6lu millis\n",
      "seekFile", fileName, (unsigned) numSeeks, (unsigned long) readSize,
      elapsedMillis);
}

//-----------------------------------------------------------------------------

void setup() {
  delay(1000);
  Serial.begin(115200);

  Serial.println(F("==== Initializing file system."));
  if (! FILE_SYSTEM.begin()) {
    Serial.println(F("File system failed."));
    exit(1);
  }

  createFile("/file0.txt", 1000);
  createFile("/file1.txt", 2000);
  createFile("/file2.txt", 4000);
  createFile("/file3.txt", 8000);
  createFile("/file4.txt", 16000);
  createFile("/file5.txt", 32000);
  createFile("/file6.txt", 64000);

  readFile("/file0.txt", 1000);
  readFile("/file1.txt", 2000);
  readFile("/file2.txt", 4000);
  readFile("/file3.txt", 8000);
  readFile("/file4.txt", 16000);
  readFile("/file5.txt", 32000);
  readFile("/file6.txt", 64000);

  seekFile("/file0.txt", 1000, 1000, 30);
  seekFile("/file1.txt", 2000, 1000, 30);
  seekFile("/file2.txt", 4000, 1000, 30);
  seekFile("/file3.txt", 8000, 1000, 30);
  seekFile("/file4.txt", 16000, 1000, 30);
  seekFile("/file5.txt", 32000, 1000, 30);
  seekFile("/file6.txt", 64000, 1000, 30);

  seekFile("/file0.txt", 1000, 1000, 100);
  seekFile("/file1.txt", 2000, 1000, 100);
  seekFile("/file2.txt", 4000, 1000, 100);
  seekFile("/file3.txt", 8000, 1000, 100);
  seekFile("/file4.txt", 16000, 1000, 100);
  seekFile("/file5.txt", 32000, 1000, 100);
  seekFile("/file6.txt", 64000, 1000, 100);

  Serial.println("==== Done");
}

void loop() {}

The output is:

ESP8266

==== Initializing file system.
createFile(/file0.txt ,   1000):    162 millis
createFile(/file1.txt ,   2000):    151 millis
createFile(/file2.txt ,   4000):    178 millis
createFile(/file3.txt ,   8000):    231 millis
createFile(/file4.txt ,  16000):    443 millis
createFile(/file5.txt ,  32000):    899 millis
createFile(/file6.txt ,  64000):   1832 millis
readFile  (/file0.txt ,   1000):     11 millis
readFile  (/file1.txt ,   2000):     15 millis
readFile  (/file2.txt ,   4000):     25 millis
readFile  (/file3.txt ,   8000):     43 millis
readFile  (/file4.txt ,  16000):     79 millis
readFile  (/file5.txt ,  32000):    152 millis
readFile  (/file6.txt ,  64000):    298 millis
seekFile  (/file0.txt , 1000,   30):    175 millis
seekFile  (/file1.txt , 1000,   30):    176 millis
seekFile  (/file2.txt , 1000,   30):    176 millis
seekFile  (/file3.txt , 1000,   30):    177 millis
seekFile  (/file4.txt , 1000,   30):    191 millis
seekFile  (/file5.txt , 1000,   30):    212 millis
seekFile  (/file6.txt , 1000,   30):    235 millis
seekFile  (/file0.txt , 1000,  100):    476 millis
seekFile  (/file1.txt , 1000,  100):    489 millis
seekFile  (/file2.txt , 1000,  100):    497 millis
seekFile  (/file3.txt , 1000,  100):    499 millis
seekFile  (/file4.txt , 1000,  100):    516 millis
seekFile  (/file5.txt , 1000,  100):    536 millis
seekFile  (/file6.txt , 1000,  100):    561 millis
==== Done

ESP32

==== Initializing file system.
createFile(/file0.txt ,   1000):    134 millis
createFile(/file1.txt ,   2000):     99 millis
createFile(/file2.txt ,   4000):    132 millis
createFile(/file3.txt ,   8000):    234 millis
createFile(/file4.txt ,  16000):    428 millis
createFile(/file5.txt ,  32000):   1496 millis
createFile(/file6.txt ,  64000):   1569 millis
readFile  (/file0.txt ,   1000):     25 millis
readFile  (/file1.txt ,   2000):     35 millis
readFile  (/file2.txt ,   4000):     55 millis
readFile  (/file3.txt ,   8000):     95 millis
readFile  (/file4.txt ,  16000):    175 millis
readFile  (/file5.txt ,  32000):    337 millis
readFile  (/file6.txt ,  64000):    661 millis
seekFile  (/file0.txt , 1000,   30):    312 millis
seekFile  (/file1.txt , 1000,   30):    316 millis
seekFile  (/file2.txt , 1000,   30):    318 millis
seekFile  (/file3.txt , 1000,   30):    899 millis
seekFile  (/file4.txt , 1000,   30):   1466 millis
seekFile  (/file5.txt , 1000,   30):   1970 millis
seekFile  (/file6.txt , 1000,   30):   2386 millis
seekFile  (/file0.txt , 1000,  100):    947 millis
seekFile  (/file1.txt , 1000,  100):    972 millis
seekFile  (/file2.txt , 1000,  100):    991 millis
seekFile  (/file3.txt , 1000,  100):   1587 millis
seekFile  (/file4.txt , 1000,  100):   2140 millis
seekFile  (/file5.txt , 1000,  100):   2632 millis
seekFile  (/file6.txt , 1000,  100):   3163 millis
==== Done

Environments

• ESP8266 Core 3.0.2
  • D1Mini
  • esp8266:esp8266:d1_mini:xtal=80,vt=flash,exception=disabled,ssl=all,eesz=4M2M,ip=lm2f,dbg=Disabled, lvl=None____,wipe=none,baud=921600
• ESP32 Core 2.0.2
  • ESP32 Dev Kit v1
  • esp32:esp32:esp32:PSRAM=disabled,PartitionScheme=default,CPUFreq=240,FlashMode=qio,FlashFreq=80,FlashSize=4M,UploadSpeed=921600,DebugLevel=none

[VojtechBartoska]

Hello, are you able to retest this with Arduino core ESP32 version 2.0.3-rc1? Thanks!

[VojtechBartoska]

Any updates?

[guestisp]

I'm experiencing exactly the same, with arduino-esp32 2.0.2 and 2.0.3

As workound, downgrading to arduino-esp32 1.0.6 and adding https://github.com/lorol/LITTLEFS LittleFS library seems to have fixed the performance issue.

[bxparks]

@VojtechBartoska: Who is the "you" that you are asking? In my case, for various reasons, I don't have the environment and hardware to re-run the example that I uploaded, at least not for a while. But I uploaded a complete reproducible example, so anyone else can.

[VojtechBartoska]

Adding this to our Roadmap.