SPIFFS support

[tijnkooijmans]

Could you share the status of SPIFFS support? In migrating our ESP8266 projects to ESP32 this is the one thing that is currently holding us up. Looking forward to an eta update.

All the best,
Tijn

[me-no-dev]

After I'm done with my current assignment, next is SPIFFS. FS is already implemented and SPIFFS will extend it the same way that SD and SD_MMC currently do, so you can maybe start from that and switch to SPIFFS when done.

[tijnkooijmans]

Thanks for that fast response! I tried the SD library and it works well on my dev board, but unfortunately the production HW doesn't allow for an SD card.

[bestpika]

I need SPIFFS to save config.

[bbx10]

@bestpika Take a look at the Preferences library. I used it to save config info.

[bestpika]

@bbx10 Can the library save array?

Oh, I can save json string and use ArduinoJson to parse it.

[gmag11]

Maybe this helps https://github.com/joysfera/esp-idf-spiffs

[Buffalchill]

realy need the SPIFFS on the ESP32 :-( SD works but its no option because to much hardware

[copercini]

Maybe this help until the official one https://github.com/copercini/arduino-esp32-SPIFFS

[lonerzzz]

@copercini captured your reference under projects

[lonerzzz]

@me-no-dev Is the SPIFFs work something that you are actively working? Do you need assistance, for development or testing or otherwise? I will try to capture the current state under projects if you have details.

[lonerzzz]

@copercini Can you elaborate on the "Known Issue" regarding "It doesn't support file upload yet." Are you referring to the ESP8266 Sketch Data Upload tool not supporting ESP32 or something else? Are you working to resolve the issue?

[copercini]

@lonerzzz

Are you referring to the ESP8266 Sketch Data Upload tool not supporting ESP32 or something else?

Exactly!
I'm without my board now, but next week I will take a look on this =)

[copercini]

Looks like it’s just add another "if" for ESP32 here which parses partitions CSV

And recompile mkspiffs with ESP32 headers from here

But @me-no-dev started the official implementation, so it’s best wait he definitive solution.

[pbecchi]

If "official implementation is started" do you have one expected release date?
If I do start using Copercini branch I will have to change my code later?

Thanks!

[bbx10]

An alternative to the Sketch Data Upload tool is to upload the files using HTTP POST. For example, load the FSBrowser example, then

$ curl -X POST -F "[email protected]"     http://<ESP32 IP address>/edit >/dev/null
$ curl -X POST -F "[email protected]"  http://<ESP32 IP address>/edit >/dev/null
$ curl -X POST -F "[email protected]"   http://<ESP32 IP address>/edit >/dev/null
$ curl -X POST -F "[email protected]"   http://<ESP32 IP address>/edit >/dev/null

[boneskull]

great success w/ @bbx10's FSBrowser example (here) and @copercini's SPIFFS implementation

[copercini]

@pbecchi It's implemented like me-no-dev comment above

FS is already implemented and SPIFFS will extend it the same way that SD and SD_MMC currently do, so you can maybe start from that and switch to SPIFFS when done.

So I think you will not need to change some or much code

---

@boneskull nice to hear it =)

[pbecchi]

Thanks
I already started working with Spiffs! It works perfect!

[lonerzzz]

@copercini Where do we stand with the implementation of the Sketch Data Upload tool for ESP32? Did you start this, is it deferred or in some other state. I ask because if there is no plan for this or you are busied out with all your other contributions, it is something I could work on. Just let me know any status details.
@me-no-dev Is this part of the work you are planning on SPIFFS?

[me-no-dev]

It is planned but depends on mkspiffs and it being on the same spiffs version ;) will get it done ;) ota already supports it

[copercini]

As I understand @me-no-dev will re-write SPIFFS things, so unofficial development is paused, but pretty stable =)

Will talk with @me-no-dev about mkspiffs, because I have something working here...

[tijnkooijmans]

Great, I'm using your lib now @copercini for the time being and it works well.

[me-no-dev]

SPIFFS is in ;)

[PhilippRott]

Thanks me-no-dev for your work on SPIFFS!
Tried your example code with following results:
Hardware: HIMALAYA-ESP32-Dev-Board
Arduino 1.8.2

ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0010,len:4
load:0x3fff0014,len:732
load:0x40078000,len:0
load:0x40078000,len:11572
entry 0x40078a14
E (180) SPIFFS: mount failed, -10025
SPIFFS Mount Failed

Any recommendations to fix this problem?

[bbx10]

@PhilippRott @me-no-dev Try adding true as the first parameter to begin(). This fixed the problem for me. The first parameter is formatOnFail so SPIFFS is formatted if the mount fails.

SPIFFS.begin(true);

@copercini Thanks for the unofficial SPIFFS library. It has worked without fail since you released it.

[copercini]

@bbx10 Thanks! you just made my day =)

[PhilippRott]

@bbx10 Thanks, worked for me perfect!

[trycoon]

Is Preference (https://github.com/espressif/arduino-esp32/blob/master/libraries/Preferences/src/Preferences.h) reading and writing to the same memory region as SPIFFS? Is there a memory map somewhere showing the different memory regions and there default sizes for the ESP32?

[me-no-dev]

No it does not. There is partition scheme https://github.com/espressif/arduino-esp32/blob/master/tools/partitions/default.csv (Preferences go to NVS)

[trycoon]

@me-no-dev Thanks! Looking at the partition scheme, it seems like the max program size is 81920 bytes (0x140000) since that whats fits into the ota-space. So my application is located in "ota_0" and when I update the program using OTA it will be uploaded to "ota_1" and then a pointer is changes to point to "ota_1" and the device is restarted? Next time it will upload to "ota_0" and re-point to ota_0, is that how it works?
Adding up the memory-spaces gives me 81920*2+8192+20480+4096+93952=290560 bytes (if I didn't miscalculate). But according to this there should be 1280KB flash, what is the rest used for? http://platformio.org/boards. Also why does platform.io list the devices as having 280KB RAM when this states 520KB, https://en.wikipedia.org/wiki/ESP32, does the operatingsystem use up 240KB by it self?

[me-no-dev]

0x140000 = 1310720B (1.25MB)

that 520 is used for many things ;) not available to you

[igrr]

Also why does platform.io list the devices as having 280KB RAM when this states 520KB

Just to expand on what @me-no-dev said: 520KB is the total amount of RAM. That includes IRAM (instruction RAM), DRAM (data RAM), and RTC memory. "System and memory" chapter of the ESP32 Technical reference manual has a table explains this in more detail. Specifically, there is 328 KB of DRAM available on the chip (the rest is IRAM and RTC RAM). Some of that is used by the OS, some is used by tasks created during SDK startup. An empty application which just prints the amount of available heap will report ~300KB of free heap, in the latest IDF.

Enabling features such as WiFi, BT, and dual core support, reduces the amount of memory available to applications. I guess this is where the 280KB number comes from.

[trycoon]

Ok, thanks for the clarification.

[kendo55]

@bbx10 Thanks, worked for me perfect, too!

[lonerzzz]

@me-no-dev @igrr What are we using as the equivalent of the file system uploader plugin for ESP32? If the update to the esp8266 file system uploader (https://github.com/esp8266/arduino-esp8266fs-plugin) is the intended solution, please add me to that project and I can make the changes.

[me-no-dev]

@lonerzzz https://github.com/me-no-dev/arduino-esp32fs-plugin :) mkspiffs works since friday :)

[beegee-tokyo]

@me-no-dev, @igrr
how can I use the plugin on PlatformIO?

[me-no-dev]

@beegee-tokyo I don't think that you can. PIO would need a custom implementation for that.

[beegee-tokyo]

@me-no-dev
Understand.
Found a solution to use it:

1. Create an empty.ino file inside the src folder of the PlatformIO project.
2. Create the data folder inside the src folder of the PlatformIO project.
3. Move the files that should be uploaded into this data folder
4. Start ArduinoIDE, select the empty.ino as sketch and then use Tools > ESP32 Sketch Data Upload to flash the SPIFFS image to the ESP32

;) :)

[lonerzzz]

@me-no-dev Perfect, I am now marking SPIFFS as done in the Project tracking area. Didn't realize the UI portion was also completed.

[me-no-dev]

@lonerzzz didn't say anything because mkspiffs was not working :) had to get that fixed first

[henricazottes]

@me-no-dev
Understand.
Found a solution to use it:

1. Create an empty.ino file inside the src folder of the PlatformIO project.

2. Create the data folder inside the src folder of the PlatformIO project.

3. Move the files that should be uploaded into this data folder

4. Start ArduinoIDE, select the empty.ino as sketch and then use **`Tools > ESP32 Sketch Data Upload`** to flash the SPIFFS image to the ESP32

;) :)

For records: https://docs.platformio.org/en/latest/platforms/espressif32.html#uploading-files-to-file-system-spiffs