Add support for 8 and 16 MB Flash

[me-no-dev]

Add to core dev config initially
Will add to generic if it works
Requires updated esptool

[codecov-io]

Current coverage is 27.62% (diff: 100%)

Merging #2351 into master will not change coverage

@@             master      #2351   diff @@
==========================================
  Files            20         20          
  Lines          3656       3656          
  Methods         335        335          
  Messages          0          0          
  Branches        678        678          
==========================================
  Hits           1010       1010          
  Misses         2468       2468          
  Partials        178        178          

Powered by Codecov. Last update b7c7bc0...dbfcac1

[pieman64]

@me-no-dev and @igrr are 16M ESP devices now supported with this commit?

[me-no-dev]

not really in the way that you would expect :(
best is to use the 4M option then map the rest as another SPIFFS partition or something else.

[pieman64]

Thanks @me-no-dev , so when our 16M devices arrive we can at least use them.
Do you happen to have a link to any SPIFFS partitioning instructions?

[martinayotte]

I've received my Wemos D1 Mini Pro, but SPIFFS and OTA are failing if I use 16MB option.
I think there is problem with MagicID in Esp.cpp, but it didn't fix the issue.

[martinayotte]

Using the hack details here http://www.packom.org/esp8266/16mb/flash/eeprom/2016/10/14/esp8266-16mbyte-flash_handling.html, I was able to get SPIFFS working on 16MB, but there still issues : SDK still try to access WiFi configs like 4MB, so I had to move SPIFFS_START after the first 4MB, with 12MB remaining. Also, I still face issue with OTA, when e-boot is call, I get "need boot 1.4+", and hangs forever until doing a erase_flash with esptool.py.

[martinayotte]

Ok ! I took time to investigate a bit more : to avoid the issue with OTA, this OTA still need to be done in low space location, not just before the SPIFFS_START. So, I added a check in Updater.cpp in the begin() method :

    //address of the end of the space available for sketch and update
    uint32_t updateEndAddress = (uint32_t)&_SPIFFS_start - 0x40200000;
+    if (updateEndAddress > 0x100000) updateEndAddress = 0x100000;
    //size of the update rounded to a sector

So, it seems to work, I've now a 12MB partition, and OTA still working. Of course, I'm loosing the space between addresses 0x100000 and 0x400000, but it can still be use using raw EspClass::flashWrite/EspClass::flashRead for other purposes than SPIFFS.

[BobLynas]

@martinayotte, this is absolutely brilliant information ! and nearly precisely what I am looking for. I need to be using those cute little ESP-01S devices (like the diddy ESP01, but with 8MB of Flash). Can I please ask a huge favour (and maybe it may benefit a few others too) what values do I need to use the 8MB rather than the 16MB as you have documented ? and do I need to change anything in my boards.txt as it only accounts for 4MB devices ? Best of Regards, and thanks again

[martinayotte]

Some additional comments can be found here :
platformio/platform-espressif8266#8 (comment)
In your case, the _SPIFFS_end would be 0x40A00000 for 8MB.

[BobLynas]

@martinayotte - huge thank you, I shall collate all the info and carefully make the changes, test, then come back and let you know how it all went :)

[BobLynas]

@martinayotte , I am having a look at this today. I started by locating the boards.txt file and then looking at what you/previous posters were trying to do, and made this
generic.menu.FlashSize.16M12M=16M (12M SPIFFS)
generic.menu.FlashSize.4M3M.build.flash_size=1M
generic.menu.FlashSize.4M3M.build.flash_ld=eagle.flash.16m.ld
generic.menu.FlashSize.4M3M.build.spiffs_start=0x300000
generic.menu.FlashSize.4M3M.build.spiffs_end=0x17FB00
generic.menu.FlashSize.4M3M.build.spiffs_blocksize=8192
generic.menu.FlashSize.4M3M.upload.maximum_size=1044464

One thing that struck me (and probably the bit I am not getting) is why is spiffs_start > spiffs_end ?
I was wondering if it was a typo, missing a zero perhaps

Then I was looking at your comment for "_SPIFFS_end would be 0x40A00000 for 8MB" and realised that this must somehow be relating to "eagle.flash.xxm.ld" information, right ?

I'm sorry i'm not quite as up to speed as everyone else on this, but I really would love to solve this 8MB issue for the ESP01S boards, even to the point of paying for a proven solution

Thanks

[martinayotte]

First, your menu structure is broken, mixing 16M12M and 4M3M, all entries should have 16M12M.
Second spiffs_start/spiffs_end entries are only used for upload SPIFFS, not for code itself, and spiffs_end is mistakingly set here with one zero is missing, it should be 0x17FB000, but even then, it is a wrong value, it should have been 0x0FFB000 for 16MB. (0x17FB000 would be for 24MB chip which doesn't exist...)
Since we leave the WiFi config below SPIFF, it can be the full size, meaning 0x1000000, ie: 12MB.
"maximum_size=" needs to be tweak too with value of 12M, so 12582912.
The code itself relies on linker script flash_ld=eagle.flash.16m.ld, so make sure that the file .arduino15/packages/esp8266/hardware/esp8266/2.3.0/tools/sdk/ld/eagle.flash.16m.ld contains proper numbers. Mine are :

PROVIDE ( _SPIFFS_start = 0x40600000 );
PROVIDE ( _SPIFFS_end = 0x41200000 );
PROVIDE ( _SPIFFS_page = 0x100 );
PROVIDE ( _SPIFFS_block = 0x2000 );

EDIT : Oupps ! We should leave "maximum_size=1044464" since this is maximum code size ...

[davisonja]

I've started a script that will automate the generation of these files based on some defined values - at least in part to ensure that the menu items match the linker scripts with only a single point of change. Adding new flash sizes is exactly the kind of thing that's a little more involved than it really needs to be. It's background work for a change to OTA that survives a power-failure mid-update. Part of that process uses a 'page' of flash to store the parameters (which are otherwise in RTC memory). I'd also be keen to get OTA updates into SPIFFS so we can have our program memory back, but that's another thread entirely :) I don't have any larger-flash boards, but if I can pick one up somewhere I'm happy to make it go, just not sure how long it might take to get to me in NZ :)

…

On Sun, Jul 23, 2017 at 4:45 AM, martinayotte ***@***.***> wrote: First, your menu structure is broken, mixing 16M12M and 4M3M, all entries should have 16M12M. Second spiffs_start/spiffs_end entries are only used for upload SPIFFS, not for code itself. The code itself relies on linker script flash_ld=eagle.flash.16m.ld, so make sure that the file .arduino15/packages/esp8266/ hardware/esp8266/2.3.0/tools/sdk/ld/eagle.flash.16m.ld contains proper numbers. Mine are : PROVIDE ( _SPIFFS_start = 0x40600000 ); PROVIDE ( _SPIFFS_end = 0x41200000 ); PROVIDE ( _SPIFFS_page = 0x100 ); PROVIDE ( _SPIFFS_block = 0x2000 ); — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2351 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAN_A2dhgbYJZPlEt2-C8Y_otl1Ufhhzks5sQicPgaJpZM4JZDvy> .