FLASH_SIZE problem > update error

[jarda195]

I use ESP WROOM 02 and I have this problem with ESP8266httpUpdate.. First 20 pcs ESP WROOM 02 working ok.. my sketch has 320.000 byte, update ok.. I purchased next 100 pcs but update with error 'bin for wrong flash size'.. restart and update not working.. for first 20 pcs function getFlashChipRealSize returned 4.000.000 and new 100pcs returned 2.080.000 only.. ESP Wroom 02 is same, maybe not same production serie. (I assume - 4.000.000 /8/1024 = 488 kB and 2.080.00/8/1024 = 253kB) . Function system_get_flash_size_map(); returned FLASH_SIZE_32M_MAP_512_512 (both)

FlashId = 1392840 not working
FlashId = 1458337 workong

My sketch (on Arduino 1.65 and W8) is 95% sleep and 5 % sending 30 byte...
after compiling: 311 263 byte (29%) max is 1 044 464 byte. (ESPDuino )

1. How I can change this value ? (from 2.080.000 to 4.000.000)
2. Why is differenc between two ESP WROOM 02 and ESP WROOM 02 ? :(

SDK ( 1.5.3(aec24ac9) both) over function sdk = system_get_sdk_version();

Thank for responde..

[vicnevicne]

@jarda195 this is not the right place to ask for help for your particular project or module. This is an issue tracker, meant for tracking issues in the core.
Please refer to esp8266.com or stackoverflow for assistance.

[devyte]

@vicnevicne actually, I suspect this one really is a core issue, related to autodetection of the flash size. I have seen more than one issue of this type now.
@jarda195 I somehow doubt it will help, but to be thorough: please try with latest git.
@igrr do you know who wrote that part of the code? It seems the autodetection rules need to be changed.

[davisonja]

@devyte It's historically appeared because get_flash_size uses some assumptions about the format of the flash ID which doesn't always hold true for all flash vendors.
Indeed, the rule is a simple bit shift, when using getFlashChipRealSize

[devyte]

Alright, let's use this issue to cover the problem. I'll close the others that seem to be duplicates, and reference this one.
@davisonja if the current rule doesn't hold for all vendors, then we need a new uber-rule of sorts. As things stand, it seems like ESPs that don't hold to the current rule aren't usable.
If necessary, a per-vendor rule table will need to be maintained, even though that would likely get ugly.
Unless there is some other way to get the flash chip size?

[igrr]

@jarda195 could you please use esptool.py flash_id command to check flash chip device id on both types of modules? Thanks.

[devyte]

per #3618:

ESP WROOM 02 flash vendor: A1h : FM (32MB) and ESP WROOM 02 flash vendor: C8h : GD (16MB)

[davisonja]

The last time I looked at this, the existing code appeared to be using a sensible default based on a common standard usage. It's come up before, and there are references to existing discussions in other projects that have had the same issue with flash ID formats not necessarily being consistent. The end conclusion seems to be there isn't a good answer, aside from a table that gets built up over time, but the existing rule (using the third byte as X in `1 << X`) works the majority of the time. If we can get the result of `spi_flash_get_id` I'm happy to build a table :)

…

On Mon, Sep 18, 2017 at 11:06 AM, Ivan Grokhotkov ***@***.***> wrote: @jarda195 <https://github.com/jarda195> could you please use esptool.py flash_id command to check flash chip device id on both types of modules? Thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3616 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAN_A579Asp0CT5FpMpNDEkp9zqACG6fks5sjaYPgaJpZM4PaG1j> .

[devyte]

@davisonja Something I noticed, not sure how usefule it may be:

returned 2.080.000

As afar as I know, flash chips don't come in fractionary numbers, but in powers of 2, as in 512KB, 1MB, 2MB, 4MB, 8MB, 16MB, 32MB. A calculated flash size like 2.080.000 is obviously invalid. This info could be used to assert or at least a warning.

[davisonja]

@devyte I translated that into 2,080,000 as in a number using '.'s as thousands separators rather than ','

[devyte]

@davisonja it seems that the flash detection thing is getting worse very quickly. Are you available to help out with it?

[ondabeach]

Arduino 1.8.5
Core 2.4

Several different esp-01 makers from several Chinese suppliers are all using this PUYA 4Mbit flash chip:
http://www.puyasemi.com/attached/file/20171027/20171027183954_35795.pdf

Out of the last 100 esp-01's I've received in the last couple of weeks, only 18 had chips other than the PUYA. So it seems likely that lots more modules will be appearing with this flash chip family on-board.

The "Upload Sketch Data" plug in works. It can write to these flash chips no problem. A sketch can then read the data, but sketches can not write to SPIFFS.

How is it that the plug in can write to these chips but a sketch can't???

[igrr]

Could you please run esptool.py flash_id command on the board which has this issue?

[ondabeach]

I have the flash ID from a Get_Flash_Data sketch. If the below doesn't have what you need I can run up esptool. Just let me know.

Flash chip information:
Flash chip Id: 00146085 (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))
Sketch thinks Flash RAM is size: 0.50 MB
Actual size based on chip Id: 1.00 MB ... given by (2^( "Device" - 1) / 8 / 1024
Flash frequency: 40.00 MHz
Flash write mode: QIO

[weswitt]

I'm seeing something similar as I documented in #4061.

What I see is that after flashing a SPIFFS and then running a sketch, the sketch does not find any files in the FS. Below is the output from a sketch that dumps & tests the flash/SPIFFS (see #4061). Flash ID included below.

#####################

---

Firmware:
Chip Id: 0042A910
Core version: 2_4_0
SDK version: 2.1.0(deb1901)
Boot version: 31
Boot mode: 1

---

Flash chip information:
Flash chip Id: 001440E0 (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))
Sketch thinks Flash RAM is size: 0.50 MB
Actual size based on chip Id: 1.00 MB ... given by (2^( "Device" - 1) / 8 / 1024
Flash frequency: 40.00 MHz
Flash write mode: DIO

---

File system (SPIFFS):
Total KB: 51.72 KB
Used KB: 0.00 KB
Block size: 4096
Page size: 256
Maximum open files: 5
Maximum path length: 32

SPIFFS directory {/} :

---

CPU frequency: 80 MHz

#####################file open failed
====== Reading from SPIFFS file =======
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:

[ondabeach]

@weswitt I just replied on #4061. @igrr has the ChipID of the new flash chips that is an issue and he or others are looking into a solution. I'll help you as far as I'm able over on #4061, it'll save cluttering this issue for now.

[igrr]

@ondabeach

Several different esp-01 makers from several Chinese suppliers are all using this PUYA 4Mbit flash chip:
http://www.puyasemi.com/attached/file/20171027/20171027183954_35795.pdf

The linked datasheet is for an 8 Mbit part. I'm a bit confused, what is the real flash size of the chip you have on these new boards?

[ondabeach]

@igrr Sorry Ivan, that was a typo, my bad ;) The chips on all 75 odd boards are 8Mbit chips as per the datasheet. I don't think it's a size issue though. I think there's something different about this PUYA family of chips.

What I find strange is that the Arduino (1.8.0~1.8.5) Sketch Data Upload plugin can write to the PUYA chip no problem but the SPIFFS.println function cannot. This is way above my pay grade but maybe the former writes in bigger blocks than the latter. One thing I spotted in the data sheet section on the configure register. There's a DP bit that sets the page size to 256 or 512.

I'm grabbing at straws here, but my business is hanging by a thread as I've got orders coming in and I'm down to my last few modules with BergMicro flash chips on-board.

[ondabeach]

@igrr If it would help at all I could overnight post one of these modules to Roger Clarke in Melbourne. I'm sure you and he could speak the same language and get to the bottom of this in a flash (pun intended :)

[igrr]

Ok, thanks for the clarification.
I will check tomorrow if we have any P25Q80H chips in the office. If not, will order a few to do a test.

In the meantime, please run the following sketch on a board with this PUYA chip, and share the full output you get. Note that this sketch will also format SPIFFS partition. Before uploading it, please go to Arduino IDE Tools menu, and select Debug Port: Serial, Debug Level: CORE.

https://gist.github.com/igrr/964b0d139ce63fcee07c8c1f1facb24d

[igrr]

By the way, @ondabeach, let's continue with your issue in #4061. I think the issue you are seeing is not related to flash chips size detection.

[ondabeach]

Thanks @igrr, I must admit I was a bit surprised when the write fail issue was merged with this one. I'll load that sketch and report the output back on #4061.

[devyte]

Apologies for linking that one. Since I started combing through the issues, the only reasons for flash problems I have seen, where one manufacturer works and another doesn't, is the autodetection rules, and misconfiguration (e.g. QIO set when the pins are in use), and this didn't sound like the latter.

[ondabeach]

No problem @devyte, I know you guys are multitasking to the max. This is the first time I've had an issue I couldn't resolve with my usual brute force hacker approach. Any and all assistance is greatly appreciated. Being a one man band doesn't give me the time to get in under the hood and see what makes it all tick. I'm dying to know how the "Sketch Data Upload" can write to this chip when the SPIFFS.??? functions can't. I don't even know where to start looking. Anyhow, @igrr 's got the bit between his teeth so I'm confident of a resolution sooner rather than later.

[benwallacestock]

Has a solution been found to this issue yet as I have 4 ESP chis with these memory modules and was hoping to use them for a project?

[squigley]

Yes, it has. See this other bug: #4061

There's a patch for Esp.CPP which implements a different write procedure for these chips.

[earlephilhower]

The whole PUYA broken flash was fixed by @TD-er with #5493 . See that issue for the steps you need to take after updating to the latest builds. Closing.