Update esptool to version 4.2.1 #7127

me-no-dev · 2022-08-14T07:23:15Z

No description provided.

me-no-dev · 2022-08-14T07:44:43Z

@VojtechBartoska @SuGlider @P-R-O-C-H-Y please test this for building and flashing on all operating systems. We bumped the version quite a bit and also the layout for Linux has changed. Please report if there are still warnings after boot for mismatched image checksum.

P-R-O-C-H-Y · 2022-08-14T07:52:29Z

@me-no-dev After checkout this PR, I tried to get the Esptool by get.py command from tools.
I got this error printed in terminal:

prochy@Jan-MacBook-Pro tools % python3 get.py
System: Darwin, Bits: 64, Info: macOS-11.6.7-arm64-arm-64bit
Platform: x86_64-apple-darwin
Tool riscv32-esp-elf-gcc8_4_0-esp-2021r2-patch3-macos.tar.gz already downloaded
Extracting riscv32-esp-elf-gcc8_4_0-esp-2021r2-patch3-macos.tar.gz ...
Tool xtensa-esp32-elf-gcc8_4_0-esp-2021r2-patch3-macos.tar.gz already downloaded
Extracting xtensa-esp32-elf-gcc8_4_0-esp-2021r2-patch3-macos.tar.gz ...
Tool xtensa-esp32s2-elf-gcc8_4_0-esp-2021r2-patch3-macos.tar.gz already downloaded
Extracting xtensa-esp32s2-elf-gcc8_4_0-esp-2021r2-patch3-macos.tar.gz ...
Tool xtensa-esp32s3-elf-gcc8_4_0-esp-2021r2-patch3-macos.tar.gz already downloaded
Extracting xtensa-esp32s3-elf-gcc8_4_0-esp-2021r2-patch3-macos.tar.gz ...
Downloading esptool-4.2.1-macos.tar.gz ...
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 975, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1454, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1071, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1342, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/prochy/Documents/Arduino/hardware/espressif/esp32/tools/get.py", line 170, in <module>
    get_tool(tool)
  File "/Users/prochy/Documents/Arduino/hardware/espressif/esp32/tools/get.py", line 129, in get_tool
    urlretrieve(url, local_path, report_progress)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 241, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 519, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>

Jason2866 · 2022-08-14T15:14:52Z

@me-no-dev I would be surprised if the sha mismatch would be gone. Since there is no digest anymore there are FFFFF... and the MCU calculates a SHA from the current flash image. This can never match. The only way is to switch off (if possible!) the SHA calculation when starting.

VojtechBartoska · 2022-08-15T12:10:41Z

@PilnyTomas and @SuGlider Please help with testing under Windows and Linux.

SuGlider · 2022-08-15T20:26:05Z

@me-no-dev - I guess that the test is to check if ESPTOOL.exe 4.2.1 works with Windows:

I have downloaded https://github.com/espressif/arduino-esp32/releases/download/2.0.4/esptool-4.2.1-windows.zip
Copied it in the tools folder, replacing esptool.exe 3.3.0 -- on top of Core 2.0.4 installation

Tested it with ESP32-S3 - evething is OK.

Arduino IDE for Windows - Console OUPUT:

esptool.py v4.2.1
Creating esp32s3 image...
Merged 2 ELF sections
Successfully created esp32s3 image.
"C:\\Users\\user\\AppData\\Local\\Arduino15\\packages\\esp32\\hardware\\esp32\\2.0.4/tools/gen_esp32part.exe" -q "C:\\Users\\user\\AppData\\Local\\Temp\\arduino_build_541033/partitions.csv" "C:\\Users\\user\\AppData\\Local\\Temp\\arduino_build_541033/BareMinimum.ino.partitions.bin"
"C:\\Users\\user\\AppData\\Local\\Arduino15\\packages\\esp32\\tools\\xtensa-esp32s3-elf-gcc\\gcc8_4_0-esp-2021r2-patch3/bin/xtensa-esp32s3-elf-size" -A "C:\\Users\\user\\AppData\\Local\\Temp\\arduino_build_541033/BareMinimum.ino.elf"
Sketch uses 208649 bytes (15%) of program storage space. Maximum is 1310720 bytes.
Global variables use 12824 bytes (3%) of dynamic memory, leaving 314856 bytes for local variables. Maximum is 327680 bytes.
C:\Users\user\AppData\Local\Arduino15\packages\esp32\tools\esptool_py\3.3.0/esptool.exe --chip esp32s3 --port COM14 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 4MB 0x0 C:\Users\user\AppData\Local\Temp\arduino_build_541033/BareMinimum.ino.bootloader.bin 0x8000 C:\Users\user\AppData\Local\Temp\arduino_build_541033/BareMinimum.ino.partitions.bin 0xe000 C:\Users\user\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.4/tools/partitions/boot_app0.bin 0x10000 C:\Users\user\AppData\Local\Temp\arduino_build_541033/BareMinimum.ino.bin 
esptool.py v4.2.1
Serial port COM14
Connecting....
Chip is ESP32-S3
Features: WiFi, BLE
Crystal is 40MHz
MAC: 7c:df:a1:e0:10:18
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00003fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x0000e000 to 0x0000ffff...
Flash will be erased from 0x00010000 to 0x00043fff...
Flash params set to 0x022f
Compressed 15008 bytes to 10328...
Writing at 0x00000000... (100 %)
Wrote 15008 bytes (10328 compressed) at 0x00000000 in 0.3 seconds (effective 355.7 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 128...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (128 compressed) at 0x00008000 in 0.1 seconds (effective 430.7 kbit/s)...
Hash of data verified.
Compressed 8192 bytes to 47...
Writing at 0x0000e000... (100 %)
Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.1 seconds (effective 626.5 kbit/s)...
Hash of data verified.
Compressed 209040 bytes to 116579...
Writing at 0x00010000... (12 %)
Writing at 0x0001c174... (25 %)
Writing at 0x000216f6... (37 %)
Writing at 0x00026bc6... (50 %)
Writing at 0x0002c1a6... (62 %)
Writing at 0x00034839... (75 %)
Writing at 0x0003c9d0... (87 %)
Writing at 0x000423c1... (100 %)
Wrote 209040 bytes (116579 compressed) at 0x00010000 in 1.9 seconds (effective 885.4 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

PilnyTomas · 2022-08-16T08:29:51Z

Linux Fedora,
Arduino IDE 1.8.19
get.py downloads esptool without problem
Arduino reports correctly esptool.py v4.2.1
ESP32-C3 and S3 - SHA mismatch persists

SuGlider · 2022-08-16T10:20:41Z

@me-no-dev @P-R-O-C-H-Y

I have also replaced package/package_esp32_index.template.json and run python get.py within Win11, using PowerShell.
Everything runs OK.

get.py / PowerShell OUTPUT:

PS C:\Users\user\Documents\Arduino\hardware\espressif\esp32\tools> python --version
Python 3.9.13
PS C:\Users\user\Documents\Arduino\hardware\espressif\esp32\tools> python get.py
System: Windows, Bits: 64, Info: Windows-10-10.0.22000-SP0
Platform: i686-mingw32
Tool riscv32-esp-elf-gcc8_4_0-esp-2021r2-patch3-win32.zip already downloaded
Extracting riscv32-esp-elf-gcc8_4_0-esp-2021r2-patch3-win32.zip ...
Tool xtensa-esp32-elf-gcc8_4_0-esp-2021r2-patch3-win32.zip already downloaded
Extracting xtensa-esp32-elf-gcc8_4_0-esp-2021r2-patch3-win32.zip ...
Tool xtensa-esp32s2-elf-gcc8_4_0-esp-2021r2-patch3-win32.zip already downloaded
Extracting xtensa-esp32s2-elf-gcc8_4_0-esp-2021r2-patch3-win32.zip ...
Tool xtensa-esp32s3-elf-gcc8_4_0-esp-2021r2-patch3-win32.zip already downloaded
Extracting xtensa-esp32s3-elf-gcc8_4_0-esp-2021r2-patch3-win32.zip ...
Downloading esptool-4.2.1-windows.zip ...
Extracting esptool-4.2.1-windows.zip ...
Tool i686-w64-mingw32.mklittlefs-c41e51a.200706.zip already downloaded
Extracting i686-w64-mingw32.mklittlefs-c41e51a.200706.zip ...
Tool mkspiffs-0.2.3-arduino-esp32-win32.zip already downloaded
Extracting mkspiffs-0.2.3-arduino-esp32-win32.zip ...
Renaming mkspiffs-0.2.3-arduino-esp32-win32/ to mkspiffs ...
Platform Tools Installed
PS C:\Users\user\Documents\Arduino\hardware\espressif\esp32\tools>

SuGlider · 2022-08-16T10:21:07Z

@VojtechBartoska @SuGlider @P-R-O-C-H-Y please test this for building and flashing on all operating systems. We bumped the version quite a bit and also the layout for Linux has changed. Please report if there are still warnings after boot for mismatched image checksum.

Everything is OK for Windows 11.

VojtechBartoska · 2022-08-16T13:47:59Z

Thanks for testing @SuGlider & @PilnyTomas! So the problem occurs only on MAC platform.

to complete the test.

* Enable configuring target variants (espressif#7019) This change enables picking the right board configuration from the variants/ folder. Previously, we would always pick the default configuration (e.g., "esp32" instead of "heltec_wifi_lora_32_V2"). * Update .gitignore (espressif#7021) Add entries to gitignore * Update esptool to version 4.2.1 (espressif#7127) * Update esptool to version 4.2.1 * Fix esptool for MacOS * Esptool v4.2.1 for CI Platformio (espressif#7147) to complete the test. Co-authored-by: Jason2866 <[email protected]> * Update get.py to support python 3.10+ (espressif#7166) * Update get.py to support python 3.10+ * Use try/except to remove version check * fixed names on the VID and PID for boards (espressif#7144) somehow had duplicated the same name across all my boards. the PID's and VID's were correct, but the name was the same for all of them. that is fixed now. * Compile error if CONFIG_FREERTOS_HZ != 1000 (espressif#6955) * Compile error if CONFIG_FREERTOS_HZ != 1000 * add a check at the CMake level, per feedback * Set CONFIG_FREERTOS_HZ=1000 in CI test of Arduino-as-component * Adding u-blox NORA-W10 series (ESP32-S3) (espressif#7191) * Create pins_arduino.h * Update boards.txt * Update boards.txt * Fixed espressif/esp-rainmaker#152 (espressif#7171) * Update PlatformIO build scripts (espressif#7200) This update includes the following: - Implemented an additional build step that produces an adjusted bootloader image with updated headers according to selected flash mode and size values. This step is only executed for debugging or uploading via debug probes. - Implemented a basic mechanism to dynamically add an extra UF2 bootloader image if corresponding partition is selected (e.g. for Adafruit targets) - Minor code formatting Co-authored-by: Dr. Christian Kohlschütter <[email protected]> Co-authored-by: Me No Dev <[email protected]> Co-authored-by: Paul Price <[email protected]> Co-authored-by: Daniel Egnor <[email protected]> Co-authored-by: Michael Ammann <[email protected]> Co-authored-by: Sanket Wadekar <[email protected]> Co-authored-by: Valerii Koval <[email protected]>

me-no-dev added 2 commits August 14, 2022 10:22

Update esptool to version 4.2.1

8e26c2c

Fix esptool for MacOS

581e6e1

VojtechBartoska added this to the 2.0.5 milestone Aug 15, 2022

VojtechBartoska linked an issue Aug 15, 2022 that may be closed by this pull request

REGRESSION: ESP32-S3-DevKitC-1-N8R8 reboot cycle under 2.0.4, partially worked in 2.0.3 #6980

Closed

1 task

VojtechBartoska requested review from PilnyTomas and SuGlider August 15, 2022 12:09

SuGlider approved these changes Aug 16, 2022

View reviewed changes

PilnyTomas approved these changes Aug 16, 2022

View reviewed changes

VojtechBartoska added the Status: Pending label Aug 17, 2022

VojtechBartoska assigned me-no-dev Aug 17, 2022

Esptool v4.2.1 for CI Platformio (#7147)

a79c4e7

to complete the test.

me-no-dev merged commit 3ebb774 into master Aug 22, 2022

me-no-dev deleted the esptool/v4.2.1 branch August 22, 2022 14:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update esptool to version 4.2.1 #7127

Update esptool to version 4.2.1 #7127

me-no-dev commented Aug 14, 2022

me-no-dev commented Aug 14, 2022

P-R-O-C-H-Y commented Aug 14, 2022

Jason2866 commented Aug 14, 2022

VojtechBartoska commented Aug 15, 2022

SuGlider commented Aug 15, 2022 •

edited

Loading

PilnyTomas commented Aug 16, 2022 •

edited

Loading

SuGlider commented Aug 16, 2022

SuGlider commented Aug 16, 2022

VojtechBartoska commented Aug 16, 2022

Update esptool to version 4.2.1 #7127

Update esptool to version 4.2.1 #7127

Conversation

me-no-dev commented Aug 14, 2022

me-no-dev commented Aug 14, 2022

P-R-O-C-H-Y commented Aug 14, 2022

Jason2866 commented Aug 14, 2022

VojtechBartoska commented Aug 15, 2022

SuGlider commented Aug 15, 2022 • edited Loading

Arduino IDE for Windows - Console OUPUT:

PilnyTomas commented Aug 16, 2022 • edited Loading

SuGlider commented Aug 16, 2022

get.py / PowerShell OUTPUT:

SuGlider commented Aug 16, 2022

VojtechBartoska commented Aug 16, 2022

SuGlider commented Aug 15, 2022 •

edited

Loading

PilnyTomas commented Aug 16, 2022 •

edited

Loading