Upload speed a lot slower on Arduino IDE than using esptool.py #4973

dimitre · 2018-07-28T18:19:56Z

Hello, I'm using the classic FTDI1232 to upload sketches to ESP-01, both using Arduino IDE and esptool, and the possible upload speeds are so different I was wondering how to automate the upload process outside Arduino IDE.

On Arduino IDE my max baud rate possible is 230400 and it takes a while to upload some medium projects while using esptool I get 12x more speed

esptool.py --baud 3000000 --port /dev/cu.usbserial-12345678 write_flash --flash_mode dio --flash_size detect 0x0 /Volumes/ofw/TEENSYATUAL/ESP/ESTILHACO7Ota/e8266.bin

and the result

esptool.py v2.5.0
Serial port /dev/cu.usbserial-12345678
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: 5c:cf:7f:94:6a:8a
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 3000000
Changed.
Configuring flash size...
Auto-detected Flash size: 1MB
Flash params set to 0x0220
Compressed 295024 bytes to 212380...
Wrote 295024 bytes (212380 compressed) at 0x00000000 in 1.8 seconds (effective 1306.8 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

It would be great to have Arduino possible upload speeds improved. Thank you

The text was updated successfully, but these errors were encountered:

earlephilhower · 2018-07-28T19:05:40Z

I didn't know the 8266 had a compressed upload stub like the 32. PRs are always appreciated. platform.txt has the command invoked and would be worth looking at to see if a simple replacement could be done.

One worry I see is that for Linux/Mac you can expect python to exist. On Windows, though, I don't think we install any interpreter and hence esptool*.py* may be some trouble.

devyte · 2018-07-29T07:29:04Z

I have several things to say here:

@dimitre ignored the issue template.
@dimitre 's request here seems to be to somehow allow other speeds in the IDE menu. I seem to remember mention that some OSs (Windows?) limit the options, hence the IDE doesn't show all speeds. Personally, I see as high as almost 1Mbps on my IDE under Ubuntu. I stick to 230K though.
Writing at speeds above 230K or so is just asking for trouble. If it works for you, awesome, but unless you have a board with a certified flash chip, it's mostly luck. Flash issues due to high speed have been a source of headaches for a lot of users, especially with the typical chinese clones.
Writing compressed would be very useful for flashing the SPIFFS image.

@earlephilhower does your comment mean you're thinking of replacing our current esptool-ck with esptool.py?

minida28 · 2018-07-29T13:23:57Z

I am on Windows 10, but strange I don't have option for 230k baud rate...

I've been using 921600 baud rate for very long time ago and never have issues except occasionally failed upload. But if 3000000 speed is possible, it will be awesome :)

earlephilhower · 2018-07-29T21:45:00Z

...edit...looks like upload speed is used, so this is less good than I thought...

@devyte I'm not thinking of replacing the uploader, but if someone comes along with a PR to do it I think it's worth serious consideration (as long as it works cross-platform, etc. as well as the current setup). Too many other bits of ocean to boil already with File, String, WebServer, and probably ten other things.

dimitre · 2018-07-30T00:55:25Z

@devyte sorry I'll follow the template before submiting anything else.
Complementing, I'm in OS X and I can use only up to 230400. It would be great already if it worked at 921600.

devyte · 2018-07-30T01:44:45Z

I do see upload speeds in boards.txt.
@dimitre @minida28 feel free to experiment with different speeds. PRs are welcome, but please be aware that the boards.txt file is not meant to be edited directly. Instead, boards.txt.py should be edited, and boards.txt should be generated with it.

d-a-v · 2018-07-30T10:12:20Z

@dimitre @minida28 you can use more upload speed using the board generator (option --customspeed 230 (or more, see help)). Speeds are os-dependant but since windows has evolved, maybe some more can be enabled.

One worry I see is that for Linux/Mac you can expect python to exist. On Windows, though, I don't think we install any interpreter and hence esptool*.py* may be some trouble.

I think this is why we are still not using it, to keep the arduino KISS way.
But this is interesting enough to add an option for it in menu / board generator. I'm also opened to this.

minida28 · 2018-07-30T14:34:56Z

@d-a-v may I beg your pardon, but where is the 'help' ..? I searched in the front page, doc, FAQ, but can't find it...

(or more, see help))

By the way, I just tried the quick and dirty way (editing boards.txt file directly), and added 3 new speeds: 1000000, 1152000 and 3000000.

None of them seem to work and it falls back to the default 115200 baud rate.

Example output using 1M speed:

G:\arduino-1.8.5\hardware\esp8266com\esp8266/tools/esptool/esptool.exe -vv -cd nodemcu -cb 1152000 -cp COM4 -ca 0x00000 -cf C:\Users\miq\AppData\Local\Temp\arduino_build_36368/BlinkWithoutDelay.ino.bin 
esptool v0.4.13 - (c) 2014 Ch. Klippel <[email protected]>
warning: unsupported baud rate: 1000000, using 115200
Uploading 251936 bytes from C:\Users\miq\AppData\Local\Temp\arduino_build_36368/BlinkWithoutDelay.ino.bin to flash at 0x00000000
................................................................................ [ 32% ]
................................................................................ [ 64% ]
................................................................................ [ 97% ]
.......                                                                          [ 100% ]

d-a-v · 2018-07-30T14:52:05Z

#4478
:) Sorry, still have work to do

Start the board generator without option, you will get help:

$ ./tools/boards.txt.py
C:\...\boards-txt-directory> .\tools\boards.txt.py

boards.txt generator for esp8266/Arduino

usage: ./tools/boards.txt.py [options]

 -h, --help
 --lwip          - preferred default lwIP version (default 2)
 --led           - preferred default builtin led for generic boards (default 2)
 --board b       - board to modify:
 --speed s       - change default serial speed
 --customspeed s - new serial speed for all boards
 --nofloat       - disable float support in printf/scanf
 --noextra4kheap - disable extra 4k heap (will enable WPS)
 --allowWPS      - synonym for --noextra4kheap

 mandatory option (at least one):

 --boards        - show boards.txt
 --boardsgen     - replace boards.txt
 --ld            - show ldscripts
 --ldgen         - replace ldscripts
 --package       - show package
 --packagegen    - replace board:[] in package
 --doc           - shows doc/boards.rst
 --docgen        - replace doc/boards.rst
 --allgen        - generate and replace everything
                   (useful for pushing on github)

available serial speed options (kbps): 9 57 115 230 256 460 512 921 
available board names: generic(115k) esp8285(115k) espduino(115k) huzzah(115k) cw01(115k) espresso_lite_v1(115k) espresso_lite_v2(115k) phoenix_v1(115k) phoenix_v2(115k) nodemcu(115k) nodemcuv2(115k) modwifi(115k) thing(115k) thingdev(115k) esp210(57k) d1_mini(921k) d1_mini_pro(921k) d1_mini_lite(921k) d1(921k) espino(115k) espinotee(115k) wifinfo(115k) arduino-esp8266(115k) gen4iod(115k) oak(921k) wifiduino(921k) wifi_slot(115k) wiolink(115k)

minida28 · 2018-07-30T14:57:25Z

I see thank you @d-a-v :)

minida28 · 2018-07-30T15:29:03Z

Ok I tested using esptool.py (still in Windows 10 via GIT bash) using @dimitre command above, still does not work (with 1000000, 11520000 & 3000000 speed).
But it works at 921600.

$ ./esptool.py --baud 1000000 --port COM4 write_flash --flash_mode dio --flash_size detect 0x0 H:/BlinkWithoutDelay/BlinkWithoutDelay.bin
esptool.py v2.5.1-dev
Serial port COM4
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: 5c:cf:7f:12:64:2b
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 1000000
Changed.
Configuring flash size...

A fatal error occurred: Timed out waiting for packet header

Note: I tested with NodeMcu v1.0 with onboard USB serial chip (Silicon Labs CP210x)

Is this related to the type of USB serial chip being used?

dimitre · 2018-07-30T16:10:25Z

I can confirm it doesn't work here either with 3000000 or 2000000. I even tried the baud rate esptool.py returned me as actual writing speed (1306800)

./tools/boards.txt.py --boardsgen --customspeed 2000000

But I'm happy now it is working with 921600 speed here.
Now testing with a clean Arduino install with only ESP boards added. (OS X)
Thank you

e-ika · 2018-09-06T09:58:14Z

Thanks for the information!

dimitre · 2018-10-11T13:43:14Z

Arduino 1.8.7 and esp8266 from GIT HEAD improves a lot the transfer time and caching too. Thank you.
On further investigation I've discovered some older modules I have are much faster due to the Flash Memory (BergMicro 25Q80ASSIG) and mostly I've been buying on ebay now comes with the much slower PUY P25Q80H which fails on higher baudrates.

devyte · 2018-10-11T15:23:22Z

Careful with those puya chips, they require a patch to allow writing to flash (spiffs, eeprom, etc).

Is the original issue resolved then?

marcelstoer · 2019-03-06T08:20:41Z

That issue may very well be stale by now but here's my $.02 nonetheless.

Background: over two years ago I created a standalone self-contained app around esptool.py that runs on Windows even if you haven't got Python installed -> https://github.com/marcelstoer/nodemcu-pyflasher/

Yes, I too am a bit annoyed by the CK flasher used by the Arduino IDE.
- it's not stable at anything beyond 230400
- 230400 is just too slow if you frequently need to flash as working with Arduino (IDE) requires
- it hasn't seen updates in over a year; not sure what the plans of the maintainer(s) are
Yes, running esptool.py requires a Python context. So, it's a dead end for most Windows users unless the Arduino IDE would e.g. bundle my PyFlasher but currently you couldn't run it headless anyhow.
Would it be possible to ship the IDE with esptool.py and allow it to be invoked if the user configures the path to a Python binary in the IDE settings?

d-a-v · 2019-03-06T09:42:37Z

Would it be possible to ship the IDE with esptool.py and allow it to be invoked if the user configures the path to a Python binary in the IDE settings?

That's currently the case in master thanks to @earlephilhower (bundled python, submodule for esptool.py).
It works (also) on windows machine without any installed python #5635.

marcelstoer · 2019-03-06T09:46:19Z

Thanks for the pointer to the PR! Good stuff @earlephilhower 👍

dimitre · 2019-08-07T22:22:33Z

It would be great to have 3000000 as possible baud for esp8266.
I use it by default in esptool.py

earlephilhower · 2019-08-07T23:08:38Z

@dimitre, please open a new issue for this. It'll get lost here on a closed bug.

earlephilhower added type: enhancement component: uploader labels Jul 28, 2018

devyte added the waiting for feedback Waiting on additional info. If it's not received, the issue may be closed. label Jul 30, 2018

devyte mentioned this issue Jul 30, 2018

Implement compressed stub for sketch upload #4982

Closed

6 tasks

d-a-v closed this as completed Mar 6, 2019

d-a-v removed the waiting for feedback Waiting on additional info. If it's not received, the issue may be closed. label Mar 6, 2019

d-a-v added this to the 2.5.1 milestone Mar 6, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upload speed a lot slower on Arduino IDE than using esptool.py #4973

Upload speed a lot slower on Arduino IDE than using esptool.py #4973

dimitre commented Jul 28, 2018

earlephilhower commented Jul 28, 2018

devyte commented Jul 29, 2018

minida28 commented Jul 29, 2018

earlephilhower commented Jul 29, 2018 •

edited

Loading

dimitre commented Jul 30, 2018

devyte commented Jul 30, 2018

d-a-v commented Jul 30, 2018

minida28 commented Jul 30, 2018

d-a-v commented Jul 30, 2018

minida28 commented Jul 30, 2018

minida28 commented Jul 30, 2018 •

edited

Loading

dimitre commented Jul 30, 2018

e-ika commented Sep 6, 2018

dimitre commented Oct 11, 2018

devyte commented Oct 11, 2018

marcelstoer commented Mar 6, 2019

d-a-v commented Mar 6, 2019

marcelstoer commented Mar 6, 2019

dimitre commented Aug 7, 2019

earlephilhower commented Aug 7, 2019

Upload speed a lot slower on Arduino IDE than using esptool.py #4973

Upload speed a lot slower on Arduino IDE than using esptool.py #4973

Comments

dimitre commented Jul 28, 2018

earlephilhower commented Jul 28, 2018

devyte commented Jul 29, 2018

minida28 commented Jul 29, 2018

earlephilhower commented Jul 29, 2018 • edited Loading

dimitre commented Jul 30, 2018

devyte commented Jul 30, 2018

d-a-v commented Jul 30, 2018

minida28 commented Jul 30, 2018

d-a-v commented Jul 30, 2018

minida28 commented Jul 30, 2018

minida28 commented Jul 30, 2018 • edited Loading

dimitre commented Jul 30, 2018

e-ika commented Sep 6, 2018

dimitre commented Oct 11, 2018

devyte commented Oct 11, 2018

marcelstoer commented Mar 6, 2019

d-a-v commented Mar 6, 2019

marcelstoer commented Mar 6, 2019

dimitre commented Aug 7, 2019

earlephilhower commented Aug 7, 2019

earlephilhower commented Jul 29, 2018 •

edited

Loading

minida28 commented Jul 30, 2018 •

edited

Loading