Remote firmware update?

[zenmanenergy]

Is it possible to upgrade the codebase so we can remotely upload new firmware?

[lukecyca]

It should already be possible, but you would need to do it by calling the underlying functions directly. The IoT SDK has a series of system_upgrade_* functions, and Sming has a module called
HttpFirmwareUpdate and an example which might be good starting points.

It would great to eventually expose this through the ESP object with a more user-friendly API.

[zenmanenergy]

Holy crud. That looks fucking awesome!! Thank you!

[igrr]

We now have parts of OTA update infrastructure ready. Uploads of sketch from the IDE over WiFi are not ready yet, but here is an example sketch which can be updated using a python script:

sketch: https://gist.github.com/igrr/43d5c52328e955bb6b09

python script: https://gist.github.com/igrr/d35ab8446922179dc58c

EDIT: you need to use either the staging release or build from git to use that feature: http://arduino.esp8266.com/staging/package_esp8266com_index.json

[pablotix20]

I'm interested on that, the bin I have to upload is the one finished by cpp_10000.bin?

[pablotix20]

I can't upload:
C:\Users\pablo_000\Downloads>python ota_server.py 192.168.1.25 Wifi_temp_post.cp
p_00000.bin
Traceback (most recent call last):
File "ota_server.py", line 64, in
sys.exit(main(sys.argv))
File "ota_server.py", line 59, in main
return serve(args[1], args[2])
File "ota_server.py", line 19, in serve
print >>sys.stderr, 'starting up on %s port %s' % server_address
TypeError: unsupported operand type(s) for >>: 'builtin_function_or_method' and
'_io.TextIOWrapper'

EDIT: Works on linux (Ubuntu)

[igrr]

I think this print statement stuff will not work with python 3. I've written this script with 2.7.

[pablotix20]

OK thanks, on Ubuntu iḿ using 2.7.9, I'll see on Windows

[igrr]

Regarding the bin file, there is now only one .bin file generated (like Blink.cpp.bin)
You need to use either git version or this package:
http://arduino.esp8266.com/staging/package_esp8266com_index.json

[pablotix20]

I cannot update:

socket.error: [Errno 10054] Se ha forzado la interrupci¾n de una conexi¾n existente por el host remoto

I have searched in Google but it seems that the connection is closed by the ESP and i have also tried with HERCULES for windows to see if it sended it and the python programm sended it correctly

[igrr]

You have the OTATest.ino sketch loaded?

On Tue, Jun 16, 2015, 17:42 Pablotis [email protected] wrote:

I cannot update:
socket.error: [Errno 10054] Se ha forzado la interrupci¾n de una conexi¾n
existente por el host remoto

—
Reply to this email directly or view it on GitHub
#268 (comment).

[pablotix20]

Yes, it shows Update failed when the cmd shows the error

[igrr]

Can you please post complete output from both sides?
Also it would be helpful if you would mention what kind of board you are using, what is the flash size, what is the size of the sketch you are trying to load via OTA. Thanks.

[olivernz]

This is way cool! I have ESP's that are built into walls behind switches
and upgrading is a....
So this'd be great!

On 17 June 2015 at 03:18, Ivan Grokhotkov [email protected] wrote:

Can you please post complete output from both sides?
Also it would be helpful if you would mention what kind of board you are
using, what is the flash size, what is the size of the sketch you are
trying to load via OTA. Thanks.

—
Reply to this email directly or view it on GitHub
#268 (comment).

[Duality4Y]

this seems to work here :)
that it resets the module (wdt) is what is supposed to happen after it is finished uploading ?

[pablotix20]

Here is the log and the bin file:
https://drive.google.com/folderview?id=0BxPwW0n9dYaFfkVFTy1tUG96QkVCQi1xSC1YdEJ2d2o3Xy1KZ09wbFpoRjdPQnBHVWhVSG8&usp=sharing
I have also tried uploading the blink example with no success

[pablotix20]

Any new?

[igrr]

From your log:

Free size: 184320
...
upload size: 262416

It fails because free space size is less than the size of the sketch you are trying to upload.

[pablotix20]

But I tried to do it with Blink example and still not working, and doesn't it rewrite de current program?

[igrr]

In that case please attach the log for the case when you are downloading the blink example. It's pretty hard trying to guess what's happening on your side :)

[chadouming]

I'm pretty sure that even using a blank sketch won't work with 512KB space,
since blank sketch and library takes roughly 200K

On Thu, Jun 18, 2015, 10:05 AM Ivan Grokhotkov [email protected]
wrote:

In that case please attach the log for the case when you are downloading
the blink example. It's pretty hard trying to guess what's happening on
your side :)

—
Reply to this email directly or view it on GitHub
#268 (comment).

[thewiep]

I was having, what seems to be, the same issue and I suspect it has something to do with my ESP module or flakey wifi signal.
I frequently have issues getting the ESP into flashing mode.

this is the output in my terminal:

root@desktop:/tmp# python ota.py 192.168.1.119 ESP8266_Remote_Sketch_Upload_OTA.cpp.bin 
starting up on 0.0.0.0 port 48266
upload size: 253280
sending invitation
waiting for a connection
connection from ('192.168.1.119', 4097)
opening file ESP8266_Remote_Sketch_Upload_OTA.cpp.bin
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
Traceback (most recent call last):
  File "ota.py", line 64, in <module>
    sys.exit(main(sys.argv))
  File "ota.py", line 59, in main
    return serve(args[1], args[2])
  File "ota.py", line 48, in serve
    connection.sendall(chunk)
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 32] Broken pipe

root@desktop:/tmp# ls -lah | grep ESP8266_Remote_Sketch_Upload_OTA.cpp.bin 
-rw-r--r--  1 user   user   248K jun 18 17:50 ESP8266_Remote_Sketch_Upload_OTA.cpp.bin

root@desktop:/tmp# python --version
Python 2.7.6

this is the output on the ESP:

OTA test 3
scandone
f 0, ....scandone
.add 0
aid 4
pm open phy_2,type:2 0 0
cnt 

connected with cybertron_main_g, channel 6
dhcp client start...
.ip:192.168.1.119,mask:255.255.255.0,gw:192.168.1.1
.
Connected to cybertron_main_g
IP address: 192.168.1.119
Sketch size: 253268
Free size: 794624
Got packet
0 48266 253280
Update failed

when it works I get following output in terminal:

...
sending 4096
sending 4096
sending 4096
sending 3424
done!

and this from the ESP:

OTA test 3
scandone
f 0, scandone
.add 0
aid 4
pm open phy_2,type:2 0 0
cnt 

connected with cybertron_main_g, channel 6
dhcp client start...
.ip:192.168.1.119,mask:255.255.255.0,gw:192.168.1.1
.
Connected to cybertron_main_g
IP address: 192.168.1.119
Sketch size: 253268
Free size: 794624
Got packet
0 48266 253280
rm 0
pm close 7 0 0/7924696
Fatal exception (28):
epc1=0x402029ce, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, depc=0x00000000

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1464, room 16 
tail 8
chksum 0x7a
csum 0x7a
@
cp
��ՁMEM CHECK FAIL!!!
�rl
OTA test 3
...

the Fatal exception seems scary but seems to indicated the update succeeded..

This is a really nice feature!

[CptanPanic]

Wow this could be a great feature that I will need to try. Does this utilize http://www.esp8266.com/viewtopic.php?f=12&t=3115&start=10 ?

[igrr]

@CptanPanic i wrote my own bootloader, called eboot. It's a bit more efficient because it requires only one segment for code and data, plus it avoids an extra copying step which both rboot and Espressif's original bootloader choose to do for some reason. Also the code is slimmer and a bit easier to digest. You can check the source code in hardware/esp8266com/esp8266/bootloaders/eboot.

[prussiap]

@igrr I saw it in the git repo but how do you make use of it? Was there an example or samples in the wiki? I saw another post you had about using the ota_server.py. Do you flash the eboot bootloader then use the ota_server.py ipaddress any_arduino_sketch method ?

[igrr]

The binary which is generated by IDE contains the bootloader in the first
4kb sector, so you don't need anything extra — the bootloader will be
flashed along with the sketch.
To use OTA you can flash OTATest.ino linked above, and then use
ota_server.py to upload the new sketch.

In a short while this will be integrated into the IDE so you will just have
to hit "upload" button for OTA to work.

On Sun, Jun 21, 2015, 03:41 prussiap [email protected] wrote:

@igrr https://github.com/igrr I saw it in the git repo but how do you
make use of it? Was there an example or samples in the wiki? I saw another
post you had about using the ota_server.py. Do you flash the eboot
bootloader then use the ota_server.py ipaddress any_arduino_sketch method ?

—
Reply to this email directly or view it on GitHub
#268 (comment).

[duncan-a]

This is a wonderful project - and it just keeps getting better and better...

Thank you everyone who is involved..................

On 21 June 2015 at 09:01, Ivan Grokhotkov [email protected] wrote:

The binary which is generated by IDE contains the bootloader in the first
4kb sector, so you don't need anything extra — the bootloader will be
flashed along with the sketch.
To use OTA you can flash OTATest.ino linked above, and then use
ota_server.py to upload the new sketch.

In a short while this will be integrated into the IDE so you will just have
to hit "upload" button for OTA to work.

On Sun, Jun 21, 2015, 03:41 prussiap [email protected] wrote:

@igrr https://github.com/igrr I saw it in the git repo but how do you

make use of it? Was there an example or samples in the wiki? I saw
another
post you had about using the ota_server.py. Do you flash the eboot
bootloader then use the ota_server.py ipaddress any_arduino_sketch
method ?

—
Reply to this email directly or view it on GitHub
#268 (comment).

—
Reply to this email directly or view it on GitHub
#268 (comment).

[probonopd]

Using Arduino IDE 1.6.4 with ESP8266 1.6.4-835.g77d77e8 I get on the ESP-01:

OTA test
scandone
f 0, ....scandone
.add 0
aid 3
pm open phy_2,type:2 0 0
cnt 

connected with <SSID>, channel 6
dhcp client start...
...ip:192.168.0.16,mask:255.255.255.0,gw:192.168.0.1
.
Connected to h48
IP address: 192.168.0.16
Sketch size: 248580
Free size: 188416
Got packet
0 48266 248592
Update failed

And on the Mac:

python /Users/user/Downloads/ota_server.py 192.168.0.16 OTAtest.cpp.bin
starting up on 0.0.0.0 port 48266
upload size: 248592
sending invitation
waiting for a connection
connection from ('192.168.0.16', 4097)
opening file OTAtest.cpp.bin
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
Traceback (most recent call last):
  File "/Users/user/Downloads/ota_server.py", line 64, in <module>
    sys.exit(main(sys.argv))
  File "/Users/user/Downloads/ota_server.py", line 59, in main
    return serve(args[1], args[2])
  File "/Users/user/Downloads/ota_server.py", line 48, in serve
    connection.sendall(chunk)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 32] Broken pipe

[igrr]

Free size: 188416
upload size: 248592

You need to have at least 1M flash for the OTA to fit.

On Sun, Jun 21, 2015, 13:22 probonopd [email protected] wrote:

I get on the ESP-01:

OTA test
scandone
f 0, ....scandone
.add 0
aid 3
pm open phy_2,type:2 0 0
cnt

connected with h48, channel 6
dhcp client start...
...ip:192.168.0.16,mask:255.255.255.0,gw:192.168.0.1
.
Connected to h48
IP address: 192.168.0.16
Sketch size: 248580
Free size: 188416
Got packet
0 48266 248592
Update failed

And on the Mac:

python /Users/user/Downloads/ota_server.py 192.168.0.16 OTAtest.cpp.bin
starting up on 0.0.0.0 port 48266
upload size: 248592
sending invitation
waiting for a connection
connection from ('192.168.0.16', 4097)
opening file OTAtest.cpp.bin
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
Traceback (most recent call last):
File "/Users/user/Downloads/ota_server.py", line 64, in
sys.exit(main(sys.argv))
File "/Users/user/Downloads/ota_server.py", line 59, in main
return serve(args[1], args[2])
File "/Users/user/Downloads/ota_server.py", line 48, in serve
connection.sendall(chunk)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 32] Broken pipe

—
Reply to this email directly or view it on GitHub
#268 (comment).

[probonopd]

This means I need a different hardware module, correct? So OTA will not work on ESP-01? (If so it would be good to document that fact or put in a check). Also, what is not clear to me: When I upload OTAtest, does that mean I can upload blink.cpp.bin OTA and it will still be possible to load another sketch via OTA afterwards, or does every sketch that I upload via OTA need to contain the OTAtest functionality (in order to allow for subsequent OTA updates)?

[igrr]

This is a very much work-in-progress feature so please forgive the lack of
documentation. You need at least 1M flash for this to work.
Many ESP12/ESP07 modules right now ship with 4M flash.
For now the OTA function is implemented in the sketch, so if you upload
Blink sketch, it won't be able to update. I'm open to suggestions how to
handle this better though.

One more thing that you will certainly need in a sketch in order for OTA to
be possible is ESP8266WiFi library.

On Sun, Jun 21, 2015, 13:38 probonopd [email protected] wrote:

This means I need a different hardware module, correct? So OTA will not
work on ESP-01? (If so it would be good to document that fact or put in a
check). Also, what is not clear to me: When I upload OTAtest, does that
mean I can upload blink.cpp.bin OTA and it will still be possible to load
another sketch via OTA afterwards, or does every sketch that I upload via
OTA need to contain the OTAtest functionality (in order to allow for
subsequent OTA updates)?

—
Reply to this email directly or view it on GitHub
#268 (comment).

[whyameye]

@igrr you said earlier about TLS:

the trick is that you have to configure your TLS server to use smaller fragment size (16k is the default for TLS, and we can support up to 4k at the moment)

Would you have any hints for me about how I would configure Apache2 to do this? Google is coming up dry for me.

[igrr]

See second comment to the answer: http://stackoverflow.com/a/29850611

On Fri, Nov 6, 2015, 01:45 whyameye [email protected] wrote:

@igrr https://github.com/igrr you said earlier about TLS:

the trick is that you have to configure your TLS server to use smaller
fragment size (16k is the default for TLS, and we can support up to 4k at
the moment)

Would you have any hints for me about how I would configure Apache2 to do
this? Google is coming up dry for me.

—
Reply to this email directly or view it on GitHub
#268 (comment).

[Links2004]

i start to document the different OTA processes here:
https://github.com/esp8266/Arduino/blob/master/doc/ota_updates.md

[tlerdrden]

Hi!

I have tried the OTATest.ino / OTAserver.py sketches from @igrr. See>

ESP sketch: https://gist.github.com/igrr/43d5c52328e955bb6b09

SERVER python script: https://gist.github.com/igrr/d35ab8446922179dc58c

This is what I got from the ESP Side>

Got packet
0 48266 343072
Fatal exception (9):
epc1=0x40236451, epc2=0x00000000, epc3=0x00000000, excvaddr=0x3106a145, depc=0x00000000

Exception (9):
epc1=0x40236451 epc2=0x00000000 epc3=0x00000000 excvaddr=0x3106a145 depc=0x00000000

ctx: sys 
sp: 3ffffd50 end: 3fffffb0 offset: 01a0

>>>stack>>>
3ffffef0:  00000000 3ffeecf4 3ffeecfb 3ffeed2e  
3fffff00:  3fff0000 4020aa4a 3ffeffb0 3fff2588  
3fffff10:  3ffeda2c 3fff258c 3fff3090 40209ee9  
3fffff20:  3fff2498 3fff2c48 3fff2c48 00000147  
3fffff30:  00000000 3fffff60 00000001 40236872  
3fffff40:  ffffffff 00000000 00000001 3ffe8034  
3fffff50:  3fffff60 00000000 00000000 0000005c  
3fffff60:  444a4e52 40103840 00000000 00000001  
3fffff70:  402178fa 3fffdab0 00000002 00000000  
3fffff80:  3ffefb78 40217411 3fffdab0 00000000  
3fffff90:  3fffdcc0 3ffec4e0 00000000 40201795  
3fffffa0:  3ffec4e0 40000f49 3fffdab0 40000f49  
<<<stack<<<

ets Jan 8 2013,rst cause:1, boot mode:(1,7)

ets Jan 8 2013,rst cause:4, boot mode:(1,7)

wdt reset

And this is what I got from the

C:\Users\E1202097>C:\Users\E1202097\Desktop\d35ab8446922179dc58c-2f36f852162e6fe
1385c707c0e425c555d51a857\ota_server.py 192.168.0.5 C:\Users\E1202097\Desktop\d3
5ab8446922179dc58c-2f36f852162e6fe1385c707c0e425c555d51a857\OTATest.cpp.bin
starting up on 0.0.0.0 port 48266
upload size: 343072
sending invitation
waiting for a connection
connection from ('192.168.0.5', 4097)
opening file C:\Users\E1202097\Desktop\d35ab8446922179dc58c-2f36f852162e6fe1385c
707c0e425c555d51a857\OTATest.cpp.bin
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
sending 4096
Traceback (most recent call last):
File "C:\Users\E1202097\Desktop\d35ab8446922179dc58c-2f36f852162e6fe1385c707c0
e425c555d51a857\ota_server.py", line 64, in <module>
    sys.exit(main(sys.argv))
File "C:\Users\E1202097\Desktop\d35ab8446922179dc58c-2f36f852162e6fe1385c707c0
e425c555d51a857\ota_server.py", line 59, in main
    return serve(args[1], args[2])
File "C:\Users\E1202097\Desktop\d35ab8446922179dc58c-2f36f852162e6fe1385c707c0
e425c555d51a857\ota_server.py", line 48, in serve
    connection.sendall(chunk)
File "C:\Python27\lib\socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 10054] Se ha forzado la interrupci¾n de una conexi¾n existe
nte por el host remoto

@Links2004 @igrr thanks!!

[whyameye]

Following up on the earlier conversation to get ESPhttpUpdate working with TLS, I have a server now restricting its TLS fragment (record) size to 4K. (notes about how I did it at https://sites.google.com/site/linuxwebcomputer/lamp-etc). I hacked up the ESP8266httpUpdate.cpp code for TLS connection. The GET request seems to work well and the server happily sends the update. However ESP.updateSketch(tcp, len, false, false) is returning false. Is this method documented somewhere? I couldn't find the code for it.

Update: I did find the code to updateSketch in /cores/esp8266/ I also was able to get the updates to work with TLS about 50% of the time by shutting off compression on the webserver and lowering the max record size to 128. From the other 50% of the time, about 45% of the time it timed out in writeStream on the call to readBytes in updater.cpp. A few times it thought it succeeded but the update was corrupt. I think we need a checksum calculated on the update.

With compression on the webserver on, updates still succeeded but less often. Higher and lower values for the max record size also sometimes worked but less often. 4095 (4K-1) never worked.

If the update fails, subsequent calls to ESP8266httpUpdate immediately fail until the 8266 is rebooted. Is there a way to avoid this? Also, why does ESP8266httpUpdate crash the uC when in loop() but not in setup()?

Given that the read() function is in Udp.h, is this really udp and not tcp? That seems like that could be a problem for reliability. And, is there a Udp.cpp or is Udp.h a header for a binary blob?

I've been working with the understanding that TLS fragment size is the same as TLS record size and is not related to packet size.

[Links2004]

@tlerdrden you linked files are out to date, try to use this: https://github.com/esp8266/Arduino/blob/master/libraries/ArduinoOTA/examples/BasicOTA/BasicOTA.ino with Arduino IDE 1.6.5.

@whyameye the http update is running in TCP no UDP is involved here.
you can try to enable the debug output for the ESP.updateSketch()
here:
https://github.com/esp8266/Arduino/blob/master/cores/esp8266/Updater.h#L17
the ESPhttpUpdate has some debuging too:
https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266httpUpdate/src/ESP8266httpUpdate.h#L34

may we then see better where the problem is coming from.

for debugging the tcp and ssl/tls layer the define is here:

Arduino/cores/esp8266/debug.h

Line 5 in 342c4ae

//#define DEBUGV(...) ets_printf(__VA_ARGS__)

for malloc debug inside the tls set the define DEBUG_TLS_MEM

the ESP8266httpUpdate is working in the setup and the loop at least the non TLS variant.
may its a memory problem.

Serial.setDebugOutput(true);

shut also help to get the exact hex address where the ESP is rebooting, if its not the ESP.restart() call here:

Arduino/libraries/ESP8266httpUpdate/src/ESP8266httpUpdate.cpp

Line 141 in 342c4ae

ESP.restart();

[igrr]

Most useful would be to enable DEBUG_SSL in WiFiClientSecure:
https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/src/WiFiClientSecure.cpp
and DEBUGV in debug.h
Also add Serial.setDebugOutput(true); to setup()

On Sat, Nov 7, 2015, 12:18 Markus [email protected] wrote:

@tlerdrden https://github.com/tlerdrden you liked files are out to
date, try to use this:
https://github.com/esp8266/Arduino/blob/master/libraries/ArduinoOTA/examples/BasicOTA/BasicOTA.ino
with Arduino IDE 1.6.5.

@whyameye https://github.com/whyameye the http update is running in TCP
no UDP is involved here.
you can try to enable the debug output for the ESP.updateSketch()
here:
https://github.com/esp8266/Arduino/blob/master/cores/esp8266/Updater.h#L17
the ESPhttpUpdate has some debuging too:

https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266httpUpdate/src/ESP8266httpUpdate.h#L34

may we then see better where the problem is coming from.

for debugging the tcp and ssl/tls layer the define is here:

Arduino/cores/esp8266/debug.h

Line 5 in 342c4ae

//#define DEBUGV(...) ets_printf(__VA_ARGS__)

for malloc debug inside the tls set the define DEBUG_TLS_MEM

the ESP8266httpUpdate is working in the setup and the loop at least the
non TLS variant.
may its a memory problem.

Serial.setDebugOutput(true);

shut also help to get the exact hex address wehre the ESP is rebooting, if
its not the ESP.restart() call here:

Arduino/libraries/ESP8266httpUpdate/src/ESP8266httpUpdate.cpp

Line 141 in 342c4ae

ESP.restart();

—
Reply to this email directly or view it on GitHub
#268 (comment).

[whyameye]

With the debug settings igrr recommended, I see this message repeatedly with any TLS fragment sizes above 1023:
WiFiClientSecure rx overflow*** rb: 0 + 1535 = 1535

Setting the fragment size at 1023 the overflow message goes away. Instead I get rb: 0 + 1023 = 1023 multiple times. With this setting I haven't gotten an OTA firmware update failure yet so it's hard to debug further. I will keep trying in different environments etc. and report what I find.

[igrr]

There is bug in WiFiClientSecure, I'll think about possible fix for this.

In the meantime, I think your security requirement may be implemented
without sending firmware over HTTPS. Get the firmware URL and SHA hash over
HTTPS (response will likely not exceed 1k), and then use normal OTA
procedure to download the firmware over HTTP. One thing you will have to
add is hash calculation for the firmware being downloaded, but that should
not be difficult to do. If you're willing to make such an enhancement, let
me know. Otherwise, @mangelajo had a plan to add SHA hash verification to
Arduino OTA support, which may be reused for http OTA as well.

On Sat, Nov 7, 2015, 19:52 whyameye [email protected] wrote:

With the debug settings igrr recommended, I see this message repeatedly
with any TLS fragment sizes above 1023:
WiFiClientSecure rx overflow*** rb: 0 + 1535 = 1535

Setting the fragment size at 1023 the overflow message goes away. Instead
I get rb: 0 + 1023 = 1023 multiple times. With this setting I haven't
gotten an OTA firmware update failure yet so it's hard to debug further. I
will keep trying in different environments etc. and report what I find.

—
Reply to this email directly or view it on GitHub
#268 (comment).

[whyameye]

@igrr so your feeling is that keeping the fragment size to 1023 (which eliminated the overflow errors) is not a good workaround for the bug in WiFiClientSecure? That's the easiest solution for me if you feel it is robust.

[igrr]

Sorry I didn't get it from your previous post that it finally worked when
you made fragment size less than 1024. I thought the error was gone, but
OTA still failed.

If it works and you're satisfied with this solution, that's awesome.

On Sat, Nov 7, 2015, 23:42 whyameye [email protected] wrote:

@igrr https://github.com/igrr so your feeling is that keeping the
fragment size to 1023 (which eliminated the overflow errors) is not a good
workaround for the bug in WiFiClientSecure? That's the easiest solution for
me if you feel it is robust.

—
Reply to this email directly or view it on GitHub
#268 (comment).

[starlino]

Will OTA work with current stable version: Stable version 1.6.5-947-g39819f0 ?
If yes what is the appropriate sample code link, will this one work: https://gist.github.com/igrr/43d5c52328e955bb6b09

[whyameye]

@igrr with fragment size < 1024 I'm finding in one environment it works 100%. In another environment it fails about 1/2 the time. Both environments have very good wifi and internet connectivity. When the update fails I don't see an error. Here's the end of the debug I see:

*** rb: 0 + 1023 = 1023
.:rch 1460, 1460
:rd 5, 2920, 222
:rdi 1238, 5
:rd 1072, 2920, 227
:rdi 1233, 1072
*** rb: 0 + 1023 = 1023
:rd 5, 2920, 1299
:rdi 161, 5
:rd 1072, 2920, 1304
:rdi 156, 156
:c 156, 1460, 2920
:rdi 1460, 916
*** rb: 0 + 1023 = 1023
:rd 5, 1460, 916
:rdi 544, 5
:rd 539, 1460, 921
:rdi 539, 539
:c0 539, 1460
..TIMEOUT

(The TIMEOUT is my own added debug which is printed when timedRead times out.)

I'm not sure how to interpret any of this.

-John

[igrr]

With stable version you might get some issues with OTA.

On Sat, Nov 7, 2015, 23:58 whyameye [email protected] wrote:

@igrr https://github.com/igrr with fragment size < 1024 I'm finding in
one environment it works 100%. In another environment it fails about 1/2
the time. Both environments have very good wifi and internet connectivity.
When the update fails I don't see an error. Here's the end of the debug I
see:

*** rb: 0 + 1023 = 1023
.:rch 1460, 1460
:rd 5, 2920, 222
:rdi 1238, 5
:rd 1072, 2920, 227
:rdi 1233, 1072
*** rb: 0 + 1023 = 1023
:rd 5, 2920, 1299
:rdi 161, 5
:rd 1072, 2920, 1304
:rdi 156, 156
:c 156, 1460, 2920
:rdi 1460, 916
*** rb: 0 + 1023 = 1023
:rd 5, 1460, 916
:rdi 544, 5
:rd 539, 1460, 921
:rdi 539, 539
:c0 539, 1460
..TIMEOUT

(The TIMEOUT is my own added debug which is printed when timedRead times
out.)

I'm not sure how to interpret any of this.

-John

—
Reply to this email directly or view it on GitHub
#268 (comment).

[igrr]

@whyameye
What is the difference between environments?
Debug output looks okay, but perhaps there is some issue with TLS which doesn't surface in logs.

[whyameye]

@igrr I'm not sure the difference in the environments. The one that always works is when the 8266 tethers to my Verizon phone. The one that only sometimes works is when the 8266 tethers to my router, which is connected to the cable company's modem. I'll keep experimenting.

[mtnbrit]

This resembles issues I had when trying to do OTA over MQTT. Would work fine sometimes, and be quite unreliable at others, never could narrow down the conditions that affected it one way or the other. Someone pointed out that the esp doesn't do fragmented packets very well and suggested trying to reduce MTU, but thats not always practical obviously. Would be nice if we could just fix the fragged packet re-assembly issue.

By the way thanks for the fix for MQTT over SSL, its working great!

[igrr]

@mtnbrit IP fragment re-assembly is not related to this issue. TLS fragment size is a different entity and it's just too large by default (16k). The only way to fit TLS in ESP8266 RAM is to reduce TLS fragment size.

[mtnbrit]

understood, I'm referring to underlying tcp fragmentation

[dee101k]

I am working on esp8266 module.I am implementing FOTA update operation using HTTP PUSH operation in Arduino IDE.
I am running httpUpdate.ino code in folder ESP8266httpUpdate of Arduino examples.

But i am getting follwing error:-
HTTP_UPDATE_FAILD Error (-1): HTTP error: connection refused

So anyone please guide me to solve this kind of problem..

Thanks

[Links2004]

i am not sure what you mean by "HTTP PUSH operation in Arduino IDE"
the ESP8266httpUpdate download a firmware from a http server, the IDE is only needed for compiling.

the error shows that your server refused the connection.
check your http server and network settings.

[dee101k]

@Links2004 yup Arduino IDE is only used for compiling our code.

My .bin file is located in following link
http://dl.dropboxusercontent.com/s/k4iq3ztndbml7mk/Parse_notification.bin.

So using my esp8266 module i am trying to download this .bin file from HTTP server.But i am encountering with problem:
HTTP_UPDATE_FAILD Error (-1): HTTP error: connection refused

I have also checked my network settings and it is perfectly alright...
I have tried many ways to debug this problem but ended with fail.Please guide me if you have any suggestions to this.

[d3alek]

I have seen this error when the chip does not have a valid wifi connection. Try pinging something to make sure wifi works:

WiFiClient client;

if (client.connect("www.google.com", 80)) {
   // you are connected to the internet
}
else {
   // you are not
}

[dee101k]

@d3alek I have checked wifi connection.It is properly connecting to www.google.com.
So my wifi is working properly.

[dee101k]

Now i have uploaded my .bin file on different server.Following is the link
http://teamhawk.parseapp.com/Parse_notification.bin
So now different error is coming
HTTP_UPDATE_FAILD Error (8): Update error: ERROR[8]: Flash config wrong real: 2097152 IDE: 4194304

Please anyone guide me to resolve this issue
@igrr any suggestions for my problem

[Links2004]

your flash settings from the IDE not match your real hardware
Hardware: 2MB flash
IDE selected: 4MB

[choudhas]

Getting Socket.error 10054: An existing connection was forcibly closed by the remote port
Here are the logs:

ESP Side:
OTA test
scandone
f 0, scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt

connected with test_pod, channel 11
dhcp client start...
...ip:192.168.1.12,mask:255.255.255.0,gw:192.168.1.1
.
Connected to test_pod
IP address: 192.168.1.12
Sketch size: 237728
Free size: 806912
pm open,type:2 0
Got packet
0 48267 237744
sleep disable
Update failed

PC Logs:
starting up on 192.168.1.13 port 48267
upload size: 237744
sending invitation
waiting for a connection
connection from ('192.168.1.12', 6433)
opening file OTATest.ino.bin
sending 4096
sending 4096
sending 4096
Traceback (most recent call last):
File "ota_server.py", line 64, in
sys.exit(main(sys.argv))
File "ota_server.py", line 59, in main
return serve(args[1], args[2])
File "ota_server.py", line 48, in serve
connection.sendall(chunk)
File "C:\Python27\lib\socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 10054] An existing connection was forcibly closed by the re
mote host

[buzzy]

Could someone please explain exactly how the Updater knows that the .bin I am trying to upload is for a certain flash size? I am using a esp07 and it fails because the Updater says the flash size is only 1MB and the .bin file is compiled for 4MB. The file itself is only 300kb and I already specified "build_flags = -Wl,-Teagle.flash.1m0.ld" in PlatformIO. Not sure why the resulting .bin-file is regarded as 4MB still. What am I missing? I have even tried to specify the board as "d1_mini_lite" as that one has 1MB flash. Updater still gives me the same error:
Update error: ERROR[8]: Flash config wrong real: 1048576 IDE: 4194304

[eduardomarcos]

BasicOTA working in ESP-01s (1MB Flash)