OTA update with HTTPS is not working #4400

waghekapil · 2018-02-20T11:50:29Z

Basic Infos

Hardware

Hardware: ?ESP8266-12E?
Core Version: ?2.4.0?

Description

OTA update without SSL is working fine. With SSL is it throwing error.

I generates FP through following command.

openssl x509 -noout -fingerprint -sha1 -inform pem -in C:\HTTPS\sipl.crt

Settings in IDE

Module: ?Generic ESP8266 Module?
Flash Size: ?4MB(3M SPIFFS)?
CPU Frequency: ?80Mhz?
Flash Mode: ?DIO?
Flash Frequency: ?40Mhz?
Upload Using: ?SERIAL?
Reset Method: ?ck?
Arduino SDK: ?1.8.5?

Sketch

/*
Name:    OTA.ino
Created: 2/13/2018 11:57:55 AM
Author:  kapil
*/

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <ESP8266httpUpdate.h>

void setup() {
	Serial.begin(115200);
	Serial.setDebugOutput(true);
	Serial.println("Hussain...");

	WiFi.begin("SSID", "PASS");
	while (WiFi.status() != WL_CONNECTED) {
		delay(1000);
	}
	Serial.println("Connected, updating");
	delay(1000);

	t_httpUpdate_return ret = ESPhttpUpdate.update("https://13.127.174.106:443/Blink.ino.bin", "", "07 CB 9F 52 17 54 4C 5D 84 93 31 49 F7 9A E0 43 69 FB 49 03");

	Serial.println();
	Serial.print("OTA free heap : ");
	Serial.print(ESP.getFreeHeap());

	Serial.println(F("\n---- ota update done ----\n"));
	switch (ret)
	{
	case HTTP_UPDATE_FAILED:
		Serial.printf("HTTP_UPDATE_FAILD Error (%d): %s", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
		break;
	case HTTP_UPDATE_NO_UPDATES:
		Serial.println(F("HTTP_UPDATE_NO_UPDATES"));
		break;
	case HTTP_UPDATE_OK:
		Serial.println(F("HTTP_UPDATE_OK"));
		break;
	default:
		Serial.printf("Unexpected response code %d from ESPhttpUpdate.update\n", (int)ret);
		break;

	}
}
void loop() {
}

Debug Messages

scandone
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt

connected with SIPLEC-IOT, channel 1
dhcp client start...
ip:192.168.100.5,mask:255.255.255.0,gw:192.168.100.2
Connected, updating
State: sending Client Hello (1)
State: receiving Server Hello (2)
State: receiving Certificate (11)
=== CERTIFICATE ISSUED TO ===
Common Name (CN): S
Organization (O): SYSTEMATIX INFOTECH PVT LTD
Organizational Unit (OU): SIPL
Location (L): INDORE
Country (C): IN
State (ST): MP
Basic Constraints: CA:TRUE, pathlen:10000
=== CERTIFICATE ISSUED BY ===
Common Name (CN): S
Organization (O): SYSTEMATIX INFOTECH PVT LTD
Organizational Unit (OU): SIPL
Location (L): INDORE
Country (C): IN
State (ST): MP
Not Before: Tue Feb 20 09:36:08 2018
Not After: Wed Feb 20 09:36:08 2019
RSA bitsize: 2048
Sig Type: SHA256
State: receiving Server Hello Done (14)
State: sending Client Key Exchange (16)
State: sending Finished (16)
State: receiving Finished (16)
cert FP: F4 9C BE 82 ED 63 F3 D9 F3 8A 60 1D 9C 40 FE 06 8D C4 47 F7
test FP: 07 CB 9F 52 17 54 4C 5D 84 93 31 49 F7 9A E0 43 69 FB 49 03

OTA free heap : 38648
---- ota update done ----

HTTP_UPDATE_FAILD Error (-1): HTTP error: connection refused

The text was updated successfully, but these errors were encountered:

kristinpaget · 2018-02-21T03:05:18Z

Same behaviour here. I've tried all the overloaded forms of ESPhttpUpdate.update and they all fail in various ways. The sketch below will:

Fetch the HTTPS update URL with an HTTPClient first, just to make sure it works
Attempts an update over SSL, which claims connection refused from the same URL
Attempts the same update over plaintext, demonstrating that the server is working OK.

Sketch

#include "ESP8266WiFi.h"
#include "gpio.h"
#include "WiFiClientSecure.h"
#include "ESP8266httpUpdate.h"

#define LOG Serial.print

const char HTTPURL[] = "<something>";
const char HTTPSURL[] = "<something with an extra S>";
const char Fingerprint[] = "<An appropriate fingerprint>";

const char SSID[] = "SSID";
const char Pass[] = "Password";

void setup() 
{

  Serial.begin(115200, SERIAL_8N1);
  delay(2000); // Allow the other end to attach

  WiFi.mode(WIFI_STA);
  WiFi.begin(SSID, Pass);

  while (WiFi.status() != WL_CONNECTED)
  {
    LOG(".");
    delay(100);
  }
        
  LOG("Wifi connected!\r\n");
  LOG("My IP is ");
  LOG(WiFi.localIP());
  LOG("\r\n");

  gpio_init();
  pinMode(0,INPUT_PULLUP);

  LOG("Ready to go!\r\n");
}

void loop() 
{
  unsigned char Buffer[1024];
  HTTPClient Client;

  if (digitalRead(0) == HIGH)
  {
    // Waiting for button
    delay(100);
    return;
  }

  // Start the sequence.
  Client.begin(HTTPSURL, Fingerprint);

  LOG("About to GET ");
  LOG(HTTPSURL);
  LOG("\r\n");
  int Result = Client.GET();

  if(Result > 0)
  {
    sprintf((char *)Buffer, "TLS URL fetched!\r\nHTTP response code: %u\r\n%u bytes received\r\n", Result, Client.getSize());
    LOG((char *)Buffer);
  }
  else
  {
    LOG("HTTP request failed!\r\nError: ");
    LOG(Result);
    LOG("\r\n");
  }

  // Check for a firmware update
  LOG("Checking for firmware, same URL.\r\n");
  Result = ESPhttpUpdate.update(HTTPSURL, Fingerprint);

  // if successful, ESP will restart
  LOG("Oh noes, update failed!\r\nResult:");
  LOG(Result);
  LOG("\r\n");

  switch(Result) {
    case HTTP_UPDATE_FAILED:
        sprintf((char *)Buffer, "Update failed, error %d: %s\r\n", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
        LOG((char *)Buffer);
        break;
    case HTTP_UPDATE_NO_UPDATES:
        LOG("No update available.\r\n");
        break;
    case HTTP_UPDATE_OK:
        LOG("You shouldn't see this message because I should be rebooting :)\r\n");
        break;
  }  

  // Try again in plaintext...
  LOG("Checking for firmware insecurely...\r\n");
  LOG("Updating from ");
  LOG(HTTPURL);
  LOG("\r\n");

  Result = ESPhttpUpdate.update(HTTPURL);
  
  // if successful, ESP will restart
  LOG("Oh noes, update failed!\r\nResult:");
  LOG(Result);
  LOG("\r\n");

  switch(Result) {
    case HTTP_UPDATE_FAILED:
        sprintf((char *)Buffer, "Update failed, error %d: %s\r\n", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
        LOG((char *)Buffer);
        break;
    case HTTP_UPDATE_NO_UPDATES:
        LOG("No update available.\r\n");
        break;
    case HTTP_UPDATE_OK:
        LOG("You shouldn't see this message because I should be rebooting :)\r\n");
        break;
  }  
}

Debug output:

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1384, room 16 
tail 8
chksum 0x2d
csum 0x2d
v4ceabea9
@cp:0
ld
......................Wifi connected!
My IP is 192.168.30.4
Ready to go!
About to GET <SSL URL>
TLS URL fetched!
HTTP response code: 200
344064 bytes received
Checking for firmware, same URL.
[httpUpdate] HTTP error: connection refused
Oh noes, update failed!
Result:0
Update failed, error -1: HTTP error: connection refused
Checking for firmware insecurely...
Updating from <plaintext URL>
[httpUpdate] Header read fin.
[httpUpdate] Server header:
[httpUpdate]  - code: 200
[httpUpdate]  - len: 344064
[httpUpdate] ESP8266 info:
[httpUpdate]  - free Space: 684032
[httpUpdate]  - current Sketch Size: 344064
[httpUpdate] runUpdate flash...
sleep disable
pm open,type:0 0
[httpUpdate] Update ok
state: 5 -> 0 (0)
rm 0
pm close 7
del if0
usl

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

lucasromeiro · 2018-03-19T17:32:45Z

I have a same problem.... Anyone solve this??

earlephilhower · 2018-03-19T17:43:01Z

Is this the same as #3908 or #4501?

One has to do with an out of memory error during HTTPS update (3908), while the other has found a problem in axTLS where the SSL handshake is expecting that each message is sent in a separate TCP packet but some servers pack multiple SSL handshake messages into a single fragment (4501).

Can you connect to the specified HTTPS sites using a simple WiFiClientSecure::connect? If not then I'd think it was #4501.

aderusha · 2018-04-28T14:58:44Z

Has anyone made this work successfully? I'm seeing the same behavior using the stock httpUpdate example against https://raw.githubusercontent.com.

lucasromeiro · 2018-05-02T14:29:43Z

I have not yet encaged a way to do OTA updates using HTTPS. nothing worked.

me-no-dev · 2018-05-02T14:53:55Z

I would say that 8266 does not have the memory to handle a whole firmware update over SSL. If you can devise a way to get the update in 4K chunks for example, it could work. Though it might take quite a bit longer...

aderusha · 2018-05-02T21:03:35Z

That's pretty much what I had expected. I've implemented a ridiculous workaround by way of a reverse proxy that will deliver the binary from GitHub over plain HTTP.

earlephilhower · 2018-05-02T21:20:32Z

There is a PR for a different SSL which doesn't need dynamic memory allocation after it's initiated (#4273) and supports the http update (you need to pass in a uint8_t[20] instead of a hex string to the constructor and everything else is identical as far as the updater API).

It may not work in your case, but it may be worth a try to avoid the workaround you've had to implement...

lucasromeiro · 2018-05-05T18:39:49Z

Has anyone managed to make it work ?? how to do this using http proxy?

aderusha · 2018-05-05T18:57:32Z

@lucasromeiro my case is pretty simple and the solution I've applied is straightforward. I've spun up an Ubuntu server in Azure, deployed NGINX, registered haswitchplate.com, pointed it at this host, and I've applied the following configuration:

server {
    listen 80;
    server_name haswitchplate.com;

    location /HASwitchPlate.ino.d1_mini.bin {
        proxy_pass https://raw.githubusercontent.com/aderusha/HASwitchPlate/master/Arduino_Sketch/HASwitchPlate.ino.d1_mini.bin;
    }
    location / {
        return 301 https://github.com/aderusha/HASwitchPlate;
    }
}

When a user requests http://haswitchplate.com/HASwitchPlate.ino.d1_mini.bin NGINX will proxy the request for the user, so it will connect to GitHub, start downloading https://raw.githubusercontent.com/aderusha/HASwitchPlate/master/Arduino_Sketch/HASwitchPlate.ino.d1_mini.bin, and hand that over to the connected user via HTTP. Any other URL is redirected to my GitHub repo for this project.

I've also put CloudFlare in front of this to cache the HASwitchPlate.ino.d1_mini.bin file in their CDN to improve global response time and cut down on the bandwidth consumption on my cloud server.

apicquot · 2018-05-09T15:08:42Z

If ESP8266's memory is too limited to work with SSL, it may useful to add in the httpUpdate an option to sign the binary or even fully crypt it? Connection may not be secure but at least the file could be authenticated.

mischmerz · 2018-05-15T16:38:10Z

Ok guys - still not working. My server only "speaks" https, I added the required ciphers and

lucasromeiro · 2018-05-15T16:53:30Z

I could not do it either. I think I will have to create another domain with another server that is http to be able to do the update ota ... because the current library can not manage the memoia.

mischmerz · 2018-05-15T16:53:42Z

It seems to be working now. My server only speaks https, I added the required codes and test the connection:

WiFiClientSecure httpClient;

 if (!httpClient.connect(host, 8080)) {
                Serial.println("connection failed");
                return;
                }
 if (httpClient.verify(fprint,host)) {
              Serial.println("certificate match");
  } else {
            Serial.println("certificate doesn't match");
            return;
            }

It returns with "certificate match" . However, continuing with

result = ESPhttpUpdate.update(host,8080,URI, String(""),String(fprint));

returns with the dreaded "connection refused". I tried

HTTPClient httpClient;
httpClient.begin(host,8080,URI,fprint)

and that also worked ok. I was able to see the URI in the servers log-file. But any attempt to do

ESPhttpUpdate.update(..)

fails with connection refused without log-file entry on the server. So - ESPhttpUpdate.update(..) doesn't even come to the point where it actually requests anything.

HOWEVER: Changing the URI to something that ends in ".bin"

EC:11 PID:21297 OTA Update request: [/ota/test/works.bin]

allowed the update-process to proceed (server log entry above). So it seems the "Connection refused" error is .. emm .. bogus. Has nothing to do with a connection, but the filename(URL, URI) needs to be correct.

Hope that helps.

CRCinAU · 2018-08-03T04:25:04Z

So as this is still open, does anyone have an example of using the BearSSL implementation with httpUpdate?

I'm happy to start tinkering here, but can't quite figure out how to make it all fit together...

liebman · 2018-08-03T12:40:44Z

You could try the changes in #4980

lucasromeiro · 2018-12-16T21:26:57Z

Any news on that?
Can you upgrade OTA with an https link?

mischmerz · 2018-12-17T03:46:58Z

Absolutely. Been doing that for several months now. However, you need to set the HTTPS params somewhat different (server side). My server runs PHP and I set the ciphers to: DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:AES256-SHA:KRB5-DES-CBC3-MD5:KRB5-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:EDH-DSS-DES-CBC3-SHA:DES-CBC3-SHA:DES-CBC3-MD5:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES128-SHA:RC2-CBC-MD5:KRB5-RC4-MD5:KRB5-RC4-SHA:RC4-SHA:RC4-MD5:RC4-MD5:KRB5-DES-CBC-MD5:KRB5-DES-CBC-SHA:EDH-RSA-DES-CBC-SHA:EDH-DSS-DES-CBC-SHA:DES-CBC-SHA:DES-CBC-MD5:EXP-KRB5-RC2-CBC-MD5:EXP-KRB5-DES-CBC-MD5:EXP-KRB5-RC2-CBC-SHA:EXP-KRB5-DES-CBC-SHA:EXP-EDH-RSA-DES-CBC-SHA:EXP-EDH-DSS-DES-CBC-SHA:EXP-DES-CBC-SHA:EXP-RC2-CBC-MD5:EXP-RC2-CBC-MD5:EXP-KRB5-RC4-MD5:EXP-KRB5-RC4-SHA:EXP-RC4-MD5:EXP-RC4-MD5

CRCinAU · 2018-12-17T03:56:25Z

On this topic, what TLS version and which specific ciphers does lwip support?

devyte · 2018-12-17T04:33:57Z

@CRCinAU lwip doesn't, It's just the TCP/IP stack and low level sockets. The secure layer is built on top of that and is provided by BearSSL. If you want to know the ciphers, please check their webpage.

CRCinAU · 2018-12-17T04:39:40Z

Ahhh good.

From: https://bearssl.org/support.html#supported-versions
BearSSL implements TLS 1.0, 1.1 and 1.2.

The supported ciphers are available here:
https://www.bearssl.org/gitweb/?p=BearSSL;a=blob;f=src/ssl/ssl_client_full.c#l52

lucasromeiro · 2018-12-17T10:22:26Z

Hello, I do not understand how it works. is there any complete example? I need to update ota through the file link.

earlephilhower · 2019-01-16T18:39:06Z

Closing as there are examples w/OTA https update using BearSSL thanks to @d-a-v.

CRCinAU · 2019-02-05T10:57:19Z

So I'm starting to revisit this again as in theory its fixed - but I'll be buggered if I can find the examples referenced and attributed to @d-a-v ...

Can someone post a link to these? I can find this issue in Google, but not the new OTA examples using BearSSL etc...

CRCinAU · 2019-02-05T12:27:05Z

Searching further, I came up with this:
https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266httpUpdate/examples/httpUpdateSecure/httpUpdateSecure.ino

Currently, I'm using the following code:

void checkForUpdate() {
	update_status = F("Checking for update...\n");
	BearSSL::WiFiClientSecure UpdateClient;
	UpdateClient.setInsecure();

	t_httpUpdate_return result = ESPhttpUpdate.update(UpdateClient, "https://www.example.com/arduino/update/");

	update_status += F("Returned: ");
	switch(result) {
		case HTTP_UPDATE_FAILED:
			update_status += "Update failed:\nLastError: ";
			update_status += ESPhttpUpdate.getLastError();
			update_status += "\nError: ";
			update_status += ESPhttpUpdate.getLastErrorString().c_str();
			update_status += "\n";
			break;

		case HTTP_UPDATE_NO_UPDATES:
			update_status += F("No Update Available.\n");
			break;

		case HTTP_UPDATE_OK:
			update_status += F("Updated OK.\n");
			break;
	}
}

If I use www.example.com, I get a the following output:
Checking for update...
Returned: Update failed:
LastError: -1
Error: HTTP error: connection refused

If I change this to the IP address of the host, ie a 10.1.1.1 address, then everything works fine. I'm trying to get some debug happening to figure out why this is the case - will post if / when I find anything - but hoping someone has a bit more insight than me :\

d-a-v · 2019-02-05T12:48:06Z

attributed to @d-a-v ...

@earlephilhower was in fact explaining I requested availability of such example for the community. He did the job.

You found the good example.

www.example.com has to be changed in this example to a known webserver offering a binary update. Usually one of your webserver, serving your compiled sketches.

If I change this to the IP address of the host, ie a 10.1.1.1 address, then everything works fine. I'm trying to get some debug happening to figure out why this is the case

"then everything works fine" => I don't get your issue

CRCinAU · 2019-02-05T13:04:33Z

GAH. This turned out to be a networking issue.

The server in question is behind a DMZ. There is a DNAT for public IP -> 10.x.x.x address in a different part of the network. This was masked from debugging on a desktop because this was still working fine over IPv6 - which does not use DNAT. As the ESP8266 bits don't support IPv6, it was failing because the DNAT was causing a RST to be sent back to the initial SYN.

Once I disabled IPv6 on my desktop, I could replicate the same problem. Dammit.

Sorry for the noise - the code I posted works fine - as long as your network is behaving :)

CRCinAU · 2019-02-05T13:50:10Z

On another note though, the binary size has gone from 361344 bytes to 473728 bytes just by including the BearSSL updater.

Is this just a fact of life, or is there optimisations that can be achieved to not interfere with these updates and program size?

EDIT: Further, on a 4Mb flash chip, if I use -Wl,-Teagle.flash.4m1m.ld, does that use the empty space to store retrieved images to then flash to the first 1Mb of the chip? I know I can't do this on 1Mb ESP-01 devices, but this would at least help with the D1 mini's etc..

EDIT2: It seems even a very minimalistic program that only has a simple web page and does an OTA from a https source produces the following:

DATA:    [====      ]  36.3% (used 29728 bytes from 81920 bytes)
PROGRAM: [====      ]  41.2% (used 430632 bytes from 1044464 bytes)

The dependency graph for this simple program is:

Dependency Graph
|-- <ESP8266WiFi> 1.0
|-- <ESP8266WebServer> 1.0
|   |-- <ESP8266WiFi> 1.0
|-- <ESP8266httpUpdate> 1.3
|   |-- <ESP8266HTTPClient> 1.2
|   |   |-- <ESP8266WiFi> 1.0
|   |-- <ESP8266WiFi> 1.0

Ouch.

d-a-v · 2019-02-05T14:11:22Z

As the ESP8266 bits don't support IPv6

I claim it does :)

d-a-v · 2019-02-05T14:30:09Z

does that use the empty space to store retrieved images to then flash to the first 1Mb of the chip?

yes

CRCinAU · 2019-02-05T14:45:20Z

Interesting. Does anyone seem to make an ESP01 with a 4MB flash chip? Is it only a DIY to replace the chip by hand? I can see myself coming across pain in the future ;)

mischmerz · 2019-02-05T16:41:36Z

I don't understand your troubles. I am communicating and OTA'ing all sorts of ESPs via SSL without any problems. Just adapt the SSL/TLS protocols in your server setup.

m.

CRCinAU · 2019-02-07T07:03:41Z

I'm aware I'm starting to wander off-topic here - but if I replace the flash chip with a 4MB one (using there: https://www.aliexpress.com/item/10piece-W25Q32FVSIG-W25Q32FVSSIG-25Q32FVSIG-W25Q32-SOP-8-new-original/32946698708.html ) - what profile do I use for the new flash layout?

I notice esp01.json and esp01_1m.json.

Would I end up having to duplicate esp01_1m.json to something like esp01_4m.json and just change the values of 1048576 to 4194304 and use -Wl,-Teagle.flash.4m1m.ld ?

d-a-v · 2019-02-07T12:44:39Z

Probably, best would be to run the arduino IDE, build and check command line.
We are not aware of your json files.

CRCinAU · 2019-02-08T00:22:10Z

Ok - so back to the problem with OTA updates... I've been using the stage branch in building my code, but even uploading a .bin to $ip/update is causing issues. From a tcpdump, it seems that things are running way too slow for what they should be....

11:17:24.665111 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [S], seq 729678129, win 63360, options [mss 3960,sackOK,TS val 3422414935 ecr 0,nop,wscale 7], length 0
11:17:24.723930 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [S.], seq 6599, ack 729678130, win 5840, options [mss 1250,nop,nop,sackOK], length 0
11:17:24.723982 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], ack 1, win 63360, length 0
11:17:24.724123 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 1:2501, ack 1, win 63360, length 2500: HTTP: POST /update HTTP/1.1
11:17:24.724136 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [P.], seq 2501:5001, ack 1, win 63360, length 2500: HTTP
11:17:24.851079 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [P.], seq 3751:5001, ack 1, win 63360, length 1250: HTTP
11:17:24.928808 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 1, win 5840, options [nop,nop,sack 1 {3751:5001}], length 0
11:17:24.928844 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 1:1251, ack 1, win 63360, length 1250: HTTP: POST /update HTTP/1.1
11:17:24.932083 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 1251, win 4590, options [nop,nop,sack 1 {3751:5001}], length 0
11:17:24.932106 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 1251:2501, ack 1, win 63360, length 1250: HTTP
11:17:24.932115 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 2501:3751, ack 1, win 63360, length 1250: HTTP
11:17:25.195089 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 1251:2501, ack 1, win 63360, length 1250: HTTP
11:17:25.197320 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 2501, win 4590, options [nop,nop,sack 1 {3751:5001}], length 0
11:17:25.197350 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 2501:3751, ack 1, win 63360, length 1250: HTTP
11:17:25.197358 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 5001:6251, ack 1, win 63360, length 1250: HTTP
11:17:25.747092 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 2501:3751, ack 1, win 63360, length 1250: HTTP
11:17:25.752941 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 5001, win 3550, length 0
11:17:25.752968 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 5001:6251, ack 1, win 63360, length 1250: HTTP
11:17:25.752978 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 6251:7501, ack 1, win 63360, length 1250: HTTP
11:17:25.754406 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 5001, win 5010, length 0
11:17:26.835238 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 5001:6251, ack 1, win 63360, length 1250: HTTP
11:17:26.846367 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 6251, win 5220, length 0
11:17:26.846398 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 6251:7501, ack 1, win 63360, length 1250: HTTP
11:17:26.846409 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 7501:8751, ack 1, win 63360, length 1250: HTTP
11:17:28.947233 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 6251:7501, ack 1, win 63360, length 1250: HTTP
11:17:29.186504 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 7501, win 3970, length 0
11:17:29.186539 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 7501:8751, ack 1, win 63360, length 1250: HTTP
11:17:29.186546 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [P.], seq 8751:10001, ack 1, win 63360, length 1250: HTTP
11:17:29.186600 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 7501, win 5430, length 0
11:17:33.683091 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 7501:8751, ack 1, win 63360, length 1250: HTTP
11:17:33.690704 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 8751, win 5640, length 0
11:17:33.690738 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [P.], seq 8751:10001, ack 1, win 63360, length 1250: HTTP
11:17:33.690749 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 10001:11251, ack 1, win 63360, length 1250: HTTP
11:17:42.387072 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [P.], seq 8751:10001, ack 1, win 63360, length 1250: HTTP
11:17:42.437017 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 10001, win 5840, length 0
11:17:42.437047 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 10001:11251, ack 1, win 63360, length 1250: HTTP
11:17:42.437055 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 11251:12501, ack 1, win 63360, length 1250: HTTP
11:17:59.283226 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 10001:11251, ack 1, win 63360, length 1250: HTTP
11:17:59.513470 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 11251, win 4590, length 0
11:17:59.513503 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 11251:12501, ack 1, win 63360, length 1250: HTTP
11:17:59.513512 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 12501:13751, ack 1, win 63360, length 1250: HTTP
11:18:33.075084 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 11251:12501, ack 1, win 63360, length 1250: HTTP
11:18:33.080710 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 12501, win 4800, length 0
11:18:33.080783 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 12501:13751, ack 1, win 63360, length 1250: HTTP
11:18:33.080800 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [P.], seq 13751:15001, ack 1, win 63360, length 1250: HTTP
11:18:38.195061 ARP, Request who-has 10.1.1.156 tell 10.1.1.82, length 28
11:18:38.195201 ARP, Reply 10.1.1.156 is-at 84:f3:eb:0e:fd:24, length 46
11:19:42.707096 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 12501:13751, ack 1, win 63360, length 1250: HTTP
11:19:42.710061 IP 10.1.1.156.http > 10.1.1.82.46978: Flags [.], ack 13751, win 5010, length 0
11:19:42.710099 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [P.], seq 13751:15001, ack 1, win 63360, length 1250: HTTP
11:19:42.710109 IP 10.1.1.82.46978 > 10.1.1.156.http: Flags [.], seq 15001:16251, ack 1, win 63360, length 1250: HTTP
11:19:47.827221 ARP, Request who-has 10.1.1.156 tell 10.1.1.82, length 28
11:19:47.827392 ARP, Reply 10.1.1.156 is-at 84:f3:eb:0e:fd:24, length 46

10.1.1.156 is an ESP01, 10.1.1.82 is my desktop.

If I try to flash via the ESP01 retrieving via a web server, it comes up with the Error 6 read timeout. If I try to upload to the ESP01's onboard web server, that eventually times out in the browser.

I cracked open a new ESP01S to flash via USB and test, and I get the same results when trying to OTA (using both methods).

Where should I start to debug this?

CRCinAU · 2019-02-08T02:29:38Z

Further to my last, if I change the update code to the following, all works well:

WiFiClient UpdateClient;
//BearSSL::WiFiClientSecure UpdateClient;
//UpdateClient.setInsecure();
t_httpUpdate_return result = ESPhttpUpdate.update(UpdateClient, F("http://10.1.1.93/arduino/update/"));

ie - not using a https source is fine.

CRCinAU · 2019-02-08T05:15:33Z

To add more information - the logs from my server showing timestamps during the update process. Changing ONLY the below lines of code vs the result:

Using:

WiFiClient UpdateClient;
t_httpUpdate_return result = ESPhttpUpdate.update(UpdateClient, F("http://10.1.1.93/arduino/update/"));

Log results:

Fri Feb  8 16:11:28 2019 - 10.1.1.6: Host: 10.1.1.93 - Session started. MAC Address: 84:F3:EB:10:75:2C, SDK Version: 3.0.0-dev(c0f7b44)
Fri Feb  8 16:11:28 2019 - 10.1.1.6: [update] Known device - proceeding...
Fri Feb  8 16:11:28 2019 - 10.1.1.6: [update] File associated with device: ClimateControl.bin
Fri Feb  8 16:11:28 2019 - 10.1.1.6: [update] Firmware differs from loaded. Sending File. Header: a332d61201bb8a8d3222c5aa92365a3a, File: 59ac6c233378c9eb3c5cc1ce494df3e3
Fri Feb  8 16:11:31 2019 - 10.1.1.6: [update] Firmware Send complete...
Fri Feb  8 16:11:48 2019 - 10.1.1.6: Host: 10.1.1.93 - Session started. MAC Address: 84:F3:EB:10:75:2C, SDK Version: 3.0.0-dev(c0f7b44)
Fri Feb  8 16:11:48 2019 - 10.1.1.6: [update] Known device - proceeding...
Fri Feb  8 16:11:48 2019 - 10.1.1.6: [update] File associated with device: ClimateControl.bin
Fri Feb  8 16:11:48 2019 - 10.1.1.6: [update] Firmware version matches: 59ac6c233378c9eb3c5cc1ce494df3e3

When switching to BearSSL / https code:

BearSSL::WiFiClientSecure UpdateClient;
UpdateClient.setInsecure();
t_httpUpdate_return result = ESPhttpUpdate.update(UpdateClient, F("https://10.1.1.93/arduino/update/"));

Logs show:

Fri Feb  8 11:32:31 2019 - 10.1.1.254: Host: <hostname> - Session started. MAC Address: 84:F3:EB:10:75:2C, SDK Version: 3.0.0-dev(c0f7b44)
Fri Feb  8 11:32:31 2019 - 10.1.1.254: [update] Known device - proceeding...
Fri Feb  8 11:32:31 2019 - 10.1.1.254: [update] File associated with device: ClimateControl.bin
Fri Feb  8 11:32:31 2019 - 10.1.1.254: [update] Firmware differs from loaded. Sending File. Header: 7df6648c6b82cdee92219cc3bff4ed44, File: 6820e4ece7898d702ef72a98d3a992c5
Fri Feb  8 11:33:57 2019 - 10.1.1.254: [update] Error sending file. Transfer incomplete!
Fri Feb  8 11:36:14 2019 - 10.1.1.254: Host: <hostname> - Session started. MAC Address: 84:F3:EB:10:75:2C, SDK Version: 3.0.0-dev(c0f7b44)
Fri Feb  8 11:36:14 2019 - 10.1.1.254: [update] Known device - proceeding...
Fri Feb  8 11:36:14 2019 - 10.1.1.254: [update] File associated with device: ClimateControl.bin
Fri Feb  8 11:36:14 2019 - 10.1.1.254: [update] Firmware differs from loaded. Sending File. Header: 7df6648c6b82cdee92219cc3bff4ed44, File: 366e4771ee5fcab893bf087cc7ecfcfc
Fri Feb  8 11:37:20 2019 - 10.1.1.254: [update] Error sending file. Transfer incomplete!

CRCinAU · 2019-02-08T05:28:42Z

And now that I've posted all this, I just updated to:

Updating framework-arduinoespressif8266  @ c08efb5

And the problem seems to have gone away....

Both http and https works again:

Fri Feb  8 16:22:33 2019 - 10.1.1.6: Called URL: http://10.1.1.93/arduino/update/ - Session started. MAC Address: 84:F3:EB:10:75:2C, SDK Version: 3.0.0-dev(c0f7b44)
Fri Feb  8 16:22:33 2019 - 10.1.1.6: [update] Known device - proceeding...
Fri Feb  8 16:22:33 2019 - 10.1.1.6: [update] File associated with device: ClimateControl.bin
Fri Feb  8 16:22:33 2019 - 10.1.1.6: [update] Firmware differs from loaded. Sending File. Header: 5dced22d5b1a591ae96b9787e8d4ba13, File: 2381bf8a6fc64b1154dd5d5c19d1fc39
Fri Feb  8 16:22:52 2019 - 10.1.1.6: [update] Firmware Send complete...

Fri Feb  8 16:24:12 2019 - 10.1.1.6: Called URL: https://10.1.1.93/arduino/update/ - Session started. MAC Address: 84:F3:EB:10:75:2C, SDK Version: 3.0.0-dev(c0f7b44)
Fri Feb  8 16:24:12 2019 - 10.1.1.6: [update] Known device - proceeding...
Fri Feb  8 16:24:12 2019 - 10.1.1.6: [update] File associated with device: ClimateControl.bin
Fri Feb  8 16:24:12 2019 - 10.1.1.6: [update] Firmware differs from loaded. Sending File. Header: 2381bf8a6fc64b1154dd5d5c19d1fc39, File: 0c29c9a21d20a5cbe8d6626136d9482a
Fri Feb  8 16:24:39 2019 - 10.1.1.6: [update] Firmware Send complete...

Maybe I just seem to find things at the wrong times 👎

I've also found that the latest lot of updates has broken FastLED as well - apparent due to new typedefs?

FastLED/FastLED#733

hkarthik97 · 2019-11-09T16:47:06Z

Refer this HTTPS is working for me
https://otaesp8266.blogspot.com/2019/11/https-ota-for-esp8266.html

devyte assigned earlephilhower Dec 17, 2018

earlephilhower closed this as completed Jan 16, 2019

OTA update with HTTPS is not working #4400

OTA update with HTTPS is not working #4400

Comments

waghekapil commented Feb 20, 2018

Basic Infos

Hardware

Description

Settings in IDE

Sketch

Debug Messages

kristinpaget commented Feb 21, 2018

Sketch

Debug output:

lucasromeiro commented Mar 19, 2018

earlephilhower commented Mar 19, 2018

aderusha commented Apr 28, 2018

lucasromeiro commented May 2, 2018

me-no-dev commented May 2, 2018

aderusha commented May 2, 2018

earlephilhower commented May 2, 2018

lucasromeiro commented May 5, 2018

aderusha commented May 5, 2018

apicquot commented May 9, 2018 • edited Loading

mischmerz commented May 15, 2018

lucasromeiro commented May 15, 2018

mischmerz commented May 15, 2018

CRCinAU commented Aug 3, 2018

liebman commented Aug 3, 2018

lucasromeiro commented Dec 16, 2018

mischmerz commented Dec 17, 2018

CRCinAU commented Dec 17, 2018

devyte commented Dec 17, 2018

CRCinAU commented Dec 17, 2018

lucasromeiro commented Dec 17, 2018

earlephilhower commented Jan 16, 2019

CRCinAU commented Feb 5, 2019

CRCinAU commented Feb 5, 2019

d-a-v commented Feb 5, 2019

CRCinAU commented Feb 5, 2019

CRCinAU commented Feb 5, 2019 • edited Loading

d-a-v commented Feb 5, 2019

d-a-v commented Feb 5, 2019

CRCinAU commented Feb 5, 2019

mischmerz commented Feb 5, 2019

CRCinAU commented Feb 7, 2019

d-a-v commented Feb 7, 2019

CRCinAU commented Feb 8, 2019 • edited Loading

CRCinAU commented Feb 8, 2019

CRCinAU commented Feb 8, 2019

CRCinAU commented Feb 8, 2019 • edited Loading

hkarthik97 commented Nov 9, 2019

apicquot commented May 9, 2018 •

edited

Loading

CRCinAU commented Feb 5, 2019 •

edited

Loading

CRCinAU commented Feb 8, 2019 •

edited

Loading

CRCinAU commented Feb 8, 2019 •

edited

Loading