Receiving 'invalid handshake' during establishing SSL connection to my web server

[sukretniy]

Basic Infos

Hardware

Hardware: ?ESP8266?
Core Version: ?git version?

Description

I'm receiving 'invalid handshake' message during receiving cert from my(www.gsmcounters.com and portal.gsmcounters.com) web servers, but when I try to connect to 'api.github.com', everything goes well.
I've checked supported ciphers on my web server and i have this
SNI: www.gsmcounters.com
TLSv1.0:
server selection: uses client preferences
3-- (key: RSA) RSA_WITH_AES_128_CBC_SHA
3-- (key: RSA) RSA_WITH_AES_256_CBC_SHA
3f- (key: RSA) ECDHE_RSA_WITH_AES_128_CBC_SHA
3f- (key: RSA) ECDHE_RSA_WITH_AES_256_CBC_SHA
TLSv1.1: idem
TLSv1.2:
server selection: uses client preferences
3-- (key: RSA) RSA_WITH_AES_128_CBC_SHA
3-- (key: RSA) RSA_WITH_AES_256_CBC_SHA
3-- (key: RSA) RSA_WITH_AES_128_CBC_SHA256
3-- (key: RSA) RSA_WITH_AES_256_CBC_SHA256
3-- (key: RSA) RSA_WITH_AES_128_GCM_SHA256
3-- (key: RSA) RSA_WITH_AES_256_GCM_SHA384
3f- (key: RSA) ECDHE_RSA_WITH_AES_128_CBC_SHA
3f- (key: RSA) ECDHE_RSA_WITH_AES_256_CBC_SHA
3f- (key: RSA) ECDHE_RSA_WITH_AES_128_CBC_SHA256
3f- (key: RSA) ECDHE_RSA_WITH_AES_256_CBC_SHA384

As I understand ESP8266 supports following ciphers:

TLS_RSA_WITH_AES_128_CBC_SHA256
TLS_RSA_WITH_AES_256_CBC_SHA256
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA

What I'm missing?

Settings in IDE

Module: ?Generic ESP8266 Module?
Flash Size: ?1MB?
CPU Frequency: ?80Mhz?
Flash Mode: ?dio?
Flash Frequency: ?40Mhz?
Upload Using: ?SERIAL?
Reset Method: ?ck?

Sketch

#include <time.h>
#include <ESP8266WiFi.h>
#include <WiFiClient.h> 

WiFiClientSecure client;
char* host = "www.gsmcounters.com";
const int httpsPort = 443;

void setup() {  
  testSSL();
}

void testSSL(){
  Serial.begin(115200);
  Serial.setDebugOutput(true);

  Serial.println("Starting SSL testing. Connecting to wifi.");
  WiFi.begin();
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }  

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

  // Synchronize time useing SNTP. This is necessary to verify that
  // the TLS certificates offered by the server are currently valid.
  Serial.println("Setting time using SNTP");
  configTime(8 * 3600, 0, "pool.ntp.org", "time.nist.gov");
  time_t now = time(nullptr);
  while (now < 1000) {
    delay(500);
    Serial.print(".");
    now = time(nullptr);
  }
  Serial.println("");
  struct tm timeinfo;
  gmtime_r(&now, &timeinfo);
  Serial.print("Current time: ");
  Serial.print(asctime(&timeinfo));

  
  host = "www.gsmcounters.com"; 
  
  Serial.print("connecting to ");
  Serial.println(host);
  if (!client.connect(host, httpsPort)) {
    Serial.println("connection failed");    
  } else {
    Serial.println("connection succeed");
  }

  host = "portal.gsmcounters.com";

  Serial.print("connecting to ");
  Serial.println(host);
  if (!client.connect(host, httpsPort)) {
    Serial.println("connection failed");
  } else {
    Serial.println("connection succeed");
  }

  host = "api.github.com";

  Serial.print("connecting to ");
  Serial.println(host);
  if (!client.connect(host, httpsPort)) {
    Serial.println("connection failed");
  } else {
    Serial.println("connection succeed");
  }  
}
void loop() {
    
}

Debug Messages

Starting SSL testing. Connecting to wifi.
scandone
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 2
cnt

connected with TheKGWNetwork, channel 7
dhcp client start...
wifi evt: 0
.ip:192.168.0.114,mask:255.255.255.0,gw:192.168.0.1
wifi evt: 3
.
WiFi connected
IP address:
192.168.0.114
Setting time using SNTP
please start sntp first !
.
Current time: Sat Sep 30 06:16:52 2017
connecting to www.gsmcounters.com
[hostByName] request IP for: www.gsmcounters.com
[hostByName] Host: www.gsmcounters.com IP: 191.235.177.30
:ref 1
State: sending Client Hello (1)
:wr 106 106 0
:wrc 106 106 0
:sent 106
:rn 1460
:rch 1460, 1219
:rd 5, 2679, 0
:rdi 1460, 5
:rd 2674, 2679, 5
:rdi 1455, 1455
:c 1455, 1460, 2679
:rdi 1219, 1219
:c0 1219, 1219
State: receiving Server Hello (2)
State: receiving Certificate (11)
Error: invalid handshake
:wr 7 7 0
:wrc 7 7 0
Alert: unexpected message
:wr 7 7 0
:wrc 7 7 0
Alert: close notify
connection failed
connecting to portal.gsmcounters.com
[hostByName] request IP for: portal.gsmcounters.com
:er -9 00000000
[hostByName] Host: portal.gsmcounters.com IP: 191.235.177.30
:ur 1
:del
:ref 1
State: sending Client Hello (1)
:wr 109 109 0
:wrc 109 109 0
:sent 109
:rn 1460
:rd 5, 1460, 0
:rdi 1460, 5
:rd 1455, 1460, 5
:rdi 1455, 1455
:c0 1455, 1460
:rn 1240
:rd 1240, 1240, 0
:rdi 1240, 1240
:c0 1240, 1240
State: receiving Server Hello (2)
State: receiving Certificate (11)
Error: invalid handshake
:wr 7 7 0
:wrc 7 7 0
Alert: unexpected message
:wr 7 7 0
:wrc 7 7 0
Alert: close notify
connection failed
connecting to api.github.com
[hostByName] request IP for: api.github.com
:er -9 00000000
[hostByName] Host: api.github.com IP: 192.30.253.117
:ur 1
:del
:ref 1
State: sending Client Hello (1)
:wr 101 101 0
:wrc 101 101 0
:sent 101
:rn 1436
:rch 1436, 1436
:rd 5, 2872, 0
:rdi 1436, 5
:rd 80, 2872, 5
:rdi 1431, 80
State: receiving Server Hello (2)
:rd 5, 2872, 85
:rdi 1351, 5
:rch 2872, 318

:rd 3091, 3190, 90
:rdi 1346, 1346
:c 1346, 1436, 3190
:rdi 1436, 1436
:c 1436, 1436, 1754
:rdi 318, 309
State: receiving Certificate (11)
:rd 5, 318, 309
:rdi 9, 5
:rd 4, 318, 314
:rdi 4, 4
:c0 4, 318
State: receiving Server Hello Done (14)
State: sending Client Key Exchange (16)
:wr 267 267 0
:wrc 256 267 0
:wrc 11 11 0
:wr 6 6 0
:wrc 6 6 0
State: sending Finished (16)
:wr 85 85 0
:wrc 85 85 0
:sent 267
:sent 91
:rn 91
:rd 5, 91, 0
:rdi 91, 5
:rd 1, 91, 5
:rdi 86, 1
:rd 5, 91, 6
:rdi 85, 5
:rd 80, 91, 11
:rdi 80, 80
:c0 80, 91
State: receiving Finished (16)
connection succeed
pm open,type:2 0
:rn 69
:rcl
:abort

messages here

[igrr]

From a very brief check, it looks like axTLS fails while parsing the server certificate handshake message. Wireshark is able to parse the same message correctly, so it must be a bug on axTLS side.

Further debugging needs to be done in process_certificate function in tls1.c.

[sukretniy]

Tried to investigate a bit in Wireshark

The main difference between api.github.com and www.gsmcounters.com, that
api.github.com auto selects Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c) among four supported
and www.gsmcounters.com auto selects Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)

Is there any way to force axTLS to say to web server that it supports only TLS_RSA_WITH_AES_128_CBC_SHA256 (I want server not to choose among the other three)?

[igrr]

Did you get past the server certificate message when connecting to gsmcounters with axTLS? I couldn't, and for the record same behavior happens in the Linux port of axTLS.

[sukretniy]

How can I debug axTLS? Or compile it on windows 10 at least.

[sukretniy]

I managed to setup debug environment (compile axTLS lib). This bug has nothing with my previous assumption. I'll try to investigate deeper to see difference between server handshake messages from my server and api.github.com.

[sukretniy]

Bug in this line

uint8_t *buf = &ssl->bm_data[ssl->dc->bm_proc_index];

It's related to features of handshake protocol:

axTLS sends Client Hello message to server and expects Server Hello message after that Certificate and Server Hello Done messages (api.git.hub behaviour) at this point ssl->dc->bm_proc_index = 0 always.
But instead it receives Multiple Handshake Messages, which contains Server Hello, Certificate and Server Hello Done messages at this point ssl->dc->bm_proc_index=74, that points to the end of previous (Server Hello) message. After that total_cert_len calculated as 11 (because the last byte of previous package=0 and 11 - is type of Certificate message and start of the whole message package)

It works, if that line replace with
uint8_t *buf = &ssl->bm_data[ssl->dc->bm_proc_index > 0 ? ssl->dc->bm_proc_index + 6 : ssl->dc->bm_proc_index];

[jtmoderate876]

+1

I think I am experiencing this problem but don't (yet) have the skills/experience/setup to try and recompile from:
uint8_t *buf = &ssl->bm_data[ssl->dc->bm_proc_index];

To:
uint8_t *buf = &ssl->bm_data[ssl->dc->bm_proc_index > 0 ? ssl->dc->bm_proc_index + 6 : ssl->dc->bm_proc_index];

[jtmoderate876]

@sukretniy - Any chance that you can fix this in github.com/igrr/axtls-8266.
I think the process is something like:
Fork the above, make changes, commit, and wait for @igrr to accept?

[sukretniy]

I can send you compiled axtls lib with correction, so you can check if this is your issue.

[jtmoderate876]

So sorry for delays and I really appreciate your help.
I'm travelling with spotty internet for a a few weeks.
Can you post it somewhere public for a couple weeks or email me or are there other ways here at github (I'm a novice)?