Skip to content

can't load x509 certificates #2470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
copercini opened this issue Sep 1, 2016 · 22 comments
Closed

can't load x509 certificates #2470

copercini opened this issue Sep 1, 2016 · 22 comments

Comments

@copercini
Copy link

copercini commented Sep 1, 2016

Basic Infos

Hardware

Hardware: ESP-12
Core Version: git

Description

I am trying use a x509 certificate to connect with a TLS1.2 host, but in if(espClient.loadCertificate(ca)) I get Success to open ca file and not loaded

I have tried with PEM and DER formats

Settings in IDE

Module: Generic ESP8266 Module
Flash Size: 4MB/1MB
CPU Frequency: 80Mhz
Flash Mode: dio
Flash Frequency: 40Mhz
Upload Using: SERIAL
Reset Method: ck

Sketch

...
#include "FS.h"
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

WiFiClientSecure espClient;

void setup() {
  Serial.begin(115200);
  setup_wifi();
  delay(1000);
  if (!SPIFFS.begin()) {
    Serial.println("Failed to mount file system");
    return;
  }

File ca = SPIFFS.open("/ca.crt", "r"); //replace ca.crt eith your uploaded file name
  if (!ca) {
    Serial.println("Failed to open ca file");
  }
  else
  Serial.println("Success to open ca file");

if(espClient.loadCertificate(ca))
  Serial.println("loaded");
  else
  Serial.println("not loaded");

}
...

I am trying use a x509 certificate to connect with a TLS1.2 host, but in if(espClient.loadCertificate(ca)) I get Success to open ca file and not loaded

I have tried with PEM and DER formats

@igrr
Copy link
Member

igrr commented Sep 1, 2016

Certificate should be in DER format. Could you please upload the one you have tried somewhere and post a link?

@igrr
Copy link
Member

igrr commented Sep 1, 2016

Also, please enable debug output (Core+SSL) and add Serial.setDebugOutput(true); after Serial.begin. Then post full output you get.

@copercini
Copy link
Author

Thanks for the fast reply:

WiFi connected
IP address:
192.168.137.109
SPIFFSImpl: allocating 512+180+1400=2092 bytes
SPIFFSImpl: mounting fs @300000, size=fb000, block=2000, page=100
SPIFFSImpl: mount rc=0
Success to open ca file
not loaded
SPIFFS_close: fd=1
Attempting MQTT connection...:ref 1
please start sntp first !
State: sending Client Hello (1)
:sent 72
:rn 1460
:rd 5, 1460, 0
:rdi 1460, 5
:rd 1455, 1460, 5
:rdi 1455, 1455
:c0 1455, 1460
:rn 1310
:rd 1310, 1310, 0
:rdi 1310, 1310
:c0 1310, 1310
State: receiving Server Hello (2)
State: receiving Certificate (11)
Error: invalid handshake
Alert: unexpected message
Alert: close notify
failed, rc=-2 try again in 5 seconds
:sent 7
:rn 7
:rcl
:abort

Here is the zip with certs and the complete sketch
certificates.zip

@chaeplin
Copy link
Contributor

chaeplin commented Sep 1, 2016

I have checked with der format.
openssl x509 -in ca.crt -outform der -out ca.der

if(espClient.loadCertificate(ca)) with

bool WiFiClientSecure::loadCertificate(Stream& stream, size_t size)
{
    if (!_ssl) {
        DEBUGV("is _ssl?");
        return false;
    }
    return _ssl->loadObject(SSL_OBJ_X509_CERT, stream, size);
}
SPIFFSImpl: mount rc=0
Success to open ca file
825
is _ssl?not loaded
SPIFFS_close: fd=1
pm open,type:2 0

if (espClient.loadCACert(ca, ca.size())) with

bool WiFiClientSecure::loadCACert(Stream& stream, size_t size)
{
    if (!_ssl) {
        DEBUGV("is _ssl?");
        return false;
    }
    return _ssl->loadObject(SSL_OBJ_X509_CACERT, stream, size);
}
SPIFFSImpl: mount rc=0
Success to open ca file
825
is _ssl?not loaded
SPIFFS_close: fd=1
pm open,type:2 0

@igrr
Copy link
Member

igrr commented Sep 1, 2016

Ah, this is a regression I think... Previously you were able to load certs before calling connect. Now you can only load them after connect which isn't very useful if you would like to use a client side cert :)
Thanks for pointing this out, will fix.

@copercini
Copy link
Author

@igrr downgrading this commit b412660 the certificate is loaded, but without support to CA root cert

@copercini
Copy link
Author

And downgraded version still doesn't handshake =/

@madpilot
Copy link

It looks like a problem with last axTLS upgrade (35ee060).

The last commit that works for me is d6e38f0

@copercini
Copy link
Author

I think this bug is caused in upgrade to axtls v2. 0.0, here: igrr/axtls-8266#19

@igrr
Copy link
Member

igrr commented Sep 14, 2016

Not exactly. The bug was added to WiFiClientSecure, axTLS is okay.

@hkartadi
Copy link

hkartadi commented Oct 7, 2016

Hi all...

Just wondering if the bug already solved? I think I also encounter the same thing, I am able to load the file from FFS, however it is not able to load the certificate in WifiSecureClient.

Thank you for the help.
Below is some portions of my code:

void certver() {
  SPIFFS.begin();

  Dir dir = SPIFFS.openDir("/");
  while (dir.next()) {
      Serial.print(dir.fileName());
      File f = dir.openFile("r");
      Serial.println(f.size());
  }

  File ca = SPIFFS.open("/ca.crt.der", "r");
  if (!ca) {
    Serial.println("Couldn't load ca cert");
    return;
  }
  if (espClientSecure.loadCACert(ca, ca.size())) {
    Serial.println("Loaded ca cert");
  } else {
    Serial.println("Didn't load ca cert");
    return;
  }

  File cl = SPIFFS.open("/client.crt.der", "r");
  if (!cl) {
    Serial.println("Couldn't load client cert");
    return;
  }
  if (espClientSecure.loadCertificate(cl)) {
    Serial.println("Loaded client cert");
  } else {
    Serial.println("Didn't load client cert");
    return;
  }

  File key = SPIFFS.open("/client.key.der", "r");
  if (!key) {
    Serial.println("Couldn't load key");
    return;
  }

  if (espClientSecure.loadPrivateKey(key)) {
    Serial.println("Loaded Key");
  } else {
    Serial.println("Didn't load Key");
  }
}

OUTPUT:

WiFi connected
IP address:
[192.168.10.94]
/ca.crt.der759
/client.crt.der702
/client.key.der1192
Didn't load ca cert
Attempting MQTT connection...please start sntp first !
failed, rc=-2 try again in 5 seconds

@copercini
Copy link
Author

copercini commented Oct 7, 2016

@hkartadi

The WiFiClientSecure just create a SSLContext in https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/src/WiFiClientSecure.cpp#L315 when you call _connectSSL.

To load the certificates before connect, it should create a SSLContext in loadCACert or loadCertificate or loadPrivateKey.... and verify when called _connectSSL if it has already been created.

@igrr
Copy link
Member

igrr commented May 21, 2017

PR #3271 has the fix for this issue.

@jogaraobommana
Copy link

i am getting this error

Success to open private cert file
loadObject: ssl_obj_memory_load returned -269
private key not loaded

@chegewara
Copy link

Hi @igrr
sorry for bothering with such trivial question, but how to convert pem or der certificate to array like in this example:
https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/examples/HTTPSRequestCACert/CACert.ino

Also what is difference in use setCACert and setCACert_P?

@chadouming
Copy link
Contributor

chadouming commented Aug 27, 2018 via email

@chegewara
Copy link

@chadouming thanks a lot

@DerGuteWolf
Copy link

What is difference in use setCACert and setCACert_P?

@ateeb327
Copy link

ateeb327 commented Mar 20, 2019

Hello, I was having the same issue and finally solved it by using
espClient.loadCACertificate(ca)
after the connect function.
@copercini
Look at my following working code:

`#include "FS.h"
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Update these with values suitable for your network.

const char* ssid = "Your SSID name";
const char* password = "Your SSID password";
const char* mqtt_server = "test.mosquitto.org"; //MQTT broker ip 
//IPAddress server(172, 16, 0, 95);
void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();

}
WiFiClientSecure espClient;
//SPIFFS.begin();
//File ca = SPIFFS.open("/mosquitto_or.der", "r");
//if(!ca) {
//  Serial.println("FIle not Found!");
//  return;  
//}
//
//if(espClient.loadCertificate(ca)) {
//  Serial.println("Loaded Cert");
//} else {
//  Serial.println("Didn't load cert");
//  return;
//}

PubSubClient client(mqtt_server,8883,callback,espClient); //set  MQTT port number to 8883 as per //standard
long lastMsg = 0;
char msg[50];
int value = 0;




void setup_wifi() {
  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}


void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
 
    File ca = SPIFFS.open("/mosquitto_or.der", "r"); //replace ca.crt eith your uploaded file name
  if (!ca) {
   Serial.println("Failed to open ca file");
  }
  else
 Serial.println("Success to open ca file");


    
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("ESP8266Client1")) {
      Serial.println("connected");
      if(espClient.loadCACert(ca))
  Serial.println("loaded");
  else
 Serial.println("not loaded");
      
      // Once connected, publish an announcement...
      client.publish("outTopic", "hello world");
      // ... and resubscribe
      client.subscribe("inTopic");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(115200);
  SPIFFS.begin();
  setup_wifi();
  delay(1000);
  if (!SPIFFS.begin()) {
    Serial.println("Failed to mount file system");
    return;
  }

  client.setServer(mqtt_server, 8883);
client.setCallback(callback);

}



void loop() {

  if (!client.connected()) {
    reconnect(); 
  }
  client.loop();

  long now = millis();
  if (now - lastMsg > 2000) {
    lastMsg = now;
    ++value;
    snprintf (msg, 75, "hello world #%ld", value);
    Serial.print("Publish message: ");
    Serial.println(msg);
    client.publish("outTopic", msg);
  }
}`

@tianmanik
Copy link

in my arduino IDE at tools menu, there is no spiffs flash size to upload the skecth, is it the problem?
my code :
#include "FS.h"
#include <ESP8266WiFi.h>
#include <NTPClient.h>
#include <WiFiUdp.h>
const char* ssid = "MANIK";
const char* password = "manik123";

//-----pzem define-----//
#include <PZEM004Tv30.h>
PZEM004Tv30 pzem(2, 0);
//-----pzem define end-----//

#include <PubSubClient.h>

const char* AWS_endpoint = "xxxxx-ats.iot.ap-southeast-1.amazonaws.com"; //MQTT broker ip

void callback(char* topic, byte* payload, unsigned int length) {
Serial.print("Message arrived [");
Serial.print(topic);
Serial.print("] ");
for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
Serial.println();

}

WiFiClientSecure espClient;
PubSubClient client(AWS_endpoint, 8883,espClient); //set MQTT port number to 8883 as per //standard
long lastMsg = 0;
char msg[90];
int value = 0;

void setup() {
Serial.begin(115200);
Serial.setDebugOutput(true);

espClient.setBufferSizes(512, 512);
Serial.println();
Serial.print("Connecting to ");
Serial.println(ssid);

WiFi.begin(ssid, password);

while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}

Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());

//Initialize File system
if(SPIFFS.begin())
{
Serial.println("SPIFFS got Initialized successfully");
}
else
{
Serial.println("SPIFFS Initialization is failed");
}
//Format File system
if(SPIFFS.format())
{
Serial.println("File system Formatted");
}
else
{
Serial.println("File system formatting error");
}

// Load certificate file
File cert = SPIFFS.open("cert.der", "r"); //replace cert.crt eith your uploaded file name
if (!cert) {
Serial.println("Failed to open cert file");
}
else
Serial.println("Success to open cert file");

delay(1000);

if (espClient.loadCertificate(cert))
Serial.println("cert loaded");
else
Serial.println("cert not loaded");

// Load private key file
File private_key = SPIFFS.open("/private.cer", "r"); //replace private eith your uploaded file name
if (!private_key) {
Serial.println("Failed to open private file");
}
else
Serial.println("Success to open private file");

delay(1000);

if (espClient.loadPrivateKey(private_key))
Serial.println("private key loaded");
else
Serial.println("private key not loaded");

// Load CA file
File ca = SPIFFS.open("/CA.der", "r"); //replace ca eith your uploaded file name
if (!ca) {
Serial.println("Failed to open ca ");
}
else
Serial.println("Success to open ca");

delay(1000);

if(espClient.loadCACert(ca))
Serial.println("ca loaded");
else
Serial.println("ca not loaded");

Serial.print("Heap: "); Serial.println(ESP.getFreeHeap());
}

void loop() {

//-----pzem-----//
float voltage = pzem.voltage();
float current = pzem.current();
float power = pzem.power();
float energy = pzem.energy();
float frequency = pzem.frequency();
float pf = pzem.pf();
if(voltage == NAN || current == NAN || voltage == NAN || power == NAN || energy == NAN || frequency == NAN || pf == NAN){ Serial.println("Reading failed.");}
else{
//publishing data
snprintf (msg, 75, "{"Voltage": #%ld"}", voltage);
snprintf (msg, 75, "{"Current": #%ld"}", current);
snprintf (msg, 75, "{"Power": #%ld"}", power);
snprintf (msg, 75, "{"Energy": #%ld"}", energy);
snprintf (msg, 75, "{"Frequency": #%ld"}", frequency);
snprintf (msg, 75, "{"PF": #%ld"}", pf);
client.publish("outTopic", msg);
Serial.println("Publishing:- ");
Serial.print("Voltage: "); Serial.print(voltage); Serial.println("V");
Serial.print("Current: "); Serial.print(current); Serial.println("A");
Serial.print("Power: "); Serial.print(power); Serial.println("W");
Serial.print("Energy: "); Serial.print(energy, 3); Serial.println("kWh");
Serial.print("Frequency: "); Serial.print(frequency, 1); Serial.println("Hz");
Serial.print("PF: "); Serial.println(pf);

  Serial.println("Success\n");
}

delay(10);
}

output
cant open cert file
fail to loaded cert file
cant open private file
fail to loaded private file
cant open ca file
fail to loaded ca file

@ateebpk2
Copy link

@tianmanik
You need to follow the following steps in order to enable SPIFFS in your Arduino IDE.

  1. Download the tool: https://github.com/esp8266/arduino-esp8266fs-plugin/releases/download/0.5.0/ESP8266FS-0.5.0.zip

  2. In your Arduino sketchbook directory, create tools directory if it doesn’t exist yet.

  3. Unpack the tool into tools directory (the path will look like <home_dir>/Arduino/tools/ESP8266FS/tool/esp8266fs.jar) If upgrading, overwrite the existing JAR file with the newer version.

  4. Restart Arduino IDE.

  5. Open a sketch (or create a new one and save it).

  6. Go to sketch directory (choose Sketch > Show Sketch Folder).

  7. Create a directory named data and any files you want in the file system there.

  8. Make sure you have selected a board, port, and closed Serial Monitor.

  9. If your board requires you to press a button (or other action) to enter bootload mode for flashing a sketch, do that now.

  10. Select Tools > ESP8266 Sketch Data Upload. This should start uploading the files into ESP8266 flash file system. When done, IDE status bar will display SPIFFS Image Uploaded message.

You also need to select FileSystem Size of your CHip in order to upload data to SPIFFS. Look at the following image for Flash Sizes:
flashsize

For more information kindly refer to the ESP8266 Filesystem Documentation at following link:
https://arduino-esp8266.readthedocs.io/en/latest/filesystem.html

@xmoulin
Copy link

xmoulin commented Nov 18, 2020

Work fine for me, over a board ESP8266 : Wemos D1 R1
Thanks you all. I use this topic to do this, some more code to add Root certificate, private cert and private key.

For this test, I use https://www.httpcs.com/fr/convertisseur-ssl to convert the AWS IoT Certificated to ".DER" file

/**
 * Tester sur Wemos D1 R1
 * Utilisation des certificat AWS
 * Code basé sur l'exemple : https://github.com/esp8266/Arduino/issues/2470
 **/
#include "FS.h"
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Update these with values suitable for your network.

const char* ssid = "MySSID";
const char* password = "MyPASSWORD";
const char* mqtt_server = "xxxxx.iot.eu-west-1.amazonaws.com"; //MQTT broker ip

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();

}
WiFiClientSecure espClient;


PubSubClient client(mqtt_server, 8883, callback, espClient); //set  MQTT port number to 8883 as per //standard
long lastMsg = 0;
char msg[50];
int value = 0;


void setup_wifi() {
  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}


void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("reconnect - Heap - before : "); Serial.println(ESP.getFreeHeap());

    // -------------------
    // ------- CA --------
    // -------------------
    File ca = SPIFFS.open("/CAcertificate.der", "r"); //replace ca.crt eith your uploaded file name
    if (!ca)
      Serial.println("Failed to open ca file");
    else
      Serial.println("Success to open ca file");
    if (espClient.loadCACert(ca))
      Serial.println("loaded CA");
    else
      Serial.println("not loaded CA");
    delay (100);

    // -------------------
    // ------- Certif-----
    // -------------------
    File certificate = SPIFFS.open("/certificate.der", "r"); //replace ca.crt eith your uploaded file name
    if (!certificate)
      Serial.println("Failed to open certificate file");
    else
      Serial.println("Success to open certificate file");
    //https://github.com/esp8266/Arduino/issues/2470
    //Serial.println("LoadCertificate");
    //espClient.loadCertificate(ca);
    if (espClient.loadCertificate(certificate))
      Serial.println("loaded certificate");
    else
      Serial.println("not loaded certificate");
    delay(100);

    // -------------------
    // ------- privateKey-
    // -------------------
    //Load private key file
    File private_key = SPIFFS.open("/092b035fee-private.pem.key", "r"); //replace private eith your uploaded file name
    if (!private_key)
      Serial.println("Failed to open private file");
    else
      Serial.println("Success to open private file");
    if (espClient.loadPrivateKey(private_key))
      Serial.println("private key loaded");
    else
      Serial.println("private key not loaded");
    delay(100);


    Serial.print("Attempting MQTT connection...");
    // Attempt to connect

    if (client.connect("ESP8266Client1")) {
      Serial.println("connected");


      // Once connected, publish an announcement...
      client.publish("topic/topic_1", "hello world");
      // ... and resubscribe
      client.subscribe("topic/topic_2");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
  Serial.print("reconnect - Heap - after : "); Serial.println(ESP.getFreeHeap());
}

void setup() {
  Serial.begin(115200);
  SPIFFS.begin();
  setup_wifi();
  delay(1000);
  if (!SPIFFS.begin()) {
    Serial.println("Failed to mount file system");
    return;
  }

  client.setServer(mqtt_server, 8883);
  client.setCallback(callback);

}



void loop() {
  if (!client.connected()) {
    reconnect();
  }
  client.loop();

  long now = millis();
  if (now - lastMsg > 5000) {
    lastMsg = now;
    ++value;
    snprintf (msg, 75, "hello world #%ld", value);
    Serial.print("Publish message: ");
    Serial.println(msg);
    client.publish("topic/topic_1", msg);
    Serial.print("loop - Heap : "); Serial.println(ESP.getFreeHeap());
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests