Skip to content

"Chain could not be linked to a trust anchor" using mongoDB stitch #6209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
5 of 6 tasks
AugustoCiuffoletti opened this issue Jun 18, 2019 · 14 comments
Closed
5 of 6 tasks

"Chain could not be linked to a trust anchor" using mongoDB stitch #6209

AugustoCiuffoletti opened this issue Jun 18, 2019 · 14 comments
Assignees
@earlephilhower

Comments

@AugustoCiuffoletti
Copy link

Basic Infos

  • This issue complies with the issue POLICY doc.
  • I have read the documentation at readthedocs and the issue is not addressed there.
  • I have tested that the issue is present in current master branch (aka latest git).
  • I have searched the issue tracker for a similar issue.
  • If there is a stack dump, I have decoded it.
  • I have filled out all fields below.

Platform

  • Hardware: [Wemos D1-mini]
  • Core Version: [2.5.2]
  • Development Env: [Arduino IDE]
  • Operating System: [Ubuntu]

Settings in IDE

  • Module: [Wemos D1 mini r2]
  • Flash Mode: [dio?]
  • Flash Size: [4MB]
  • lwip Variant: [v2 Lower Memory]
  • Reset Method: [?]
  • Flash Frequency: [?]
  • CPU Frequency: [80Mhz]
  • Upload Using: [SERIAL]
  • Upload Speed: [921600] (serial upload only)

I am trying to interface a Wemos with a mongoDB stitch using HTTPS with fingerprint authentication, and I meet the problem in the title. I am successfully running a similar interface with mLab, another database service from the same provider.

To reproduce my problem, start from the HTTPSRequest example in ESP8266WiFi, and replace

const char* host = "api.github.com";
const int httpsPort = 443;

// Use web browser to view and copy
// SHA1 fingerprint of the certificate
const char fingerprint[] PROGMEM = "5F F1 60 31 09 04 3E F2 90 D2 B0 8A 50 38 04 E8 37 9F BC 76";

with

const char* host = "eu-west-1.aws.webhooks.mongodb-stitch.com";
const int httpsPort = 443;

// Use web browser to view and copy
// SHA1 fingerprint of the certificate
const char fingerprint[] PROGMEM = "73 5D 6B A2 F7 ED 7C 72 74 AC A3 F5 67 F0 56 6B 68 3B 4B 47";

Adding the following lines just before the "connection failed" printout around line 60 we have some debugging info. The output on Serial is the following:

...
WiFi connected
IP address: 
192.168.113.133
connecting to eu-west-1.aws.webhooks.mongodb-stitch.com
Using fingerprint '73 5D 6B A2 F7 ED 7C 72 74 AC A3 F5 67 F0 56 6B 68 3B 4B 47'
Chain could not be linked to a trust anchor.
connection failed

Replacing the ''fingerprint()'' method with ''setInsecure()'' everything works fine with no errors.

Inspecting the SSL protocol with

openssl s_client -connect <host>:443 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin

when is eu-west-1.aws.webhooks.mongodb-stitch.com I obtain (only the chain"):

Certificate chain
 0 s:/C=US/ST=New York/L=New York/O=MongoDB, Inc./CN=*.mongodb.com
   i:/C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA
 1 s:/C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA

With the mLab server api.mlab.com (which does not exhibit the problem) the chain is similar:

Certificate chain
 0 s:/C=US/ST=California/L=San Francisco/O=ObjectLabs Corporation/CN=api.mlab.com
   i:/C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA
 1 s:/C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA

My suspect is that either the library or the certificate are not exactly compliant. I started asking you.

Thank you for your attention.
@earlephilhower
Copy link
Collaborator

FWIW the URL for the eu-west-1 stitch can't negotiate a proper connection with Firefox, either, but that might just be some regional restrictions in place so I can't get to it from US-West.

Fingerprints don't actually check chains, they just take the SHA fingerprint of the server and the one you send in and compare. I imagine there are load balancers in between you and the service, so it might be that you're getting the cert for the load balancer which is not always going to be the same as you bounce around.

A quick OpenSSL run on my own server seems to match your fingerprint, though.

The other thing that could be happening is that, technically, the SHA1 fingerprint is taken on a specially formatted ASN.1 document (since reordering elements could cause a different fingerprint). BearSSL doesn't support reordering elements inside the ASN.1 since it could require 10s of KB of RAM and breaks the very cool stream setup they have.

Since you're tying yourself to a specific cert with specific pub/private key already with the fingerprint, I would suggest just extracting the public key from the cert and using it as a KnownKey. You're just as secure as before, because you can't negotiate with anyone who doesn't have that server's private key, anyway.

@AugustoCiuffoletti
Copy link
Author

In fact, I was wondering about the solution in the last lines of your reply.
After posting I noticed that the Common Name in the chain does not match the FQDN (mongodb-stitch.com vs *.mongodb.com)? How strict (or needed) is the match?

@earlephilhower
Copy link
Collaborator

Fingerprint doesn't check anything in the certificate, only that the final hash of its contents matches what is expected. So SNI is ignored, expiry dates, etc. Using knownkey is simple enough and should get you what you want without doing a full validation.

@AugustoCiuffoletti
Copy link
Author

I guess your reply closes the issue.
Thank you very much!

@AugustoCiuffoletti
Copy link
Author

My guess was wrong.

I have tried to use the pubkey as suggested, but the problem is still there. From a reply on the thread I opened on MongoDB community, your words, and a number of posts that I found googling around, my new guess is that the problem is that SNI thing, which is not implemented in BearSSL. Broadly speaking, it corresponds to the "load balancer" situation you mentioned. In the case of MongoDB-Stitch, which is a serverless cloud provision, it is totally justified and, IMHO, unavoidable.

From the other thread (https://jira.mongodb.org/browse/SUPPORT-2675, but you need to be registered to have access) I obtain the following reply to a direct answer:

"Hi Augusto Ciuffoletti, we don't believe there is an issue as we require clients to support SNI such that the right certificate is served. I believe this would be an issue on the Arduino side."

Is there a way out?

@earlephilhower
Copy link
Collaborator

BearSSL supports SNI https://bearssl.org/gitweb/?p=BearSSL;a=blob;f=inc/bearssl_ssl.h;h=f2ecc3782828bb54a18e625b72db46aefb061c5b;hb=HEAD#l2814 which means the connect(host, port) host needs to correspond to the hostname that SNI will present.

We call the API with the hostname you used in your connect() call at

Arduino/libraries/ESP8266WiFi/src/WiFiClientSecureBearSSL.cpp

Line 1075 in 5a47cab

if (!br_ssl_client_reset(_sc.get(), hostName, _session?1:0)) {

@earlephilhower
Copy link
Collaborator

@d-a-v, was there a way to get packet dumps from the 8266? This would be very simple to diagnosis using Wireshark or even plain hex dumps of the SSL handshake.

@AugustoCiuffoletti
Copy link
Author

AugustoCiuffoletti commented Jun 28, 2019 via email

@d-a-v
Copy link
Collaborator

d-a-v commented Jun 28, 2019 via email

@earlephilhower
Copy link
Collaborator

@AugustoCiuffoletti , any luck yet? Again, since it's against your private shard I can't reproduce anything locally so without packet traces I don't think we're going to be able to get very far here...

@AugustoCiuffoletti
Copy link
Author

AugustoCiuffoletti commented Jul 6, 2019 via email

@AugustoCiuffoletti
Copy link
Author

Dear @earle and others,

I've collected the trace for the fingerprint case, which is the initial issue in this thread.

This is the story, useful to reproduce:

  • I created a webhook on my MongoDB service that responds with the string "Hello world" to a GET. It is the skeleton example that is initialized upon webhook creation;

  • I modified as follows the HTTPSRequest example referenced above:

#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <NetDump.h>
#include "secret.h"

const char* ssid = STASSID;
const char* password = STAPSK;

const char* host = "eu-west-1.aws.webhooks.mongodb-stitch.com";
const int httpsPort = 443;

// Use web browser to view and copy
// SHA1 fingerprint of the certificate
const char fingerprint[] PROGMEM = "73 5D 6B A2 F7 ED 7C 72 74 AC A3 F5 67 F0 56 6B 68 3B 4B 47";
//const char fingerprint[] PROGMEM = "5F F1 60 31 09 04 3E F2 90 D2 B0 8A 50 38 04 E8 37 9F BC 76";

void dump (int netif_idx, const char* data, size_t len, int out, int success) {
  (void)success;
  Serial.print(out ? F("out ") : F(" in "));
  Serial.printf("%d ", netif_idx);

  // optional filter example: if (netDump_is_ARP(data))
  {
    //netDump(Serial, data, len);
    netDumpHex(Serial, data, len);
  }
}

void setup() {
  Serial.begin(115200);
  Serial.println();
  Serial.print("connecting to ");
  Serial.println(ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

//  phy_capture = dump;

  // Use WiFiClientSecure class to create TLS connection
  WiFiClientSecure client;
  Serial.print("connecting to ");
  Serial.println(host);

  Serial.printf("Using fingerprint '%s'\n", fingerprint);
  client.setFingerprint(fingerprint);
//  client.setInsecure();

  if (!client.connect(host, httpsPort)) {
    Serial.println("connection failed");
    return;
  }

  String url = "/api/client/v2.0/app/manage-jjtug/service/exampleHTTPS/incoming_webhook/webhook0";
  Serial.print("requesting URL: ");
  Serial.println(url);

  client.print(String("GET ") + url + " HTTP/1.1\r\n" +
               "Host: " + host + "\r\n" +
               "User-Agent: BuildFailureDetectorESP8266\r\n" +
               "Connection: close\r\n\r\n");

  Serial.println("request sent");
  while (client.connected()) {
    String line = client.readStringUntil('\n');
    if (line == "\r") {
      Serial.println("headers received");
      break;
    }
  }
  String line = client.readStringUntil('\n');
  if (line.startsWith("{\"state\":\"success\"")) {
    Serial.println("esp8266/Arduino CI successfull!");
  } else {
    Serial.println("esp8266/Arduino CI has failed");
  }
  Serial.println("reply was:");
  Serial.println("==========");
  Serial.println(line);
  Serial.println("==========");
  Serial.println("closing connection");
}

void loop() {
}

Notes: I pasted in the "dump" function from @d-a-v examples and added the definition of the phy_capture hook, moved the WiFi credential in a secret.h file, changed the definition of host, url and fingerprint.

  • I uploaded the sketch to a WEMOS D1 R1 using the Arduino IDE (V1.8.9) and tested with the setInsecure method first, and then with the setFingerprint. With dump disabled in the first case I obtained the expected response:
WiFi connected
IP address: 
192.168.113.122
connecting to eu-west-1.aws.webhooks.mongodb-stitch.com
Using fingerprint '73 5D 6B A2 F7 ED 7C 72 74 AC A3 F5 67 F0 56 6B 68 3B 4B 47'
requesting URL: /api/client/v2.0/app/manage-jjtug/service/exampleHTTPS/incoming_webhook/webhook0
request sent
headers received
esp8266/Arduino CI has failed
reply was:
==========
"Hello World!"
==========
closing connection

while using the fingerprint:

WiFi connected
IP address: 
192.168.113.122
connecting to eu-west-1.aws.webhooks.mongodb-stitch.com
Using fingerprint '73 5D 6B A2 F7 ED 7C 72 74 AC A3 F5 67 F0 56 6B 68 3B 4B 47'
connection failed

  • I ran the fingerprint case with dump enabled to obtain a trace that I imported in Wireshark so to remove unwanted traffic, with the result shown in the screenshot

  • Finally, from Wireshark I saved the trace in a pcap file that you can analyze

I leave the thread open, right?

Thank you for your help,

Augusto

@earlephilhower
Copy link
Collaborator

earlephilhower commented Jul 14, 2019 •
edited
Loading

I added some debugging info to the FP mismatch case, will do a PR later, but the problem is simply the FP doesn't match:

WiFi connected
IP address: 
192.168.1.154
connecting to eu-west-1.aws.webhooks.mongodb-stitch.com
Using fingerprint '73 5D 6B A2 F7 ED 7C 72 74 AC A3 F5 67 F0 56 6B 68 3B 4B 47'
[hostByName] request IP for: eu-west-1.aws.webhooks.mongodb-stitch.com
[hostByName] Host: eu-west-1.aws.webhooks.mongodb-stitch.com IP: 63.34.75.222
BSSL:_connectSSL: start connection
BSSL:insecure_end_chain: Received cert FP doesn't match
BSSL:insecure_end_chain: wanted   73 5d 6b a2 f7 ed 7c 72 74 ac a3 f5 67 f0 56 6b 68 3b 4b 47 
BSSL:insecure_end_chain: received 85 41 43 19 ca 5a 2e 57 4b 0b d1 c9 44 93 5d f8 eb 30 00 73 
BSSL:_wait_for_handshake: failed
BSSL:Couldn't connect. Error = 'Chain could not be linked to a trust anchor.'
connection failed

A binary dump of the certificate shows it's correct (no SNI issue), and that the SHA1 over it is the received one above.

openssl x509 -in cert.bin -inform der -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            0e:47:fa:df:67:2b:c2:81:f5:61:8a:98:88:f5:b7:32
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
        Validity
            Not Before: Sep  6 00:00:00 2018 GMT
            Not After : Sep 10 12:00:00 2020 GMT
        Subject: C = US, ST = New York, L = New York, O = "MongoDB, Inc.", OU = SRE, CN = *.aws.webhooks.mongodb-stitch.com
        Subject Public Key Info:

-----BEGIN CERTIFICATE-----
MIIG6TCCBdGgAwIBAgIQDkf632crwoH1YYqYiPW3MjANBgkqhkiG9w0BAQsFADBN
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMScwJQYDVQQDEx5E
aWdpQ2VydCBTSEEyIFNlY3VyZSBTZXJ2ZXIgQ0EwHhcNMTgwOTA2MDAwMDAwWhcN
MjAwOTEwMTIwMDAwWjCBhTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3Jr
MREwDwYDVQQHEwhOZXcgWW9yazEWMBQGA1UEChMNTW9uZ29EQiwgSW5jLjEMMAoG
A1UECxMDU1JFMSowKAYDVQQDDCEqLmF3cy53ZWJob29rcy5tb25nb2RiLXN0aXRj
aC5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC8Okm3ychJKodB
2L2pMY6qVrauyeihjq99vQSCLHrFthylrBRU6lYW9bpkh8OdsGDs+YpJL7goPA6t
c+Djqcc/6KOzkRxG/a76iMFx4fDpp3LnUIMX/oZpc+FDA+RBDwWXTTJp6bz2F07p
116m1ywPmB90BmSx3ZKDaEYdOh1KYflP7D3d841ADIwydkax/s15W3Ar7D7fLjHz
hzWLGWdK+NoND/Aw9i+xWvoVL9jYAw1WpWQbWbvGA2YYL0guXnXAdI/DF7S4KhPq
1JbDR5hx10u1dCBsRa/RtqH2VssQ5LcWuqAQc7+mVgflcf8q3MitJ+XD6OVG5iK9
NkfQSfPVAgMBAAGjggOKMIIDhjAfBgNVHSMEGDAWgBQPgGEcgjFh1S8o541GOLQs
4cbZ4jAdBgNVHQ4EFgQUvBsD/4vetZ+oQY1kaUBqlHqEegswTQYDVR0RBEYwRIIh
Ki5hd3Mud2ViaG9va3MubW9uZ29kYi1zdGl0Y2guY29tgh9hd3Mud2ViaG9va3Mu
bW9uZ29kYi1zdGl0Y2guY29tMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggr
BgEFBQcDAQYIKwYBBQUHAwIwawYDVR0fBGQwYjAvoC2gK4YpaHR0cDovL2NybDMu
ZGlnaWNlcnQuY29tL3NzY2Etc2hhMi1nNi5jcmwwL6AtoCuGKWh0dHA6Ly9jcmw0
LmRpZ2ljZXJ0LmNvbS9zc2NhLXNoYTItZzYuY3JsMEwGA1UdIARFMEMwNwYJYIZI
AYb9bAEBMCowKAYIKwYBBQUHAgEWHGh0dHBzOi8vd3d3LmRpZ2ljZXJ0LmNvbS9D
UFMwCAYGZ4EMAQICMHwGCCsGAQUFBwEBBHAwbjAkBggrBgEFBQcwAYYYaHR0cDov
L29jc3AuZGlnaWNlcnQuY29tMEYGCCsGAQUFBzAChjpodHRwOi8vY2FjZXJ0cy5k
aWdpY2VydC5jb20vRGlnaUNlcnRTSEEyU2VjdXJlU2VydmVyQ0EuY3J0MAwGA1Ud
EwEB/wQCMAAwggF9BgorBgEEAdZ5AgQCBIIBbQSCAWkBZwB2AKS5CZC0GFgUh7sT
osxncAo8NZgE+RvfuON3zQ7IDdwQAAABZbAzw1gAAAQDAEcwRQIhAJnFTbzLOUeR
EqARpFH9OFHyPGNr4aMM8n2N6QBt6weFAiBqDqkIUp6qWqJLtvqTQgTLu0ZlTzRP
2A/ezDzqY5XfXQB1AId1v+dZfPiMQ5lfvfNu/1aNR1Y2/0q1YMG06v9eoIMPAAAB
ZbAzxDAAAAQDAEYwRAIgHqSr3tRf5tK133VC5q/UTyHX0iyLkMoqhDmVcXLZSQ4C
IHhHPS4HTSfLITijnTwsLz2Y6nMajgAKYQk/rL9rK3oMAHYAu9nfvB+KcbWTlCOX
qpJ7RzhXlQqrUugakJZkNo4e0YUAAAFlsDPEZQAABAMARzBFAiEApRIwVWnap++7
OYIr/Ius/SM9iKjsnbDMRM57oRwzjYECIC9mO915ScvIStK4lk/f6dROFNHk3Gpj
8qqWXR5VqnERMA0GCSqGSIb3DQEBCwUAA4IBAQDAroxFUn8efzxo9ZzRxyNhTA11
9BNyyUVgROjBbPU7BjSXcw56F0lWGVrCJ/f9YmrqTdTuKJJxX2ALjquIvetBsbD0
dhZMrzmN7c/vUQX+spPnnhZ3ksK7pQwWEhb2kyqzePlMngY7QZN05vvUXX9fJJT7
I4EyHqttEnr06cRAohMZS2bgpn2q+TZGgGDD7NEYm2V+YQ9xKQky6/bSDNE8kP3d
tW24P9TF4nv3YMKC/1fG+7miAgws/B7gcfjtz/fwhvYTnogObv/ez4bZYqXRJNUb
OPyjgAP0jenHMyNYn3OBD8+kvNxUg33F0V3lQm4+9JS45Hqs/ajWy/LMy8Ic
-----END CERTIFICATE-----

If I take the SHA1 over the DER form of the cert, it matches what BearSSL calculated, so BSSL is doing the math right, as far as it knows.

But, OpenSSL calculates the FP you're giving, so there's obviously some difference and OpenSSL is normalizing/dropping fields before doing the SHA1.

As I said before, BearSSL code streams the cert and doesn't do things like reordering or normalizing in order to calculate a FP. It just doesn't have the memory.

You could either use the BSSL calculated FP or use the cert itself and do full X509 validation to connect securely.

@earlephilhower
Copy link
Collaborator

earlephilhower commented Jul 14, 2019 •
edited
Loading

This explains the issue you're hitting:
https://stackoverflow.com/questions/33496031/mismatch-in-keytool-and-openssl-certificate-fingerprint

@or78rXxZ or78rXxZ mentioned this issue Nov 11, 2019
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@earlephilhower earlephilhower

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@earlephilhower @d-a-v @AugustoCiuffoletti