Skip to content

Commit 03eb38e

Browse files
authored
Support PKCS#11 for mutual TLS on Unix platforms (#259)
- Update to latest `aws-crt-python`, which exposes PKCS#11 functionality (see awslabs/aws-crt-python#323) - Add `pkcs11_pubsub.py` sample, demonstrating an MQTT connection where the private key is stored in PKCS#11 token. - Add docs for sample
1 parent 39606e2 commit 03eb38e

File tree

4 files changed

+281
-17
lines changed

4 files changed

+281
-17
lines changed

awsiot/mqtt_connection_builder.py

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -249,6 +249,61 @@ def mtls_from_bytes(cert_bytes, pri_key_bytes, **kwargs) -> awscrt.mqtt.Connecti
249249
return _builder(tls_ctx_options, **kwargs)
250250

251251

252+
def mtls_with_pkcs11(*,
253+
pkcs11_lib: awscrt.io.Pkcs11Lib,
254+
user_pin: str,
255+
slot_id: int = None,
256+
token_label: str = None,
257+
private_key_label: str = None,
258+
cert_filepath: str = None,
259+
cert_bytes=None,
260+
**kwargs) -> awscrt.mqtt.Connection:
261+
"""
262+
This builder creates an :class:`awscrt.mqtt.Connection`, configured for an mTLS MQTT connection to AWS IoT,
263+
using a PKCS#11 library for private key operations.
264+
265+
This function takes all :mod:`common arguments<awsiot.mqtt_connection_builder>`
266+
described at the top of this doc, as well as...
267+
268+
Keyword Args:
269+
pkcs11_lib (awscrt.io.Pkcs11Lib): Use this PKCS#11 library
270+
271+
user_pin (Optional[str]): User PIN, for logging into the PKCS#11 token.
272+
Pass `None` to log into a token with a "protected authentication path".
273+
274+
slot_id (Optional[int]): ID of slot containing PKCS#11 token.
275+
If not specified, the token will be chosen based on other criteria (such as token label).
276+
277+
token_label (Optional[str]): Label of the PKCS#11 token to use.
278+
If not specified, the token will be chosen based on other criteria (such as slot ID).
279+
280+
private_key_label (Optional[str]): Label of private key object on PKCS#11 token.
281+
If not specified, the key will be chosen based on other criteria
282+
(such as being the only available private key on the token).
283+
284+
cert_filepath (Optional[str]): Use this X.509 certificate (file on disk).
285+
The certificate must be PEM-formatted. The certificate may be
286+
specified by other means instead (ex: `cert_file_contents`)
287+
288+
cert_bytes (Optional[bytes-like object]):
289+
Use this X.509 certificate (contents in memory).
290+
The certificate must be PEM-formatted. The certificate may be
291+
specified by other means instead (ex: `cert_file_path`)
292+
"""
293+
_check_required_kwargs(**kwargs)
294+
295+
tls_ctx_options = awscrt.io.TlsContextOptions.create_client_with_mtls_pkcs11(
296+
pkcs11_lib=pkcs11_lib,
297+
user_pin=user_pin,
298+
slot_id=slot_id,
299+
token_label=token_label,
300+
private_key_label=private_key_label,
301+
cert_file_path=cert_filepath,
302+
cert_file_contents=cert_bytes)
303+
304+
return _builder(tls_ctx_options, **kwargs)
305+
306+
252307
def websockets_with_default_aws_signing(
253308
region,
254309
credentials_provider,

samples/README.md

Lines changed: 71 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,13 @@
11
# Sample apps for the AWS IoT Device SDK v2 for Python
22

3-
* [pubsub](#pubsub)
4-
* [shadow](#shadow)
5-
* [fleet provisioning](#fleet-provisioning)
3+
* [PubSub](#pubsub)
4+
* [PKCS#11 PubSub](#pkcs11-pubsub)
5+
* [Shadow](#shadow)
6+
* [Jobs](#jobs)
7+
* [Fleet Provisioning](#fleet-provisioning)
68
* [Greengrass Discovery](#greengrass-discovery)
79

8-
## Pubsub
10+
## PubSub
911

1012
This sample uses the
1113
[Message Broker](https://docs.aws.amazon.com/iot/latest/developerguide/iot-message-broker.html)
@@ -67,6 +69,59 @@ and receive.
6769
</pre>
6870
</details>
6971

72+
## PKCS#11 PubSub
73+
74+
This sample is similar to the [Pub-Sub](#pubsub),
75+
but the private key for mutual TLS is stored on a PKCS#11 compatible smart card or Hardware Security Module (HSM)
76+
77+
WARNING: Unix only. Currently, TLS integration with PKCS#11 is only available on Unix devices.
78+
79+
source: `samples/pkcs11_pubsub.py`
80+
81+
To run this sample using [SoftHSM2](https://www.opendnssec.org/softhsm/) as the PKCS#11 device:
82+
83+
1) Create an IoT Thing with a certificate and key if you haven't already.
84+
85+
2) Convert the private key into PKCS#8 format
86+
```sh
87+
openssl pkcs8 -topk8 -in <private.pem.key> -out <private.p8.key> -nocrypt
88+
```
89+
90+
3) Install [SoftHSM2](https://www.opendnssec.org/softhsm/):
91+
```sh
92+
sudo apt install softhsm
93+
```
94+
95+
Check that it's working:
96+
```sh
97+
softhsm2-util --show-slots
98+
```
99+
100+
If this spits out an error message, create a config file:
101+
* Default location: `~/.config/softhsm2/softhsm2.conf`
102+
* This file must specify token dir, default value is:
103+
```
104+
directories.tokendir = /usr/local/var/lib/softhsm/tokens/
105+
```
106+
107+
4) Create token and import private key.
108+
109+
You can use any values for the labels, PINs, etc
110+
```sh
111+
softhsm2-util --init-token --free --label <token-label> --pin <user-pin> --so-pin <so-pin>
112+
```
113+
114+
Note which slot the token ended up in
115+
116+
```sh
117+
softhsm2-util --import <private.p8.key> --slot <slot-with-token> --label <key-label> --id <hex-chars> --pin <user-pin>
118+
```
119+
120+
5) Now you can run the sample:
121+
```sh
122+
python3 pkcs11_pubsub.py --endpoint <xxxx-ats.iot.xxxx.amazonaws.com> --root-ca <AmazonRootCA1.pem> --cert <certificate.pem.crt> --pkcs11-lib <libsofthsm2.so> --pin <user-pin> --token-label <token-label> --key-label <key-label>
123+
124+
70125
## Shadow
71126
72127
This sample uses the AWS IoT
@@ -306,14 +361,14 @@ and receive.
306361

307362
### Fleet Provisioning Detailed Instructions
308363

309-
#### Aws Resource Setup
364+
#### AWS Resource Setup
310365

311366
Fleet provisioning requires some additional AWS resources be set up first. This section documents the steps you need to take to
312367
get the sample up and running. These steps assume you have the AWS CLI installed and the default user/credentials has
313368
sufficient permission to perform all of the listed operations. These steps are based on provisioning setup steps
314369
that can be found at [Embedded C SDK Setup](https://docs.aws.amazon.com/freertos/latest/lib-ref/c-sdk/provisioning/provisioning_tests.html#provisioning_system_tests_setup).
315370

316-
First, create the IAM role that will be needed by the fleet provisioning template. Replace `RoleName` with a name of the role you want to create.
371+
First, create the IAM role that will be needed by the fleet provisioning template. Replace `RoleName` with a name of the role you want to create.
317372
``` sh
318373
aws iam create-role \
319374
--role-name [RoleName] \
@@ -325,17 +380,17 @@ aws iam attach-role-policy \
325380
--role-name [RoleName] \
326381
--policy-arn arn:aws:iam::aws:policy/service-role/AWSIoTThingsRegistration
327382
```
328-
Finally, create the template resource which will be used for provisioning by the demo application. This needs to be done only
329-
once. To create a template, the following AWS CLI command may be used. Replace `TemplateName` with the name of the fleet
330-
provisioning template you want to create. Replace `RoleName` with the name of the role you created previously. Replace
331-
`TemplateJSON` with the template body as a JSON string (containing escape characters). Replace `account` with your AWS
332-
account number.
383+
Finally, create the template resource which will be used for provisioning by the demo application. This needs to be done only
384+
once. To create a template, the following AWS CLI command may be used. Replace `TemplateName` with the name of the fleet
385+
provisioning template you want to create. Replace `RoleName` with the name of the role you created previously. Replace
386+
`TemplateJSON` with the template body as a JSON string (containing escape characters). Replace `account` with your AWS
387+
account number.
333388
``` sh
334389
aws iot create-provisioning-template \
335390
--template-name [TemplateName] \
336391
--provisioning-role-arn arn:aws:iam::[account]:role/[RoleName] \
337392
--template-body "[TemplateJSON]" \
338-
--enabled
393+
--enabled
339394
```
340395
The rest of the instructions assume you have used the following for the template body:
341396
``` sh
@@ -345,13 +400,13 @@ If you use a different body, you may need to pass in different template paramete
345400

346401
#### Running the sample and provisioning using a certificate-key set from a provisioning claim
347402

348-
To run the provisioning sample, you'll need a certificate and key set with sufficient permissions. Provisioning certificates are normally
403+
To run the provisioning sample, you'll need a certificate and key set with sufficient permissions. Provisioning certificates are normally
349404
created ahead of time and placed on your device, but for this sample, we will just create them on the fly. You can also
350405
use any certificate set you've already created if it has sufficient IoT permissions and in doing so, you can skip the step
351406
that calls `create-provisioning-claim`.
352407

353408
We've included a script in the utils folder that creates certificate and key files from the response of calling
354-
`create-provisioning-claim`. These dynamically sourced certificates are only valid for five minutes. When running the command,
409+
`create-provisioning-claim`. These dynamically sourced certificates are only valid for five minutes. When running the command,
355410
you'll need to substitute the name of the template you previously created, and on Windows, replace the paths with something appropriate.
356411

357412
(Optional) Create a temporary provisioning claim certificate set:
@@ -364,7 +419,7 @@ aws iot create-provisioning-claim \
364419
```
365420

366421
The provisioning claim's cert and key set have been written to `/tmp/provision*`. Now you can use these temporary keys
367-
to perform the actual provisioning. If you are not using the temporary provisioning certificate, replace the paths for `--cert`
422+
to perform the actual provisioning. If you are not using the temporary provisioning certificate, replace the paths for `--cert`
368423
and `--key` appropriately:
369424
370425
``` sh
@@ -415,7 +470,7 @@ python3 fleetprovisioning.py \
415470
--key /tmp/provision.private.key \
416471
--templateName [TemplateName] \
417472
--templateParameters "{\"SerialNumber\":\"1\",\"DeviceLocation\":\"Seattle\"}" \
418-
--csr /tmp/deviceCert.csr
473+
--csr /tmp/deviceCert.csr
419474
```
420475

421476
## Greengrass Discovery

samples/pkcs11_pubsub.py

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: Apache-2.0.
3+
4+
import argparse
5+
from awscrt import io, mqtt
6+
from awsiot import mqtt_connection_builder
7+
import sys
8+
import threading
9+
import time
10+
from uuid import uuid4
11+
import json
12+
13+
# This sample is similar to `samples/pubsub.py` but the private key
14+
# for mutual TLS is stored on a PKCS#11 compatible smart card or
15+
# Hardware Security Module (HSM).
16+
#
17+
# See `samples/README.md` for instructions on setting up your PKCS#11 device
18+
# to run this sample.
19+
#
20+
# WARNING: Unix only. Currently, TLS integration with PKCS#11 is only available on Unix devices.
21+
22+
parser = argparse.ArgumentParser(description="Send and receive messages through and MQTT connection.")
23+
parser.add_argument('--endpoint', required=True, help="Your AWS IoT custom endpoint, not including a port. " +
24+
"Ex: \"abcd123456wxyz-ats.iot.us-east-1.amazonaws.com\"")
25+
parser.add_argument('--port', type=int, help="Specify port. AWS IoT supports 443 and 8883. (default: auto)")
26+
parser.add_argument('--cert', required=True, help="File path to your client certificate, in PEM format.")
27+
parser.add_argument('--pkcs11-lib', required=True, help="Path to PKCS#11 library.")
28+
parser.add_argument('--pin', required=True, help="User PIN for logging into PKCS#11 token.")
29+
parser.add_argument('--token-label', help="Label of PKCS#11 token to use. (default: None) ")
30+
parser.add_argument('--slot-id', help="Slot ID containing PKCS#11 token to use. (default: None)")
31+
parser.add_argument('--key-label', help="Label of private key on the PKCS#11 token. (default: None)")
32+
parser.add_argument('--root-ca', help="File path to root certificate authority, in PEM format. (default: None)")
33+
parser.add_argument('--client-id', default="test-" + str(uuid4()),
34+
help="Client ID for MQTT connection. (default: 'test-*')")
35+
parser.add_argument('--topic', default="test/topic",
36+
help="Topic to subscribe to, and publish messages to. (default: 'test/topic')")
37+
parser.add_argument('--message', default="Hello World!",
38+
help="Message to publish. Specify empty string to publish nothing. (default: 'Hello World!')")
39+
parser.add_argument('--count', default=10, type=int, help="Number of messages to publish/receive before exiting. " +
40+
"Specify 0 to run forever. (default: 10)")
41+
parser.add_argument('--verbosity', choices=[x.name for x in io.LogLevel], default=io.LogLevel.NoLogs.name,
42+
help="Logging level. (default: 'NoLogs')")
43+
44+
# Using globals to simplify sample code
45+
args = parser.parse_args()
46+
47+
io.init_logging(getattr(io.LogLevel, args.verbosity), 'stderr')
48+
49+
received_count = 0
50+
received_all_event = threading.Event()
51+
52+
53+
def on_connection_interrupted(connection, error, **kwargs):
54+
# Callback when connection is accidentally lost.
55+
print("Connection interrupted. error: {}".format(error))
56+
57+
58+
def on_connection_resumed(connection, return_code, session_present, **kwargs):
59+
# Callback when an interrupted connection is re-established.
60+
print("Connection resumed. return_code: {} session_present: {}".format(return_code, session_present))
61+
62+
63+
# Callback when the subscribed topic receives a message
64+
def on_message_received(topic, payload, dup, qos, retain, **kwargs):
65+
print("Received message from topic '{}': {}".format(topic, payload))
66+
global received_count
67+
received_count += 1
68+
if received_count == args.count:
69+
received_all_event.set()
70+
71+
72+
if __name__ == '__main__':
73+
# Spin up resources
74+
event_loop_group = io.EventLoopGroup(1)
75+
host_resolver = io.DefaultHostResolver(event_loop_group)
76+
client_bootstrap = io.ClientBootstrap(event_loop_group, host_resolver)
77+
78+
print(f"Loading PKCS#11 library '{args.pkcs11_lib}' ...")
79+
pkcs11_lib = io.Pkcs11Lib(
80+
file=args.pkcs11_lib,
81+
behavior=io.Pkcs11Lib.InitializeFinalizeBehavior.STRICT)
82+
print("Loaded!")
83+
84+
# Create MQTT connection
85+
mqtt_connection = mqtt_connection_builder.mtls_with_pkcs11(
86+
pkcs11_lib=pkcs11_lib,
87+
user_pin=args.pin,
88+
slot_id=int(args.slot_id) if args.slot_id else None,
89+
token_label=args.token_label,
90+
private_key_label=args.key_label,
91+
cert_filepath=args.cert,
92+
endpoint=args.endpoint,
93+
port=args.port,
94+
client_bootstrap=client_bootstrap,
95+
ca_filepath=args.root_ca,
96+
on_connection_interrupted=on_connection_interrupted,
97+
on_connection_resumed=on_connection_resumed,
98+
client_id=args.client_id,
99+
clean_session=False,
100+
keep_alive_secs=30)
101+
102+
print("Connecting to {} with client ID '{}'...".format(
103+
args.endpoint, args.client_id))
104+
105+
connect_future = mqtt_connection.connect()
106+
107+
# Future.result() waits until a result is available
108+
connect_future.result()
109+
print("Connected!")
110+
111+
# Subscribe
112+
print("Subscribing to topic '{}'...".format(args.topic))
113+
subscribe_future, packet_id = mqtt_connection.subscribe(
114+
topic=args.topic,
115+
qos=mqtt.QoS.AT_LEAST_ONCE,
116+
callback=on_message_received)
117+
118+
subscribe_result = subscribe_future.result()
119+
print("Subscribed with {}".format(str(subscribe_result['qos'])))
120+
121+
# Publish message to server desired number of times.
122+
# This step is skipped if message is blank.
123+
# This step loops forever if count was set to 0.
124+
if args.message:
125+
if args.count == 0:
126+
print("Sending messages until program killed")
127+
else:
128+
print("Sending {} message(s)".format(args.count))
129+
130+
publish_count = 1
131+
while (publish_count <= args.count) or (args.count == 0):
132+
message = "{} [{}]".format(args.message, publish_count)
133+
print("Publishing message to topic '{}': {}".format(args.topic, message))
134+
message_json = json.dumps(message)
135+
mqtt_connection.publish(
136+
topic=args.topic,
137+
payload=message_json,
138+
qos=mqtt.QoS.AT_LEAST_ONCE)
139+
time.sleep(1)
140+
publish_count += 1
141+
142+
# Wait for all messages to be received.
143+
# This waits forever if count was set to 0.
144+
if args.count != 0 and not received_all_event.is_set():
145+
print("Waiting for all messages to be received...")
146+
147+
received_all_event.wait()
148+
print("{} message(s) received.".format(received_count))
149+
150+
# Disconnect
151+
print("Disconnecting...")
152+
disconnect_future = mqtt_connection.disconnect()
153+
disconnect_future.result()
154+
print("Disconnected!")

setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ def _load_version():
4040
"Operating System :: OS Independent",
4141
],
4242
install_requires=[
43-
'awscrt==0.12.1',
43+
'awscrt==0.13.0',
4444
],
4545
python_requires='>=3.6',
4646
)

0 commit comments

Comments
 (0)