From 40b760e4f38189abc10721329c820448cdd63f7b Mon Sep 17 00:00:00 2001 From: Steve Kim Date: Tue, 30 Jan 2024 10:50:41 -0800 Subject: [PATCH 1/2] mqtt5 samples prioritization in readme --- documents/MQTT5_Userguide.md | 56 ++++++++--------- samples/README.md | 31 +++++---- samples/mqtt5_pubsub.md | 119 +++++++++++++++++++++++++++++++++++ 3 files changed, 166 insertions(+), 40 deletions(-) diff --git a/documents/MQTT5_Userguide.md b/documents/MQTT5_Userguide.md index 188259fb..30a7a9bf 100644 --- a/documents/MQTT5_Userguide.md +++ b/documents/MQTT5_Userguide.md @@ -10,12 +10,12 @@ * [Connecting to AWS IoT Core](#connecting-to-aws-iot-core) * [How to create a MQTT5 Client based on desired connection method](#how-to-create-a-mqtt5-client-based-on-desired-connection-method) * [Direct MQTT with X509-based mutual TLS](#direct-mqtt-with-x509-based-mutual-tls) - * [MQTT over Websockets with Sigv4 authentication](#mqtt-over-websockets-with-sigv4-authentication) * [Direct MQTT with Custom Authentication](#direct-mqtt-with-custom-authentication) * [Direct MQTT with PKCS11 Method](#direct-mqtt-with-pkcs11-method) * [Direct MQTT with PKCS12 Method](#direct-mqtt-with-pkcs12-method) + * [MQTT over Websockets with Sigv4 authentication](#mqtt-over-websockets-with-sigv4-authentication) * [MQTT over Websockets with Cognito authentication](#mqtt-over-websockets-with-cognito-authentication) - * [HTTP Proxy](#http-proxy) + * [Adding an HTTP Proxy](#adding-an-http-proxy) * [Client Lifecycle Management](#client-lifecycle-management) * [Lifecycle Events](#lifecycle-events) * [Client Operations](#client-operations) @@ -90,30 +90,6 @@ For X509 based mutual TLS, you can create a client where the certificate and pri pri_key_filepath=private_key_filePath)) ``` -#### **MQTT over Websockets with Sigv4 authentication** -Sigv4-based authentication requires a credentials provider capable of sourcing valid AWS credentials. Sourced credentials -will sign the websocket upgrade request made by the client while connecting. The default credentials provider chain supported by -the SDK is capable of resolving credentials in a variety of environments according to a chain of priorities: - -```Environment -> Profile (local file system) -> STS Web Identity -> IMDS (ec2) or ECS``` - -If the default credentials provider chain and built-in AWS region extraction logic are sufficient, you do not need to specify -any additional configuration: - -```python - # The signing region. e.x.: 'us-east-1' - signing_region = "" - credentials_provider = auth.AwsCredentialsProvider.new_default_chain() - - # other builder configurations can be added using **kwargs in the builder - - # Create an MQTT5 Client using mqtt5_client_builder - client = mqtt5_client_builder.websockets_with_default_aws_signing( - endpoint = "", - region = signing_region, - credentials_provider=credentials_provider)) -``` - #### **Direct MQTT with Custom Authentication** AWS IoT Core Custom Authentication allows you to use a lambda to gate access to IoT Core resources. For this authentication method, you must supply an additional configuration structure containing fields relevant to AWS IoT Core Custom Authentication. @@ -129,7 +105,7 @@ If your custom authenticator does not use signing, you don't specify anything re auth_password = ) ``` -If your custom authorizer uses signing, you must specify the three signed token properties as well. The token signature must be the URI-encoding of the base64 encoding of the digital signature of the token value via the private key associated with the public key that was registered with the custom authorizer. It is your responsibility to URI-encode the token signature. +If your custom authorizer uses signing, you must specify the three signed token properties as well. It is your responsibility to URI-encode the auth_username, auth_authorizer_name, and auth_token_key_name parameters. ```python # other builder configurations can be added using **kwargs in the builder @@ -182,6 +158,30 @@ A MQTT5 direct connection can be made using a PKCS12 file rather than using a PE **Note**: Currently, TLS integration with PKCS#12 is only available on MacOS devices. +#### **MQTT over Websockets with Sigv4 authentication** +Sigv4-based authentication requires a credentials provider capable of sourcing valid AWS credentials. Sourced credentials +will sign the websocket upgrade request made by the client while connecting. The default credentials provider chain supported by +the SDK is capable of resolving credentials in a variety of environments according to a chain of priorities: + +```Environment -> Profile (local file system) -> STS Web Identity -> IMDS (ec2) or ECS``` + +If the default credentials provider chain and built-in AWS region extraction logic are sufficient, you do not need to specify +any additional configuration: + +```python + # The signing region. e.x.: 'us-east-1' + signing_region = "" + credentials_provider = auth.AwsCredentialsProvider.new_default_chain() + + # other builder configurations can be added using **kwargs in the builder + + # Create an MQTT5 Client using mqtt5_client_builder + client = mqtt5_client_builder.websockets_with_default_aws_signing( + endpoint = "", + region = signing_region, + credentials_provider=credentials_provider)) +``` + #### **MQTT over Websockets with Cognito authentication** A MQTT5 websocket connection can be made using Cognito to authenticate rather than the AWS credentials located on the device or via key and certificate. Instead, Cognito can authenticate the connection using a valid Cognito identity ID. This requires a valid Cognito identity ID, which can be retrieved from a Cognito identity pool. A Cognito identity pool can be created from the AWS console. @@ -211,7 +211,7 @@ To create a MQTT5 builder configured for this connection, see the following code **Note**: A Cognito identity ID is different from a Cognito identity pool ID and trying to connect with a Cognito identity pool ID will not work. If you are unable to connect, make sure you are passing a Cognito identity ID rather than a Cognito identity pool ID. -#### **HTTP Proxy** +### **Adding an HTTP Proxy** No matter what your connection transport or authentication method is, you may connect through an HTTP proxy by adding the http_proxy_options keyword argument to the builder: diff --git a/samples/README.md b/samples/README.md index 747fe46b..c35e638f 100644 --- a/samples/README.md +++ b/samples/README.md @@ -1,24 +1,31 @@ # Sample apps for the AWS IoT Device SDK v2 for Python - +## MQTT5 Samples +#### MQTT5 is the recommended MQTT Client. It has many benefits over MQTT311 outlined in the [MQTT5 User Guide](../documents/MQTT5_Userguide.md) * [MQTT5 PubSub](./mqtt5_pubsub.md) -* [PubSub](./pubsub.md) + + [Direct MQTT with X509-based mutual TLS](./mqtt5_pubsub.md#direct-mqtt-with-x509-based-mutual-tls) + + [Direct MQTT with PKCS12 Method](./mqtt5_pubsub.md#direct-mqtt-with-pkcs12-method) + + [MQTT over Websockets with Sigv4 authentication](./mqtt5_pubsub.md#mqtt-over-websockets-with-sigv4-authentication) + + [MQTT over Websockets with Cognito authentication](./mqtt5_pubsub.md#mqtt-over-websockets-with-cognito-authentication) * [MQTT5 Shared Subscription](./mqtt5_shared_subscription.md) +* [MQTT5 PKCS#11 Connect](./mqtt5_pkcs11_connect.md) +* [MQTT5 Custom Authorizer Connect](./mqtt5_custom_authorizer_connect.md) +* [MQTT5 Shadow](./shadow_mqtt5.md) +* [MQTT5 Jobs](./jobs_mqtt5.md) +* [MQTT5 Fleet Provisioning](./fleetprovisioning_mqtt5.md) +## MQTT311 Samples +* [PubSub](./pubsub.md) * [Basic Connect](./basic_connect.md) * [Websocket Connect](./websocket_connect.md) -* [MQTT5 PKCS#11 Connect](./mqtt5_pkcs11_connect.md) * [PKCS#11 Connect](./pkcs11_connect.md) * [PKCS#12 Connect](./pkcs12_connect.md) * [Windows Certificate Connect](./windows_cert_connect/README.md) -* [MQTT5 Custom Authorizer Connect](./mqtt5_custom_authorizer_connect.md) * [Custom Authorizer Connect](./custom_authorizer_connect.md) * [Cognito Connect](./cognito_connect.md) * [X509 Connect](./x509_connect.md) * [Shadow](./shadow.md) -* [MQTT5 Shadow](./shadow_mqtt5.md) * [Jobs](./jobs.md) -* [MQTT5 Jobs](./jobs_mqtt5.md) * [Fleet Provisioning](./fleetprovisioning.md) -* [MQTT5 Fleet Provisioning](./fleetprovisioning_mqtt5.md) +## Other * [Greengrass Discovery](./basic_discovery.md) * [Greengrass IPC](./ipc_greengrass.md) @@ -26,11 +33,11 @@ First, install the `aws-iot-devices-sdk-python-v2` with following the instructions from [Installation](../README.md#Installation). -Each sample README has instructions on how to run each sample with the same name as the sample itself. For example, the [PubSub README](./pubsub.md) is `pubsub.md` and it can be run with the following: +Each sample README has instructions on how to run each sample with the same name as the sample itself. For example, the [MQTT5 PubSub README](./mqtt5_pubsub.md) is `mqtt5_pubsub.md` and it can be run with the following: ``` sh # For Windows: replace 'python3' with 'python' and '/' with '\' -python3 pubsub.py --endpoint --cert --key +python3 mqtt5_pubsub.py --endpoint --cert --key ``` ### Sample Help @@ -39,7 +46,7 @@ All samples will show their options by passing in `--help`. For example: ``` sh # For Windows: replace 'python3' with 'python' and '/' with '\' -python3 pubsub.py --help +python3 mqtt5_pubsub.py --help ``` Which will result in output showing all of the options that can be passed in at the command line, along with descriptions of what each does and whether they are optional or not. @@ -48,9 +55,9 @@ Which will result in output showing all of the options that can be passed in at To enable logging in the samples, you need to pass the `--verbosity` as an additional argument. `--verbosity` controls the level of logging shown. `--verbosity` can be set to `Trace`, `Debug`, `Info`, `Warn`, `Error`, `Fatal`, or `None`. -For example, to run [PubSub](./pubsub/README.md) sample with logging you could use the following: +For example, to run [MQTT5 PubSub](./mqtt5_pubsub.md) sample with logging you could use the following: ``` sh # For Windows: replace 'python3' with 'python' and '/' with '\' -python3 pubsub.py --verbosity Debug +python3 mqtt5_pubsub.py --verbosity Debug ``` diff --git a/samples/mqtt5_pubsub.md b/samples/mqtt5_pubsub.md index ae803659..919b3f28 100644 --- a/samples/mqtt5_pubsub.md +++ b/samples/mqtt5_pubsub.md @@ -71,3 +71,122 @@ You can also pass a Certificate Authority file (CA) if your certificate and key # For Windows: replace 'python3' with 'python' and '/' with '\' python3 mqtt5_pubsub.py --endpoint --cert --key --ca_file ``` + +## **Alternate Connection Configuration Methods supported by AWS IoT Core** +### **Optional Keyword Arguments** +All lifecycle events and the callback for publishes received by the MQTT5 Client should be added to the builder on creation of the Client. A full list of accepted arguments can be found in the API guide. + +* [Direct MQTT with X509-based mutual TLS](#direct-mqtt-with-x509-based-mutual-tls) +* [Direct MQTT with Custom Authentication](./mqtt5_custom_authorizer_connect.md) +* [Direct MQTT with PKCS11 Method](./mqtt5_pkcs11_connect.md) +* [Direct MQTT with PKCS12 Method](#direct-mqtt-with-pkcs12-method) +* [MQTT over Websockets with Sigv4 authentication](#mqtt-over-websockets-with-sigv4-authentication) +* [MQTT over Websockets with Cognito authentication](#mqtt-over-websockets-with-cognito-authentication) +### HTTP Proxy +* [Adding an HTTP Proxy](#adding-an-http-proxy) + +#### **Direct MQTT with X509-based mutual TLS** +For X509 based mutual TLS, you can create a client where the certificate and private key are configured by path: + +```python + # X.509 based certificate file + certificate_file_path = "" + # PKCS#1 or PKCS#8 PEM encoded private key file + private_key_filePath = "" + + # other builder configurations can be added using **kwargs in the builder + + # Create an MQTT5 Client using mqtt5_client_builder + client = mqtt5_client_builder.mtls_from_path( + endpoint = "", + cert_filepath=certificate_file_path, + pri_key_filepath=private_key_filePath)) +``` + +#### **Direct MQTT with PKCS12 Method** + +A MQTT5 direct connection can be made using a PKCS12 file rather than using a PEM encoded private key. To create a MQTT5 builder configured for this connection, see the following code: + +```python + # other builder configurations can be added using **kwargs in the builder + + client = mqtt5_client_builder.mtls_with_pkcs12( + pkcs12_filepath = ", + pkcs12_password = " + endpoint = "") +``` + +**Note**: Currently, TLS integration with PKCS#12 is only available on MacOS devices. + +#### **MQTT over Websockets with Sigv4 authentication** +Sigv4-based authentication requires a credentials provider capable of sourcing valid AWS credentials. Sourced credentials +will sign the websocket upgrade request made by the client while connecting. The default credentials provider chain supported by +the SDK is capable of resolving credentials in a variety of environments according to a chain of priorities: + +```Environment -> Profile (local file system) -> STS Web Identity -> IMDS (ec2) or ECS``` + +If the default credentials provider chain and built-in AWS region extraction logic are sufficient, you do not need to specify +any additional configuration: + +```python + # The signing region. e.x.: 'us-east-1' + signing_region = "" + credentials_provider = auth.AwsCredentialsProvider.new_default_chain() + + # other builder configurations can be added using **kwargs in the builder + + # Create an MQTT5 Client using mqtt5_client_builder + client = mqtt5_client_builder.websockets_with_default_aws_signing( + endpoint = "", + region = signing_region, + credentials_provider=credentials_provider)) +``` + +#### **MQTT over Websockets with Cognito authentication** + +A MQTT5 websocket connection can be made using Cognito to authenticate rather than the AWS credentials located on the device or via key and certificate. Instead, Cognito can authenticate the connection using a valid Cognito identity ID. This requires a valid Cognito identity ID, which can be retrieved from a Cognito identity pool. A Cognito identity pool can be created from the AWS console. + +To create a MQTT5 builder configured for this connection, see the following code: + +```python + # The signing region. e.x.: 'us-east-1' + signing_region = "" + + # See https://docs.aws.amazon.com/general/latest/gr/cognito_identity.html for Cognito endpoints + cognito_endpoint = "cognito-identity." + signing_region + ".amazonaws.com" + cognito_identity_id = "" + credentials_provider = auth.AwsCredentialsProvider.new_cognito( + endpoint=cognito_endpoint, + identity=cognito_identity_id, + tls_ctx=io.ClientTlsContext(TlsContextOptions())) + + # other builder configurations can be added using **kwargs in the builder + + # Create an MQTT5 Client using mqtt5_client_builder + client = mqtt5_client_builder.websockets_with_default_aws_signing( + endpoint = "", + region = signing_region, + credentials_provider=credentials_provider)) +``` + +**Note**: A Cognito identity ID is different from a Cognito identity pool ID and trying to connect with a Cognito identity pool ID will not work. If you are unable to connect, make sure you are passing a Cognito identity ID rather than a Cognito identity pool ID. + +## **Adding an HTTP Proxy** +No matter what your connection transport or authentication method is, you may connect through an HTTP proxy +by adding the http_proxy_options keyword argument to the builder: + +```python + http_proxy_options = http.HttpProxyOptions( + host_name = "", + port = ) + + # Create an MQTT5 Client using mqtt5_client_builder with proxy options as keyword argument + client = mqtt5_client_builder.mtls_from_path( + endpoint = "", + cert_filepath = "", + pri_key_filepath = "", + http_proxy_options = http_proxy_options)) +``` + +SDK Proxy support also includes support for basic authentication and TLS-to-proxy. SDK proxy support does not include any additional +proxy authentication methods (kerberos, NTLM, etc...) nor does it include non-HTTP proxies (SOCKS5, for example). \ No newline at end of file From 617b0697a2a538f54ea7ea01f365d74d745d04e3 Mon Sep 17 00:00:00 2001 From: Steve Kim Date: Tue, 30 Jan 2024 13:40:22 -0800 Subject: [PATCH 2/2] argument correction --- documents/MQTT5_Userguide.md | 4 ++-- samples/mqtt5_pubsub.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/documents/MQTT5_Userguide.md b/documents/MQTT5_Userguide.md index 30a7a9bf..d6944c4f 100644 --- a/documents/MQTT5_Userguide.md +++ b/documents/MQTT5_Userguide.md @@ -77,9 +77,9 @@ For X509 based mutual TLS, you can create a client where the certificate and pri ```python # X.509 based certificate file - certificate_file_path = "" + cert_file_path = "" # PKCS#1 or PKCS#8 PEM encoded private key file - private_key_filePath = "" + pri_key_filepath = "" # other builder configurations can be added using **kwargs in the builder diff --git a/samples/mqtt5_pubsub.md b/samples/mqtt5_pubsub.md index 919b3f28..08c59e72 100644 --- a/samples/mqtt5_pubsub.md +++ b/samples/mqtt5_pubsub.md @@ -90,9 +90,9 @@ For X509 based mutual TLS, you can create a client where the certificate and pri ```python # X.509 based certificate file - certificate_file_path = "" + cert_file_path = "" # PKCS#1 or PKCS#8 PEM encoded private key file - private_key_filePath = "" + pri_key_filepath = "" # other builder configurations can be added using **kwargs in the builder