Skip to content

prioritize mqtt5 samples #553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 30 additions & 30 deletions documents/MQTT5_Userguide.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@
* [Connecting to AWS IoT Core](#connecting-to-aws-iot-core)
* [How to create a MQTT5 Client based on desired connection method](#how-to-create-a-mqtt5-client-based-on-desired-connection-method)
* [Direct MQTT with X509-based mutual TLS](#direct-mqtt-with-x509-based-mutual-tls)
* [MQTT over Websockets with Sigv4 authentication](#mqtt-over-websockets-with-sigv4-authentication)
* [Direct MQTT with Custom Authentication](#direct-mqtt-with-custom-authentication)
* [Direct MQTT with PKCS11 Method](#direct-mqtt-with-pkcs11-method)
* [Direct MQTT with PKCS12 Method](#direct-mqtt-with-pkcs12-method)
* [MQTT over Websockets with Sigv4 authentication](#mqtt-over-websockets-with-sigv4-authentication)
* [MQTT over Websockets with Cognito authentication](#mqtt-over-websockets-with-cognito-authentication)
* [HTTP Proxy](#http-proxy)
* [Adding an HTTP Proxy](#adding-an-http-proxy)
* [Client Lifecycle Management](#client-lifecycle-management)
* [Lifecycle Events](#lifecycle-events)
* [Client Operations](#client-operations)
Expand Down Expand Up @@ -77,9 +77,9 @@ For X509 based mutual TLS, you can create a client where the certificate and pri

```python
# X.509 based certificate file
certificate_file_path = "<certificate file path>"
cert_file_path = "<certificate file path>"
# PKCS#1 or PKCS#8 PEM encoded private key file
private_key_filePath = "<private key file path>"
pri_key_filepath = "<private key file path>"

# other builder configurations can be added using **kwargs in the builder

Expand All @@ -90,30 +90,6 @@ For X509 based mutual TLS, you can create a client where the certificate and pri
pri_key_filepath=private_key_filePath))
```

#### **MQTT over Websockets with Sigv4 authentication**
Sigv4-based authentication requires a credentials provider capable of sourcing valid AWS credentials. Sourced credentials
will sign the websocket upgrade request made by the client while connecting. The default credentials provider chain supported by
the SDK is capable of resolving credentials in a variety of environments according to a chain of priorities:

```Environment -> Profile (local file system) -> STS Web Identity -> IMDS (ec2) or ECS```

If the default credentials provider chain and built-in AWS region extraction logic are sufficient, you do not need to specify
any additional configuration:

```python
# The signing region. e.x.: 'us-east-1'
signing_region = "<signing region>"
credentials_provider = auth.AwsCredentialsProvider.new_default_chain()

# other builder configurations can be added using **kwargs in the builder

# Create an MQTT5 Client using mqtt5_client_builder
client = mqtt5_client_builder.websockets_with_default_aws_signing(
endpoint = "<account-specific endpoint>",
region = signing_region,
credentials_provider=credentials_provider))
```

#### **Direct MQTT with Custom Authentication**
AWS IoT Core Custom Authentication allows you to use a lambda to gate access to IoT Core resources. For this authentication method,
you must supply an additional configuration structure containing fields relevant to AWS IoT Core Custom Authentication.
Expand All @@ -129,7 +105,7 @@ If your custom authenticator does not use signing, you don't specify anything re
auth_password = <Binary data value of the password field to be passed to the authorizer lambda>)
```

If your custom authorizer uses signing, you must specify the three signed token properties as well. The token signature must be the URI-encoding of the base64 encoding of the digital signature of the token value via the private key associated with the public key that was registered with the custom authorizer. It is your responsibility to URI-encode the token signature.
If your custom authorizer uses signing, you must specify the three signed token properties as well. It is your responsibility to URI-encode the auth_username, auth_authorizer_name, and auth_token_key_name parameters.

```python
# other builder configurations can be added using **kwargs in the builder
Expand Down Expand Up @@ -182,6 +158,30 @@ A MQTT5 direct connection can be made using a PKCS12 file rather than using a PE

**Note**: Currently, TLS integration with PKCS#12 is only available on MacOS devices.

#### **MQTT over Websockets with Sigv4 authentication**
Sigv4-based authentication requires a credentials provider capable of sourcing valid AWS credentials. Sourced credentials
will sign the websocket upgrade request made by the client while connecting. The default credentials provider chain supported by
the SDK is capable of resolving credentials in a variety of environments according to a chain of priorities:

```Environment -> Profile (local file system) -> STS Web Identity -> IMDS (ec2) or ECS```

If the default credentials provider chain and built-in AWS region extraction logic are sufficient, you do not need to specify
any additional configuration:

```python
# The signing region. e.x.: 'us-east-1'
signing_region = "<signing region>"
credentials_provider = auth.AwsCredentialsProvider.new_default_chain()

# other builder configurations can be added using **kwargs in the builder

# Create an MQTT5 Client using mqtt5_client_builder
client = mqtt5_client_builder.websockets_with_default_aws_signing(
endpoint = "<account-specific endpoint>",
region = signing_region,
credentials_provider=credentials_provider))
```

#### **MQTT over Websockets with Cognito authentication**

A MQTT5 websocket connection can be made using Cognito to authenticate rather than the AWS credentials located on the device or via key and certificate. Instead, Cognito can authenticate the connection using a valid Cognito identity ID. This requires a valid Cognito identity ID, which can be retrieved from a Cognito identity pool. A Cognito identity pool can be created from the AWS console.
Expand Down Expand Up @@ -211,7 +211,7 @@ To create a MQTT5 builder configured for this connection, see the following code

**Note**: A Cognito identity ID is different from a Cognito identity pool ID and trying to connect with a Cognito identity pool ID will not work. If you are unable to connect, make sure you are passing a Cognito identity ID rather than a Cognito identity pool ID.

#### **HTTP Proxy**
### **Adding an HTTP Proxy**
No matter what your connection transport or authentication method is, you may connect through an HTTP proxy
by adding the http_proxy_options keyword argument to the builder:

Expand Down
31 changes: 19 additions & 12 deletions samples/README.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,43 @@
# Sample apps for the AWS IoT Device SDK v2 for Python

## MQTT5 Samples
#### MQTT5 is the recommended MQTT Client. It has many benefits over MQTT311 outlined in the [MQTT5 User Guide](../documents/MQTT5_Userguide.md)
* [MQTT5 PubSub](./mqtt5_pubsub.md)
* [PubSub](./pubsub.md)
+ [Direct MQTT with X509-based mutual TLS](./mqtt5_pubsub.md#direct-mqtt-with-x509-based-mutual-tls)
+ [Direct MQTT with PKCS12 Method](./mqtt5_pubsub.md#direct-mqtt-with-pkcs12-method)
+ [MQTT over Websockets with Sigv4 authentication](./mqtt5_pubsub.md#mqtt-over-websockets-with-sigv4-authentication)
+ [MQTT over Websockets with Cognito authentication](./mqtt5_pubsub.md#mqtt-over-websockets-with-cognito-authentication)
* [MQTT5 Shared Subscription](./mqtt5_shared_subscription.md)
* [MQTT5 PKCS#11 Connect](./mqtt5_pkcs11_connect.md)
* [MQTT5 Custom Authorizer Connect](./mqtt5_custom_authorizer_connect.md)
* [MQTT5 Shadow](./shadow_mqtt5.md)
* [MQTT5 Jobs](./jobs_mqtt5.md)
* [MQTT5 Fleet Provisioning](./fleetprovisioning_mqtt5.md)
## MQTT311 Samples
* [PubSub](./pubsub.md)
* [Basic Connect](./basic_connect.md)
* [Websocket Connect](./websocket_connect.md)
* [MQTT5 PKCS#11 Connect](./mqtt5_pkcs11_connect.md)
* [PKCS#11 Connect](./pkcs11_connect.md)
* [PKCS#12 Connect](./pkcs12_connect.md)
* [Windows Certificate Connect](./windows_cert_connect/README.md)
* [MQTT5 Custom Authorizer Connect](./mqtt5_custom_authorizer_connect.md)
* [Custom Authorizer Connect](./custom_authorizer_connect.md)
* [Cognito Connect](./cognito_connect.md)
* [X509 Connect](./x509_connect.md)
* [Shadow](./shadow.md)
* [MQTT5 Shadow](./shadow_mqtt5.md)
* [Jobs](./jobs.md)
* [MQTT5 Jobs](./jobs_mqtt5.md)
* [Fleet Provisioning](./fleetprovisioning.md)
* [MQTT5 Fleet Provisioning](./fleetprovisioning_mqtt5.md)
## Other
* [Greengrass Discovery](./basic_discovery.md)
* [Greengrass IPC](./ipc_greengrass.md)

### Build instructions

First, install the `aws-iot-devices-sdk-python-v2` with following the instructions from [Installation](../README.md#Installation).

Each sample README has instructions on how to run each sample with the same name as the sample itself. For example, the [PubSub README](./pubsub.md) is `pubsub.md` and it can be run with the following:
Each sample README has instructions on how to run each sample with the same name as the sample itself. For example, the [MQTT5 PubSub README](./mqtt5_pubsub.md) is `mqtt5_pubsub.md` and it can be run with the following:

``` sh
# For Windows: replace 'python3' with 'python' and '/' with '\'
python3 pubsub.py --endpoint <endpoint> --cert <path to certificate> --key <path to private key>
python3 mqtt5_pubsub.py --endpoint <endpoint> --cert <path to certificate> --key <path to private key>
```

### Sample Help
Expand All @@ -39,7 +46,7 @@ All samples will show their options by passing in `--help`. For example:

``` sh
# For Windows: replace 'python3' with 'python' and '/' with '\'
python3 pubsub.py --help
python3 mqtt5_pubsub.py --help
```

Which will result in output showing all of the options that can be passed in at the command line, along with descriptions of what each does and whether they are optional or not.
Expand All @@ -48,9 +55,9 @@ Which will result in output showing all of the options that can be passed in at

To enable logging in the samples, you need to pass the `--verbosity` as an additional argument. `--verbosity` controls the level of logging shown. `--verbosity` can be set to `Trace`, `Debug`, `Info`, `Warn`, `Error`, `Fatal`, or `None`.

For example, to run [PubSub](./pubsub/README.md) sample with logging you could use the following:
For example, to run [MQTT5 PubSub](./mqtt5_pubsub.md) sample with logging you could use the following:

``` sh
# For Windows: replace 'python3' with 'python' and '/' with '\'
python3 pubsub.py <other arguments> --verbosity Debug
python3 mqtt5_pubsub.py <other arguments> --verbosity Debug
```
119 changes: 119 additions & 0 deletions samples/mqtt5_pubsub.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,3 +71,122 @@ You can also pass a Certificate Authority file (CA) if your certificate and key
# For Windows: replace 'python3' with 'python' and '/' with '\'
python3 mqtt5_pubsub.py --endpoint <endpoint> --cert <file> --key <file> --ca_file <file>
```

## **Alternate Connection Configuration Methods supported by AWS IoT Core**
### **Optional Keyword Arguments**
All lifecycle events and the callback for publishes received by the MQTT5 Client should be added to the builder on creation of the Client. A full list of accepted arguments can be found in the API guide.

* [Direct MQTT with X509-based mutual TLS](#direct-mqtt-with-x509-based-mutual-tls)
* [Direct MQTT with Custom Authentication](./mqtt5_custom_authorizer_connect.md)
* [Direct MQTT with PKCS11 Method](./mqtt5_pkcs11_connect.md)
* [Direct MQTT with PKCS12 Method](#direct-mqtt-with-pkcs12-method)
* [MQTT over Websockets with Sigv4 authentication](#mqtt-over-websockets-with-sigv4-authentication)
* [MQTT over Websockets with Cognito authentication](#mqtt-over-websockets-with-cognito-authentication)
### HTTP Proxy
* [Adding an HTTP Proxy](#adding-an-http-proxy)

#### **Direct MQTT with X509-based mutual TLS**
For X509 based mutual TLS, you can create a client where the certificate and private key are configured by path:

```python
# X.509 based certificate file
cert_file_path = "<certificate file path>"
# PKCS#1 or PKCS#8 PEM encoded private key file
pri_key_filepath = "<private key file path>"

# other builder configurations can be added using **kwargs in the builder

# Create an MQTT5 Client using mqtt5_client_builder
client = mqtt5_client_builder.mtls_from_path(
endpoint = "<account-specific endpoint>",
cert_filepath=certificate_file_path,
pri_key_filepath=private_key_filePath))
```

#### **Direct MQTT with PKCS12 Method**

A MQTT5 direct connection can be made using a PKCS12 file rather than using a PEM encoded private key. To create a MQTT5 builder configured for this connection, see the following code:

```python
# other builder configurations can be added using **kwargs in the builder

client = mqtt5_client_builder.mtls_with_pkcs12(
pkcs12_filepath = "<PKCS12 file path>,
pkcs12_password = "<PKCS12 password>
endpoint = "<account-specific endpoint>")
```

**Note**: Currently, TLS integration with PKCS#12 is only available on MacOS devices.

#### **MQTT over Websockets with Sigv4 authentication**
Sigv4-based authentication requires a credentials provider capable of sourcing valid AWS credentials. Sourced credentials
will sign the websocket upgrade request made by the client while connecting. The default credentials provider chain supported by
the SDK is capable of resolving credentials in a variety of environments according to a chain of priorities:

```Environment -> Profile (local file system) -> STS Web Identity -> IMDS (ec2) or ECS```

If the default credentials provider chain and built-in AWS region extraction logic are sufficient, you do not need to specify
any additional configuration:

```python
# The signing region. e.x.: 'us-east-1'
signing_region = "<signing region>"
credentials_provider = auth.AwsCredentialsProvider.new_default_chain()

# other builder configurations can be added using **kwargs in the builder

# Create an MQTT5 Client using mqtt5_client_builder
client = mqtt5_client_builder.websockets_with_default_aws_signing(
endpoint = "<account-specific endpoint>",
region = signing_region,
credentials_provider=credentials_provider))
```

#### **MQTT over Websockets with Cognito authentication**

A MQTT5 websocket connection can be made using Cognito to authenticate rather than the AWS credentials located on the device or via key and certificate. Instead, Cognito can authenticate the connection using a valid Cognito identity ID. This requires a valid Cognito identity ID, which can be retrieved from a Cognito identity pool. A Cognito identity pool can be created from the AWS console.

To create a MQTT5 builder configured for this connection, see the following code:

```python
# The signing region. e.x.: 'us-east-1'
signing_region = "<signing region>"

# See https://docs.aws.amazon.com/general/latest/gr/cognito_identity.html for Cognito endpoints
cognito_endpoint = "cognito-identity." + signing_region + ".amazonaws.com"
cognito_identity_id = "<Cognito identity ID>"
credentials_provider = auth.AwsCredentialsProvider.new_cognito(
endpoint=cognito_endpoint,
identity=cognito_identity_id,
tls_ctx=io.ClientTlsContext(TlsContextOptions()))

# other builder configurations can be added using **kwargs in the builder

# Create an MQTT5 Client using mqtt5_client_builder
client = mqtt5_client_builder.websockets_with_default_aws_signing(
endpoint = "<account-specific endpoint>",
region = signing_region,
credentials_provider=credentials_provider))
```

**Note**: A Cognito identity ID is different from a Cognito identity pool ID and trying to connect with a Cognito identity pool ID will not work. If you are unable to connect, make sure you are passing a Cognito identity ID rather than a Cognito identity pool ID.

## **Adding an HTTP Proxy**
No matter what your connection transport or authentication method is, you may connect through an HTTP proxy
by adding the http_proxy_options keyword argument to the builder:

```python
http_proxy_options = http.HttpProxyOptions(
host_name = "<proxy host>",
port = <proxy port>)

# Create an MQTT5 Client using mqtt5_client_builder with proxy options as keyword argument
client = mqtt5_client_builder.mtls_from_path(
endpoint = "<account-specific endpoint>",
cert_filepath = "<certificate file path>",
pri_key_filepath = "<private key file path>",
http_proxy_options = http_proxy_options))
```

SDK Proxy support also includes support for basic authentication and TLS-to-proxy. SDK proxy support does not include any additional
proxy authentication methods (kerberos, NTLM, etc...) nor does it include non-HTTP proxies (SOCKS5, for example).