Skip to content

Pub sub sample split #296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Apr 1, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
c316ff5
Simplified command line parsing
TwistedTwigleg Feb 21, 2022
721b992
Fixed tests not working correctly after simplified command line parsing
TwistedTwigleg Feb 21, 2022
bea34e7
Added action support to command line parser
TwistedTwigleg Feb 22, 2022
e3d2350
Added additional command groups
TwistedTwigleg Mar 14, 2022
b52fb25
Merge branch 'main' of https://github.com/aws/aws-iot-device-sdk-pyth…
TwistedTwigleg Mar 16, 2022
5f7ea01
Merge branch 'main' of https://github.com/aws/aws-iot-device-sdk-pyth…
TwistedTwigleg Mar 18, 2022
c2a78bb
Split PubSub sample into smaller connect samples
TwistedTwigleg Mar 23, 2022
623435b
Fixed codebuild tests to work with sample split
TwistedTwigleg Mar 23, 2022
fdd7256
Merge branch 'main' of https://github.com/aws/aws-iot-device-sdk-pyth…
TwistedTwigleg Mar 23, 2022
e560f4c
Merge branch 'main' of https://github.com/aws/aws-iot-device-sdk-pyth…
TwistedTwigleg Mar 24, 2022
ca2e042
Adjusted code based on code review:
TwistedTwigleg Mar 24, 2022
75764fc
Converted Windows Cert PubSub to Windows Cert Connect
TwistedTwigleg Mar 24, 2022
8b8f447
Merge branch 'main' into PubSubSampleSplit
TwistedTwigleg Mar 29, 2022
2777cc0
Merge branch 'main' into PubSubSampleSplit
TwistedTwigleg Apr 1, 2022
4163ca7
Adjusted README to use consistent command format
TwistedTwigleg Apr 1, 2022
ec7d320
Merge branch 'main' into PubSubSampleSplit
TwistedTwigleg Apr 1, 2022
ec0b9bf
Fixed diff being included in fleet provision sample
TwistedTwigleg Apr 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added codebuild/.DS_Store
Binary file not shown.
17 changes: 17 additions & 0 deletions codebuild/samples/connect-linux.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/bin/bash

set -e

env

pushd $CODEBUILD_SRC_DIR/samples/

ENDPOINT=$(aws secretsmanager get-secret-value --secret-id "unit-test/endpoint" --query "SecretString" | cut -f2 -d":" | sed -e 's/[\\\"\}]//g')

echo "Mqtt Direct test"
python3 basic_connect.py --endpoint $ENDPOINT --key /tmp/privatekey.pem --cert /tmp/certificate.pem

echo "Websocket test"
python3 websocket_connect.py --endpoint $ENDPOINT --signing_region us-east-1

popd
3 changes: 2 additions & 1 deletion codebuild/samples/linux-smoke-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ phases:
commands:
- echo Build started on `date`
- $CODEBUILD_SRC_DIR/codebuild/samples/setup-linux.sh
- $CODEBUILD_SRC_DIR/codebuild/samples/connect-linux.sh
- $CODEBUILD_SRC_DIR/codebuild/samples/pubsub-linux.sh
post_build:
commands:
- echo Build completed on `date`
- echo Build completed on `date`
5 changes: 1 addition & 4 deletions codebuild/samples/pubsub-linux.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,7 @@ pushd $CODEBUILD_SRC_DIR/samples/

ENDPOINT=$(aws secretsmanager get-secret-value --secret-id "unit-test/endpoint" --query "SecretString" | cut -f2 -d":" | sed -e 's/[\\\"\}]//g')

echo "Mqtt Direct test"
echo "PubSub test"
python3 pubsub.py --endpoint $ENDPOINT --key /tmp/privatekey.pem --cert /tmp/certificate.pem

echo "Websocket test"
python3 pubsub.py --endpoint $ENDPOINT --use_websocket --signing_region us-east-1

popd
192 changes: 165 additions & 27 deletions samples/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,25 @@
# Sample apps for the AWS IoT Device SDK v2 for Python

* [PubSub](#pubsub)
* [PKCS#11 PubSub](#pkcs11-pubsub)
* [Windows Certificate PubSub](#windows-certificate-pubsub)
* [Basic Connect](#basic-connect)
* [Websocket Connect](#websocket-connect)
* [PKCS#11 Connect](#pkcs11-connect)
* [Windows Certificate Connect](#windows-certificate-connect)
* [Shadow](#shadow)
* [Jobs](#jobs)
* [Fleet Provisioning](#fleet-provisioning)
* [Greengrass Discovery](#greengrass-discovery)

## Build instructions

First, install the aws-iot-devices-sdk-python-v2 with following the instructions from [Installation](../README.md#Installation).

Then change into the samples directory to run the Python commands to execute the samples. You can view the commands of a sample like this:

``` sh
python3 pubsub.py --help
```

## PubSub

This sample uses the
Expand All @@ -21,11 +33,6 @@ Status updates are continually printed to the console.

Source: `samples/pubsub.py`

Run the sample like this:
``` sh
python3 pubsub.py --endpoint <endpoint> --ca_file <file> --cert <file> --key <file>
```

Your Thing's
[Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html)
must provide privileges for this sample to connect, subscribe, publish,
Expand Down Expand Up @@ -70,14 +77,120 @@ and receive.
</pre>
</details>

## PKCS#11 PubSub
Run the sample like this:
``` sh
python3 pubsub.py --endpoint <endpoint> --ca_file <file> --cert <file> --key <file>
```

## Basic Connect

This sample makes an MQTT connection using a certificate and key file. On startup, the device connects to the server using the certificate and key files, and then disconnects.
This sample is for reference on connecting via certificate and key files.

This sample is similar to the basic [PubSub](#pubsub),
Source: `samples/basic_connect.py`

Your Thing's
[Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html)
must provide privileges for this sample to connect, subscribe, publish,
and receive.

<details>
<summary>(see sample policy)</summary>
<pre>
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:client/test-*"
]
}
]
}
</pre>
</details>

Run the sample like this:
``` sh
python3 basic_connect.py --endpoint <endpoint> --ca_file <file> --cert <file> --key <file>
```

## Websocket Connect

This sample makes an MQTT connection via websockets and then disconnects. On startup, the device connects to the server via websockets and then disconnects.
This sample is for reference on connecting via websockets.

Source: `samples/websocket_connect.py`

Your Thing's
[Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html)
must provide privileges for this sample to connect, subscribe, publish,
and receive.

<details>
<summary>(see sample policy)</summary>
<pre>
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:client/test-*"
]
}
]
}
</pre>
</details>

Run the sample like this:
``` sh
python3 websocket_connect.py --endpoint <endpoint> --ca_file <file> --signing_region <signing region>
```

Note that using Websockets will attempt to fetch the AWS credentials from your enviornment variables or local files. See the [authorizing direct AWS](https://docs.aws.amazon.com/iot/latest/developerguide/authorizing-direct-aws.html) page for documentation on how to get the AWS credentials, which then you can set to the `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS`, and `AWS_SESSION_TOKEN` environment variables.

## PKCS#11 Connect

This sample is similar to the [Basic Connect](#basic-connect),
but the private key for mutual TLS is stored on a PKCS#11 compatible smart card or Hardware Security Module (HSM)

WARNING: Unix only. Currently, TLS integration with PKCS#11 is only available on Unix devices.

source: `samples/pkcs11_pubsub.py`
source: `samples/pkcs11_connect.py`

Your Thing's
[Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html)
must provide privileges for this sample to connect, subscribe, publish,
and receive.

<details>
<summary>(see sample policy)</summary>
<pre>
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:client/test-*"
]
}
]
}
</pre>
</details>

To run this sample using [SoftHSM2](https://www.opendnssec.org/softhsm/) as the PKCS#11 device:

Expand Down Expand Up @@ -120,14 +233,14 @@ To run this sample using [SoftHSM2](https://www.opendnssec.org/softhsm/) as the

5) Now you can run the sample:
```sh
python3 pkcs11_pubsub.py --endpoint <xxxx-ats.iot.xxxx.amazonaws.com> --ca_file <AmazonRootCA1.pem> --cert <certificate.pem.crt> --pkcs11_lib <libsofthsm2.so> --pin <user-pin> --token_label <token-label> --key_label <key-label>

python3 pkcs11_connect.py --endpoint <endpoint> --ca_file <path to root CA> --cert <path to certificate> --pkcs11_lib <path to PKCS11 lib> --pin <user-pin> --token_label <token-label> --key_label <key-label>
```

## Windows Certificate PubSub
## Windows Certificate Connect

WARNING: Windows only

This sample is similar to the basic [PubSub](#pubsub),
This sample is similar to the basic [Connect](#basic-connect),
but your certificate and private key are in a
[Windows certificate store](https://docs.microsoft.com/en-us/windows-hardware/drivers/install/certificate-stores),
rather than simply being files on disk.
Expand All @@ -141,7 +254,32 @@ If your certificate and private key are in a
[TPM](https://docs.microsoft.com/en-us/windows/security/information-protection/tpm/trusted-platform-module-overview),,
you would use them by passing their certificate store path.

source: `samples/windows_cert_pubsub.py`
source: `samples/windows_cert_connect.py`

Your Thing's
[Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html)
must provide privileges for this sample to connect, subscribe, publish,
and receive.

<details>
<summary>(see sample policy)</summary>
<pre>
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:client/test-*"
]
}
]
}
</pre>
</details>

To run this sample with a basic certificate from AWS IoT Core:

Expand Down Expand Up @@ -183,7 +321,7 @@ To run this sample with a basic certificate from AWS IoT Core:
4) Now you can run the sample:

```sh
python3 windows_cert_pubsub.py --endpoint xxxx-ats.iot.xxxx.amazonaws.com --root-ca AmazonRootCA.pem --cert CurrentUser\My\A11F8A9B5DF5B98BA3508FBCA575D09570E0D2C6
python3 windows_cert_connect.py --endpoint <endpoint> --ca_file <path to root CA> --cert <path to certificate>
```

## Shadow
Expand Down Expand Up @@ -488,11 +626,11 @@ and `--key` appropriately:

``` sh
python3 fleetprovisioning.py \
--endpoint [your endpoint]-ats.iot.[region].amazonaws.com \
--ca_file [pathToRootCA] \
--cert /tmp/provision.cert.pem \
--key /tmp/provision.private.key \
--template_name [TemplateName] \
--endpoint <endpoint> \
--ca_file <path to root CA> \
--cert <path to certificate> \
--key <path to private key> \
--template_name <template name> \
--template_parameters "{\"SerialNumber\":\"1\",\"DeviceLocation\":\"Seattle\"}"
```

Expand Down Expand Up @@ -528,13 +666,13 @@ using a permanent certificate set, replace the paths specified in the `--cert` a

``` sh
python3 fleetprovisioning.py \
--endpoint [your endpoint]-ats.iot.[region].amazonaws.com \
--ca_file [pathToRootCA] \
--cert /tmp/provision.cert.pem \
--key /tmp/provision.private.key \
--template_name [TemplateName] \
--endpoint <endpoint> \
--ca_file <path to root CA> \
--cert <path to certificate> \
--key <path to key> \
--template_name <template name> \
--template_parameters "{\"SerialNumber\":\"1\",\"DeviceLocation\":\"Seattle\"}" \
--csr /tmp/deviceCert.csr
--csr <path to csr file>
```

## Greengrass Discovery
Expand Down
55 changes: 55 additions & 0 deletions samples/basic_connect.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0.

from uuid import uuid4

# This sample shows how to create a MQTT connection using a certificate file and key file.
# This sample is intended to be used as a reference for making MQTT connections.

# Parse arguments
import command_line_utils
cmdUtils = command_line_utils.CommandLineUtils("Basic Connect - Make a MQTT connection.")
cmdUtils.add_common_mqtt_commands()
cmdUtils.add_common_proxy_commands()
cmdUtils.add_common_logging_commands()
cmdUtils.register_command("key", "<path>", "Path to your key in PEM format.", True, str)
cmdUtils.register_command("cert", "<path>", "Path to your client certificate in PEM format.", True, str)
cmdUtils.register_command("port", "<int>",
"Connection port for direct connection. " +
"AWS IoT supports 433 and 8883 (optional, default=8883).",
False, int)
cmdUtils.register_command("client_id", "<str>",
"Client ID to use for MQTT connection (optional, default='test-*').",
default="test-" + str(uuid4()))
# Needs to be called so the command utils parse the commands
cmdUtils.get_args()

# Callback when connection is accidentally lost.
def on_connection_interrupted(connection, error, **kwargs):
print("Connection interrupted. error: {}".format(error))

# Callback when an interrupted connection is re-established.
def on_connection_resumed(connection, return_code, session_present, **kwargs):
print("Connection resumed. return_code: {} session_present: {}".format(return_code, session_present))


if __name__ == '__main__':
# Create a connection using a certificate and key.
# Note: The data for the connection is gotten from cmdUtils.
# (see build_direct_mqtt_connection for implementation)
mqtt_connection = cmdUtils.build_direct_mqtt_connection(on_connection_interrupted, on_connection_resumed)

print("Connecting to {} with client ID '{}'...".format(
cmdUtils.get_command(cmdUtils.m_cmd_endpoint), cmdUtils.get_command("client_id")))

connect_future = mqtt_connection.connect()

# Future.result() waits until a result is available
connect_future.result()
print("Connected!")

# Disconnect
print("Disconnecting...")
disconnect_future = mqtt_connection.disconnect()
disconnect_future.result()
print("Disconnected!")
Loading