Examples refresh, take 1

examples/README.md

@@ -0,0 +1,91 @@
+# AWS Encryption SDK Examples
+
+This section features examples that show you
+how to use the AWS Encryption SDK.
+We demonstrate how to use the encryption and decryption APIs
+and how to set up some common configuration patterns.
+
+## APIs
+
+The AWS Encryption SDK provides two high-level APIs:
+one-step APIs that process the entire operation in memory
+and streaming APIs.
+
+You can find examples that demonstrate these APIs
+in the [`examples/src/`](./src) directory root.
+
+## Configuration
+
+To use the library APIs,
+you must provide a configuration that defines
+how you want to protect your data keys.
+
+### Keyrings
+
+Keyrings are the most common way for you to configure the AWS Encryption SDK.
+They determine how the AWS Encryption SDK protects your data.
+You can find these examples in [`examples/src/keyring`](./src/keyring).
+
+### Cryptographic Materials Managers
+
+Keyrings define how your data keys are protected,
+but there is more going on here than just protecting data keys.
+
+Cryptographic materials managers give you higher-level controls
+over how the AWS Encryption SDK protects your data.
+This can include things like
+enforcing the use of certain algorithm suites or encryption context settings,
+reusing data keys across messages,
+or changing how you interact with keyrings.
+You can find these examples in
+[`examples/src/crypto_materials_manager`](./src/crypto_materials_manager).
+
+### Master Key Providers
+
+Before there were keyrings, there were master key providers.
+Master key providers were the original configuration structure
+that we provided for defining how you want to protect your data keys.
+Keyrings provide a simpler experience and often more powerful configuration options,
+but if you need to use master key providers,
+need help migrating from master key providers to keyrings,
+or simply want to see the difference between these configuration experiences,
+you can find these examples in [`examples/src/master_key_provider`](./src/master_key_provider).
+
+## Legacy
+
+This section includes older examples, including examples of using master keys and master key providers in Java and Python.
+You can use them as a reference,
+but we recommend looking at the newer examples
+but we recommend looking at the newer examples, which explain the preferred ways of using this library.
+You can find these examples in [`examples/src/legacy`](./src/legacy).
+
+# Writing Examples
+
+If you want to contribute a new example, that's awesome!
+To make sure that your example is tested in our CI,
+please make sure that it meets the following requirements:
+
+1. The example MUST be a distinct module in the [`examples/src/`](./src) directory.
+1. The example MAY be nested arbitrarily deeply,
+    but every intermediate directory MUST contain a `__init__.py` file
+    so that CPython 2.7 will recognize it as a module.
+1. Every example MUST be CPython 2.7 compatible.
+1. Each example file MUST contain exactly one example.
+1. Each example file MUST contain a function called `run` that runs the example.
+1. If your `run` function needs any of the following inputs,
+    the parameters MUST have the following names:
+    * `aws_kms_cmk` (`str`) : A single AWS KMS CMK ARN.
+        * NOTE: You can assume that automatically discovered credentials have
+            `kms:GenerateDataKey`, `kms:Encrypt`, and `kms:Decrypt` permissions on this CMK.
+    * `aws_kms_generator_cmk` (`str`) : A single AWS KMS CMK ARN to use as a generator key.
+        * NOTE: You can assume that automatically discovered credentials have
+            `kms:GenerateDataKey`, `kms:Encrypt`, and `kms:Decrypt` permissions on this CMK.
+    * `aws_kms_child_cmks` (`List[str]`) : A list of AWS KMS CMK ARNs to use as child keys.
+        * NOTE: You can assume that automatically discovered credentials have
+            `kms:Encrypt` and `kms:Decrypt` permissions on these CMKs.
+    * `source_plaintext` (`bytes`) : Plaintext data to encrypt.
+    * `source_plaintext_filename` (`str`) : A path to a file containing plaintext to encrypt.
+        * NOTE: You can assume that you have write access to the parent directory
+            and that anything you do in that directory will be cleaned up
+            by our test runners.
+1. Any additional parameters MUST be optional.

examples/src/file_streaming_defaults.py

@@ -0,0 +1,82 @@
+# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
+"""
+This example shows how to use the streaming encrypt and decrypt APIs when working with files.
+
+One benefit of using the streaming API is that
+we can check the encryption context in the header before we start decrypting.
+
+This example uses an AWS KMS CMK,
+but you can use other key management options with the AWS Encryption SDK.
+For examples that demonstrate how to use other key management configurations,
+see the ``keyring`` and ``mater_key_provider`` directories.
+"""
+import filecmp
+
+import aws_encryption_sdk
+from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring
+
+
+def run(aws_kms_cmk, source_plaintext_filename):
+    # type: (str, str) -> None
+    """Demonstrate an encrypt/decrypt cycle using the streaming encrypt/decrypt APIs to work with files.
+
+    :param str aws_kms_cmk: AWS KMS CMK ARN to use to protect data keys
+    :param str source_plaintext_filename: Path to plaintext file to encrypt
+    """
+    # We assume that you can also write to the directory containing the plaintext file,
+    # so that is where we will put all of the results.
+    ciphertext_filename = source_plaintext_filename + ".encrypted"
+    decrypted_filename = ciphertext_filename + ".decrypted"
+
+    # Prepare your encryption context.
+    # https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html#encryption-context
+    encryption_context = {
+        "encryption": "context",
+        "is not": "secret",
+        "but adds": "useful metadata",
+        "that can help you": "be confident that",
+        "the data you are handling": "is what you think it is",
+    }
+
+    # Create the keyring that determines how your data keys are protected.
+    keyring = KmsKeyring(generator_key_id=aws_kms_cmk)
+
+    # Open the files you want to work with.
+    with open(source_plaintext_filename, "rb") as plaintext, open(ciphertext_filename, "wb") as ciphertext:
+        # The streaming API provides a context manager.
+        # You can read from it just as you read from a file.
+        with aws_encryption_sdk.stream(
+            mode="encrypt", source=plaintext, encryption_context=encryption_context, keyring=keyring
+        ) as encryptor:
+            # Iterate through the segments in the context manager
+            # and write the results to the ciphertext.
+            for segment in encryptor:
+                ciphertext.write(segment)
+
+    # Verify that the ciphertext and plaintext are different.
+    assert not filecmp.cmp(source_plaintext_filename, ciphertext_filename)
+
+    # Open the files you want to work with.
+    with open(ciphertext_filename, "rb") as ciphertext, open(decrypted_filename, "wb") as decrypted:
+        # Decrypt your encrypted data.
+        #
+        # We do not need to specify the encryption context on decrypt
+        # because the header message includes the encryption context.
+        with aws_encryption_sdk.stream(mode="decrypt", source=ciphertext, keyring=keyring) as decryptor:
+            # Check the encryption context in the header before we start decrypting.
+            #
+            # Verify that the encryption context used in the decrypt operation includes
+            # the encryption context that you specified when encrypting.
+            # The AWS Encryption SDK can add pairs, so don't require an exact match.
+            #
+            # In production, always use a meaningful encryption context.
+            assert set(encryption_context.items()) <= set(decryptor.header.encryption_context.items())
+
+            # Now that we are more confident that we will decrypt the right message,
+            # we can start decrypting.
+            for segment in decryptor:
+                decrypted.write(segment)
+
+    # Verify that the decrypted plaintext is identical to the original plaintext.
+    assert filecmp.cmp(source_plaintext_filename, decrypted_filename)

examples/src/in_memory_streaming_defaults.py

@@ -0,0 +1,79 @@
+# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
+"""
+This example shows how to use the streaming encrypt and decrypt APIs when working in memory.
+
+One benefit of using the streaming API is that
+we can check the encryption context in the header before we start decrypting.
+
+For the purposes of this example, we demonstrate using AWS KMS,
+but you can use other key management options with the AWS Encryption SDK.
+For examples that demonstrate how to use other key management configurations,
+see the ``keyring`` and ``mater_key_provider`` directories.
+"""
+import io
+
+import aws_encryption_sdk
+from aws_encryption_sdk.keyrings.aws_kms import KmsKeyring
+
+
+def run(aws_kms_cmk, source_plaintext):
+    # type: (str, bytes) -> None
+    """Demonstrate an encrypt/decrypt cycle using the streaming encrypt/decrypt APIs in-memory.
+
+    :param str aws_kms_cmk: AWS KMS CMK ARN to use to protect data keys
+    :param bytes source_plaintext: Plaintext to encrypt
+    """
+    # Prepare your encryption context.
+    # https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html#encryption-context
+    encryption_context = {
+        "encryption": "context",
+        "is not": "secret",
+        "but adds": "useful metadata",
+        "that can help you": "be confident that",
+        "the data you are handling": "is what you think it is",
+    }
+
+    # Create the keyring that determines how your data keys are protected.
+    keyring = KmsKeyring(generator_key_id=aws_kms_cmk)
+
+    ciphertext = io.BytesIO()
+
+    # The streaming API provides a context manager.
+    # You can read from it just as you read from a file.
+    with aws_encryption_sdk.stream(
+        mode="encrypt", source=source_plaintext, encryption_context=encryption_context, keyring=keyring
+    ) as encryptor:
+        # Iterate through the segments in the context manager
+        # and write the results to the ciphertext.
+        for segment in encryptor:
+            ciphertext.write(segment)
+
+    # Verify that the ciphertext and plaintext are different.
+    assert ciphertext.getvalue() != source_plaintext
+
+    # Reset the ciphertext stream position so that we can read from the beginning.
+    ciphertext.seek(0)
+
+    # Decrypt your encrypted data.
+    #
+    # We do not need to specify the encryption context on decrypt
+    # because the header message includes the encryption context.
+    decrypted = io.BytesIO()
+    with aws_encryption_sdk.stream(mode="decrypt", source=ciphertext, keyring=keyring) as decryptor:
+        # Check the encryption context in the header before we start decrypting.
+        #
+        # Verify that the encryption context used in the decrypt operation includes
+        # the encryption context that you specified when encrypting.
+        # The AWS Encryption SDK can add pairs, so don't require an exact match.
+        #
+        # In production, always use a meaningful encryption context.
+        assert set(encryption_context.items()) <= set(decryptor.header.encryption_context.items())
+
+        # Now that we are more confident that we will decrypt the right message,
+        # we can start decrypting.
+        for segment in decryptor:
+            decrypted.write(segment)
+
+    # Verify that the decrypted plaintext is identical to the original plaintext.
+    assert decrypted.getvalue() == source_plaintext

examples/src/legacy/__init__.py

@@ -0,0 +1,11 @@
+# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
+"""
+Legacy examples.
+
+These are any examples that were already defined
+before we started revamping our examples.
+We are keeping them around for anyone who needs them as reference material,
+but we recommend looking at the newer examples
+that should provide a clearer picture of how to use this library.
+"""

examples/src/basic_encryption.py → examples/src/legacy/basic_encryption.py

@@ -1,29 +1,19 @@
-# Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"). You
-# may not use this file except in compliance with the License. A copy of
-# the License is located at
-#
-# http://aws.amazon.com/apache2.0/
-#
-# or in the "license" file accompanying this file. This file is
-# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
-# ANY KIND, either express or implied. See the License for the specific
-# language governing permissions and limitations under the License.
+# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
 """Example showing basic encryption and decryption of a value already in memory."""
 import aws_encryption_sdk
 
 
-def cycle_string(key_arn, source_plaintext, botocore_session=None):
+def run(aws_kms_cmk, source_plaintext, botocore_session=None):
     """Encrypts and then decrypts a string under a KMS customer master key (CMK).
 
-    :param str key_arn: Amazon Resource Name (ARN) of the KMS CMK
+    :param str aws_kms_cmk: Amazon Resource Name (ARN) of the KMS CMK
     :param bytes source_plaintext: Data to encrypt
     :param botocore_session: existing botocore session instance
     :type botocore_session: botocore.session.Session
     """
     # Create a KMS master key provider
-    kms_kwargs = dict(key_ids=[key_arn])
+    kms_kwargs = dict(key_ids=[aws_kms_cmk])
     if botocore_session is not None:
         kms_kwargs["botocore_session"] = botocore_session
     master_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(**kms_kwargs)

examples/src/basic_file_encryption_with_multiple_providers.py → examples/src/legacy/basic_file_encryption_with_multiple_providers.py

@@ -1,15 +1,5 @@
-# Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"). You
-# may not use this file except in compliance with the License. A copy of
-# the License is located at
-#
-# http://aws.amazon.com/apache2.0/
-#
-# or in the "license" file accompanying this file. This file is
-# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
-# ANY KIND, either express or implied. See the License for the specific
-# language governing permissions and limitations under the License.
+# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
 """Example showing creation of a RawMasterKeyProvider, how to use multiple
 master key providers to encrypt, and demonstrating that each master key
 provider can then be used independently to decrypt the same encrypted message.
@@ -60,12 +50,12 @@ def _get_raw_key(self, key_id):
         )
 
 
-def cycle_file(key_arn, source_plaintext_filename, botocore_session=None):
+def run(aws_kms_cmk, source_plaintext_filename, botocore_session=None):
     """Encrypts and then decrypts a file using a KMS master key provider and a custom static master
     key provider. Both master key providers are used to encrypt the plaintext file, so either one alone
     can decrypt it.
 
-    :param str key_arn: Amazon Resource Name (ARN) of the KMS Customer Master Key (CMK)
+    :param str aws_kms_cmk: Amazon Resource Name (ARN) of the KMS Customer Master Key (CMK)
     (http://docs.aws.amazon.com/kms/latest/developerguide/viewing-keys.html)
     :param str source_plaintext_filename: Filename of file to encrypt
     :param botocore_session: existing botocore session instance
@@ -77,7 +67,7 @@ def cycle_file(key_arn, source_plaintext_filename, botocore_session=None):
     cycled_static_plaintext_filename = source_plaintext_filename + ".static.decrypted"
 
     # Create a KMS master key provider
-    kms_kwargs = dict(key_ids=[key_arn])
+    kms_kwargs = dict(key_ids=[aws_kms_cmk])
     if botocore_session is not None:
         kms_kwargs["botocore_session"] = botocore_session
     kms_master_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(**kms_kwargs)

examples/src/basic_file_encryption_with_raw_key_provider.py → examples/src/legacy/basic_file_encryption_with_raw_key_provider.py

@@ -1,15 +1,5 @@
-# Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"). You
-# may not use this file except in compliance with the License. A copy of
-# the License is located at
-#
-# http://aws.amazon.com/apache2.0/
-#
-# or in the "license" file accompanying this file. This file is
-# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
-# ANY KIND, either express or implied. See the License for the specific
-# language governing permissions and limitations under the License.
+# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
 """Example showing creation and use of a RawMasterKeyProvider."""
 import filecmp
 import os
@@ -48,7 +38,7 @@ def _get_raw_key(self, key_id):
         )
 
 
-def cycle_file(source_plaintext_filename):
+def run(source_plaintext_filename):
     """Encrypts and then decrypts a file under a custom static master key provider.
 
     :param str source_plaintext_filename: Filename of file to encrypt