ddb-sdk-integration.md

DynamoDB SDK Integration

Version

1.0.0

Changelog

• 1.0.0

  • Initial record

Definitions

Conventions used in this document

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Overview

Customers want to be able to use the AWS SDK to call the DynamoDB APIs they are familiar with, while having encryption and decryption of their items happen transparently.

The AWS SDK is available in a variety of languages, and the ways to configure and construct the client in each differs. Thus, providing this behavior in each AWS SDK language will not be standardized.

There MUST be some process by which the customer can create this AWS SDK with additional encryption behavior, which this document will refer to as a DynamoDB Encryption Client. There is no restriction on what this process is, other than the fact that at some point the customer MUST specify a DynamoDB Encryption Client Configuration. This configuration MUST be the configuration ultimately used by the behaviors within the DynamoDB Encryption Client.

For example, the AWS SDK for Java (as of 2.x) defines a Interceptor interface which can be used to add logic before and after API calls. In order to provide an idiomatic experience for AWS SDK for Java customers, to implement this for the Java SDK we would implement an Interceptor that requires the DynamoDB Encryption Client Configuration on construction. This interceptor can then itself be use as input when building the AWS SDK client in order to create a client which performs the necessary encryption and decryption logic during API calls.

For AWS SDKs that do not support an idiomatic way to hook new behavior before and after API calls, this MAY be implemented by providing an implementation of AWS SDK APIs that wrap the relevant APIs with the necessary encryption and decryption logic. This new implementation SHOULD be a drop-in replacement of the regular AWS SDK.

DynamoDB Encryption Client Configuration

The DynamoDB Encryption Client Configuration is a structure that the customer MUST provide in the process of creating a DynamoDB Encryption Client.

The client configuration consists of the following REQUIRED field:

• DynamoDb Table Encryption Configs

DynamoDb Table Encryption Configs

A map of DynamoDb table names to a structure that MUST contain data as described by DynamoDb Table Encryption Config. Note that this data does not need to contain the Table Name.

During initialization, this client MUST construct a DynamoDb Item Encryptor per configured table, using these table encryption configs.

Encrypted Table

If there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to a given TableName, that table is said to be an Encrypted Table.

DynamoDB Encryption Client

The DynamoDB Encryption Client is an AWS SDK client for DynamoDB, such that client-side encryption and decryption happens transparently during AWS SDK API calls to DynamoDB.

General Modification Behavior

If no Item Encryptor is specified for the table, the call to DynamoDB is unmodified. Otherwise:

No DDB API legacy parameters (such as AttributesToGet) can be specified.

Any Condition Expression must be checked for validity. Any Update Expression must be checked for validity

If data is to be written

• the input data must validated
• beacons must be added
• the input data must be encrypted

If data is being read

• the output data must be decrypted
• beacons must be removed

For Query and Scan, more complex processing must be invoked for both input (before encryption), and output (after decryption).

API modification

DynamoDB API calls MUST have the following modified behavior:

• Encrypt before PutItem
• Encrypt before BatchWriteItem
• Encrypt before TransactWriteItems
• Decrypt after BatchWriteItem
• Decrypt after GetItem
• Decrypt after PutItem
• Decrypt after UpdateItem
• Decrypt after DeleteItem
• Decrypt after BatchGetItem
• Decrypt after Scan
• Decrypt after Query
• Decrypt after TransactGetItem
• Validate before UpdateItem
• Validate before DeleteItem
• Validate before ExecuteStatement
• Validate before BatchExecuteStatement
• Validate before ExecuteTransaction
• Modify before Scan
• Modify before Query

The Allowed Passthrough DynamoDB APIs MUST NOT be modified.

Any DynamoDB API not specified is this document either MUST NOT be callable or MUST yield an error, and that error SHOULD indicate that the operation is not supported with DynamoDB client-side encryption.

Encrypt before PutItem

Before the PutItem call is made to DynamoDB :

If the TableName in the request does not refer to an encrypted-table, the PutItem request MUST be unchanged.

Otherwise,

The PutItem request MUST NOT refer to any legacy parameters, specifically Expected and ConditionalOperator MUST NOT be set.

The Item MUST be writable.

The ConditionExpression MUST be valid.

Signed Beacons MUST be added.

If the Beacon Key Source is a Multi Key Store the Beacon Key Field Name MUST be handled.

If the Beacon Key Source is a Multi Key Store the Item Encryptor MUST perform Encrypt Item, where the input DynamoDB Item is output of handling the beacon key field name operation.

If the Beacon Key Source is a Single Key Store the Item Encryptor MUST perform Encrypt Item, where the input DynamoDB Item is output of adding the signed beacons operation.

If the Beacon Key Source is a Multi Key Store and a branch key id was returned from handling the beacon key field name this branch key id MUST match the value returned from Get beacon key id from Parsed Header.

Encrypted Beacons MUST be added.

If any of the above fails, the client MUST NOT make a network call to DynamoDB, and PutItem MUST yield an error.

The PutItem request's Item field MUST be replaced with a value that is equivalent to the output of the add encrypted beacons operation calculated above.

Encrypt before BatchWriteItem

Before the BatchWriteItem call is made to DynamoDB :

For each table name in RequestItems

If the table name does not refer to an encrypted-table, the list of operations MUST be unchanged.

Otherwise,

For each operation associated with the table name, if there is a PutRequest:

The Item MUST be writable.

Signed Beacons MUST be added.

If the Beacon Key Source is a Multi Key Store the Beacon Key Field Name MUST be handled.

If the Beacon Key Source is a Multi Key Store the Item Encryptor MUST perform Encrypt Item, where the input DynamoDB Item is output of handling the beacon key field name operation.

If the Beacon Key Source is a Single Key Store the Item Encryptor MUST perform Encrypt Item, where the input DynamoDB Item is output of adding the signed beacons operation.

If the Beacon Key Source is a Multi Key Store and a branch key id was returned from handling the beacon key field name this branch key id MUST match the value returned from Get beacon key id from Parsed Header.

Encrypted Beacons MUST be added.

If any of the above fails, the client MUST NOT make a network call to DynamoDB, and BatchWriteItem MUST yield an error.

The PutRequest request's Item field MUST be replaced with a value that is equivalent to the result Encrypted DynamoDB Item calculated above.

Encrypt before TransactWriteItems

Before the TransactWriteItems call is made to DynamoDB, for every entry under TransactItems:

To protect against a possible fifth field being added to the TransactWriteItem structure in the future, the client MUST fail if none of the Update, ConditionCheck, Delete and Put fields are set.

Any actions other than Put, MUST be unchanged.

Any Put actions with a TableName that does not refer to an encrypted-table, MUST be unchanged.

If there is an Update that refers to a TableName that refers to an encrypted-table

• The UpdateExpression of the Update MUST be valid.
• The ConditionExpression of the Update MUST be valid.

If there is a ConditionCheck that refers to a TableName that refers to an encrypted-table

• The ConditionExpression of the ConditionCheck MUST be valid.

If there is a Delete that refers to a TableName that refers to an encrypted-table

• The ConditionExpression of the Delete MUST be valid.

If there is a Put that refers to a TableName that refers to an encrypted-table

• The Item MUST be writable.
• The ConditionExpression Put MUST be valid.
• Signed Beacons MUST be added.
• If the Beacon Key Source is a Multi Key Store the Beacon Key Field Name MUST be handled.
• If the Beacon Key Source is a Multi Key Store the Item Encryptor MUST perform Encrypt Item, where the input DynamoDB Item is output of handling the beacon key field name operation.
• If the Beacon Key Source is a Single Key Store the Item Encryptor MUST perform Encrypt Item, where the input DynamoDB Item is output of adding the signed beacons operation.
• If the Beacon Key Source is a Multi Key Store and a branch key id was returned from handling the beacon key field name this branch key id MUST match the value returned from Get beacon key id from Parsed Header.
• Encrypted Beacons MUST be added.
• If any of the above fails, the client MUST NOT make a network call to DynamoDB, and PutItem MUST yield an error.
• The PutItem request's Item field MUST be replaced with a value that is equivalent to the result Encrypted DynamoDB Item calculated above.

Decrypt after BatchWriteItem

If there are no UnprocessedItems, then the BatchWriteItemOutput MUST be returned unchanged.

Each item in UnprocessedItems MUST be replaced by its original plaintext value.

Decrypt after GetItem

After a GetItem call is made to DynamoDB, the resulting response MUST be modified before being returned to the caller if there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to the TableName on the GetItem request.

The Item Encryptor MUST perform Decrypt Item where the input DynamoDB Item is the Item field in the original response

Beacons MUST be removed from the result.

The GetItem response's Item field MUST be replaced by the encrypted DynamoDb Item outputted above.

Decrypt after PutItem

After a PutItem call is made to DynamoDB, the resulting response MUST be modified before being returned to the caller if:

• there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to the TableName on the PutItem request.
• the response contains Attributes. The response will contain Attributes if the related PutItem request's ReturnValues had a value of ALL_OLD and the PutItem call replaced a pre-existing item.

In this case, the Item Encryptor MUST perform Decrypt Item where the input DynamoDB Item is the Attributes field in the original response

Beacons MUST be removed from the result.

The PutItem response's Attributes field MUST be replaced by the encrypted DynamoDb Item outputted above.

Decrypt after DeleteItem

After a DeleteItem call is made to DynamoDB, the resulting response MUST be modified before being returned to the caller if:

• there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to the TableName on the DeleteItem request.
• the response contains Attributes. The response will contain Attributes if the related DeleteItem request's ReturnValues had a value of ALL_OLD and an item was deleted.

In this case, the Item Encryptor MUST perform Decrypt Item where the input DynamoDB Item is the Attributes field in the original response

Beacons MUST be removed from the result.

The DeleteItem response's Attributes field MUST be replaced by the encrypted DynamoDb Item outputted above.

Decrypt after UpdateItem

After a UpdateItem call is made to DynamoDB, the resulting response MUST be modified before being returned to the caller if:

• there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to the TableName on the UpdateItem request.
• the response contains Attributes.
• the original UpdateItem request had a ReturnValues with a value of ALL_OLD or ALL_NEW.

In this case, the Item Encryptor MUST perform Decrypt Item where the input DynamoDB Item is the Attributes field in the original response

Beacons MUST be removed from the result.

The UpdateItem response's Attributes field MUST be replaced by the encrypted DynamoDb Item outputted above.

In all other cases, the UpdateItem response MUST NOT be modified.

Additionally, if a value of UPDATED_OLD or UPDATED_NEW was used, and any Attributes in the response are authenticated per the DynamoDB Encryption Client Config, an error MUST be raised. Given that we validate UpdateItem requests, and thus updates will not modify any signed field, an error here would indicate a bug in our library or a bug within DynamoDB.

Decrypt after BatchGetItem

After a BatchGetItem call is made to DynamoDB :

For each list item under each key in Responses, if there is a configured Item Encryptor with DynamoDB Table Name equal to the key, that Item Encryptor MUST perform Decrypt Item where the input DynamoDB Item is the Item field in the original response.

Beacons MUST be removed from the result.

Each of these items on the original response MUST be replaced with a value that is equivalent to this result.

If any Decrypt Item operation fails, BatchGetItem MUST yield an error.

Decrypt after Scan

After a Scan call is made to DynamoDB :

For each list entry in Items in the response, if there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to the TableName on the request, the corresponding Item Encryptor MUST perform Decrypt Item where the input DynamoDB Item is this list entry.

Each of these entries on the original response MUST be replaced with the resulting decrypted DynamoDB Item.

If any Decrypt Item operation fails, Scan MUST yield an error.

The resulting decrypted response MUST be filtered from the result.

Decrypt after Query

After a Query call is made to DynamoDB :

For each list entry in Items in the response, if there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to the TableName on the request, the corresponding Item Encryptor MUST perform Decrypt Item where the input DynamoDB Item is this list entry.

Each of these entries on the original response MUST be replaced with the resulting decrypted DynamoDB Item.

If any Decrypt Item fails, Query MUST yield an error.

The resulting decrypted response MUST be filtered from the result.

Decrypt after TransactGetItems

After a TransactGetItems call is made to DynamoDB :

For each list item under each key in Responses, if there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to the key, the Item Encryptor that corresponds to the key in the request MUST perform Decrypt Item where the input DynamoDB Item is the Item in the original response.

Beacons MUST be removed from the result.

Each of these items on the original response MUST be replaced with a value that is equivalent to the resulting item.

If any Decrypt Item fails, TransactGetItems MUST yield an error.

Validate Before UpdateItem

Before an UpdateItem call is made to DynamoDB,

If the TableName in the request does not refer to an encrypted-table, the UpdateItem request MUST be unchanged.

Otherwise,

The UpdateItem request MUST NOT refer to any legacy parameters, specifically Expected, AttributeUpdates and ConditionalOperator MUST NOT be set.

The UpdateExpression MUST be valid.

If all of the above validation succeeds, the UpdateItem request MUST be unchanged.

Validate Before DeleteItem

Before a DeleteItem call is made to DynamoDB,

If the TableName in the request does not refer to an encrypted-table, the DeleteItem request MUST be unchanged.

Otherwise,

The DeleteItem request MUST NOT refer to any legacy parameters, specifically Expected and ConditionalOperator MUST NOT be set.

The ConditionExpression MUST be valid.

If all of the above validation succeeds, the DeleteItem request MUST be unchanged.

Validate Before ExecuteStatement

Before an ExecuteStatement call is made to DynamoDB :

The request MUST fail, and the client make no network call to DynamoDB, if there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to table named in the request.

If no such Item Encryptor exists, there MUST NOT be any modification to the ExecuteStatement request.

Validate Before BatchExecuteStatement

Before an BatchExecuteStatement call is made to DynamoDB :

The request MUST fail, and the client make no network call to DynamoDB, if there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to table named in any of the Statements of the request.

If no such Item Encryptor exists, there MUST NOT be any modification to the BatchExecuteStatement request.

Validate Before ExecuteTransaction

Before an ExecuteTransaction call is made to DynamoDB :

The request MUST fail, and the client make no network call to DynamoDB, if there exists an Item Encryptor specified within the DynamoDB Encryption Client Config with a DynamoDB Table Name equal to table named in any of the TransactStatements of the request.

If no such Item Encryptor exists, there MUST NOT be any modification to the ExecuteTransaction request.

Modify before Scan

Before the Scan call is made to DynamoDB :

If the TableName in the request does not refer to an encrypted-table, the Scan request MUST be unchanged.

Otherwise

The Scan request MUST NOT refer to any legacy parameters, specifically AttributesToGet, ScanFilter and ConditionalOperator MUST NOT be set.

The request MUST be altered to transform any references to encrypted attributes into references to beacons.

Modify before Query

Before the Query call is made to DynamoDB :

If the TableName in the request does not refer to an encrypted-table, the Query request MUST be unchanged.

Otherwise

The Query request MUST NOT refer to any legacy parameters, specifically AttributesToGet, KeyConditions, QueryFilter and ConditionalOperator MUST NOT be set.

The request MUST be altered to transform any references to encrypted attributes into references to beacons.

Allowed Passthrough DynamoDB APIs

• CreateBackup
• CreateGlobalTable
• CreateTable
• DeleteBackup
• DeleteTable
• DescribeBackup
• DescribeContinuousBackups
• DescribeContributorInsights
• DescribeEndpoints
• DescribeExport
• DescribeGlobalTable
• DescribeGlobalTableSettings
• DescribeImport
• DescribeKinesisStreamingDestination
• DescribeLimits
• DescribeTable
• DescribeTableReplicaAutoScaling
• DescribeTimeToLive
• DisableKinesisStreamingDestination
• EnableKinesisStreamingDestination
• ExportTableToPointInTime
• ImportTable
• ListBackups
• ListContributorInsights
• ListExports
• ListGlobalTables
• ListImports
• ListTables
• ListTagsOfResource
• RestoreTableFromBackup
• RestoreTableToPointInTime
• TagResource
• UntagResource
• UpdateContinuousBackups
• UpdateContributorInsights
• UpdateGlobalTable
• UpdateGlobalTableSettings
• UpdateTable
• UpdateTableReplicaAutoScaling
• UpdateTimeToLive