- 1.1.0
- Update Cache Entry Identifier Formulas to shared cache across multiple Beacon Key Sources
- New optional parameter
Partition IDused to distinguish Cryptographic Material Providers (i.e: Beacon Key Sources) writing to a cache - New optional parameter
cacheallowed while creating aSingleKeyStore
- 1.0.0
- Initial record
The search config holds the configuration for all of the searchable configuration functionality.
The search config MUST be an optional part of the item encryptor config.
Although versioning is not yet supported, it exists in the configuration, so that the configuration will not need to change when we add versioning.
On initialization of the Search Config, the caller MUST provide:
- A list of beacon versions
- The version number of the beacon versions to be used for writing.
Initialization MUST fail if the length of the list of beacon versions is not 1.
Initialization MUST fail if the version number is not 1.
A name refers to a configured field if it appears in the item encryptor configuration as one of the attribute actions or unauthenticated attributes, or begins with the unauthenticated attribute prefix.
On initialization of a Beacon Version, the caller MUST provide:
- A [version number](#version number)
- A Beacon Key Source
- A Keystore
- A list of standard beacons
On initialization of the Beacon Version, the caller MAY provide:
- A list of compound beacons
- A list of virtual fields
- A list of signed parts
- A list of encrypted parts
Initialization MUST fail if the [version number](#version number) is not 1.
Initialization MUST fail if at least one standard beacon is not provided.
Initialization MUST fail if the name of any virtual fields matches that of any configured field.
Initialization MUST fail if the beacon key source is a multi key store and the name of any of the following match the beacon key field name
Initialization MUST fail if the beacon key source is a multi key store and the beacon key field name is a configured field with ENCRYPT_AND_SIGN.
Initialization MUST fail if the name of any standard beacon matches that of any unencrypted configured field.
Initialization MUST fail if the name of any compound beacon matches that of any unencrypted configured field.
Initialization MUST fail if there is any duplicates among the names of the standard beacons
Initialization MUST fail if there is any duplicates among the names of the compound beacons
Initialization MUST fail if the name of a compound beacons matches the name of a standard beacons
A terminal location is considered signed if
the field that contains it is SIGN_ONLY
or ENCRYPT_AND_SIGN.
A terminal location is considered encrypted if
the field that contains it is ENCRYPT_AND_SIGN.
A virtual field is considered signed if all of its
terminal locations are signed or encrypted.
A virtual field is considered encrypted if any of its
terminal locations are encrypted.
Initialization MUST fail if any virtual field is not signed.
Initialization MUST fail if the terminal location
reference by a standard beacon is not encrypted.
Initialization MUST fail if the terminal location
reference by a signed part is encrypted,
or is not signed.
The signed parts and encrypted parts together are known as the Global Parts List. Parts specified in a compound beacon's constructor, may come from any combination of their local definitions or the Global Parts List.
A version number MUST be 1.
In the future, when versioning is supported, other positive integers will be allowed.
There are some changes that can be made to a beacon versions such that a record written with the old version might not be retrieved by a query using the new version.
If such a change is necessary, one adds another beacon versions with the next higher [version number](#version number).
When a record is written, its beacons are generated using the version designated in the search config. In addition to the configured beacons, a version tag MUST also be written.
When a querying a database, it might be necessary to make multiple queries, one per version, to successfully find all the records. For this reason, it is recommended that one rewrite all of the records from an old version with the new version.
A version can safely be removed from the search config when no records exists with the associate version tag.
For version N, the name of the version tag is aws_dbe_vN and the value of the tag is a single space.
On initialization of a Beacon Key Source, the caller MUST provide exactly one of
The single key store is intended to be used when all items in a DDB table will use the same beacon key. This can also be described as single tenant.
On initialization of a Single Key Store, the caller MUST provide:
The multi key store is intended to be used when different items in a DDB table will use the different beacon keys. This can also be described as multi tenant.
On initialization of a Multi Key Store, the caller MUST provide:
A Keystore used to obtain the configured Beacon Key. If being used with a Hierarchical Keyring to encrypt/decrypt items, this SHOULD be the same Keystore as configured to that Keyring.
A single Beacon Key Id that exists in the keystore.
A attribute name that is used to identify the beacon key id. This attribute can be passed in on the DDB item, by default it will not be stored in the table but will be stored if an attribute action of SIGN_ONLY is configured. It can be referenced in compound beacons and used to extract a beacon key id from a query.
The cacheTTL for how long a beacon key should exist locally before reauthorization.
For a Beacon Key Source a CMC MUST be created. For a Single Key Store, either the user provides a cache, or we create a cache that has Entry Capacity equal to 1. For a Multi Key Store, either the user provides a cache, or we create a cache that has Entry Capacity equal to 1000.
The Key Store Cache is shared across different Beacon Key Sources if and only if a Shared cache is used.
In all other cases, the Key Store Cache is bound to the Beacon Key Source.
In either case, Cache Identifiers MUST be unique across all key sources.
Cache Identifiers for Searchable Encryption MUST be set as per the section Searchable Encryption Cache Identifier.
An optional string that uniquely identifies the respective Beacon Key Source and is used to avoid collisions with other Beacon Key Sources.
PartitionId can be a string provided by the user. If provided, it MUST be interpreted as UTF8 bytes. If the PartitionId is NOT provided by the user, it MUST be set to the 16 byte representation of a v4 UUID.
The Partition ID MUST NOT be changed after initialization.
Please see Shared Cache Considerations on how to provide the Partition ID and Logical Key Store Name while providing a Shared Cache to the Beacon Key Source.
In order for beacons to function they need materials. Beacon keys are these materials. A beacon key forms a hierarchy of HMAC keys for all searchable components. Beacon keys MUST be obtained from the configured Beacon Key Source.
Takes a Search Config and a Parsed Header.
If the Beacon Key Source is a Single Key Store
then beacon key id MUST be the configured beacon key id
If the Beacon Key Source is a Multi Key Store
then beacon key id MUST be obtained from Get beacon key id from parsed header.
The beacon key id, Key Store Cache, and a KeyStore
MUST be passed to Get Beacon Key Materials
and the result returned for Get beacon key after encrypt.
Takes a Parsed Header.
If the Parsed Header's encrypted data keys do not contain only one encrypted data key this function MUST fail.
If this single encrypted data key's Key Provider ID does not equal the provider ID for the AWS KMS Hierarchical Keyring this function MUST fail.
This function MUST return the Key Provider Information.
Takes a Search Config and a Query Object
If the Beacon Key Source is a Single Key Store
then beacon key id MUST be the configured beacon key id
If the Beacon Key Source is a Multi Key Store
then for each Key Condition Expression or Filter Expression,
extract the value of any signed part
of any compound beacons
with the name that equals Beacon Key Field Name
and construct a list of beacon key ids.
If this list of beacon key ids is empty get beacon key for query MUST return a flag indicating this.
A query MUST fail if this flag is present, and any part of the query requires a key; that is, needs to compute a standard beacon.
If this list of beacon key ids has more than one unique beacon key id then get beacon key for query MUST fail.
beacon key id MUST be this unique value.
The beacon key id, Key Store Cache, and a KeyStore
MUST be passed to Get Beacon Key Materials
and the result returned for Get beacon key for query.
Takes a beacon key id, Key Store Cache, and a KeyStore.
Get beacon key MUST Call the associated Key Store Cache Get Cache Entry with the cache identifier defined in the Searchable Encryption Cache Identifier section.
If a cache entry exists, get beacon key MUST return the entry materials.
The beacon key id MUST be passed to the configured KeyStore's GetBeaconKey operation.
If GetBeaconKey fails get beacon key MUST fail.
If using a Shared cache across multiple Beacon Key Sources,
different Beacon Key Sources having the same beaconKey can have different TTLs.
In such a case, the expiry time in the cache is set according to the Beacon Key Source that populated the cache.
There MUST be a check (cacheEntryWithinLimits) to make sure that for the cache entry found, who's TTL has NOT expired,
time.now() - cacheEntryCreationTime <= ttlSeconds is true and
valid for TTL of the Beacon Key Source getting the cache entry.
If this is NOT true, then we MUST treat the cache entry as expired.
For every standard beacons, an HMAC key MUST be generated in accordance with HMAC Key Generation.
Beacon Key Materials MUST be generated
with the beacon key id equal to the beacon key id
and the HMAC Keys equal to a map
of every standard beacons name to its generated HMAC key.
These materials MUST be put into the associated Key Store Cache with an Expiry Time equal to now + configured cacheTTL.
The Searchable Encryption cache identifier used to Put Cache Entry MUST be the same as the identifier that was used to attempt Get Cache Entry.
These cached materials MUST be returned.
Each beacon requires a key for use in its HMAC calculation.
For each beacon HKDF-SHA512 MUST be used
to calculate the key for individual beacon,
using the beacon key retrieved above as the initial key material with no salt.
The info MUST be the concatenation of "AWS_DBE_SCAN_BEACON" encoded as UTF8
and the beacon name.
The expectedLength MUST be 64 bytes.
This section is consistent with the Cache Entry Identifier formulas for the Hierarchical Keyring and Caching CMM in the Material Providers spec. These cache identifier formulas should not be changed independently.
When accessing the underlying cryptographic materials cache, Searchable Encryption MUST use the formulas specified in this section in order to compute the cache entry identifier.
Each of the cache entry identifier formulas includes serialized information related to the branch key, as defined in the Key Provider Info.
We establish the following definitions for the Cache Entry Identifier formula:
Resource Identifier is defined in the Material Providers specification.
Scope Identifier is defined in the Material Providers specification.
Partition ID is an optional parameter provided to the Beacon Key Source input, which distinguishes Cryptographic Material Providers (i.e: Beacon Key Sources) writing to a cache. It can either be a String provided by the user, which MUST be interpreted as the bytes of UTF-8 Encoding of the String, or a v4 UUID, which SHOULD be interpreted as the 16 byte representation of the UUID.
Note: The cache will not know if the Partition ID is a String set by the user or the UUID. The constructor of the Beacon Key Source MUST record these bytes at construction time.
Please see Shared Cache Considerations on how to provide the Partition ID and Logical Key Store Name of the beacon versions while providing a Shared Cache to the Beacon Key Source.
The resource suffixes for the Searchable Encryption is as follows:
logicalKeyStoreName + NULL_BYTE + UTF8Encode(branchKeyId)
The aforementioned 4 definitions (Resource Identifier, Scope Identifier, Partition ID, and Resource Suffix) MUST be appended together with the null byte, 0x00, and the SHA384 of the result should be taken as the final cache identifier.
When the Beacon Key Source receives a getKeyMap request,
the cache entry identifier MUST be calculated as the
SHA-384 hash of the following byte strings, in the order listed:
- MUST be the Resource ID for the Hierarchical Keyring (0x02)
- MUST be the Scope ID for Searchable Encryption (0x03)
- MUST be the Partition ID for the Beacon Key Source
- Resource Suffix
- MUST be the UTF8 encoded Logical Key Store Name of the keystore for the beacon versions
- MUST be the UTF8 encoded branch-key-id
All the above fields must be separated by a single NULL_BYTE 0x00.
| Field | Length (bytes) | Interpreted as |
|---|---|---|
| Resource ID | 1 | bytes |
| Null Byte | 1 | 0x00 |
| Scope ID | 1 | bytes |
| Null Byte | 1 | 0x00 |
| Partition ID | Variable | bytes |
| Null Byte | 1 | 0x00 |
| Logical Key Store Name | Variable | UTF-8 Encoded Bytes |
| Null Byte | 1 | 0x00 |
| Branch Key ID | Variable | UTF-8 Encoded Bytes |
As a formula:
resource-id = [0x02]
scope-id = [0x03]
logical-key-store-name = UTF8Encode(beaconVersion.keystore.LogicalKeyStoreName)
branch-key-id = UTF8Encode(BranchKeyIdentifier)
NULL_BYTE = [0x00]
ENTRY_ID = SHA384(
resource-id
+ NULL_BYTE
+ scope-id
+ NULL_BYTE
+ partition-id
+ NULL_BYTE
+ logical-key-store-name
+ NULL_BYTE
+ branch-key-id
)
If a user has two or more beacon versions with:
- Same Partition ID
- Same Logical Key Store Name of the Key Store
- Same Branch Key ID
then they WILL share the cache entries in the Shared Cache.
Any keyring that has access to the Shared cache may be able to use materials
that it may or may not have direct access to.
Users should make sure that all of Partition ID, Logical Key Store Name of the Key Store and Branch Key ID are set to be the same for two beacon versions if and only they want the keyrings to share cache entries.
Therefore, there are two important parameters that users need to carefully set while providing the shared cache:
Partition ID is an optional parameter provided to the Beacon Key Source input, which distinguishes Cryptographic Material Providers (i.e: Beacon Key Sources) writing to a cache.
- (Default) A a random 16-byte UUID, which makes it unique for every Beacon Key Source. In this case, two Beacon Key Sources (or another Material Provider) WILL NOT share the same cache entries in the cache.
- If the Partition ID is set by the user and is the same for two Beacon Key Sources (or another Material Provider), they MAY share the same cache entries in the cache.
- If the Partition ID is set by the user and is different for two Beacon Key Sources (or another Material Provider), they WILL NOT share the same cache entries in the cache.
Note: Users should not have two different physical Key Stores with the same Logical Key Store Name.
Logical Key Store Name is set by the user when configuring the Key Store for the beacon versions. This is a logical name for the key store. Logical Key Store Name MUST be converted to UTF8 Bytes to be used in the cache identifiers.
Suppose there's a physical Key Store on DynamoDB (K). Two Key Store clients of K (K1 and K2) are created. Now, two beacon versions (BV1 and BV2) are created with these Key Store clients (K1 and K2 respectively).
- If we want to share cache entries across these two BeaconVersions BV1 and BV2, the Logical Key Store Names for both the Key Store clients (K1 and K2) should be set to be the same.
- If we set the Logical Key Store Names for K1 and K2 to be different, BV1 (which uses Key Store client K1) and BV2 (which uses Key Store client K2) will NOT be able to share cache entries.
Notice that both K1 and K2 are clients for the same physical Key Store (K).