Intermittent success when decrypting using KMSMasterKey, on object encrypted using Multiple CMKs

[johnwalker]

import aws_encryption_sdk
from aws_encryption_sdk.key_providers.kms import KMSMasterKey

CMK_ARNS = {
 "A": "arn1",
 "B": "arn2"
}
plaintext=b'abdc'
context = {'purpose': 'encryption context'}

master_key_provider_A = aws_encryption_sdk.KMSMasterKeyProvider()
master_key_provider_A.add_master_key(CMK_ARNS['A'])
master_key_provider_A.add_master_key(CMK_ARNS['B'])

master_key_provider_B = KMSMasterKey(key_id=CMK_ARNS['B'])

encrypted_message, header = aws_encryption_sdk.encrypt(
  encryption_context=context,
  key_provider=master_key_provider_A,
  source=plaintext
)

decrypted_message, decrypted_header = aws_encryption_sdk.decrypt(
        key_provider=master_key_provider_B,
  source=encrypted_message
)

Decrypting using master_key_provider_B fails intermittently with:

aws_encryption_sdk.exceptions.IncorrectMasterKeyError: Provided data key provider
MasterKeyInfo(provider_id='aws-kms', key_info=b'$ARN2') does not match Master Key
provider MasterKeyInfo(provider_id='aws-kms', key_info=b'$ARN1')

The expected behavior is that decryption always succeeds, since B's CMK is among those used in A's.

[mattsb42-aws]

@johnwalker Could you include the full stacktrace from the reproduction case you ran?

It looks like this is probably caused by a too-narrow catch of exceptions in MasterKeyProvider.decrypt_data_key_from_list, but the stack trace will verify where the split is happening.

https://github.com/aws/aws-encryption-sdk-python/blob/master/src/aws_encryption_sdk/key_providers/base.py#L278-L284

[johnwalker]

Yes!

  File "/Users/jhwal/aws/pycall/lib/python3.7/site-packages/aws_encryption_sdk/streaming_client.py", line 189, in __exit__
    self.close()
  File "/Users/jhwal/aws/pycall/lib/python3.7/site-packages/aws_encryption_sdk/streaming_client.py", line 884, in close
    raise SerializationError("Footer not read")
aws_encryption_sdk.exceptions.SerializationError: Footer not read
Traceback (most recent call last):
  File "experiment.py", line 25, in <module>
    source=encrypted_message
  File "/Users/jhwal/aws/pycall/lib/python3.7/site-packages/aws_encryption_sdk/__init__.py", line 124, in decrypt
    plaintext = decryptor.read()
  File "/Users/jhwal/aws/pycall/lib/python3.7/site-packages/aws_encryption_sdk/streaming_client.py", line 221, in read
    self._prep_message()
  File "/Users/jhwal/aws/pycall/lib/python3.7/site-packages/aws_encryption_sdk/streaming_client.py", line 701, in _prep_message
    self._header, self.header_auth = self._read_header()
  File "/Users/jhwal/aws/pycall/lib/python3.7/site-packages/aws_encryption_sdk/streaming_client.py", line 732, in _read_header
    decryption_materials = self.config.materials_manager.decrypt_materials(request=decrypt_materials_request)
  File "/Users/jhwal/aws/pycall/lib/python3.7/site-packages/aws_encryption_sdk/materials_managers/default.py", line 139, in decrypt_materials
    encryption_context=request.encryption_context,
  File "/Users/jhwal/aws/pycall/lib/python3.7/site-packages/aws_encryption_sdk/key_providers/base.py", line 280, in decrypt_data_key_from_list
    data_key = self.decrypt_data_key(encrypted_data_key, algorithm, encryption_context)
  File "/Users/jhwal/aws/pycall/lib/python3.7/site-packages/aws_encryption_sdk/key_providers/base.py", line 485, in decrypt_data_key
    self._key_check(encrypted_data_key)
  File "/Users/jhwal/aws/pycall/lib/python3.7/site-packages/aws_encryption_sdk/key_providers/base.py", line 404, in _key_check
    key=data_key.key_provider, master=self.key_provider
aws_encryption_sdk.exceptions.IncorrectMasterKeyError: Provided data key provider MasterKeyInfo(provider_id='aws-kms', key_info=b'arn:aws:kms:us-east-1:915293556405:key/3e6c82c5-d2df-4c11-84b1-cb95b9528d88') does not match Master Key provider MasterKeyInfo(provider_id='aws-kms', key_info=b'arn:aws:kms:us-east-1:915293556405:key/b5d3368d-626e-4d5f-9bd7-8f6336c10253')

[mattsb42-aws]

Yup. Root cause here is that we need to catch both DecryptKeyError and IncorrectMasterKeyError in MasterKeyProvider.decrypt_data_key_from_list.

https://github.com/aws/aws-encryption-sdk-python/blob/master/src/aws_encryption_sdk/key_providers/base.py#L281

To test this, we will need two EDKs and a master key that matches one of them. We then need to pass in a list of EDKs that has the matching EDK second.

[mattsb42-aws]

This fix is available in published version 1.4.0