Skip to content

Commit be6870b

Browse files
lavalerimattsb42-awsrobin-aws
authored
feat: Add Decrypt inputs max body size and max header size (#145)
* feat: Add Decrypt inputs max body size and max header size * Apply suggestions from code review Co-authored-by: Matt Bullock <[email protected]> * Apply suggestions from code review Co-authored-by: Matt Bullock <[email protected]> * PR feedback * Apply suggestions from code review Co-authored-by: Matt Bullock <[email protected]> * Apply suggestions from code review Co-authored-by: Matt Bullock <[email protected]> * Apply suggestions from code review Co-authored-by: Robin Salkeld <[email protected]> Co-authored-by: Matt Bullock <[email protected]> Co-authored-by: Robin Salkeld <[email protected]>
1 parent cbc0f67 commit be6870b

File tree

1 file changed

+272
-0
lines changed
  • proposals/2020-06-26_decrypt-max-header-size-max-body-size

1 file changed

+272
-0
lines changed

‎proposals/2020-06-26_decrypt-max-header-size-max-body-size/proposal.md

Lines changed: 272 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,272 @@
1+
[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved."
2+
[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0"
3+
4+
# Decrypt Inputs Max Body Size and Max Header Size
5+
6+
## Affected Features
7+
8+
This serves as a reference of all features that this change affects.
9+
10+
| Feature |
11+
| --------------------------------------- |
12+
| [Decrypt](../../client-apis/decrypt.md) |
13+
14+
## Affected Specifications
15+
16+
This serves as a reference of all specification documents that this change affects.
17+
18+
| Specification |
19+
| --------------------------------------- |
20+
| [Decrypt](../../client-apis/decrypt.md) |
21+
22+
## Affected Implementations
23+
24+
This serves as a reference for all implementations that this change affects.
25+
26+
| Language | Repository |
27+
| ---------- | ------------------------------------------------------------------------------------- |
28+
| Python | [aws-encryption-sdk-python](https://github.com/aws/aws-encryption-sdk-python) |
29+
| Java | [aws-encryption-sdk-java](https://github.com/aws/aws-encryption-sdk-java) |
30+
| C | [aws-encryption-sdk-c](https://github.com/aws/aws-encryption-sdk-c) |
31+
| Javascript | [aws-encryption-sdk-javascript](https://github.com/aws/aws-encryption-sdk-javascript) |
32+
33+
## Definitions
34+
35+
### Conventions used in this document
36+
37+
The key words
38+
"MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
39+
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL"
40+
in this document are to be interpreted as described in
41+
[RFC 2119](https://tools.ietf.org/html/rfc2119).
42+
43+
## Summary
44+
45+
Processing ESDK messages of unknown size introduces a risk of highly-bounded resource consumption
46+
due to variable length fields in the message format.
47+
This is bad for users that need to decrypt such messages,
48+
especially for such users that care about the memory usage of their systems.
49+
50+
Most ESDK implementations have a way to mitigate some of these concerns through
51+
user specified controls on the Decrypt operation,
52+
however these controls are not described in our specification
53+
and are not consistent across implementations.
54+
55+
This change clearly defines the specification for two inputs,
56+
`Max Body Size` and `Max Header Size`,
57+
on Decrypt which provide a way for users to put more restrictive bounds
58+
on the messages the ESDK will process.
59+
60+
## Out of Scope
61+
62+
- Providing a control that is a 1:1 memory limit on
63+
what the ESDK MUST hold in memory in order to perform its operations is out of scope.
64+
The goal is to provide a control that is precise
65+
(behaves consistently and scales in expected ways),
66+
not a control that is accurate (closely resembles the actual memory usage of the ESDK).
67+
68+
- Updating the logic of the ESDK to not hold objects
69+
in memory for the duration of the decryption operation is not in scope.
70+
71+
- Additional controls to limit the memory used to process messages with invalid signature lengths
72+
is out of scope.
73+
The bounds of the signature length field are small enough (2^16 - 1 bytes)
74+
that they are not a concern.
75+
76+
## Motivation
77+
78+
ESDK implementations provide some controls that mitigate high resource consumption
79+
when decrypting messages.
80+
However, none of these mitigations behave consistently across implementations
81+
and none of them bound the fields in the message header specifically.
82+
83+
This change proposes two standard controls
84+
that bound resource consumption.
85+
86+
## Drawbacks
87+
88+
- Requires users to use two controls to mitigate high resource consumption concerns.
89+
- Requires users to be aware of the message structure
90+
and requires them to consider what messages they expect when decrypting.
91+
92+
## Security Implications
93+
94+
This change SHOULD NOT have any security implications.
95+
96+
## Operational Implications
97+
98+
This change adds operational controls that more strictly bound some fields in the
99+
message header.
100+
Users who are especially concerned with the memory usage of their systems should consider
101+
setting these controls to restrict resource consumption.
102+
103+
This change breaks Javascript and Node.js users who currently set `Max Body Size`
104+
and expect a message with an encrypted content field length of exactly `Max Body Size`
105+
to fail.
106+
This change makes `Max Body Size` inclusive,
107+
allowing encryption content fields of that exact length.
108+
109+
## Guide-level Explanation
110+
111+
Specify two optional inputs `Max Body Size` and `Max Header Size` for Decrypt.
112+
`Max Body Size` sets a limit on the maximum encryption content length
113+
the ESDK allows in a message
114+
and `Max Header Size` sets a limit on the maximum header length
115+
the ESDK allows in a message.
116+
117+
The intention of these controls is to provide a way for users to
118+
bound the resources the ESDK can consume in order to mitigate
119+
high resource consumption when decrypting a large message.
120+
121+
### Why do users care about these controls?
122+
123+
In order to decrypt encrypted messages,
124+
the ESDK MUST buffer variable length fields from the ESDK message format.
125+
Some variable length fields can be large enough that they
126+
require a significant amount of memory to process.
127+
128+
The message format contains the following variable length fields with maximum bounds high enough to be of concern:
129+
130+
1. The Encrypted Content field in an unframed message.
131+
The message format allows this to be up to 2^36 - 32 bytes (~64 GiB).
132+
For some implementations this is further restricted
133+
(e.g. Java arrays cannot have more than 2^31 - 1 members).
134+
In order to decrypt an unframed message,
135+
the ESDK MUST buffer the entire Encrypted Content field.
136+
In order to release the plaintext to the user safely,
137+
the ESDK MUST hold both the plaintext and the ciphertext in memory at the same time.
138+
Assuming an encryption algorithm where the length of the output equals the length of the input,
139+
this bumps the memory requirement to process this by a factor of 2.
140+
1. The Encrypted Content field in a frame in a framed message.
141+
The message format allows this to be up to 2^32 - 1 bytes (4 GiB).
142+
In order to decrypt a frame, the ESDK MUST buffer the entire Encrypted Content field in that frame.
143+
In order to release the plaintext to the user safely,
144+
the ESDK MUST hold both the plaintext and the ciphertext in memory at the same time.
145+
Assuming an encryption algorithm where the length of the output equals the length of the input,
146+
this bumps the memory requirement to process this by a factor of 2.
147+
1. The Encrypted Data Keys field in the message header.
148+
The message format allows this to be up to `(2^16 - 1) * (6 + 3 * (2^16 - 1))` bytes (~12 GiB).
149+
As such, 12 GiB represents a lower bound on the amount of memory required to
150+
process the largest EDK set the message format allows.
151+
The actual memory required depends on how a specific language and implementation
152+
represents these objects in memory.
153+
These objects MUST be held in memory for the duration of the decryption operation
154+
because the decrypt operation MUST output the parsed header.
155+
156+
Messages with the above fields at their size limits, when processed,
157+
can exhaust many memory runtime limits.
158+
As such, customers who are concerned with the stability and memory usage of their systems
159+
may care about providing stricter bounds to these fields.
160+
161+
The Encryption Context field in the header
162+
and the Signature Length field in the footer are also variable length
163+
but they cannot exceed 2^16 - 1 bytes (64 KiB).
164+
This is well within the bounds of memory runtime limits for the most languages,
165+
so we do not consider the bounds of these fields a concern.
166+
167+
Note that all implementations dynamically allocate resources while processing messages.
168+
169+
### When should users set this value?
170+
171+
Users should set this value if they need to process
172+
messages of unknown size
173+
or messages from an untrusted source.
174+
175+
### When should users NOT set this value?
176+
177+
This control should be considered strictly an operational control that can
178+
mitigate highly-bounded resource consumption for certain use cases.
179+
180+
If a user has a security requirement to not decrypt messages with a certain content length
181+
or message header length
182+
(ex, to enforce a “sign only” use case by only decrypting zero length messages),
183+
they SHOULD meet that security requirement using
184+
a custom CMM that only allows desired requests.
185+
186+
### What value should they set it to?
187+
188+
Users SHOULD choose reasonable values for their use case and
189+
the messages that they expect to decrypt.
190+
They can expect the memory usage of the ESDK to scale linearly with this control,
191+
and SHOULD performance test their application to determine
192+
the ESDKs memory cost in practice for their use case.
193+
194+
### Why not just provide one control that “does the right thing” for most use cases?
195+
196+
The two controls mitigate two different resource consumption concerns.
197+
`Max Body Size` limits short term memory consumption.
198+
This can be used to ensure that the decryption of a frame or
199+
an unframed message doesn't immediately exhaust memory linear to a factor of that value.
200+
201+
`Max Header Size` has a fuzzier implication on memory that
202+
depends on the implementation
203+
and represents a memory requirement that exists for the duration of the decryption operation.
204+
205+
Separating these controls
206+
keeps the purpose and affect of each control clear and simple,
207+
so that they can be easily understood and applied.
208+
209+
### Should there be a lower limit for Max Header Size?
210+
211+
To allow for the smallest possible message header
212+
(no AAD, one EDK but the EDK fields themselves are empty),
213+
the `Max Header Size` MUST NOT be less than 40 bytes.
214+
However, because future message format revisions might have different lower limits,
215+
we cannot know if a limit is too low until we start processing a message.
216+
217+
### Will this value still be valid if/when we update the message format?
218+
219+
This control does not introduce any one-way doors that would limit
220+
how we might update the message format in the future.
221+
222+
Conceptually, this should apply nicely to any new message format.
223+
With a new message format, our concerns about memory limits should be similar.
224+
There is no way to avoid processing a whole frame before releasing any plaintext,
225+
meaning that we will always want some control on the encrypted content size.
226+
The message header will always contain some metadata that the ESDK might need to hold in memory,
227+
meaning that a control on the header size would still be useful.
228+
229+
## Reference-level Explanation
230+
231+
Specify two optional inputs on Decrypt, `Max Header Size` and `Max Body Size`.
232+
233+
### Max Body Size
234+
235+
If `Max Body Size` is set on Decrypt, then the operation:
236+
237+
- MUST halt and fail if processing an unframed message with encrypted content field
238+
of a length greater than (>) Max Body Size, and MUST NOT attempt decryption on such a message.
239+
- This means that when decrypting an unframed message,
240+
if Max Body Size is set then the ESDK MUST fail if the encrypted content length field
241+
in the message body is greater than (>) Max Body Size.
242+
- MUST halt and fail if processing a message frame with an encrypted content field
243+
of a length greater than (>) Max Body Size and MUST NOT attempt decryption on any such frame.
244+
- This means that when decrypting a regular frame in a framed message,
245+
if Max Body Size is set then the ESDK MUST fail if the frame length
246+
set in the header is greater than Max Body Size.
247+
- This means that when decrypting a final frame in a framed message,
248+
if Max Body Size is set then the ESDK MUST fail if the encrypted content length field
249+
set in the final frame is greater than Max Body Size.
250+
- Note that there is an edge case for messages with bodies that only consist of a final frame.
251+
If a message body contains a single frame,
252+
that final frame MAY have a content length less then (<) the frame size in the header.
253+
For this reason, the implementation MUST NOT fail
254+
based on the frame size in the message header alone
255+
and MUST check the first frame in the message.
256+
Such a message is valid because the Frame Size is set on encrypt
257+
while Max Body Size is set only on decrypt.
258+
259+
### Max Header Size
260+
261+
If `Max Header Size` is set on Decrypt, then the operation:
262+
263+
- MUST halt and fail if processing a message with a message header of a length
264+
greater than (>) `Max Header Size`,
265+
and MUST NOT attempt to parse greater than (>) `Max Header Size` bytes of a message header.
266+
- Note that the message header does not contain a field that describes its total length,
267+
and thus implementations MUST attempt to process the message header until
268+
it either succeeds parsing the message header
269+
or it can determine that the total length of the header exceeds `Max Header Size` and fails.
270+
For example, if the amount of bytes it would take to process the next field in the header
271+
would make the total bytes in the header exceed `Max Header Size`,
272+
then the ESDK MUST immediately fail instead of attempting to process those bytes.

0 commit comments

Comments
 (0)