Skip to content

Commit aedf9cb

Browse files
committed
Add initial design tenets/goals for S3 TransferManager
1 parent d17d35f commit aedf9cb

File tree

1 file changed

+66
-0
lines changed
  • docs/design/services/s3/transfermanager

1 file changed

+66
-0
lines changed
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Project Tenets (unless you know better ones)
2+
3+
1. Meeting customers in their problem space allows them to deliver value
4+
quickly.
5+
2. Meeting customer expectations drives usability.
6+
3. Discoverability drives usage.
7+
8+
# Introduction
9+
10+
This project provides a much improved experience for S3 customers needing to
11+
easily perform uploads and downloads of objects to and from S3 by providing the
12+
S3 `S3TransferManager`, a high level library built on the S3 client.
13+
14+
# Project Goals
15+
16+
1. For the use cases it addresses, i.e. the transfer of objects to and from S3,
17+
S3TransferManager is the preferred solution. It is easier and more intuitive
18+
than using the S3 client. In the majority of situations, it is more
19+
performant.
20+
1. S3TransferManager provides a truly asynchronous, non-blocking API that
21+
conforms to the norms present in the rest of the SDK.
22+
1. S3TransferManager makes efficient use of system resources.
23+
1. S3TransferManager supplements rather than replaces the lower level S3 client.
24+
25+
# Non Project Goals
26+
27+
1. Ability to use the blocking, synchronous client.
28+
29+
Using a blocking client would severely impede the ability to deliver on goals
30+
#2 and #3.
31+
32+
# Customer-Requested Changes from 1.11.x
33+
34+
* S3TransferManager supports progress listeners that are easier to use.
35+
36+
Ref: https://github.com/aws/aws-sdk-java-v2/issues/37#issuecomment-316218667
37+
38+
* S3TransferManager provides bandwidth limiting of uploads and downloads.
39+
40+
Ref: https://github.com/aws/aws-sdk-java/issues/1103
41+
42+
* The size of resources used by Transfermanager and configured by the user
43+
should not affect its stability.
44+
45+
For example, the configured size of a threadpool should be irellevant to its
46+
ability to successfuly perform an operation.
47+
48+
Ref: https://github.com/aws/aws-sdk-java/issues/939
49+
50+
* S3TransferManager supports parallel downloads of any object.
51+
52+
Any object stored in S3 should be downloadable in multiple parts
53+
simultaneously, not just those uploaded using the Multipart API.
54+
55+
* S3TransferManager has the ability to upload to and download from a pre-signed
56+
URL.
57+
58+
* S3TransferManager allows uploads and downloads from and to memory.
59+
60+
Ref: https://github.com/aws/aws-sdk-java/issues/474
61+
62+
* Ability to easily use canned ACL policies with all transfers to S3.
63+
64+
Ref: https://github.com/aws/aws-sdk-java/issues/1207
65+
66+
* Trailing checksums for parallel uploads and downloads.

0 commit comments

Comments
 (0)