Skip to content

Commit 0f2e575

Browse files
author
Bennett Lynch
committed
[S3] Add support for more user-friendly CopyObject source parameters
## Motivation and Context The current S3Client interface has a cumbersome API for invoking CopyObjectRequests. We require users to define the bucket name, key name, and version ID in a raw string format. We require that the string conform to the S3 API, which forces users to know the intricate details for how to join these values together. Additionally, portions (but not all) of the value must be URL encoded, further increasing the burden. https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html#API_CopyObject_RequestParameters In the Java SDK v1, users are given explicit parameters for the different copy source attributes. But in v2, parity for this support is clearly lacking. E.g., v1: ``` s3.copyObject(new CopyObjectRequest() .withSourceBucketName(SOURCE_BUCKET) .withSourceKey(key) .withSourceVersionId(versionId) .withDestinationBucketName(DESTINATION_BUCKET) .withDestinationKey(key)); ``` v2: ``` s3.copyObject(CopyObjectRequest.builder() .copySource(SOURCE_BUCKET + "/" + key + "?versionId=" + versionId) .destinationBucket(DESTINATION_BUCKET) .destinationKey(key) .build()); ``` The v1 SDK will also URL encode on the user's behalf, allowing users to use the same input values as they would for a PutObjectRequest. The v2 code snippet above may appear to work for most users until they run into unexpected source keys that require URL encoding, at which point they will typically be given `NoSuchKey` errors. This API deficiency has been called out by users in at least the following issues: * aws#1313 * aws#1452 * aws#1656 * awsdocs/aws-doc-sdk-examples#740 ## Description * For both CopyObjectRequest and UploadPartCopyRequest, add explicit parameters for: SourceBucket, SourceKey, SourceVersionId * If specified, these values will be used to construct a CopySource on the user's behalf, including URL encoding the relevant portions. * These values are introduced in a backwards compatible fashion. Users who are already using CopySource today will see no change in behavior, but these new fields may not be used in conjunction with CopySource. * A follow-up PR will be submitted to propose deprecating the current CopySource parameter. It is excluded from this PR since our current code gen configuration lacks appropriate support. * Add support for "DestinationBucket" & "DestinationKey" to UploadPartCopyRequest (this support already existed for CopyObjectRequest) * Utility function added to detect if an ARN is for a particular S3 resource type. This is to conform with the S3 API requirements of inserting "/object" in the path of these requests. ## Testing * New unit tests added * New integration tests added
1 parent cf7f797 commit 0f2e575

File tree

1 file changed

+3
-8
lines changed
  • services/s3/src/test/java/software/amazon/awssdk/services/s3/internal/resource

1 file changed

+3
-8
lines changed

services/s3/src/test/java/software/amazon/awssdk/services/s3/internal/resource/S3ArnUtilsTest.java

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -136,23 +136,18 @@ public void parseOutpostArn_malformedArnEmptyOutpostId_shouldThrowException() {
136136
@Test
137137
public void getArnType_shouldRecognizeAccessPointArn() {
138138
String arnString = "arn:aws:s3:us-west-2:123456789012:accesspoint/my-access-point";
139-
Optional<S3ResourceType> arnType = S3ArnUtils.getArnType(arnString);
140-
assertThat(arnType.isPresent(), is(true));
141-
assertThat(arnType.get(), is(S3ResourceType.ACCESS_POINT));
139+
assertThat(S3ArnUtils.getArnType(arnString), is(Optional.of(S3ResourceType.ACCESS_POINT)));
142140
}
143141

144142
@Test
145143
public void getArnType_shouldRecognizeOutpostArn() {
146144
String arnString = "arn:aws:s3-outposts:us-west-2:123456789012:outpost/my-outpost/bucket/my-bucket";
147-
Optional<S3ResourceType> arnType = S3ArnUtils.getArnType(arnString);
148-
assertThat(arnType.isPresent(), is(true));
149-
assertThat(arnType.get(), is(S3ResourceType.OUTPOST));
145+
assertThat(S3ArnUtils.getArnType(arnString), is(Optional.of(S3ResourceType.OUTPOST)));
150146
}
151147

152148
@Test
153149
public void getArnType_shouldNotThrow_onRandomInput() {
154150
String arnString = UUID.randomUUID().toString();
155-
Optional<S3ResourceType> arnType = S3ArnUtils.getArnType(arnString);
156-
assertThat(arnType.isPresent(), is(false));
151+
assertThat(S3ArnUtils.getArnType(arnString), is(Optional.empty()));
157152
}
158153
}

0 commit comments

Comments
 (0)