S3TransferManager getS3AsyncClient

[aceeric]

Describe the feature

Using the V1 SDK:

TransferManager tm = TransferManagerBuilder.standard()...build();

Then it was possible to:

ObjectListing objectListing = tm.getAmazonS3Client().listObjects(...);
etc.

Using V2, one would naturally think of:

S3TransferManager tm = S3TransferManager.builder()...build();
S3AsyncClient client = tm.getS3AsyncClient();
CompletableFuture<ListObjectsV2Response> future = client.listObjectsV2(r -> r.bucket("foo"));
etc.

But it does not appear to be possible to get the client from the transfer manager. Can this capability be added?

Is your Feature Request related to a problem?

Without this capability, one needs to separately use an S3AsyncClient instance. Since the DefaultS3TransferManager already has the s3CrtAsyncClient field, providing a getter for that would help.

Proposed Solution

Add method getS3AsyncClient to either S3TransferManager interface or DefaultS3TransferManager class (requires downcast...)

Describe alternatives you've considered

Create and use a separate S3AsyncClient instance.

Acknowledge

• I may be able to implement this feature request

AWS Java SDK version used

s3-transfer-manager 2.17.96-PREVIEW

JDK version used

1.8

Operating System and version

Centos 7

[debora-ito]

Thank you for reaching out @aceeric, adding this to our backlog.

[debora-ito]

@aceeric Actually, we are refactoring some aspects of the TransferManager configuration, and are planning to provide a way to pass a S3Client to it. Would this work for your use case?

[aceeric]

Indeed. And - additionally - we want to use the V2 transfer manager with the ability to set an endpoint as a URI like with the S3AsyncClientBuilder to support using the SDK against an S3-compliant backend inside Kubernetes. Presently, the SDK+CRT seems hard-wired to s3.awazonaws.... and 443/TLS. We would want to be able to use different host names and ports to support running both in-cluster against a Kubernetes service endpoint, and also on the desktop with port-forwarding (http://localhost:8080 or https://localhost:8443) and so on. Do you have a sense of how imminent this refactoring is that would make the transfer manager equivalent with the S3AsyncClientBuilder + S3AsyncClient as described? Thank you.

[debora-ito]

Yes, when we add the ability to provide a custom S3Client you can define the necessary endpoints in it. We are tracking the feature here: #2740

We don't have a timeline for when the refactoring will be finished but we are actively working on the TransferManager v2.

I'll go ahead and close this, feel free to reach out with other questions or requests.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.