GetBucketPolicy with async S3 client doesn't work #1577

avgdorom · 2020-01-06T14:59:26Z

I'm trying to use the S3AsyncClient in order to get bucket policies from my AWS account.
I created a client, giving it the correct credentials and region, but I keep getting null in the content of the response.

Just to be sure that I initialized the client correctly, I tried other requests like, getBucketPolicyStatus, listBuckets and more. Every other request I tried returned the expected data, except for getBucketPolicy... In that case the response body is always null, which then causes an XML parsing error (I debugged until I came to GetBucketPolicyInterceptor.modifyHttpResponseContent()).

NOTE: For Exactly the same request, the S3 sync client works like a charm.

The text was updated successfully, but these errors were encountered:

debora-ito · 2020-01-06T18:44:44Z

Hy @avgdorom Can you share a sample code where you create and use the client? And also the stacktrace with the error you're seeing?

avgdorom · 2020-01-06T21:31:14Z

Hi @debora-ito, of course and thanks for the quick response.
First, a sample code, then I will get the stacktrace and post it in another comment:

public static void main(String[] args) {
        S3AsyncClient s3AsyncClient = S3AsyncClient.builder()
            .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(accessKey, secretKey)))
            .build();

        S3Client s3SyncClient = S3Client.builder()
            .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(accessKey, secretKey)))
            .build();

        Bucket bucket = Bucket.builder().name("my-test-bucket").build();

        try {
            
            GetBucketPolicyRequest bucketPolicyRequest = GetBucketPolicyRequest.builder()
                .bucket(bucket.name())
                .build();

            CompletableFuture<GetBucketPolicyResponse> bucketPolicyFuture = s3AsyncClient.getBucketPolicy(bucketPolicyRequest);
            System.out.println(bucketPolicyFuture.join());

            GetBucketPolicyResponse bucketPolicySync = s3SyncClient.getBucketPolicy(bucketPolicyRequest);
            System.out.println(bucketPolicySync);
            
        } catch (Throwable e) {
            e.printStackTrace();
        }
        s3AsyncClient.close();
        s3SyncClient.close();
}

NOTE: I put a sync call right next to it in my test, as I mentioned before, to compare the two. The sync call worked just fine.

avgdorom · 2020-01-06T21:34:58Z

The stacktrace:

java.util.concurrent.CompletionException: software.amazon.awssdk.core.exception.SdkClientException: Could not parse XML response.
	at software.amazon.awssdk.utils.CompletableFutureUtils.errorAsCompletionException(CompletableFutureUtils.java:61)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.lambda$execute$0(AsyncExecutionFailureExceptionReportingStage.java:51)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
	at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:75)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.retryErrorIfNeeded(AsyncRetryableStage.java:175)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.retryIfNeeded(AsyncRetryableStage.java:126)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.lambda$execute$0(AsyncRetryableStage.java:107)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$1(MakeAsyncHttpRequestStage.java:140)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: software.amazon.awssdk.core.exception.SdkClientException: Could not parse XML response.
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:97)
	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:43)
	at software.amazon.awssdk.protocols.query.unmarshall.XmlDomParser.parse(XmlDomParser.java:48)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.XmlProtocolUnmarshaller.unmarshall(XmlProtocolUnmarshaller.java:56)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlResponseHandler.unmarshallResponse(AwsXmlResponseHandler.java:82)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlResponseHandler.handle(AwsXmlResponseHandler.java:61)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlResponseHandler.handle(AwsXmlResponseHandler.java:41)
	at software.amazon.awssdk.core.client.handler.BaseClientHandler.lambda$interceptorCalling$2(BaseClientHandler.java:151)
	at software.amazon.awssdk.core.client.handler.AttachHttpMetadataResponseHandler.handle(AttachHttpMetadataResponseHandler.java:40)
	at software.amazon.awssdk.core.client.handler.AttachHttpMetadataResponseHandler.handle(AttachHttpMetadataResponseHandler.java:28)
	at software.amazon.awssdk.core.internal.http.async.AsyncResponseHandler.lambda$prepare$0(AsyncResponseHandler.java:88)
	at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
	at software.amazon.awssdk.core.internal.http.async.AsyncResponseHandler$BaosSubscriber.onComplete(AsyncResponseHandler.java:129)
	at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.runAndLogError(ResponseHandler.java:180)
	at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.access$500(ResponseHandler.java:67)
	at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler$PublisherAdapter$1.onComplete(ResponseHandler.java:296)
	at com.typesafe.netty.HandlerPublisher.complete(HandlerPublisher.java:408)
	at com.typesafe.netty.HandlerPublisher.handlerRemoved(HandlerPublisher.java:395)
	at io.netty.channel.AbstractChannelHandlerContext.callHandlerRemoved(AbstractChannelHandlerContext.java:972)
	at io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:638)
	at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:481)
	at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:427)
	at com.typesafe.netty.http.HttpStreamsHandler.removeHandlerIfActive(HttpStreamsHandler.java:328)
	at com.typesafe.netty.http.HttpStreamsHandler.handleReadHttpContent(HttpStreamsHandler.java:189)
	at com.typesafe.netty.http.HttpStreamsHandler.channelRead(HttpStreamsHandler.java:165)
	at com.typesafe.netty.http.HttpStreamsClientHandler.channelRead(HttpStreamsClientHandler.java:148)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:328)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:302)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	... 1 more
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,1]
Message: Content is not allowed in prolog.
	at java.xml/com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:652)
	at java.xml/com.sun.xml.internal.stream.XMLEventReaderImpl.nextEvent(XMLEventReaderImpl.java:83)
	at software.amazon.awssdk.protocols.query.unmarshall.XmlDomParser.parse(XmlDomParser.java:44)
	... 68 more

As I understand, this XML parse error stems from the content of the response being null.

debora-ito · 2020-01-07T17:54:16Z

Transferred the issue to Java V2 repo.

debora-ito · 2020-01-07T19:21:04Z

@avgdorom thank you for reporting this, we'll investigate.

avgdorom · 2020-01-11T07:14:29Z

@bmaizels Thanks! This is great. One question though: how do I get the fix? Do I need to wait for the next Java SDK release? If so, when is that due?

bmaizels · 2020-01-13T20:17:38Z

@avgdorom This should be included with the 2.10.47 release on 1/10.

avgdorom · 2020-01-13T21:59:12Z

@bmaizels Thanks, but in that case, I don't think it works... Tried it again twice, with version 2.10.47, and it just doesn't work. I get exactly what I reported in the bug in the first place.
Can you please check?

debora-ito · 2020-01-13T23:06:54Z

@avgdorom actually it is available in today's release, version 2.10.48 - https://github.com/aws/aws-sdk-java-v2/blob/master/CHANGELOG.md#21048-2020-01-13

Apologies for the confusion.

bmaizels · 2020-01-14T17:53:35Z

@avgdorom That was my fault, sorry about that confusion. I've retested it to be sure and I see broken in 2.10.47 and fixed in 2.10.48. Please let me know if you have any more problems with it.

avgdorom · 2020-01-15T06:04:24Z

@bmaizels It's fine, it happens 😄 Works like a charm now! thank you.

…929bad28d Pull request: release <- staging/171c037d-ce75-4c0b-aa92-79f929bad28d

debora-ito transferred this issue from aws/aws-sdk-java Jan 7, 2020

debora-ito added the bug This issue is a bug. label Jan 7, 2020

bmaizels mentioned this issue Jan 10, 2020

Bugfix: s3 asynchronous client GetBucketPolicy now works and does not… #1586

Merged

13 tasks

bmaizels closed this as completed in #1586 Jan 10, 2020

aws-sdk-java-automation added a commit that referenced this issue Jul 30, 2021

Merge pull request #1577 from aws/staging/171c037d-ce75-4c0b-aa92-79f…

f762d7e

…929bad28d Pull request: release <- staging/171c037d-ce75-4c0b-aa92-79f929bad28d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

GetBucketPolicy with async S3 client doesn't work #1577

GetBucketPolicy with async S3 client doesn't work #1577

avgdorom commented Jan 6, 2020

debora-ito commented Jan 6, 2020

Uh oh!

avgdorom commented Jan 6, 2020 •

edited

Loading

Uh oh!

avgdorom commented Jan 6, 2020

Uh oh!

debora-ito commented Jan 7, 2020

Uh oh!

debora-ito commented Jan 7, 2020

Uh oh!

avgdorom commented Jan 11, 2020

Uh oh!

bmaizels commented Jan 13, 2020 •

edited

Loading

Uh oh!

avgdorom commented Jan 13, 2020

Uh oh!

debora-ito commented Jan 13, 2020

Uh oh!

bmaizels commented Jan 14, 2020

Uh oh!

avgdorom commented Jan 15, 2020

Uh oh!

GetBucketPolicy with async S3 client doesn't work #1577

GetBucketPolicy with async S3 client doesn't work #1577

Comments

avgdorom commented Jan 6, 2020

debora-ito commented Jan 6, 2020

Uh oh!

avgdorom commented Jan 6, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

avgdorom commented Jan 6, 2020

Uh oh!

debora-ito commented Jan 7, 2020

Uh oh!

debora-ito commented Jan 7, 2020

Uh oh!

avgdorom commented Jan 11, 2020

Uh oh!

bmaizels commented Jan 13, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

avgdorom commented Jan 13, 2020

Uh oh!

debora-ito commented Jan 13, 2020

Uh oh!

bmaizels commented Jan 14, 2020

Uh oh!

avgdorom commented Jan 15, 2020

Uh oh!

avgdorom commented Jan 6, 2020 •

edited

Loading

bmaizels commented Jan 13, 2020 •

edited

Loading