NoSuchMethodError using SqsLargeMessageAspect and PayloadS3Pointer

[mtwrocky]

It basically has disabled our ability to submit events over 256KB

What were you trying to accomplish?

Expected Behavior

Running a test with upgrading from 1.4.0 to 1.12 powertools-sqs we hit this issue.
The expected behavior is the test should pass.

Current Behavior

Runtime error

Possible Solution

I think the change would be to change SqsLargeMessageAspect line 71 to match what the return is in PayloadS3Pointer or remove the PayloadS3Pointer from the jar file then this would match in what is in aws sqs extended 2.0.2

Steps to Reproduce (for bugs)

Basically provide data > 256KB to SqsUtils.enrichedMessageFromS3
Note: I am using Amazon Corretto 11

Environment

• **Powertools version used 1.12 **:
• Packaging format (Layers, Maven/Gradle):
  Gradle aspect ('software.amazon.lambda:powertools-sqs:1.12.0')
• AWS Lambda function runtime:
• Debugging logs

How to enable debug mode**

'java.util.Optional software.amazon.payloadoffloading.PayloadS3Pointer.fromJson(java.lang.String)': java.lang.NoSuchMethodError
java.lang.NoSuchMethodError: 'java.util.Optional software.amazon.payloadoffloading.PayloadS3Pointer.fromJson(java.lang.String)'
at software.amazon.lambda.powertools.sqs.internal.SqsLargeMessageAspect.processMessages(SqsLargeMessageAspect.java:71)
at software.amazon.lambda.powertools.sqs.SqsUtils.enrichedMessageFromS3(SqsUtils.java:80)

[pankajagrawal16]

Thanks for reporting this @mtwrocky . I will have a look at this.

[pankajagrawal16]

Hey @mtwrocky , Does your Lambda function project also depends on payloadoffloading-common library? It seems these two conflicts. Powertools in itself does not require payloadoffloading-common anymore, so is there a reason why project depends upon it?

[mtwrocky]

The only place that we are using it that I am aware of is using it within the power tools. I am not that great with gradle, I was trying to figure out how to suppress one of the PayloadS3Pointer but could not figure it out. Correct me if I am wrong but I believe AWS sdk 2.X has dependencies on the payload-common. Thanks for the response. It is greatly appreciated.

…

-Toe

On Thu, Apr 21, 2022 at 2:49 AM Pankaj Agrawal ***@***.***> wrote: Hey @mtwrocky <https://github.com/mtwrocky> , Does your Lambda function project also depends on payloadoffloading-common library? It seems these two conflicts. Powertools in itself does not require payloadoffloading-common anymore, so is there a reason why project depends upon it? — Reply to this email directly, view it on GitHub <#835 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALZ4AVL7PC2X5JVSEZWRCGDVGECAPANCNFSM5TQQ5V5A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Toemas Wersinger Cell: 612.986.0049

[pankajagrawal16]

Ok I understand. I would be very interested in learning from where payloadoffloading-common dependency is being pulled in. As far as I can see, V2 SDK, does not directly depend upon it but might be its pulling it inn transitively. Would it be possible for you to look at dependency tree for the project? If the project does not use it, then the issue should be fixed by excluding it. That being said, I will also look into the possibilities of how we can solve this dependency conflict in powertools itself.

[mtwrocky]

Sorry I stated the piece It is in the SQS Extended client lib https://frontbackend.com/maven/artifact/com.amazonaws/amazon-sqs-java-extended-client-lib/2.0.2 under the compile dependencies. Thanks

…

On Fri, Apr 22, 2022 at 2:46 AM Pankaj Agrawal ***@***.***> wrote: Ok I understand. I would be very interested in learning from where payloadoffloading-common dependency is being pulled in. As far as I can see, V2 SDK, does not directly depend upon it but might be its pulling it inn transitively. Would it be possible for you to look at dependency tree for the project? If the project does not use it, then the issue should be fixed by excluding it. That being said, I will also look into the possibilities of how we can solve this dependency conflict in powertools itself. — Reply to this email directly, view it on GitHub <#835 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALZ4AVOWTX4GAAJOCNWMYUDVGJKLJANCNFSM5TQQ5V5A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Toemas Wersinger Cell: 612.986.0049

[pankajagrawal16]

Ok i understand, so does the application/consumer lambda function need to depend on SQS Extended client lib ?

[mtwrocky]

Yes we are using sqs extended 2.0.2. (AmazonSQSExtendedClient, ExtendedClientConfig) -Toe

…

On Fri, Apr 22, 2022 at 5:54 AM Pankaj Agrawal ***@***.***> wrote: Ok i understand, so does the application/consumer lambda function need to depend on SQS Extended client lib ? — Reply to this email directly, view it on GitHub <#835 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALZ4AVKHUN6LG7Y5GSA7HQTVGKAOHANCNFSM5TQQ5V5A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Toemas Wersinger Cell: 612.986.0049

[github-actions]

This is now released under 1.12.2 version!