ChecksumValidatingPublisher deals with any packetization

[vancobuca]

… of incoming data

• fixes issue ChecksumValidatingPublisher incorrectly processes buffers that only contain the checksum #965 (ChecksumValidatingPublisher incorrectly processes buffers that only contain the checksum #965)
• Unit test added

Description

Instead of assuming the checksum will be tacked onto an existing data packet (and incorrectly re-allocating the checksum array), the fixed code simply populates the checksum array with bytes that belong to the checksum, and only sends to the wrapped Subscriber the bytes that belong to the user data

Motivation and Context

Solves issue #965

Testing

Unit test added that:

• sends a single packet of data
• sends two packets of data with varying boundary (1 and (N-1); 2 and (N-2); 3 and (N-3); ...; (N-1) and 1)
• sends data in tiny packets: 1 byte each; 2 bytes each; ...; 15 bytes each
• checks special behavior for publishers that do not know what is the expected content length (this is a use case from the EndpointOverrideTest)

Screenshots (if appropriate)

Types of changes

• [X ] Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Checklist

• [ X] I have read the CONTRIBUTING document
• [ X] Local run of mvn install succeeds
• [ X] My code follows the code style of this project
• My change requires a change to the Javadoc documentation
• I have updated the Javadoc documentation accordingly
• [ X] I have read the README document
• [ X] I have added tests to cover my changes
• [ X] All new and existing tests passed
• [ X] A short description of the change has been added to the CHANGELOG
• My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

• [ X] I confirm that this pull request can be released under the Apache 2 license

[spfink]

Thanks for the submission! We'll get this out in our next release.

[zoewangg]

This is not the same copyright profile that we are using. Could you update it?

Here's the intelliJ copyright file template link, or you can copy paste from other files in this repo.

[vancobuca]

Hi Zoe,

I'm not affiliated with Amazon, so I thought putting an Amazon copyright there would not be kosher. Am I wrong? I don't mind doing it -- just seems odd.

[vancobuca]

Done

[zoewangg]

aws-java-sdk-v2 is under the Apache License 2.0 and we wanted to make sure all the files follow the same style.

Thank you for updating it!

[codecov-io]

Codecov Report

Merging #966 into master will increase coverage by 0.04%.
The diff coverage is 91.3%.

@@             Coverage Diff              @@
##             master     #966      +/-   ##
============================================
+ Coverage     55.34%   55.39%   +0.04%     
- Complexity     4492     4494       +2     
============================================
  Files           799      799              
  Lines         27469    27487      +18     
  Branches       2221     2224       +3     
============================================
+ Hits          15204    15227      +23     
+ Misses        11545    11542       -3     
+ Partials        720      718       -2

Impacted Files	Coverage Δ	Complexity Δ
...ices/s3/checksums/ChecksumValidatingPublisher.java	80.7% <91.3%> (+19.16%)	3 <2> (+1)	⬆️
...are/amazon/awssdk/core/internal/util/Mimetype.java	80.48% <0%> (+2.43%)	15% <0%> (+1%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 79fcb11...bc27546. Read the comment docs.

[afterthought]

Is it possible that this also relates to: #428 ? I'm testing code that uses: ifNoneMatch, which according to documentation should return a 304 response code. I have tested 2.2.0 and this branch. My code does this:

currentETag.ifPresent(builder::ifNoneMatch);
Observable.fromFuture(
            s3Client.getObject(
                    builder.build(),
                    AsyncResponseTransformer.toBytes()
            ))
            .map(response -> {
                if (response.asByteArray().length > 0) {
                    Files.write(filePath, response.asByteArray());
                    Files.write(eTagPath, response.response().eTag().getBytes());
                }

                return null;
            });

With this branch I get:

Caused by: java.lang.IllegalArgumentException: 0 > -16
at java.util.Arrays.copyOfRange(Arrays.java:3519)
at software.amazon.awssdk.services.s3.checksums.ChecksumValidatingPublisher$ChecksumSkippingSubscriber.onNext(ChecksumValidatingPublisher.java:156)
at software.amazon.awssdk.services.s3.checksums.ChecksumValidatingPublisher$ChecksumSkippingSubscriber.onNext(ChecksumValidatingPublisher.java:139)
at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler$FullResponseContentPublisher$1.request(ResponseHandler.java:329)

On 2.2.0 I get:

Caused by: java.lang.ArrayIndexOutOfBoundsException: null
at java.lang.System.arraycopy(Native Method)
at java.util.Arrays.copyOfRange(Arrays.java:3521)
at software.amazon.awssdk.services.s3.checksums.ChecksumValidatingPublisher$ChecksumValidatingSubscriber.onNext(ChecksumValidatingPublisher.java:90)
at software.amazon.awssdk.services.s3.checksums.ChecksumValidatingPublisher$ChecksumValidatingSubscriber.onNext(ChecksumValidatingPublisher.java:51)
at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler$FullResponseContentPublisher$1.request(ResponseHandler.java:329)
at software.amazon.awssdk.core.internal.http.async.SyncResponseHandlerAdapter$BaosSubscriber.onSubscribe(SyncResponseHandlerAdapter.java:106)

I'm hoping to figure out if I can get a short term workaround or if I have to skip etag support right now. Thanks. Chuck