From 314071f557f4255fb395aca64590e6dc24fef9db Mon Sep 17 00:00:00 2001 From: "Jonathan M. Henson" Date: Tue, 7 Nov 2023 16:07:40 -0800 Subject: [PATCH 1/9] Synchronous CRT Http Client SPI with tests (#4405) * implemented synchronous crt java http client with tests. * Forgot about informing the loader of what we did. * Still not completed with PR, but the large refactors have been done and I want to go ahead and get a review. Still WIP on moving the anonymous classes to another class. * Addressed refactor comments. Going to make another pass at adding some more integ tests. * make the client base abstract to force users to override it, since that's its intention. * Fixed but in the response stream handler for the async to sync stream adapter to properly complete and close upon end of the response. Began adding some tests but they don't all pass yet. * Updates to use new http exception error info for tls negotiation, changes to test running. passing sync client tests. * Revert to using the error code. Updated crt version to include the needed changes. * Refactored benchmarks and s3 stability tests to handle both async and sync runs. Added sync variants for crt clients for the http client benchmarks and s3 stability test. * Update the console output to have accurate test name. * Further code review fixes as well as updates for the linters. * Fixed some checkstyle errors I hope? * More checkstyle fixes. * PR Feedback * add volatile * Close threadpool at end of CRT tests --------- Co-authored-by: Waqar Ahmed Khan Co-authored-by: Dongie Agnir <261310+dagnir@users.noreply.github.com> --- .idea/inspectionProfiles/AWS_Java_SDK_2_0.xml | 14 +- .../http/crt/AwsCrtAsyncHttpClient.java | 293 ++-------------- .../awssdk/http/crt/AwsCrtHttpClient.java | 285 ++++++++++++++++ .../awssdk/http/crt/AwsCrtSdkHttpService.java | 11 +- .../crt/internal/AwsCrtClientBuilderBase.java | 135 ++++++++ .../crt/internal/AwsCrtHttpClientBase.java | 180 ++++++++++ .../crt/internal/CrtAsyncRequestContext.java | 84 +++++ .../http/crt/internal/CrtRequestContext.java | 12 +- .../http/crt/internal/CrtRequestExecutor.java | 211 +++++++++++- .../internal/request/CrtRequestAdapter.java | 68 +++- .../request/CrtRequestInputStreamAdapter.java | 78 +++++ ...CrtAsyncHttpClientSpiVerificationTest.java | 315 ++++++++++++++++++ .../AwsCrtAsyncHttpClientWireMockTest.java | 128 +++++++ .../AwsCrtHttpClientSpiVerificationTest.java | 241 +++----------- .../awssdk/http/crt/AwsCrtHttpClientTest.java | 61 ++++ .../crt/AwsCrtHttpClientWireMockTest.java | 54 +-- ...t.java => H1SAsyncServerBehaviorTest.java} | 2 +- .../crt/internal/CrtRequestExecutorTest.java | 40 +-- .../http/SdkHttpClientDefaultTestSuite.java | 8 +- .../awssdk/http/SdkHttpClientTestSuite.java | 20 +- .../awssdk/benchmark/BenchmarkRunner.java | 4 +- .../sync/ApacheHttpClientBenchmark.java | 2 + .../sync/CrtHttpClientBenchmark.java | 118 +++++++ .../UrlConnectionHttpClientBenchmark.java | 2 + .../tests/s3/S3AsyncBaseStabilityTest.java | 208 ++++++++++++ ...cWithCrtAsyncHttpClientStabilityTest.java} | 4 +- .../tests/s3/S3BaseStabilityTest.java | 99 ++++-- .../s3/S3CrtAsyncClientStabilityTest.java | 6 +- .../tests/s3/S3CrtClientStabilityTest.java | 90 +++++ .../tests/s3/S3NettyAsyncStabilityTest.java | 2 +- .../tests/utils/StabilityTestRunner.java | 17 +- 31 files changed, 2202 insertions(+), 590 deletions(-) create mode 100644 http-clients/aws-crt-client/src/main/java/software/amazon/awssdk/http/crt/AwsCrtHttpClient.java create mode 100644 http-clients/aws-crt-client/src/main/java/software/amazon/awssdk/http/crt/internal/AwsCrtClientBuilderBase.java create mode 100644 http-clients/aws-crt-client/src/main/java/software/amazon/awssdk/http/crt/internal/AwsCrtHttpClientBase.java create mode 100644 http-clients/aws-crt-client/src/main/java/software/amazon/awssdk/http/crt/internal/CrtAsyncRequestContext.java create mode 100644 http-clients/aws-crt-client/src/main/java/software/amazon/awssdk/http/crt/internal/request/CrtRequestInputStreamAdapter.java create mode 100644 http-clients/aws-crt-client/src/test/java/software/amazon/awssdk/http/crt/AwsCrtAsyncHttpClientSpiVerificationTest.java create mode 100644 http-clients/aws-crt-client/src/test/java/software/amazon/awssdk/http/crt/AwsCrtAsyncHttpClientWireMockTest.java create mode 100644 http-clients/aws-crt-client/src/test/java/software/amazon/awssdk/http/crt/AwsCrtHttpClientTest.java rename http-clients/aws-crt-client/src/test/java/software/amazon/awssdk/http/crt/{H1ServerBehaviorTest.java => H1SAsyncServerBehaviorTest.java} (95%) create mode 100644 test/sdk-benchmarks/src/main/java/software/amazon/awssdk/benchmark/apicall/httpclient/sync/CrtHttpClientBenchmark.java create mode 100644 test/stability-tests/src/it/java/software/amazon/awssdk/stability/tests/s3/S3AsyncBaseStabilityTest.java rename test/stability-tests/src/it/java/software/amazon/awssdk/stability/tests/s3/{S3WithCrtAsyncHttpClientStabilityTest.java => S3AsyncWithCrtAsyncHttpClientStabilityTest.java} (93%) create mode 100644 test/stability-tests/src/it/java/software/amazon/awssdk/stability/tests/s3/S3CrtClientStabilityTest.java diff --git a/.idea/inspectionProfiles/AWS_Java_SDK_2_0.xml b/.idea/inspectionProfiles/AWS_Java_SDK_2_0.xml index 95ed390c8a7c..eb56a8103000 100644 --- a/.idea/inspectionProfiles/AWS_Java_SDK_2_0.xml +++ b/.idea/inspectionProfiles/AWS_Java_SDK_2_0.xml @@ -83,7 +83,6 @@ - @@ -132,6 +131,9 @@ + + @@ -249,7 +251,6 @@ - @@ -306,7 +307,6 @@ -