Skip to content

Commit 9798b4f

Browse files
joviegascenedhryn
joviegas
and
cenedhryn
authored
S3 Cross Region Client (#4132)
* Adding a generic method invokeOperation for abstract delegating clien… (#3996) Adding a generic method invokeOperation for abstract delegating client to allow overriding behavior changes for all operations * Add endpointProvider in ClientConfig and RequestOverrideConfig. Also add crossRegionBucketAccess in S3Configuration (#3978) * Add endpointProvider in ClientConfig and RequestOverrideConfig. Also add crossRegionBucketAccess in S3Configuration * updated review comments 1 * updated review comments 2 * Adds S3CrossRegion clients and codegen for retrieving bucket name (#4008) * Adds S3CrossRegion clients and codegen for retrieving bucket name * Removes codegen of bucket parameter since getValueForField() is better * Add back overloads * Adds cross region client logic for decorating endpoint provider (#4026) * Adds cross region client logic for decorating endpoint provider * Paginated methods returning publisher or iterable implement logic in … (#4046) * Paginated methods returning publisher or iterable implement logic in interface instead of throwing unsupported exception * Added internal annotation to class * Wraps s3 client in cross regional client when enabled (#4080) * Wraps s3 client in cross regional client when enabled * Adds composer interface * Add User Agent Api name for Cross region API calls (#4105) * Add User Agent Api name for Cross region API calls * Added test case for default client not to have user agent related to cross region * S3CrossRegion Sync and Async Clients Redirect implementation (#4089) * S3CrossRegionSyncClient Redirect implementation * Added implementation for Async client Decorator * Updated older Cross region test cases * Added paramterized test * Async Exception checged to completableException * Updated test cases and changes the Exception handling when HeadBucket Call fails * Handled Anna-karin's comments * Removed async execution of HeadBucket and attached it to the completableFuture of main request * Handled Zoe's comments * Added test case when Redirected after the Region is cached * Changed region constant to Region Type in Tests * Cross region support for CRT Client (#4129) * Cross region support for CRT Client * removing common class * handled review comments * Integration test cases for Cross region Async and Sync Clients (#4128) * Integration test cases for Cross region Sync and Sync Clients * Renamed files as Integration test * Handled CR comments to take care of retries thus captureInterceptor is removed * rebased and added CRT in Integ test * removed sout from integ * Attach crossRegionAccessEnabled as CustomClientParams to a client Builder, also mention of Composers are renamed as decorators (#4151) * Attach crossRegionAccessEnabled as CustomClientParams to a client Builder, also mention of Composers are renamed as decorators * Removed from S3Configurations * Move the optional parameter in getter --------- Co-authored-by: Anna-Karin Salander <[email protected]>
1 parent 6ad5e92 commit 9798b4f

File tree

94 files changed

+4608
-1509
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

94 files changed

+4608
-1509
lines changed

‎.changes/next-release/feature-AWSSDKforJavav2-70eb163.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"type": "feature",
3+
"category": "AWS SDK for Java v2",
4+
"contributor": "",
5+
"description": "Cross region bucket access for S3 Client. This feature will allow users to access buckets of different region using a single cross region configured client."
6+
}

‎codegen/src/main/java/software/amazon/awssdk/codegen/emitters/tasks/CommonGeneratorTasks.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,6 @@ class CommonGeneratorTasks extends CompositeGeneratorTask {
2828
new ModelClassGeneratorTasks(params),
2929
new PackageInfoGeneratorTasks(params),
3030
new BaseExceptionClassGeneratorTasks(params),
31-
new ClientOptionsClassGeneratorTasks(params));
31+
new CommonInternalGeneratorTasks(params));
3232
}
3333
}

‎codegen/src/main/java/software/amazon/awssdk/codegen/emitters/tasks/ClientOptionsClassGeneratorTasks.java renamed to ‎codegen/src/main/java/software/amazon/awssdk/codegen/emitters/tasks/CommonInternalGeneratorTasks.java

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,26 +15,35 @@
1515

1616
package software.amazon.awssdk.codegen.emitters.tasks;
1717

18-
import java.util.Collections;
18+
import java.util.Arrays;
1919
import java.util.List;
2020
import software.amazon.awssdk.codegen.emitters.GeneratorTask;
2121
import software.amazon.awssdk.codegen.emitters.GeneratorTaskParams;
2222
import software.amazon.awssdk.codegen.emitters.PoetGeneratorTask;
2323
import software.amazon.awssdk.codegen.poet.client.SdkClientOptions;
24+
import software.amazon.awssdk.codegen.poet.common.UserAgentUtilsSpec;
2425

25-
public class ClientOptionsClassGeneratorTasks extends BaseGeneratorTasks {
26+
public class CommonInternalGeneratorTasks extends BaseGeneratorTasks {
2627
private final GeneratorTaskParams params;
2728

28-
public ClientOptionsClassGeneratorTasks(GeneratorTaskParams params) {
29+
public CommonInternalGeneratorTasks(GeneratorTaskParams params) {
2930
super(params);
3031
this.params = params;
3132
}
3233

3334
@Override
3435
protected List<GeneratorTask> createTasks() throws Exception {
35-
return Collections.singletonList(
36-
new PoetGeneratorTask(clientOptionsDir(), params.getModel().getFileHeader(), new SdkClientOptions(params.getModel()))
37-
);
36+
return Arrays.asList(createClientOptionTask(), createUserAgentTask());
37+
}
38+
39+
private PoetGeneratorTask createClientOptionTask() {
40+
return new PoetGeneratorTask(clientOptionsDir(), params.getModel().getFileHeader(),
41+
new SdkClientOptions(params.getModel()));
42+
}
43+
44+
private PoetGeneratorTask createUserAgentTask() {
45+
return new PoetGeneratorTask(clientOptionsDir(), params.getModel().getFileHeader(),
46+
new UserAgentUtilsSpec(params.getModel()));
3847
}
3948

4049
private String clientOptionsDir() {

‎codegen/src/main/java/software/amazon/awssdk/codegen/model/config/customization/CustomizationConfig.java

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
import java.util.HashMap;
2020
import java.util.List;
2121
import java.util.Map;
22+
import software.amazon.awssdk.codegen.model.service.ClientContextParam;
2223
import software.amazon.awssdk.core.retry.RetryMode;
2324
import software.amazon.awssdk.core.traits.PayloadTrait;
2425
import software.amazon.awssdk.utils.AttributeMap;
@@ -212,6 +213,20 @@ public class CustomizationConfig {
212213
*/
213214
private boolean delegateSyncClientClass;
214215

216+
/**
217+
* Fully qualified name of a class that given the default sync client instance can return the final client instance,
218+
* for instance by decorating the client with specific-purpose implementations of the client interface.
219+
* See S3 customization.config for an example.
220+
*/
221+
private String syncClientDecorator;
222+
223+
/**
224+
* Fully qualified name of a class that given the default async client instance can return the final client instance,
225+
* for instance by decorating the client with specific-purpose implementations of the client interface.
226+
* See S3 customization.config for an example.
227+
*/
228+
private String asyncClientDecorator;
229+
215230
/**
216231
* Whether to skip generating endpoint tests from endpoint-tests.json
217232
*/
@@ -236,6 +251,11 @@ public class CustomizationConfig {
236251
*/
237252
private boolean requiredTraitValidationEnabled = false;
238253

254+
/**
255+
* Customization to attach map of Custom client param configs that can be set on a client builder.
256+
*/
257+
private Map<String, ClientContextParam> customClientContextParams;
258+
239259
private CustomizationConfig() {
240260
}
241261

@@ -566,6 +586,22 @@ public void setDelegateAsyncClientClass(boolean delegateAsyncClientClass) {
566586
this.delegateAsyncClientClass = delegateAsyncClientClass;
567587
}
568588

589+
public String getSyncClientDecorator() {
590+
return syncClientDecorator;
591+
}
592+
593+
public void setSyncClientDecorator(String syncClientDecorator) {
594+
this.syncClientDecorator = syncClientDecorator;
595+
}
596+
597+
public String getAsyncClientDecorator() {
598+
return asyncClientDecorator;
599+
}
600+
601+
public void setAsyncClientDecorator(String asyncClientDecorator) {
602+
this.asyncClientDecorator = asyncClientDecorator;
603+
}
604+
569605
public boolean isDelegateSyncClientClass() {
570606
return delegateSyncClientClass;
571607
}
@@ -621,4 +657,12 @@ public boolean isRequiredTraitValidationEnabled() {
621657
public void setRequiredTraitValidationEnabled(boolean requiredTraitValidationEnabled) {
622658
this.requiredTraitValidationEnabled = requiredTraitValidationEnabled;
623659
}
660+
661+
public Map<String, ClientContextParam> getCustomClientContextParams() {
662+
return customClientContextParams;
663+
}
664+
665+
public void setCustomClientContextParams(Map<String, ClientContextParam> customClientContextParams) {
666+
this.customClientContextParams = customClientContextParams;
667+
}
624668
}

‎codegen/src/main/java/software/amazon/awssdk/codegen/model/config/customization/ServiceConfig.java

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,8 @@ public class ServiceConfig {
4747

4848
private boolean hasAccelerateModeEnabledProperty = false;
4949

50+
private boolean hasCrossRegionAccessEnabledProperty = false;
51+
5052
public String getClassName() {
5153
return className;
5254
}
@@ -95,6 +97,14 @@ public void setHasPathStyleAccessEnabledProperty(boolean hasPathStyleAccessEnabl
9597
this.hasPathStyleAccessEnabledProperty = hasPathStyleAccessEnabledProperty;
9698
}
9799

100+
public boolean hasCrossRegionAccessEnabledProperty() {
101+
return hasCrossRegionAccessEnabledProperty;
102+
}
103+
104+
public void setHasCrossRegionAccessEnabledProperty(boolean hasCrossRegionAccessEnabledProperty) {
105+
this.hasCrossRegionAccessEnabledProperty = hasCrossRegionAccessEnabledProperty;
106+
}
107+
98108
public boolean hasAccelerateModeEnabledProperty() {
99109
return hasAccelerateModeEnabledProperty;
100110
}

‎codegen/src/main/java/software/amazon/awssdk/codegen/model/intermediate/IntermediateModel.java

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,21 @@ public String getSdkResponseBaseClassName() {
235235
}
236236
}
237237

238+
public Optional<String> syncClientDecoratorClassName() {
239+
if (customizationConfig.getSyncClientDecorator() != null) {
240+
return Optional.of(customizationConfig.getSyncClientDecorator());
241+
}
242+
return Optional.empty();
243+
}
244+
245+
public Optional<String> asyncClientDecoratorClassName() {
246+
String asyncClientDecorator = customizationConfig.getAsyncClientDecorator();
247+
if (customizationConfig.getAsyncClientDecorator() != null) {
248+
return Optional.of(asyncClientDecorator);
249+
}
250+
return Optional.empty();
251+
}
252+
238253
public String getFileHeader() {
239254
return FILE_HEADER;
240255
}

‎codegen/src/main/java/software/amazon/awssdk/codegen/poet/PoetExtension.java

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,10 @@ public ClassName getServiceConfigClass() {
7373
+ model.getMetadata().getServiceName() + "ServiceClientConfiguration");
7474
}
7575

76+
public ClassName getUserAgentClass() {
77+
return ClassName.get(model.getMetadata().getFullClientInternalPackageName(), "UserAgentUtils");
78+
}
79+
7680
/**
7781
* @param operationName Name of the operation
7882
* @return A Poet {@link ClassName} for the response type of a paginated operation in the base service package.

‎codegen/src/main/java/software/amazon/awssdk/codegen/poet/builder/AsyncClientBuilderClass.java

Lines changed: 48 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@
3232
import software.amazon.awssdk.codegen.utils.AuthUtils;
3333
import software.amazon.awssdk.core.client.config.SdkClientConfiguration;
3434
import software.amazon.awssdk.core.client.config.SdkClientOption;
35+
import software.amazon.awssdk.endpoints.EndpointProvider;
3536

3637
public class AsyncClientBuilderClass implements ClassSpec {
3738
private final IntermediateModel model;
@@ -79,7 +80,10 @@ public TypeSpec poetSpec() {
7980
builder.addMethod(bearerTokenProviderMethod());
8081
}
8182

82-
return builder.addMethod(buildClientMethod()).build();
83+
builder.addMethod(buildClientMethod());
84+
builder.addMethod(initializeServiceClientConfigMethod());
85+
86+
return builder.build();
8387
}
8488

8589
private MethodSpec endpointDiscoveryEnabled() {
@@ -119,26 +123,26 @@ private MethodSpec endpointProviderMethod() {
119123
}
120124

121125
private MethodSpec buildClientMethod() {
122-
return MethodSpec.methodBuilder("buildClient")
123-
.addAnnotation(Override.class)
124-
.addModifiers(Modifier.PROTECTED, Modifier.FINAL)
125-
.returns(clientInterfaceName)
126-
.addStatement("$T clientConfiguration = super.asyncClientConfiguration()", SdkClientConfiguration.class)
127-
.addStatement("this.validateClientOptions(clientConfiguration)")
128-
.addStatement("$T endpointOverride = null", URI.class)
129-
.addCode("if (clientConfiguration.option($T.ENDPOINT_OVERRIDDEN) != null"
130-
+ "&& $T.TRUE.equals(clientConfiguration.option($T.ENDPOINT_OVERRIDDEN))) {"
131-
+ "endpointOverride = clientConfiguration.option($T.ENDPOINT);"
132-
+ "}",
133-
SdkClientOption.class, Boolean.class, SdkClientOption.class, SdkClientOption.class)
134-
.addStatement("$T serviceClientConfiguration = $T.builder()"
135-
+ ".overrideConfiguration(overrideConfiguration())"
136-
+ ".region(clientConfiguration.option($T.AWS_REGION))"
137-
+ ".endpointOverride(endpointOverride)"
138-
+ ".build()",
139-
serviceConfigClassName, serviceConfigClassName, AwsClientOption.class)
140-
.addStatement("return new $T(serviceClientConfiguration, clientConfiguration)", clientClassName)
141-
.build();
126+
MethodSpec.Builder builder = MethodSpec.methodBuilder("buildClient")
127+
.addAnnotation(Override.class)
128+
.addModifiers(Modifier.PROTECTED, Modifier.FINAL)
129+
.returns(clientInterfaceName)
130+
.addStatement("$T clientConfiguration = super.asyncClientConfiguration()",
131+
SdkClientConfiguration.class).addStatement("this.validateClientOptions"
132+
+ "(clientConfiguration)")
133+
.addStatement("$T serviceClientConfiguration = initializeServiceClientConfig"
134+
+ "(clientConfiguration)",
135+
serviceConfigClassName);
136+
137+
builder.addStatement("$1T client = new $2T(serviceClientConfiguration, clientConfiguration)",
138+
clientInterfaceName, clientClassName);
139+
if (model.asyncClientDecoratorClassName().isPresent()) {
140+
builder.addStatement("return new $T().decorate(client, clientConfiguration, clientContextParams.copy().build())",
141+
PoetUtils.classNameFromFqcn(model.asyncClientDecoratorClassName().get()));
142+
} else {
143+
builder.addStatement("return client");
144+
}
145+
return builder.build();
142146
}
143147

144148
private MethodSpec bearerTokenProviderMethod() {
@@ -152,6 +156,29 @@ private MethodSpec bearerTokenProviderMethod() {
152156
.build();
153157
}
154158

159+
private MethodSpec initializeServiceClientConfigMethod() {
160+
return MethodSpec.methodBuilder("initializeServiceClientConfig").addModifiers(Modifier.PRIVATE)
161+
.addParameter(SdkClientConfiguration.class, "clientConfig")
162+
.returns(serviceConfigClassName)
163+
.addStatement("$T endpointOverride = null", URI.class)
164+
.addStatement("$T endpointProvider = clientConfig.option($T.ENDPOINT_PROVIDER)",
165+
EndpointProvider.class,
166+
SdkClientOption.class)
167+
.addCode("if (clientConfig.option($T.ENDPOINT_OVERRIDDEN) != null"
168+
+ "&& $T.TRUE.equals(clientConfig.option($T.ENDPOINT_OVERRIDDEN))) {"
169+
+ "endpointOverride = clientConfig.option($T.ENDPOINT);"
170+
+ "}",
171+
SdkClientOption.class, Boolean.class, SdkClientOption.class, SdkClientOption.class)
172+
.addStatement("return $T.builder()"
173+
+ ".overrideConfiguration(overrideConfiguration())"
174+
+ ".region(clientConfig.option($T.AWS_REGION))"
175+
+ ".endpointOverride(endpointOverride)"
176+
+ ".endpointProvider(endpointProvider)"
177+
+ ".build()",
178+
serviceConfigClassName, AwsClientOption.class)
179+
.build();
180+
}
181+
155182
@Override
156183
public ClassName className() {
157184
return builderClassName;

‎codegen/src/main/java/software/amazon/awssdk/codegen/poet/builder/BaseClientBuilderClass.java

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,12 @@ public TypeSpec poetSpec() {
119119
});
120120
}
121121

122+
if (hasSdkClientContextParams()) {
123+
model.getCustomizationConfig().getCustomClientContextParams().forEach((n, m) -> {
124+
builder.addMethod(clientContextParamSetter(n, m));
125+
});
126+
}
127+
122128
if (model.getCustomizationConfig().getServiceConfig().getClassName() != null) {
123129
builder.addMethod(setServiceConfigurationMethod())
124130
.addMethod(beanStyleSetServiceConfigurationMethod());
@@ -619,6 +625,12 @@ private boolean hasClientContextParams() {
619625
return clientContextParams != null && !clientContextParams.isEmpty();
620626
}
621627

628+
private boolean hasSdkClientContextParams() {
629+
return model.getCustomizationConfig() != null
630+
&& model.getCustomizationConfig().getCustomClientContextParams() != null
631+
&& !model.getCustomizationConfig().getCustomClientContextParams().isEmpty();
632+
}
633+
622634
private MethodSpec validateClientOptionsMethod() {
623635
MethodSpec.Builder builder = MethodSpec.methodBuilder("validateClientOptions")
624636
.addModifiers(PROTECTED, Modifier.STATIC)

‎codegen/src/main/java/software/amazon/awssdk/codegen/poet/builder/BaseClientBuilderInterface.java

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,12 @@ public TypeSpec poetSpec() {
8080
});
8181
}
8282

83+
if (hasSdkClientContextParams()) {
84+
model.getCustomizationConfig().getCustomClientContextParams().forEach((n, m) -> {
85+
builder.addMethod(clientContextParamSetter(n, m));
86+
});
87+
}
88+
8389
if (generateTokenProviderMethod()) {
8490
builder.addMethod(tokenProviderMethod());
8591
}
@@ -193,4 +199,10 @@ public ClassName className() {
193199
private boolean hasClientContextParams() {
194200
return model.getClientContextParams() != null && !model.getClientContextParams().isEmpty();
195201
}
202+
203+
private boolean hasSdkClientContextParams() {
204+
return model.getCustomizationConfig() != null
205+
&& model.getCustomizationConfig().getCustomClientContextParams() != null
206+
&& !model.getCustomizationConfig().getCustomClientContextParams().isEmpty();
207+
}
196208
}

0 commit comments

Comments
 (0)