Equivalent of com.amazonaws.util.json.Jackson in sdk v2

[NathanEckert]

Describe the issue

I am trying to migrate some code from the SDK v1 to the SDK v2 and one of the last hurdle is the following piece of code:

import com.amazonaws.util.json.Jackson;
...
  private static Optional<AwsConfig> deserializeConfig(final Map<String, Object> config) {
    if (config == null) {
      return Optional.empty();
    }
    for (final Class<?> clazz : new Class<?>[] {AwsKeyPairConfig.class, AwsKmsConfig.class}) {
      try {
         return Optional.of((AwsConfig) Jackson.getObjectMapper().convertValue(config, clazz));
      } catch (final IllegalArgumentException exception) {
        LOGGER.log(Level.INFO, "Failed to deserialize AWS client side encryption config.", exception);
      }
    }
    throw new Exception("Failed to deserialize AWS client side encryption config.")
  }
}

The classes involved are:

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import software.amazon.awssdk.regions.Region;

public class AwsKmsConfig extends AwsConfig {

  @JsonProperty("key_id")
  public final String keyId;

  @JsonCreator
  public AwsKmsConfig(
      @JsonProperty(value = "region", required = true) final Region region,
      @JsonProperty(value = "key_id", required = true) final String keyId) {
    super(region);
    this.keyId = keyId;
  }
}

import software.amazon.awssdk.regions.Region;

public class AwsKeyPairConfig extends AwsConfig {

  public final String publicKey;

  public final String privateKey;

  public final PublicKey deserializedPublicKey;

  public final PrivateKey deserializedPrivateKey;

  /**
   * Constructor.
   *
   * @param region the AWS region in which the S3 objects are stored
   * @param publicKey public key to read data in the bucket
   * @param privateKey private key to read data in the bucket
   */
  @JsonCreator
  public AwsKeyPairConfig(
      @JsonProperty(value = "region", required = true) final Region region,
      @JsonProperty(value = "public_key", required = true) final String publicKey,
      @JsonProperty(value = "private_key", required = true) final String privateKey) {
    super(region);
    this.privateKey = privateKey;
    this.publicKey = publicKey;
    final Pair<PublicKey, PrivateKey> deserializedKeys =
        KeyPairConfig.getDeserializedKeys(publicKey, privateKey);
    this.deserializedPublicKey = deserializedKeys.getLeft();
    this.deserializedPrivateKey = deserializedKeys.getRight();
  }
}

and

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import software.amazon.awssdk.regions.Region;

public class AwsConfig {

  @JsonProperty("region")
  public final Region region;

  @JsonCreator
  public AwsConfig(@JsonProperty(value = "region", required = true) final Region region) {
    this.region = region;
  }
}

What is the equivalent to use in the SDK v2, I did not find anything about it in the documentation, except this opened discussion: #3904 and this issue #2254

Thanks in advance

Links

https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/migration-serialization-changes.html

[imvtsl]

Hi,

I did some research and found that the AWS SDK for Java 2.7 removed its external dependency on Jackson. You can read more about this in this blog post.

The change is also discussed in these pull requests: #2598 and #2522.

I analyzed your code and it looks like a quick fix. Currently, we are using Jackson.convertValue() to convert from one object to another in the deserializeConfig() method:

return Optional.of((AwsConfig) Jackson.getObjectMapper().convertValue(config, clazz));

We can modify this line to use other third-party converters like ModelMapper, etc. to achieve the same result.

You haven’t shared the AwsKeyPairConfig class definition, so I couldn't look into it further.

[imvtsl]

In case you don't want to use third-party libraries discussed in previous comment to convert one object into another, you can use ObjectMapper class. Documentation here.

ObjectMapper mapper = new ObjectMapper();
objectMapper.convertValue(config, YourClassNameHere.class);

[NathanEckert]

I added in the description the definition of the AwsKeyPairConfig.

The solution with the ObjectMapper does not work, I already tried it and got:

java.lang.IllegalArgumentException: Cannot construct instance of `software.amazon.awssdk.regions.Region` (no Creators, like default constructor, exist): no String-argument constructor/factory method to deserialize from String value ('eu-west-3')
 at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: io.atoti.loading.s3.private_.config.AwsKeyPairConfig["region"])
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4544) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:4475) ~[atoti-aws.jar:2.15.4]
	at io.atoti.loading.s3.api.AwsPlugin.deserializeConfig(AwsPlugin.java:155) ~[atoti-aws.jar:na]
	at io.atoti.loading.s3.api.AwsPlugin.lambda$parsePath$0(AwsPlugin.java:75) ~[atoti-aws.jar:na]
	at io.atoti.loading.s3.private_.impl.S3Path.parsePath(S3Path.java:263) ~[atoti-aws.jar:na]
	at io.atoti.loading.s3.api.AwsPlugin.parsePath(AwsPlugin.java:74) ~[atoti-aws.jar:na]
	at io.atoti.runtime.private_.util.files.FilesUtil.parsePath(FilesUtil.java:77) ~[patachou-core-6.1-CI-20240528-70325d1c51.jar!/:na]
	at io.atoti.runtime.private_.loading.csv.impl.CsvDataTableFactory.createTable(CsvDataTableFactory.java:28) ~[patachou-core-6.1-CI-20240528-70325d1c51.jar!/:na]
	at io.atoti.runtime.private_.loading.csv.impl.CsvDataTableFactory.createTable(CsvDataTableFactory.java:10) ~[patachou-core-6.1-CI-20240528-70325d1c51.jar!/:na]
	at io.atoti.runtime.internal.impl.OutsideTransactionDataApiImpl.discoverCsvFileFormat(OutsideTransactionDataApiImpl.java:114) ~[patachou-core-6.1-CI-20240528-70325d1c51.jar!/:na]
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) ~[py4j-0.10.9.jar!/:na]
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) ~[py4j-0.10.9.jar!/:na]
	at py4j.Gateway.invoke(Gateway.java:282) ~[py4j-0.10.9.jar!/:na]
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) ~[py4j-0.10.9.jar!/:na]
	at py4j.commands.CallCommand.execute(CallCommand.java:79) ~[py4j-0.10.9.jar!/:na]
	at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) ~[py4j-0.10.9.jar!/:na]
	at py4j.ClientServerConnection.run(ClientServerConnection.java:106) ~[py4j-0.10.9.jar!/:na]
	at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `software.amazon.awssdk.regions.Region` (no Creators, like default constructor, exist): no String-argument constructor/factory method to deserialize from String value ('eu-west-3')
 at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: io.atoti.loading.s3.private_.config.AwsKeyPairConfig["region"])
	at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1915) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:414) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1360) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.deser.std.StdDeserializer._deserializeFromString(StdDeserializer.java:311) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromString(BeanDeserializerBase.java:1514) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:197) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1419) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[atoti-aws.jar:2.15.4]
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4539) ~[atoti-aws.jar:2.15.4]
	... 19 common frames omitted

[imvtsl]

Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of software.amazon.awssdk.regions.Region (no Creators, like default constructor, exist): no String-argument constructor/factory method to deserialize from String value ('eu-west-3')

Region is an immutable class and it doesn't have a default (no argument) constructor. So, object mapper cannot create instance of it and deserialize.

I am not sure if not having a no argument constructor is a design choice or a bug. @debora-ito can comment on this.

Having said that, there are a few other options that you can try.

1. "software.amazon.awssdk.services.ec2.model.Region" class can be serialized/deserialized. You can check if you can use this class for your use case. It implements serializable and has serializableBuilderClass() method. You can look at this for reference. The comment is for different class but the idea is the same.
2. Try the three different approaches listed in ErikE's comment on "software.amazon.awssdk.regions.Region" class.

Hope it helps!

[NathanEckert]

I implemented the deserialization manually, though I am curious to hear if not having a no argument constructor is a design choice or a bug