DefaultCredentialsProvider#create is not creating new instance when called as documentation says

[akonarska]

Describe the bug

DefaultCredentialsProvider#create does not create a new instance as documentation suggests but instead returns the same instance every time.
At the same time DefaultCredentialsProvider implements SdkAutoCloseable and has method close() - if that one is called the static instance returned will be broken for ever. We encountered this issue after upgrading SDK to 2.17.283 when WebIdentityTokenFileCredentialsProvider started to implement SdkAutoCloseable and our code stopped working.

Expected Behavior

DefaultCredentialsProvider#create returns new instance every time

Current Behavior

DefaultCredentialsProvider#create returns the same instance every time

Reproduction Steps

Running this code on eks pod should do the trick

try(DefaultCredentialsProvider defaultCredentialsProvider = DefaultCredentialsProvider.create()) {
     AwsCredentials awsCredentials = defaultCredentialsProvider.resolveCredentials();
     System.out.println(awsCredentials);
}

try(DefaultCredentialsProvider defaultCredentialsProvider = DefaultCredentialsProvider.create()) {
     AwsCredentials awsCredentials = defaultCredentialsProvider.resolveCredentials();
     System.out.println(awsCredentials);
}

Possible Solution

Create method should do what it says:

public static DefaultCredentialsProvider create() {
    return new DefaultCredentialsProvider(builder());
}

Additional Information/Context

No response

AWS Java SDK version used

2.17.283

JDK version used

amazon coretto 17.0.4.9.1

Operating System and version

alpine 3.16

[yasminetalby]

Hello @akonarska ,

Thank you very much for your submission and for bringing this up to our attention. I apologies for the delay of answer.
The close() method from SdkAutoCloseable has been added recently for WebIdentityCredentialProvider as part of the v2.17.283(see).

It seems that the current behavior is as follow:

public static DefaultCredentialsProvider create() {
        return DEFAULT_CREDENTIALS_PROVIDER;
    }

where

private static final DefaultCredentialsProvider DEFAULT_CREDENTIALS_PROVIDER = new DefaultCredentialsProvider(builder());

DefaultCredentialsProvider calls WebIdentityTokenFileCredentialsProvider.create() which has the following behavior:

public static WebIdentityTokenFileCredentialsProvider create() {

        return WebIdentityTokenFileCredentialsProvider.builder().build();
    }

Are you able to work around this by using : either DefaultCredentialsProvider.builder.build(); or WebIdentityTokenFileCredentialProvider.create() directly?

Sincerely,

Yasmine

[akonarska]

Hello @yasminetalby,

we switched to DefaultCredentialsProvider.builder.build(); when we managed to figure out where the bug is, so we are not impacted by this anymore. I took us a lot of time to figure out what was happening though and for that reason I filed this bug report so it can be fixed.

Let me know if I can be of any more help.

Best
Aleksandra

[yasminetalby]

Hello @akonarska ,

Thank you very much for bringing this up to our attention and for providing feedback.
I apologize for any inconvenience experienced with the upgrade to 2.17.283. and the difference between documented definition of the .create() method and its current behavior.

I will mark this issue as a bug and bring it up to the team.

Sincerely,

Yasmine

[0x26res]

Just like @akonarska we stumbled across similar issues, and it was very hard to point out what the issue was.

Basically our services were running fine for a while, and then suddenly lost access to s3 or RDS (getting permission errors, 403, etc).

After a while we figured out the root cause, thanks to this post #4386 and we managed to fix some of the problem by replacing DefaultCredentialsProvider.create() with DefaultCredentialsProvider.builder.build()

However, when using s3 through the hadoop library the only way to specify the credential provider is to pass its class name to the config:

import org.apache.hadoop.conf.Configuration;

Configuration conf = new Configuration();

conf.set(
  "fs.s3a.aws.credentials.provider", 
  "software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider"
);

The credential provider is created through reflection by the hadoop s3 library.

In this process, because DefaultCredentialsProvider's constructor is private, the hadoop library calls DefaultCredentialsProvider.create. So there's no way to control how it is created, or use builder.build().

I've found a hack that should fix it:

public class HackCredentialProvider implements AwsCredentialsProvider {

    @Override
    public AwsCredentials resolveCredentials() {
        throw new RuntimeException("This should not be called");
    }

    public static DefaultCredentialsProvider create() {
        return DefaultCredentialsProvider.builder().build();
    }
}

// ...


conf.set(
  "fs.s3a.aws.credentials.provider", 
  HackCredentialProvider.class.getCanonicalName()
);

[bhoradc]

Hi @akonarska,

Documentation for DefaultCredentialsProvider.create was updated via PR #5009, hence closing the issue.

Regards,
Chaitanya