Skip to content

PYTHON-2818 Add native support for AWS IAM Roles for service accounts, EKS in particular #1032

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Nov 14, 2022
Merged

PYTHON-2818 Add native support for AWS IAM Roles for service accounts, EKS in particular #1032

Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
94e4c47
PYTHON-2818 Add native support for AWS IAM Roles for service accounts…
blink1073 Aug 4, 2022
376785f
limit tasks
blink1073 Aug 4, 2022
2563b8d
fix syntax
blink1073 Aug 4, 2022
94e611b
another try at config
blink1073 Aug 4, 2022
03f0bb0
remove trailing comma
blink1073 Aug 4, 2022
90d8af4
fix creds usage
blink1073 Aug 4, 2022
4a67410
ensure proper aws lib
blink1073 Aug 4, 2022
077b942
fix install
blink1073 Aug 4, 2022
6ecd623
debug
blink1073 Aug 4, 2022
b07605c
try an export
blink1073 Aug 4, 2022
885c030
make rsa key a variable
blink1073 Aug 4, 2022
ae2d67e
clean up config
blink1073 Aug 4, 2022
ea4e36f
restore skipped task
blink1073 Aug 8, 2022
f6a5d28
add test with AWS_ROLE_SESSION_NAME
blink1073 Aug 29, 2022
cd67cf6
merge from master
blink1073 Aug 29, 2022
a365989
try with boto3
blink1073 Aug 30, 2022
9c58ef7
try with released pymongo_auth_aws
blink1073 Aug 30, 2022
aa394bf
more assurances
blink1073 Aug 30, 2022
5c0dfd0
install boto3
blink1073 Aug 30, 2022
7e5ce35
fix check
blink1073 Aug 30, 2022
889611b
more refactor
blink1073 Aug 30, 2022
4e15362
run against boto3
blink1073 Aug 31, 2022
115e702
try again
blink1073 Sep 12, 2022
ee9b82b
skip non-ubunutu variants
blink1073 Sep 12, 2022
8778563
more explicit skip
blink1073 Sep 12, 2022
7ada611
more cleanup
blink1073 Sep 12, 2022
86cb9c2
remove duplicate entry
blink1073 Oct 13, 2022
4c4b801
address review
blink1073 Oct 27, 2022
423366b
remove branch pin
blink1073 Nov 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 78 additions & 4 deletions .evergreen/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ functions:
# If this was a patch build, doing a fresh clone would not actually test the patch
cp -R ${PROJECT_DIRECTORY}/ $DRIVERS_TOOLS
else
git clone https://github.com/mongodb-labs/drivers-evergreen-tools.git $DRIVERS_TOOLS
git clone -b DRIVERS-1746 https://github.com/blink1073/drivers-evergreen-tools.git $DRIVERS_TOOLS
fi
echo "{ \"releases\": { \"default\": \"$MONGODB_BINARIES\" }}" > $MONGO_ORCHESTRATION_HOME/orchestration.config

Expand Down Expand Up @@ -572,7 +572,13 @@ functions:

"iam_auth_ec2_instance_account" : "${iam_auth_ec2_instance_account}",
"iam_auth_ec2_instance_secret_access_key" : "${iam_auth_ec2_instance_secret_access_key}",
"iam_auth_ec2_instance_profile" : "${iam_auth_ec2_instance_profile}"
"iam_auth_ec2_instance_profile" : "${iam_auth_ec2_instance_profile}",

"iam_auth_assume_web_role_name": "${iam_auth_assume_web_role_name}",
"iam_web_identity_issuer": "${iam_web_identity_issuer}",
"iam_web_identity_rsa_key": "${iam_web_identity_rsa_key}",
"iam_web_identity_jwks_uri": "${iam_web_identity_jwks_uri}",
"iam_web_identity_token_file": "${iam_web_identity_token_file}"
}
EOF

Expand Down Expand Up @@ -668,7 +674,67 @@ functions:
fi
# Write an empty prepare_mongodb_aws so no auth environment variables
# are set.
echo "" > "${PROJECT_DIRECTORY}/prepare_mongodb_aws.sh"
rm "${PROJECT_DIRECTORY}/prepare_mongodb_aws.sh" || true
PYTHON_BINARY=${PYTHON_BINARY} ASSERT_NO_URI_CREDS=true .evergreen/run-mongodb-aws-test.sh

"run aws auth test with aws web identity credentials":
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
if [ "${skip_EC2_auth_test}" = "true" ]; then
echo "This platform does not support the web identity auth test, skipping..."
exit 0
fi
cd ${DRIVERS_TOOLS}/.evergreen/auth_aws
. ./activate_venv.sh
mongo aws_e2e_web_identity.js
- command: shell.exec
type: test
params:
working_dir: "src"
silent: true
script: |
# DO NOT ECHO WITH XTRACE (which PREPARE_SHELL does)
cat <<'EOF' > "${PROJECT_DIRECTORY}/prepare_mongodb_aws.sh"
export AWS_ROLE_ARN="${iam_auth_assume_web_role_name}"
export AWS_WEB_IDENTITY_TOKEN_FILE="${iam_web_identity_token_file}"
EOF
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
if [ "${skip_web_identity_auth_test}" = "true" ]; then
echo "This platform does not support the web identity auth test, skipping..."
exit 0
fi
PYTHON_BINARY=${PYTHON_BINARY} ASSERT_NO_URI_CREDS=true .evergreen/run-mongodb-aws-test.sh
- command: shell.exec
type: test
params:
working_dir: "src"
silent: true
script: |
# DO NOT ECHO WITH XTRACE (which PREPARE_SHELL does)
cat <<'EOF' > "${PROJECT_DIRECTORY}/prepare_mongodb_aws.sh"
export AWS_ROLE_ARN="${iam_auth_assume_web_role_name}"
export AWS_WEB_IDENTITY_TOKEN_FILE="${iam_web_identity_token_file}"
export AWS_ROLE_SESSION_NAME="test"
EOF
- command: shell.exec
type: test
params:
working_dir: "src"
script: |
${PREPARE_SHELL}
if [ "${skip_web_identity_auth_test}" = "true" ]; then
echo "This platform does not support the web identity auth test, skipping..."
exit 0
fi
PYTHON_BINARY=${PYTHON_BINARY} ASSERT_NO_URI_CREDS=true .evergreen/run-mongodb-aws-test.sh

"run aws auth test with aws credentials as environment variables":
Expand Down Expand Up @@ -1751,6 +1817,7 @@ tasks:
- func: "run aws auth test with aws credentials as environment variables"
- func: "run aws auth test with aws credentials and session token as environment variables"
- func: "run aws auth test with aws EC2 credentials"
- func: "run aws auth test with aws web identity credentials"
- func: "run aws ECS auth test"

- name: "aws-auth-test-5.0"
Expand All @@ -1767,6 +1834,7 @@ tasks:
- func: "run aws auth test with aws credentials as environment variables"
- func: "run aws auth test with aws credentials and session token as environment variables"
- func: "run aws auth test with aws EC2 credentials"
- func: "run aws auth test with aws web identity credentials"
- func: "run aws ECS auth test"

- name: "aws-auth-test-6.0"
Expand All @@ -1783,6 +1851,7 @@ tasks:
- func: "run aws auth test with aws credentials as environment variables"
- func: "run aws auth test with aws credentials and session token as environment variables"
- func: "run aws auth test with aws EC2 credentials"
- func: "run aws auth test with aws web identity credentials"
- func: "run aws ECS auth test"

- name: "aws-auth-test-latest"
Expand All @@ -1799,6 +1868,7 @@ tasks:
- func: "run aws auth test with aws credentials as environment variables"
- func: "run aws auth test with aws credentials and session token as environment variables"
- func: "run aws auth test with aws EC2 credentials"
- func: "run aws auth test with aws web identity credentials"
- func: "run aws ECS auth test"
- name: "aws-auth-test-rapid"
commands:
Expand All @@ -1814,6 +1884,7 @@ tasks:
- func: "run aws auth test with aws credentials as environment variables"
- func: "run aws auth test with aws credentials and session token as environment variables"
- func: "run aws auth test with aws EC2 credentials"
- func: "run aws auth test with aws web identity credentials"
- func: "run aws ECS auth test"

- name: load-balancer-test
Expand Down Expand Up @@ -1887,6 +1958,7 @@ axes:
variables:
skip_EC2_auth_test: true
skip_ECS_auth_test: true
skip_web_identity_auth_test: true
python3_binary: /Library/Frameworks/Python.framework/Versions/3.8/bin/python3
libmongocrypt_url: https://s3.amazonaws.com/mciuploads/libmongocrypt/macos/master/latest/libmongocrypt.tar.gz
- id: macos-1100
Expand All @@ -1895,6 +1967,7 @@ axes:
variables:
skip_EC2_auth_test: true
skip_ECS_auth_test: true
skip_web_identity_auth_test: true
python3_binary: /Library/Frameworks/Python.framework/Versions/3.8/bin/python3
libmongocrypt_url: https://s3.amazonaws.com/mciuploads/libmongocrypt/macos/master/latest/libmongocrypt.tar.gz
- id: rhel62
Expand Down Expand Up @@ -1957,8 +2030,9 @@ axes:
run_on: windows-64-vsMulti-small
batchtime: 10080 # 7 days
variables:
skip_EC2_auth_test: true
skip_ECS_auth_test: true
skip_EC2_auth_test: true
skip_web_identity_auth_test: true
python3_binary: "C:/python/Python38/python.exe"
venv_bin_dir: "Scripts"
libmongocrypt_url: https://s3.amazonaws.com/mciuploads/libmongocrypt/windows-test/master/latest/libmongocrypt.tar.gz
Expand Down
1 change: 1 addition & 0 deletions pymongo/auth_aws.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ def _authenticate_aws(credentials, sock_info):
credentials.mechanism_properties.aws_session_token,
)
)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this whitespace change intentional?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, removed

client_payload = ctx.step(None)
client_first = SON(
[("saslStart", 1), ("mechanism", "MONGODB-AWS"), ("payload", client_payload)]
Expand Down