fix: mask creds from docker commands in local mode. Closes #2118 #2146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

ChoiByungWook merged 12 commits into aws:master from jmahlik:mask-creds-local-mode

Mar 2, 2021

Contributor

jmahlik commented Feb 15, 2021

Issue #, if available:
#2118

Description of changes:
Mask credentials in logging of docker compose commands. Change env vars to "[Masked]".
Pipe stderr of docker login to subprocess.DEVNULL so creds are not printed to error logs.

Testing done:
Add caplog to train and serve to ensure they contain "[Masked]".

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

[ x] I have read the CONTRIBUTING doc
[x ] I used the commit message format described in CONTRIBUTING
[ x] I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
[x ] I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

[x ] I have added tests that prove my fix is effective or that my feature works (if appropriate)
[ x] I have checked that my tests are not configured for a specific region or account (if appropriate)
[x ] I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Collaborator

sagemaker-bot commented Feb 15, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 57677e5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Contributor Author

jmahlik commented Feb 15, 2021

Let me know if anything needs changing or there are better ideas. I was on a windows machine so had limited ability to pass the full test suite. I ran the tests for tests/unit/test_image.py, the altered tests passed.

Side note, another scenario where this came up: commiting AWS credentials to git if notebook output is not cleared when using local mode. This is probably the most impactful situation this PR will fix.

Contributor Author

jmahlik commented Feb 15, 2021

Looks like there's quite a few unrelated pylint errors that already existed. Not sure how to proceed.


          fix: mask creds from docker commands in local mode. Closes aws#2118

efc6298

jmahlik force-pushed the mask-creds-local-mode branch from 57677e5 to efc6298 Compare

February 15, 2021 22:59

Collaborator

sagemaker-bot commented Feb 15, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: efc6298
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 15, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 57677e5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 15, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 57677e5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 15, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: efc6298
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 15, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: efc6298
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 15, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: efc6298
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


          Merge branch 'master' into mask-creds-local-mode

cd69d31

Collaborator

sagemaker-bot commented Feb 16, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: cd69d31
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 16, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: cd69d31
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 16, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: cd69d31
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 16, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: cd69d31
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


          Merge branch 'master' into mask-creds-local-mode

0e24c9e

Collaborator

sagemaker-bot commented Feb 16, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 0e24c9e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 16, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 0e24c9e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 16, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 0e24c9e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 16, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 0e24c9e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 16, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 0e24c9e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


          Merge branch 'master' into mask-creds-local-mode

a7df911

Collaborator

sagemaker-bot commented Feb 20, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: a7df911
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 20, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: a7df911
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 20, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: a7df911
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 20, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: a7df911
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 20, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: a7df911
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


          Merge branch 'master' into mask-creds-local-mode

91563fb

Collaborator

sagemaker-bot commented Feb 23, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 91563fb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 26, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 427ece5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 26, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 427ece5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


          Merge branch 'master' into mask-creds-local-mode

08a31b4

Collaborator

sagemaker-bot commented Feb 27, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 08a31b4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

ChoiByungWook reviewed

View reviewed changes

src/sagemaker/local/image.py Outdated Show resolved Hide resolved

Collaborator

sagemaker-bot commented Feb 27, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 08a31b4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 27, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 08a31b4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 27, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 08a31b4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Feb 27, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 08a31b4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Justin added 3 commits

March 1, 2021 19:41


          change: remove pipe to devnull for docker login on local mode

baeed34


          change: remove pipe to devnull for docker login on local mode

2e97d8f


          Merge branch 'mask-creds-local-mode' of https://github.com/jmahlik/sa…

21b37d2

…gemaker-python-sdk into mask-creds-local-mode

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: baeed34
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jmahlik mentioned this pull request

Mask credentials from stderr for failed Docker logins for local mode #2180

Closed

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 21b37d2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 21b37d2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: baeed34
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: baeed34
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: baeed34
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 21b37d2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 21b37d2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


          Merge branch 'master' into mask-creds-local-mode

67fde99

ChoiByungWook approved these changes

View reviewed changes

Contributor

ChoiByungWook left a comment

Awesome!!!

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 67fde99
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 67fde99
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 67fde99
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 67fde99
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Collaborator

sagemaker-bot commented Mar 2, 2021

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 67fde99
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

ChoiByungWook merged commit c4d71f5 into aws:master

jmahlik deleted the mask-creds-local-mode branch

March 2, 2021 04:48

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet