Skip to content

Mask creds in docker compose file for local mode #2118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jmahlik opened this issue Feb 2, 2021 · 3 comments
Closed

Mask creds in docker compose file for local mode #2118

jmahlik opened this issue Feb 2, 2021 · 3 comments
Labels
contributions welcome status: pending release The fix have been merged but not yet released to PyPI type: feature request

Comments

@jmahlik
Copy link
Contributor

jmahlik commented Feb 2, 2021

Describe the feature you'd like
Currently, line 673 in sagemaker/local/image.py logs AWS creds in the docker compose file. This could cause a credentials leak if it is running in a ci pipeline, during presentations/demos and for live streamers or youtubers creating sagemaker content.

I don't feel this is a outright security issue, just unexpected logging that could have adverse impact in certain situations.

One has to explicitly provide fake creds and local data to get around this. This limits the ability to use data from s3 and use other aws services in jobs.

How would this feature be used? Please describe.
Credentials should be masked in the logs to prevent a possible leak. This could be accomplished with a regex replace of the yaml content. I'm happy to submit a PR for this, but wanted to get thoughts because there may be better alternatives to regex. I'm not a fan of regex but it was the best I could come up with since the container needs the creds.

Describe alternatives you've considered
Currently patching out this logging in a ci pipeline.

@ajaykarpur
Copy link
Contributor

Credentials should be masked in the logs to prevent a possible leak. This could be accomplished with a regex replace of the yaml content. I'm happy to submit a PR for this, but wanted to get thoughts because there may be better alternatives to regex. I'm not a fan of regex but it was the best I could come up with since the container needs the creds.

Thanks for the suggestion! Masking the credentials definitely makes sense. It may be possible to use PyYAML for this purpose instead of a regex-- I'd recommend looking into this. Please feel free to submit a PR and we'll be happy to review.

@jmahlik
Copy link
Contributor Author

jmahlik commented Feb 2, 2021

Credentials should be masked in the logs to prevent a possible leak. This could be accomplished with a regex replace of the yaml content. I'm happy to submit a PR for this, but wanted to get thoughts because there may be better alternatives to regex. I'm not a fan of regex but it was the best I could come up with since the container needs the creds.

Thanks for the suggestion! Masking the credentials definitely makes sense. It may be possible to use PyYAML for this purpose instead of a regex-- I'd recommend looking into this. Please feel free to submit a PR and we'll be happy to review.

Excellent suggestion, that is way better. I'll take a stab at it over the weekend.

@jmahlik
Copy link
Contributor Author

jmahlik commented Feb 5, 2021

Think I have something working. The environment turned out to be a list of strings so the simple solution was to set each item in the "environment" list to "[Masked]" for all services (on a separate deepcopy for logging).

One question: where is logging configured for local mode/test suite? I can't seem to track it down. I'm only getting WARNING level logging when running the test suite. When I run jobs in local mode, it returns INFO level logging. Makes using caplog to check the INFO logs a bit hard in the tests.

Any thoughts appreciated. Will submit PR once I can get assert on the info logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contributions welcome status: pending release The fix have been merged but not yet released to PyPI type: feature request
Projects
None yet
3 participants