Skip to content

Pass in environment variables for Estimator training job #2207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
PiercingDan opened this issue Mar 12, 2021 · 7 comments
Closed

Pass in environment variables for Estimator training job #2207

PiercingDan opened this issue Mar 12, 2021 · 7 comments
Labels
component: training Relates to the SageMaker Training Platform type: feature request

Comments

@PiercingDan
Copy link

Describe the feature you'd like
I want to pass in an environment variable for Estimator like for Transformer(env=...) https://sagemaker.readthedocs.io/en/stable/api/inference/transformer.html#sagemaker.transformer.Transformer

*How would this feature be used? Please describe.
I want to pass in information to my sagemaker container during runtime

Describe alternatives you've considered
I'm forced to encode it as 'hyperparameters' to pass in the info

Additional context
N/A

@anirudhgangwal
Copy link

+1 this would be helpful. We would like to pass pip configuration as environment so that 3rd party libraries are downloaded from AWS CodeArtifact instead of PyPi.

@PiercingDan
Copy link
Author

From #2218 , looks like this is available in the native boto3 call CreateTrainingJob under Environment. Waiting for sagemaker-python-sdk support for this interface.

https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTrainingJob.html

@PiercingDan
Copy link
Author

Thanks for adding this to the boto3 call in #2218 , is it possible to also add support for passing in environment variables for local jobs? That would be really helpful

@ssmall41
Copy link
Contributor

As far as I can tell, #2218 should add support to the sagemaker python sdk for environment variables, but that seems to not be the case almost a year later with version 2.72.3. The API supports an argument environment, but the values passed are not defined in the container.

@usamec
Copy link

usamec commented Jan 20, 2022

As far as I can tell, #2218 should add support to the sagemaker python sdk for environment variables, but that seems to not be the case almost a year later with version 2.72.3. The API supports an argument environment, but the values passed are not defined in the container.

I have exactly same problem.
It can be easily confirmed by running training script, which just output os.environ content.

@ssmall41
Copy link
Contributor

I figured out that passing environment variables DOES work for me, except in local mode. I made an issue to address this. #2930

@mufaddal-rohawala
Copy link
Member

Keeping #2930 open to track Env Var Support in Local Mode. Closing this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: training Relates to the SageMaker Training Platform type: feature request
Projects
None yet
Development

No branches or pull requests

6 participants