fix: fixing propagation of tags to SageMaker endpoint

[Othmane796]

Issue #586

Description of changes:
Propagating the tags to the endpoint when calling 'model.deploy()'. This change also updates the calls to the SageMaker session to support tags when calling 'session.create_endpoint()'

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

• I have read the CONTRIBUTING doc
• I used the commit message format described in CONTRIBUTING
• I have added tests that prove my fix is effective or that my feature works (if appropriate)
• I have updated any necessary documentation (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[jesterhazy]

AWS CodeBuild CI Report

• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[icywang86rui]

Please add an integ test as well. We have a couple update_endpoint tests in test_mxnet_train.py. You can add tagging tests there.

[jesterhazy]

AWS CodeBuild CI Report

• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[jesterhazy]

AWS CodeBuild CI Report

• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[Othmane796]

Any updates on the PR ? Should I dive deeper in the CodeBuild logs to figure why it's failing ? thanks !

[icywang86rui]

create_endpoint boto call doesn't like Tag being None

______________________________ test_attach_deploy ______________________________ 
[gw3] linux2 -- Python 2.7.15 /codebuild/output/src157151743/src/github.com/aws/sagemaker-python-sdk/.tox/py27/bin/python 
mxnet_training_job = 'sagemaker-mxnet-2019-04-10-08-19-47-241' 
sagemaker_session = <sagemaker.session.Session object at 0x7f051efb1a50> 
 
    @pytest.mark.canary_quick 
    @pytest.mark.regional_testing 
    def test_attach_deploy(mxnet_training_job, sagemaker_session): 
        endpoint_name = 'test-mxnet-attach-deploy-{}'.format(sagemaker_timestamp()) 
     
        with timeout_and_delete_endpoint_by_name(endpoint_name, sagemaker_session): 
            estimator = MXNet.attach(mxnet_training_job, sagemaker_session=sagemaker_session) 
>           predictor = estimator.deploy(1, 'ml.m4.xlarge', endpoint_name=endpoint_name) 
 
tests/integ/test_mxnet_train.py:55:  
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _  
.tox/py27/local/lib/python2.7/site-packages/sagemaker/estimator.py:381: in deploy 
    tags=self.tags) 
.tox/py27/local/lib/python2.7/site-packages/sagemaker/model.py:278: in deploy 
    self.sagemaker_session.endpoint_from_production_variants(self.endpoint_name, [production_variant], tags) 
.tox/py27/local/lib/python2.7/site-packages/sagemaker/session.py:1055: in endpoint_from_production_variants 
    return self.create_endpoint(endpoint_name=name, config_name=name, tags=tags, wait=wait) 
.tox/py27/local/lib/python2.7/site-packages/sagemaker/session.py:767: in create_endpoint 
    self.sagemaker_client.create_endpoint(EndpointName=endpoint_name, EndpointConfigName=config_name, Tags=tags) 
.tox/py27/local/lib/python2.7/site-packages/botocore/client.py:357: in _api_call 
    return self._make_api_call(operation_name, kwargs) 
.tox/py27/local/lib/python2.7/site-packages/botocore/client.py:634: in _make_api_call 
    api_params, operation_model, context=request_context) 
.tox/py27/local/lib/python2.7/site-packages/botocore/client.py:682: in _convert_to_request_dict 
    api_params, operation_model) 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _  
 
self = <botocore.validate.ParamValidationDecorator object at 0x7f054e5dfc50> 
parameters = {'EndpointConfigName': 'test-mxnet-attach-deploy-2019-04-10-08-22-29-050', 'EndpointName': 'test-mxnet-attach-deploy-2019-04-10-08-22-29-050', 'Tags': None} 
operation_model = OperationModel(name=CreateEndpoint) 
 
    def serialize_to_request(self, parameters, operation_model): 
        input_shape = operation_model.input_shape 
        if input_shape is not None: 
            report = self._param_validator.validate(parameters, 
                                                    operation_model.input_shape) 
            if report.has_errors(): 
>               raise ParamValidationError(report=report.generate_report()) 
E               ParamValidationError: Parameter validation failed: 
E               Invalid type for parameter Tags, value: None, type: <type 'NoneType'>, valid types: <type 'list'>, <type 'tuple'> 
 
.tox/py27/local/lib/python2.7/site-packages/botocore/validate.py:297: ParamValidationError 

[jesterhazy]

AWS CodeBuild CI Report

• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[jesterhazy]

AWS CodeBuild CI Report

• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[laurenyu]

I'm seeing failures in the build log like:

>       self.sagemaker_client.create_endpoint(EndpointName=endpoint_name, EndpointConfigName=config_name, Tags=tags)
E       TypeError: create_endpoint() got an unexpected keyword argument 'Tags'

This is pretty strange, as Tags is definitely part of the API.

[jesterhazy]

AWS CodeBuild CI Report

• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[Othmane796]

Indeed, and even when u look at the logs it's installing 'boto3==1.9.130' which has support for Tags. Even the minimum version of boto3 in the setup.py contains support for Tags.

Note : Both the unit tests and integration test for 'test_mxnet_train.py' passed on my local environment.

[laurenyu]

ah, the tests that are failing are for Local Mode, which reimplements parts of the API. Could you add a Tags kwarg to https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/local/local_session.py#L141?

[jesterhazy]

AWS CodeBuild CI Report

• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[jesterhazy]

AWS CodeBuild CI Report

• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[laurenyu]

thanks for your contribution! sorry about the churn with the integ tests - thanks for your patience there :)

[Othmane796]

My pleasure :) Thank you for helping out !