feat: Deploy uncompressed ML model from S3 to SageMaker Hosting endpoints #4005

chiui0x18 · 2023-07-18T06:25:06Z

Issue #, if available:

n/a

Description of changes:

This change supports deployment of uncompressed ML model from S3 to SageMaker Hosting endpoints. It makes the feature available to customers via existing Estimator, Session and Model constructs and their sub-classes. Along with uncompressed model upload support in SageMaker Training, this feature enables customers to train uncompressed ML model on SageMaker Training and deploy the trained model to SageMaker Hosting with existing SageMaker python SDK use pattern.

Per https://docs.aws.amazon.com/sagemaker/latest/dg/large-model-inference-uncompressed.html

Testing done:

Unit tests
Used the feature to deploy an uncompressed ML model from S3 to SageMaker Hosting:

from sagemaker.mxnet.model import MXNetModel

mxnet_model = MXNetModel(
    role='arn:aws:iam::123456789012:role/TestSageMakerRole',
    model_data={
        'S3DataSource': {
            'CompressionType': 'None',
            'S3DataType': 'S3Prefix',
            'S3Uri': 's3://test-bucket/uncompressed/maeve-integtest-mxnet-model/',
        }
    },
    image_uri='763104351884.dkr.ecr.us-west-2.amazonaws.com/mxnet-inference:1.8.0-cpu-py37-ubuntu16.04',
)

predictor = mxnet_model.deploy(
    initial_instance_count=1,
    instance_type='ml.m5.large',
    endpoint_name='sm-py-sdk-uncompressed-test-mxnet',
)
----!

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…sting endpoints

knikure

/bot run all

sagemaker-bot · 2023-07-21T22:37:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: bc917e8
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-21T22:40:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: bc917e8
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-21T23:14:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: bc917e8
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-21T23:28:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: bc917e8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-21T23:28:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: bc917e8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-25T01:20:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: bc917e8
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-25T16:43:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 902fa80
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-25T17:12:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 902fa80
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-25T17:17:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 902fa80
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-25T17:31:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 902fa80
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

knikure · 2023-07-25T18:09:58Z

@wuyrush Can you fix the black-check errors by running tox -e black-format command?
Submit your changes after successfully running ./.githooks/pre-push command.

sagemaker-bot · 2023-07-25T20:15:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 902fa80
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

chiui0x18 · 2023-07-26T05:29:51Z

Can you fix the black-check errors by running tox -e black-format command? Submit your changes after successfully running ./.githooks/pre-push command.

Thanks for the tip. Updated the PR. I believe I had fixed all per output of ./.githooks/pre-push except some pylint error which is irrelevant to this PR.

sagemaker-bot · 2023-07-26T17:42:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: a2d3cba
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-26T17:48:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: a2d3cba
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-26T17:56:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: a2d3cba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-26T18:25:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: a2d3cba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-26T18:40:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: a2d3cba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

chiui0x18 · 2023-07-27T14:55:41Z

AWS CodeBuild CI Report

* CodeBuild project: sagemaker-python-sdk-notebook-tests

* Commit ID: [a2d3cba](https://github.com/aws/sagemaker-python-sdk/commit/a2d3cbafaa3bca5691dbeda59dbc59ef469e16aa)

* Result: FAILED

* [Build Logs](https://jr6hkjvk5d.execute-api.us-west-2.amazonaws.com/Prod/buildlogs?key=56608103-03a8-4305-b79c-f60ac91b635b%2Fbuild.log) (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

For this check I am seeing err like following:

'docker-compose' is not installed. Local Mode features will not work without docker-compose. For more information on how to install 'docker-compose'

AWS CodeBuild CI Report

* CodeBuild project: sagemaker-python-sdk-pr

* Commit ID: [a2d3cba](https://github.com/aws/sagemaker-python-sdk/commit/a2d3cbafaa3bca5691dbeda59dbc59ef469e16aa)

* Result: FAILED

* [Build Logs](https://sxsf0yqucb.execute-api.us-west-2.amazonaws.com/Prod/buildlogs?key=0a8a04b6-cc6d-4596-bf4c-6ed24067ad3a%2Fbuild.log) (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

For this check I am seeing err like following:

botocore.errorfactory.ResourceLimitExceeded: An error occurred (ResourceLimitExceeded) when calling the CreateContext operation: The account-level service limit 'Total number of contexts allowed, excluding those automatically created by SageMaker' is 600 Contexts, with current utilization of 600 Contexts and a request delta of 1 Contexts. Please use AWS Service Quotas to request an increase for this quota. If AWS Service Quotas is not available, contact AWS support to request an increase for this quota.

Neither seem relevant to the change in this PR. Please let me know if what else is necessary for us to merge this PR in. Thanks!

knikure · 2023-07-28T16:51:57Z

@wuyrush Can some one from your team also review the PR?

feat: Support deploying uncompressed ML model from S3 to SageMaker Ho…

bc917e8

…sting endpoints

chiui0x18 requested a review from a team as a code owner July 18, 2023 06:25

chiui0x18 requested review from EvelynYan21 and removed request for a team July 18, 2023 06:25

knikure reviewed Jul 21, 2023

View reviewed changes

chiui0x18 added 2 commits July 25, 2023 00:55

fix: pylint complaint

2508683

Merge branch 'aws:master' into master

902fa80

akrishna1995 self-assigned this Jul 26, 2023

akrishna1995 requested review from akrishna1995 and removed request for EvelynYan21 July 26, 2023 00:25

akrishna1995 assigned knikure Jul 26, 2023

fix: black-format complaint

a2d3cba

knikure approved these changes Jul 28, 2023

View reviewed changes

knikure merged commit 035a8ac into aws:master Jul 28, 2023

martinRenou mentioned this pull request Nov 27, 2023

S3DataSource not working with HuggingFaceModel #4248

Closed

feat: Deploy uncompressed ML model from S3 to SageMaker Hosting endpoints #4005

feat: Deploy uncompressed ML model from S3 to SageMaker Hosting endpoints #4005

Uh oh!

Conversation

chiui0x18 commented Jul 18, 2023

Merge Checklist

General

Tests

Uh oh!

knikure left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Jul 21, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 21, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 21, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 21, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 21, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 25, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 25, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 25, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 25, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 25, 2023

AWS CodeBuild CI Report

Uh oh!

knikure commented Jul 25, 2023

Uh oh!

sagemaker-bot commented Jul 25, 2023

AWS CodeBuild CI Report

Uh oh!

chiui0x18 commented Jul 26, 2023

Uh oh!

sagemaker-bot commented Jul 26, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 26, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 26, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 26, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 26, 2023

AWS CodeBuild CI Report

Uh oh!

chiui0x18 commented Jul 27, 2023

AWS CodeBuild CI Report

AWS CodeBuild CI Report

Uh oh!

knikure commented Jul 28, 2023

Uh oh!

Uh oh!

knikure left a comment •

edited

Loading