ModelBuilder to fetch local schema when no SchemaBuilder present. #4434

makungaj1 · 2024-02-17T00:30:48Z

Issue #, if available:

Description of changes:
ModelBuilder to fetch local schema when no SchemaBuilder present

Testing done:

Unit tests
Integ tests
Notebook test

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

[ x] I have read the CONTRIBUTING doc
[ x] I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
[ x] I used the commit message format described in CONTRIBUTING
[x ] I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
[ x] I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

[ x] I have added tests that prove my fix is effective or that my feature works (if appropriate)
[ x] I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
[x ] I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

ModelBuilder to fetch local schema when no SchemaBuilder present

mufaddal-rohawala · 2024-02-17T01:30:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e3361c2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-17T01:32:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: e3361c2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-17T01:33:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e3361c2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-17T02:29:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e3361c2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov · 2024-02-17T02:29:59Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.96%. Comparing base (8b206ba) to head (8f6e6b2).
Report is 32 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4434      +/-   ##
==========================================
+ Coverage   86.94%   86.96%   +0.02%     
==========================================
  Files        1203      387     -816     
  Lines      107211    35785   -71426     
==========================================
- Hits        93211    31122   -62089     
+ Misses      14000     4663    -9337

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mufaddal-rohawala · 2024-02-17T02:35:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e3361c2
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/image_uri_config/tasks.json

src/sagemaker/serve/utils/exceptions.py

src/sagemaker/serve/builder/model_builder.py

src/sagemaker/image_uri_config/tasks.json

src/sagemaker/task.py

src/sagemaker/serve/builder/model_builder.py

src/sagemaker/task.py

gwang111 · 2024-02-19T18:56:11Z

When we apply these defaults, what are the max_token_lengths we are picking?

src/sagemaker/image_uri_config/tasks.json

samruds · 2024-02-19T22:35:16Z

src/sagemaker/serve/utils/task.py

+    Raises:
+        ValueError: If no tasks config found or the task does not exist in the local config.
+    """
+    # task_io_config_path = os.path.join(os.path.dirname(__file__), "image_uri_config", "tasks.json")


Remove this comment?

mufaddal-rohawala · 2024-02-19T23:28:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 99d11cb
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

makungaj1 · 2024-02-20T19:29:09Z

so task.json is sitting in the image_uri_config?

Yes, can double-check with @knikure

mufaddal-rohawala · 2024-02-20T20:44:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 38eed2f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-20T20:45:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 38eed2f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-20T21:04:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 38eed2f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-20T21:15:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 38eed2f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-20T21:42:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 38eed2f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T01:08:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 4b3f617
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T01:14:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 4b3f617
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T01:36:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 4b3f617
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T02:06:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 4b3f617
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T03:07:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 4b3f617
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T18:14:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 8f6e6b2
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T18:24:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 8f6e6b2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T18:29:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 8f6e6b2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T19:09:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 8f6e6b2
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T19:23:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 8f6e6b2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/image_uri_config/tasks.json

knikure

/bot run notebook-tests, pr

mufaddal-rohawala · 2024-02-22T22:29:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 8f6e6b2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-02-22T22:54:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 8f6e6b2
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…t. (aws#4434) * Fetch Schema locally * Fetch Schema locally * Local schema * Test local schemas * Testing * Testing Schema * Schema for DJL * Add Integ tests * address PR comments * Address PR Review Comments * Add Unit tests * Address PR Comments * Address PR Comments --------- Co-authored-by: Jonathan Makunga <[email protected]>

Jonathan Makunga and others added 10 commits February 15, 2024 14:09

Fetch Schema locally

b2d0929

Fetch Schema locally

c4fb9f7

Local schema

3b5e699

Test local schemas

edb5716

Testing

1b2a4fb

Testing Schema

b2acab3

Schema for DJL

ad2303f

Merge branch 'aws:master' into schema-builder-detector

6ad7982

Add Integ tests

17ae5d9

Merge pull request #1 from makungaj1/schema-builder-detector

e3361c2

ModelBuilder to fetch local schema when no SchemaBuilder present

makungaj1 requested a review from a team as a code owner February 17, 2024 00:30

makungaj1 requested review from akrishna1995 and removed request for a team February 17, 2024 00:30

samruds reviewed Feb 19, 2024

View reviewed changes

src/sagemaker/task.py Outdated Show resolved Hide resolved

src/sagemaker/serve/builder/model_builder.py Show resolved Hide resolved

src/sagemaker/task.py Outdated Show resolved Hide resolved

makungaj1 requested a review from samruds February 19, 2024 18:51

jiapinw reviewed Feb 19, 2024

View reviewed changes

src/sagemaker/image_uri_config/tasks.json Outdated Show resolved Hide resolved

Jonathan Makunga and others added 3 commits February 19, 2024 14:27

address PR comments

98dd0a9

Address PR Review Comments

854db04

Merge branch 'master' into master

967c9cc

samruds reviewed Feb 19, 2024

View reviewed changes

Address PR Review Comments

99d11cb

samruds approved these changes Feb 20, 2024

View reviewed changes

mufaddal-rohawala assigned knikure Feb 21, 2024

Merge branch 'master' into master

4b3f617

Jonathan Makunga added 2 commits February 22, 2024 09:18

Address PR Comments

f78637c

Address PR Comments

8f6e6b2

mufaddal-rohawala approved these changes Feb 22, 2024

View reviewed changes

src/sagemaker/image_uri_config/tasks.json Outdated Show resolved Hide resolved

knikure reviewed Feb 22, 2024

View reviewed changes

knikure approved these changes Feb 22, 2024

View reviewed changes

mufaddal-rohawala merged commit eb49090 into aws:master Feb 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ModelBuilder to fetch local schema when no SchemaBuilder present. #4434

ModelBuilder to fetch local schema when no SchemaBuilder present. #4434

makungaj1 commented Feb 17, 2024 •

edited

Loading

mufaddal-rohawala commented Feb 17, 2024

mufaddal-rohawala commented Feb 17, 2024

mufaddal-rohawala commented Feb 17, 2024

mufaddal-rohawala commented Feb 17, 2024

codecov bot commented Feb 17, 2024 •

edited

Loading

mufaddal-rohawala commented Feb 17, 2024

gwang111 commented Feb 19, 2024

samruds Feb 19, 2024

makungaj1 Feb 19, 2024

mufaddal-rohawala commented Feb 19, 2024

makungaj1 commented Feb 20, 2024

mufaddal-rohawala commented Feb 20, 2024

mufaddal-rohawala commented Feb 20, 2024

mufaddal-rohawala commented Feb 20, 2024

mufaddal-rohawala commented Feb 20, 2024

mufaddal-rohawala commented Feb 20, 2024

mufaddal-rohawala commented Feb 22, 2024

mufaddal-rohawala commented Feb 22, 2024

mufaddal-rohawala commented Feb 22, 2024

mufaddal-rohawala commented Feb 22, 2024

mufaddal-rohawala commented Feb 22, 2024

mufaddal-rohawala commented Feb 22, 2024

mufaddal-rohawala commented Feb 22, 2024

mufaddal-rohawala commented Feb 22, 2024

mufaddal-rohawala commented Feb 22, 2024

mufaddal-rohawala commented Feb 22, 2024

knikure left a comment

mufaddal-rohawala commented Feb 22, 2024

mufaddal-rohawala commented Feb 22, 2024

ModelBuilder to fetch local schema when no SchemaBuilder present. #4434

ModelBuilder to fetch local schema when no SchemaBuilder present. #4434

Conversation

makungaj1 commented Feb 17, 2024 • edited Loading

Merge Checklist

General

Tests

mufaddal-rohawala commented Feb 17, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 17, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 17, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 17, 2024

AWS CodeBuild CI Report

codecov bot commented Feb 17, 2024 • edited Loading

Codecov Report

mufaddal-rohawala commented Feb 17, 2024

AWS CodeBuild CI Report

gwang111 commented Feb 19, 2024

samruds Feb 19, 2024

Choose a reason for hiding this comment

makungaj1 Feb 19, 2024

Choose a reason for hiding this comment

mufaddal-rohawala commented Feb 19, 2024

AWS CodeBuild CI Report

makungaj1 commented Feb 20, 2024

mufaddal-rohawala commented Feb 20, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 20, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 20, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 20, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 20, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

knikure left a comment

Choose a reason for hiding this comment

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Feb 22, 2024

AWS CodeBuild CI Report

makungaj1 commented Feb 17, 2024 •

edited

Loading

codecov bot commented Feb 17, 2024 •

edited

Loading