feature: jumpstart notebook utils -- list model ids, scripts, tasks, frameworks #2987

evakravi · 2022-03-10T22:17:33Z

Issue #, if available:

Description of changes:
This PR adds notebook utilities for JumpStart. In particular, there are new utilities in a module sagemaker.jumpstart.notebook_utils called list_jumpstart_models, list_jumpstart_scripts, list_jumpstart_tasks, list_jumpstart_frameworks, and extract_framework_task_model.

Testing done:
Unit tests written

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2022-03-10T22:30:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e499bc6
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-03-10T22:49:18Z

Codecov Report

Merging #2987 (0b69ad8) into master (51706c4) will decrease coverage by 0.02%.
The diff coverage is 91.71%.

@@            Coverage Diff             @@
##           master    #2987      +/-   ##
==========================================
- Coverage   89.47%   89.44%   -0.03%     
==========================================
  Files        1033      198     -835     
  Lines       84451    17076   -67375     
==========================================
- Hits        75563    15274   -60289     
+ Misses       8888     1802    -7086

Impacted Files	Coverage Δ
src/sagemaker/jumpstart/cache.py	`96.58% <ø> (ø)`
src/sagemaker/jumpstart/exceptions.py	`82.14% <ø> (ø)`
src/sagemaker/jumpstart/types.py	`94.37% <ø> (ø)`
src/sagemaker/jumpstart/filters.py	`90.33% <90.33%> (ø)`
src/sagemaker/jumpstart/notebook_utils.py	`93.58% <93.28%> (-6.42%)`	⬇️
src/sagemaker/jumpstart/accessors.py	`100.00% <100.00%> (ø)`
...thon3.7/site-packages/sagemaker/mxnet/estimator.py
...python3.6/site-packages/sagemaker/xgboost/model.py
...thon3.6/site-packages/sagemaker/mxnet/estimator.py
...on3.9/site-packages/sagemaker/apiutils/__init__.py
... and 832 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51706c4...0b69ad8. Read the comment docs.

sagemaker-bot · 2022-03-10T22:55:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 881273b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-10T23:31:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 881273b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-10T23:31:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: e499bc6
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-10T23:47:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e499bc6
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-10T23:47:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 881273b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

khetan2 · 2022-03-11T20:18:02Z

Why do we want to put these utilities into sagemaker.jumpstart.notebook_utils ? Should we just have them in sagemaker.jumpstart? There is nothing specific about notebooks in these utilities.

JGuinegagne

Some preliminary comments - However as discuss, I think we should pause and reflect on what we are building here. I would like to get a more general and extensible filtering mechanism.

src/sagemaker/jumpstart/accessors.py

src/sagemaker/jumpstart/notebook_utils.py

sagemaker-bot · 2022-03-23T13:39:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 553a08a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T13:40:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: fd28f22
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T13:48:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2669e83
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T13:55:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2635dd1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T13:58:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 553a08a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T14:17:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 553a08a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T14:23:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 2669e83
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T14:23:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: fd28f22
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T14:25:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 2635dd1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T14:31:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 2635dd1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T14:31:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 2669e83
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-23T14:31:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: fd28f22
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-07T20:10:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 2bb0f19
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-07T20:11:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 4b5b502
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-07T20:16:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2bb0f19
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-07T20:50:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 2bb0f19
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-07T20:50:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 4b5b502
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-07T22:36:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 4b5b502
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-08T01:43:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: f9072ed
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-08T02:52:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: f9072ed
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T01:17:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c300183
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T02:24:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: c300183
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T17:11:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c300183
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…l-id-list

…gemaker-python-sdk into feat/jumpstart-model-id-list

sagemaker-bot · 2022-04-20T13:26:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 0b69ad8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-20T13:42:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 0b69ad8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-20T13:57:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 0b69ad8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-20T14:13:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 0b69ad8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-20T14:21:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 0b69ad8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2022-04-20T19:45:20Z

src/sagemaker/jumpstart/notebook_utils.py

+    return sorted(list(model_id_version_set), key=cmp_to_key(_compare_model_version_tuples))
+
+
+def _generate_jumpstart_model_versions(  # pylint: disable=redefined-builtin


nit: This functione is very large can we make this more modular?

Thanks for the feedback. I agree this is a relatively long function. Would you mind if I create a backlog item to improve this logic?

…frameworks (aws#2987)

evakravi changed the base branch from master to dev March 10, 2022 22:17

evakravi changed the title ~~Feat/jumpstart model id list~~ Feat/jumpstart notebook utils -- list model ids, scripts, tasks, frameworks Mar 10, 2022

evakravi requested a review from JGuinegagne March 11, 2022 14:22

JGuinegagne requested changes Mar 15, 2022

View reviewed changes

ahsan-z-khan changed the base branch from dev to master March 15, 2022 16:44

evakravi added 3 commits March 23, 2022 13:25

feat: jumpstart notebook utils

2a5bb75

fix: pylint typing

a39a051

chore: new implementation for listing jumpstart models

2669e83

evakravi force-pushed the feat/jumpstart-model-id-list branch from 553a08a to 2669e83 Compare March 23, 2022 13:25

chore: resolve git comments

2635dd1

Merge branch 'master' into feat/jumpstart-model-id-list

4b5b502

evakravi force-pushed the feat/jumpstart-model-id-list branch from 9e9b746 to 4b5b502 Compare April 7, 2022 19:49

fix: read the docs capitalization

2bb0f19

Merge branch 'master' into feat/jumpstart-model-id-list

f9072ed

Merge branch 'master' into feat/jumpstart-model-id-list

c300183

JGuinegagne approved these changes Apr 14, 2022

View reviewed changes

navinsoni approved these changes Apr 19, 2022

View reviewed changes

evakravi added 2 commits April 20, 2022 13:05

Merge remote-tracking branch 'origin/master' into feat/jumpstart-mode…

8a35cba

…l-id-list

Merge branch 'feat/jumpstart-model-id-list' of github.com:evakravi/sa…

0b69ad8

…gemaker-python-sdk into feat/jumpstart-model-id-list

mufaddal-rohawala reviewed Apr 20, 2022

View reviewed changes

mufaddal-rohawala approved these changes Apr 20, 2022

View reviewed changes

mufaddal-rohawala changed the title ~~Feat/jumpstart notebook utils -- list model ids, scripts, tasks, frameworks~~ feature: jumpstart notebook utils -- list model ids, scripts, tasks, frameworks Apr 20, 2022

mufaddal-rohawala merged commit 4a94ace into aws:master Apr 20, 2022

jerrypeng7773 pushed a commit to jerrypeng7773/sagemaker-python-sdk that referenced this pull request May 13, 2022

feature: jumpstart notebook utils -- list model ids, scripts, tasks, …

c98f03c

…frameworks (aws#2987)

		return sorted(list(model_id_version_set), key=cmp_to_key(_compare_model_version_tuples))


		def _generate_jumpstart_model_versions( # pylint: disable=redefined-builtin

feature: jumpstart notebook utils -- list model ids, scripts, tasks, frameworks #2987

feature: jumpstart notebook utils -- list model ids, scripts, tasks, frameworks #2987

Conversation

evakravi commented Mar 10, 2022

Merge Checklist

General

Tests

sagemaker-bot commented Mar 10, 2022

AWS CodeBuild CI Report

codecov-commenter commented Mar 10, 2022 • edited Loading

Codecov Report

sagemaker-bot commented Mar 10, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 10, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 10, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 10, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 10, 2022

AWS CodeBuild CI Report

khetan2 commented Mar 11, 2022

JGuinegagne left a comment

Choose a reason for hiding this comment

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 20, 2022

AWS CodeBuild CI Report

codecov-commenter commented Mar 10, 2022 •

edited

Loading