feat: custom base job name for jumpstart models/estimators #2970

evakravi · 2022-03-04T13:29:47Z

Issue #, if available:

Description of changes:
Adds custom base job name for jumpstart models/estimators (sagemaker-jumpstart prefix). In addition, this PR fixes an integration test not tagging a Model so that it can be flagged for deletion.

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: Dewen Qi <[email protected]>

Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]>

…ws#2950) Co-authored-by: Payton Staub <[email protected]>

sagemaker-bot · 2022-03-04T13:41:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c639b19
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T14:47:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: c639b19
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T14:51:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c639b19
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T14:54:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 3542bb9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-03-04T15:16:19Z

Codecov Report

Merging #2970 (ea9490b) into dev (4325fcd) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##              dev    #2970   +/-   ##
=======================================
  Coverage   89.81%   89.81%           
=======================================
  Files         196      196           
  Lines       16564    16564           
=======================================
  Hits        14877    14877           
  Misses       1687     1687

Impacted Files	Coverage Δ
src/sagemaker/estimator.py	`91.32% <100.00%> (ø)`
src/sagemaker/jumpstart/cache.py	`96.58% <100.00%> (ø)`
src/sagemaker/model.py	`91.79% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4325fcd...ea9490b. Read the comment docs.

sagemaker-bot · 2022-03-04T15:27:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 3542bb9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T15:32:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 3542bb9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T15:47:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 747e234
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T16:07:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5677dcb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T16:42:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5677dcb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T16:46:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 747e234
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T16:47:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 747e234
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T22:50:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5677dcb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T22:50:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5677dcb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T23:27:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5677dcb
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

JGuinegagne

Just a few clean-up and optional comments please.

doc/overview.rst

src/sagemaker/estimator.py

src/sagemaker/jumpstart/utils.py

src/sagemaker/model.py

JGuinegagne · 2022-03-07T14:35:45Z

src/sagemaker/model.py

+                self._base_name
+                or get_jumpstart_base_name_if_jumpstart_model(script_uri, model_uri)
+                or utils.base_name_from_image(image_uri)
+            )


non-blocking: I would prefer to define a jumpstart_base_name variable here as well.

tests/unit/sagemaker/jumpstart/test_utils.py

…ws#2971)

sagemaker-bot · 2022-03-07T21:13:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: b47b1d5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-07T21:49:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b47b1d5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-07T23:06:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: b47b1d5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: Dewen Qi <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: qidewenwhen <[email protected]> Co-authored-by: Qingzi-Lan <[email protected]> Co-authored-by: Payton Staub <[email protected]>

sagemaker-bot · 2022-03-08T14:20:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 95e2ca9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T15:11:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 95e2ca9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T16:41:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 95e2ca9
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/jumpstart/utils.py

sagemaker-bot · 2022-03-08T18:24:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ea9490b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T19:17:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ea9490b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T19:52:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 95e2ca9
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T20:06:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ea9490b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T21:34:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ea9490b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T22:39:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ea9490b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

navinsoni · 2022-03-11T01:48:03Z

src/sagemaker/jumpstart/utils.py

+) -> Optional[str]:
+    """Return default JumpStart base name if a URI belongs to JumpStart.
+
+    If no URIs belong to JumpStart, return None.


Can you please address Mufis' comment

I did (*uris (Optional[str]): URI to test for association with JumpStart.). I believe you're looking at an old version.

Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: Dewen Qi <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: qidewenwhen <[email protected]> Co-authored-by: Qingzi-Lan <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Miyoung <[email protected]> Co-authored-by: Shreya Pandit <[email protected]> Co-authored-by: Ahsan Khan <[email protected]>

navinsoni and others added 7 commits March 1, 2022 14:18

change: update code to get commit_id in codepipeline (aws#2961)

f166b60

feature: Data Serializer (aws#2956)

086258d

change: reorganize test files for workflow (aws#2960)

a39b750

Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: Dewen Qi <[email protected]>

feature: TensorFlow 2.4 for Neo (aws#2861)

28fd737

Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]>

fix: Remove sagemaker_job_name from hyperparameters in TrainingStep (a…

20df3d7

…ws#2950) Co-authored-by: Payton Staub <[email protected]>

fix: Style update in DataSerializer (aws#2962)

b9f90dc

feat: custom base job name for jumpstart models/estimators

c639b19

chore: add unit tests, fix unused import

3542bb9

chore: update docs to reflect new jumpstart base name

747e234

fix: docstring

5677dcb

documentation: smddp doc update (aws#2968)

6db3774

JGuinegagne requested changes Mar 7, 2022

View reviewed changes

shreyapandit and others added 3 commits March 7, 2022 09:51

fix: container env generation for S3 URI and add test for the same (a…

d610bfb

…ws#2971)

documentation: update sagemaker training compiler docstring (aws#2969)

169dffd

chore: resolve git comments

b47b1d5

ahsan-z-khan and others added 2 commits March 7, 2022 17:28

Merge branch 'dev' into feat/custom-job-name-for-jumpstart

95e2ca9

JGuinegagne approved these changes Mar 8, 2022

View reviewed changes

evakravi marked this pull request as ready for review March 8, 2022 14:46

mufaddal-rohawala reviewed Mar 8, 2022

View reviewed changes

src/sagemaker/jumpstart/utils.py Show resolved Hide resolved

change: improve docstring

ea9490b

jeniyat approved these changes Mar 10, 2022

View reviewed changes

navinsoni suggested changes Mar 11, 2022

View reviewed changes

navinsoni approved these changes Mar 11, 2022

View reviewed changes

navinsoni changed the base branch from dev to master March 11, 2022 18:44

navinsoni merged commit dfc6eee into aws:master Mar 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: custom base job name for jumpstart models/estimators #2970

feat: custom base job name for jumpstart models/estimators #2970

evakravi commented Mar 4, 2022 •

edited

Loading

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

codecov-commenter commented Mar 4, 2022 •

edited

Loading

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

sagemaker-bot commented Mar 4, 2022

JGuinegagne left a comment

JGuinegagne Mar 7, 2022

sagemaker-bot commented Mar 7, 2022

sagemaker-bot commented Mar 7, 2022

sagemaker-bot commented Mar 7, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

navinsoni Mar 11, 2022

evakravi Mar 11, 2022

feat: custom base job name for jumpstart models/estimators #2970

feat: custom base job name for jumpstart models/estimators #2970

Conversation

evakravi commented Mar 4, 2022 • edited Loading

Merge Checklist

General

Tests

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

codecov-commenter commented Mar 4, 2022 • edited Loading

Codecov Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

JGuinegagne left a comment

Choose a reason for hiding this comment

JGuinegagne Mar 7, 2022

Choose a reason for hiding this comment

sagemaker-bot commented Mar 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

navinsoni Mar 11, 2022

Choose a reason for hiding this comment

evakravi Mar 11, 2022

Choose a reason for hiding this comment

evakravi commented Mar 4, 2022 •

edited

Loading

codecov-commenter commented Mar 4, 2022 •

edited

Loading