feat: jumpstart instance specific metric definitions #4200

evakravi · 2023-10-16T18:48:55Z

Issue #, if available:

Description of changes:
This PR adds the capability to interpret fields in JumpStart's metadata that indicate instance specific metric definitions. This enables JumpStart to launch models that require different metric definitions depending on the instance type

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2023-10-16T19:50:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: d9e9dc7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-16T19:58:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: d9e9dc7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-16T20:02:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: d9e9dc7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-16T20:42:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: d9e9dc7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-16T20:52:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: d9e9dc7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…finitions

sagemaker-bot · 2023-10-17T15:00:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: affd646
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-17T15:08:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: affd646
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-17T15:09:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: affd646
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-17T15:27:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: affd646
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-17T15:52:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: affd646
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

knikure · 2023-10-17T20:32:15Z

@evakravi Can you please resolve conflicts?

…ecific-metric-definitions

sagemaker-bot · 2023-10-18T00:27:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 59ff4a1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T00:34:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 59ff4a1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T00:56:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 59ff4a1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T01:17:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 59ff4a1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T01:21:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 59ff4a1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-10-18T01:22:37Z

Codecov Report

Attention: 46 lines in your changes are missing coverage. Please review.

Comparison is base (3057f21) 90.27% compared to head (7c82229) 90.33%.
Report is 46 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4200      +/-   ##
==========================================
+ Coverage   90.27%   90.33%   +0.06%     
==========================================
  Files         993      320     -673     
  Lines       87861    29607   -58254     
==========================================
- Hits        79313    26745   -52568     
+ Misses       8548     2862    -5686

Files	Coverage Δ
src/sagemaker/base_predictor.py	`90.90% <100.00%> (ø)`
src/sagemaker/config/config_schema.py	`100.00% <100.00%> (ø)`
src/sagemaker/estimator.py	`92.88% <ø> (ø)`
src/sagemaker/feature_store/feature_group.py	`78.82% <ø> (ø)`
...emaker/feature_store/feature_processor/__init__.py	`100.00% <100.00%> (ø)`
...eature_store/feature_processor/_config_uploader.py	`97.43% <100.00%> (ø)`
...aker/feature_store/feature_processor/_constants.py	`100.00% <100.00%> (ø)`
...er/feature_store/feature_processor/_data_source.py	`100.00% <100.00%> (ø)`
...agemaker/feature_store/feature_processor/_enums.py	`100.00% <100.00%> (ø)`
...ore/feature_processor/_feature_processor_config.py	`100.00% <100.00%> (ø)`
... and 49 more

... and 1254 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…ecific-metric-definitions

sagemaker-bot · 2023-10-18T18:03:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: fea985c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T18:13:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: fea985c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T18:27:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: fea985c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T18:31:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: fea985c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…finitions

sagemaker-bot · 2023-10-18T19:49:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 7c82229
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T20:04:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 7c82229
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T20:33:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 7c82229
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T20:45:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 7c82229
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T21:10:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 7c82229
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

feat: jumpstart instance specific metric definitions

58d5436

evakravi requested a review from a team as a code owner October 16, 2023 18:48

evakravi requested review from akrishna1995 and removed request for a team October 16, 2023 18:48

chore: update js estimator factory

d9e9dc7

Merge branch 'master' into feat/jumpstart-instance-specific-metric-de…

affd646

…finitions

knikure assigned akrishna1995 Oct 17, 2023

Merge remote-tracking branch 'origin' into feat/jumpstart-instance-sp…

59ff4a1

…ecific-metric-definitions

JGuinegagne approved these changes Oct 18, 2023

View reviewed changes

Merge remote-tracking branch 'origin' into feat/jumpstart-instance-sp…

fea985c

…ecific-metric-definitions

Merge branch 'master' into feat/jumpstart-instance-specific-metric-de…

7c82229

…finitions

knikure approved these changes Oct 18, 2023

View reviewed changes

knikure merged commit d3f2841 into aws:master Oct 18, 2023

knikure pushed a commit to knikure/sagemaker-python-sdk that referenced this pull request Oct 23, 2023

feat: jumpstart instance specific metric definitions (aws#4200)

e4f7b37

benieric pushed a commit to benieric/sagemaker-python-sdk that referenced this pull request Oct 24, 2023

feat: jumpstart instance specific metric definitions (aws#4200)

e6e6b00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: jumpstart instance specific metric definitions #4200

feat: jumpstart instance specific metric definitions #4200

evakravi commented Oct 16, 2023 •

edited

Loading

sagemaker-bot commented Oct 16, 2023

sagemaker-bot commented Oct 16, 2023

sagemaker-bot commented Oct 16, 2023

sagemaker-bot commented Oct 16, 2023

sagemaker-bot commented Oct 16, 2023

sagemaker-bot commented Oct 17, 2023

sagemaker-bot commented Oct 17, 2023

sagemaker-bot commented Oct 17, 2023

sagemaker-bot commented Oct 17, 2023

sagemaker-bot commented Oct 17, 2023

knikure commented Oct 17, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

codecov-commenter commented Oct 18, 2023 •

edited

Loading

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

sagemaker-bot commented Oct 18, 2023

feat: jumpstart instance specific metric definitions #4200

feat: jumpstart instance specific metric definitions #4200

Conversation

evakravi commented Oct 16, 2023 • edited Loading

Merge Checklist

General

Tests

sagemaker-bot commented Oct 16, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 16, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 16, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 16, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 16, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 17, 2023

AWS CodeBuild CI Report

knikure commented Oct 17, 2023

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

codecov-commenter commented Oct 18, 2023 • edited Loading

Codecov Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

evakravi commented Oct 16, 2023 •

edited

Loading

codecov-commenter commented Oct 18, 2023 •

edited

Loading