feature: support create Clarify explainer enabled endpoint for Clarify Online Explainability #3727

xiaoyi-cheng · 2023-03-17T06:26:10Z

Issue #, if available:
https://tiny.amazon.com/13fkia7b9/issuamazP578

Description of changes:
Update create_endpoint_config() and its callers to accommodate the SageMaker API change.

Out of scope of this PR, will do in upcoming PR:
Update predictor.predict() to accommodate the SageMaker Runtime API change. Deserialize for predict() response to decode the predictions.

Testing done:
Unit tests and integ tests with tox (covers sphinx for buidling doc as well).

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2023-03-17T06:30:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 769d30c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T07:02:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 769d30c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T07:20:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 769d30c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T07:21:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 769d30c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T07:22:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 769d30c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T18:31:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: fb69c6e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T19:28:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: fb69c6e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T19:28:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: fb69c6e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T19:34:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: fb69c6e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T19:37:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: fb69c6e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T21:34:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 22d8ad0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T21:35:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 22d8ad0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T21:38:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 22d8ad0
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T21:41:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 22d8ad0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T22:37:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 22d8ad0
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T22:50:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 22d8ad0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T22:57:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 22d8ad0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T22:58:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 22d8ad0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-18T00:54:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 22d8ad0
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

xgchena

General comment: besides model.py and session.py I think more files should be updated with the explainer config. E.g. the commit touches the unit test of deploy and estimator.

I think you can search for the other endpoint config parameters, e.g.

$ grep -r async_inference_config src  
$ grep -r serverless_inference_config src

General: Move the predictor related changes to the next PR, because it requires an explainer specific deserializer to work well.

Another general comment: in the Testing Done section, add a test to build the doc and review the pages.

xgchena · 2023-03-20T20:14:01Z

src/sagemaker/explainer/clarify_explainer_config.py

+
+
+class ClarifyTextConfig(object):
+    """Configuration to explain natural language processing (NLP) explainability."""


General comment: how about we copy the SageMaker API documentation for the class? E.g. in this case from ClarifyTextConfig.

src/sagemaker/explainer/explainer_config.py

src/sagemaker/explainer/clarify_explainer_config.py

src/sagemaker/model.py

tests/integ/test_explainer.py

sagemaker-bot · 2023-03-22T16:02:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: d302079
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-03-22T16:05:17Z

Codecov Report

Merging #3727 (0884cb7) into master (0473a8e) will decrease coverage by 1.09%.
The diff coverage is 97.50%.

❗ Current head 0884cb7 differs from pull request most recent head 5c85604. Consider uploading reports for the commit 5c85604 to get more accurate results

@@            Coverage Diff             @@
##           master    #3727      +/-   ##
==========================================
- Coverage   90.21%   89.12%   -1.09%     
==========================================
  Files         996      233     -763     
  Lines       94508    22973   -71535     
==========================================
- Hits        85257    20475   -64782     
+ Misses       9251     2498    -6753

Impacted Files	Coverage Δ
src/sagemaker/estimator.py	`89.89% <ø> (ø)`
src/sagemaker/huggingface/model.py	`89.87% <ø> (ø)`
src/sagemaker/tensorflow/model.py	`96.33% <ø> (ø)`
...ference_recommender/inference_recommender_mixin.py	`95.00% <50.00%> (ø)`
...rc/sagemaker/explainer/clarify_explainer_config.py	`100.00% <100.00%> (ø)`
src/sagemaker/explainer/explainer_config.py	`100.00% <100.00%> (ø)`
src/sagemaker/model.py	`93.38% <100.00%> (ø)`
src/sagemaker/session.py	`75.64% <100.00%> (ø)`

... and 1221 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2023-03-22T16:07:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: d302079
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-27T23:24:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 0884cb7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-29T17:30:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5c85604
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-29T17:31:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5c85604
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-29T18:13:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 9aa9ccf
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-29T18:31:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 9aa9ccf
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-29T18:33:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 9aa9ccf
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-29T18:38:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 9aa9ccf
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-29T20:38:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9aa9ccf
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

claytonparnell

/bot run slow-tests

sagemaker-bot · 2023-03-29T22:16:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9aa9ccf
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-30T18:55:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9aa9ccf
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-31T00:41:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9aa9ccf
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-31T04:03:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 2806995
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-31T04:20:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 2806995
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-31T04:22:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 2806995
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-31T04:23:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 2806995
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-31T04:32:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2806995
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…y Online Explainability (aws#3727) Co-authored-by: Clayton Parnell <[email protected]>

xiaoyi-cheng force-pushed the ole branch from c94e458 to 769d30c Compare March 17, 2023 06:26

xiaoyi-cheng force-pushed the ole branch from fb69c6e to 22d8ad0 Compare March 17, 2023 20:45

xiaoyi-cheng marked this pull request as ready for review March 17, 2023 21:55

xiaoyi-cheng requested a review from a team as a code owner March 17, 2023 21:55

xiaoyi-cheng requested review from claytonparnell and removed request for a team March 17, 2023 21:55

xgchena suggested changes Mar 20, 2023

View reviewed changes

xiaoyi-cheng force-pushed the ole branch from 22d8ad0 to d302079 Compare March 22, 2023 15:56

claytonparnell approved these changes Mar 28, 2023

View reviewed changes

claytonparnell and others added 3 commits March 29, 2023 10:31

Update CHANGELOG.md

b3e6399

test: add explainer config integ test

a09e6ec

fix: fix format and unit tests

e9bce93

fix: address feedbacks

e7ad3a9

xiaoyi-cheng force-pushed the ole branch from 5c85604 to 17c9d4c Compare March 29, 2023 17:32

fix: clean unit test

9aa9ccf

xiaoyi-cheng force-pushed the ole branch from 17c9d4c to 9aa9ccf Compare March 29, 2023 17:36

claytonparnell reviewed Mar 29, 2023

View reviewed changes

Merge branch 'master' into ole

2806995

trajanikant merged commit ef992e2 into aws:master Apr 4, 2023

xiaoyi-cheng deleted the ole branch April 4, 2023 23:04

xiaoyi-cheng restored the ole branch April 4, 2023 23:04

doddaspk-amzn pushed a commit to doddaspk-amzn/sagemaker-python-sdk that referenced this pull request Apr 6, 2023

feature: support create Clarify explainer enabled endpoint for Clarif…

1167339

…y Online Explainability (aws#3727) Co-authored-by: Clayton Parnell <[email protected]>



		class ClarifyTextConfig(object):
		"""Configuration to explain natural language processing (NLP) explainability."""

feature: support create Clarify explainer enabled endpoint for Clarify Online Explainability #3727

feature: support create Clarify explainer enabled endpoint for Clarify Online Explainability #3727

Conversation

xiaoyi-cheng commented Mar 17, 2023 • edited Loading

Merge Checklist

General

Tests

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 18, 2023

AWS CodeBuild CI Report

xgchena left a comment

Choose a reason for hiding this comment

xgchena Mar 20, 2023

Choose a reason for hiding this comment

sagemaker-bot commented Mar 22, 2023

AWS CodeBuild CI Report

codecov-commenter commented Mar 22, 2023 • edited Loading

Codecov Report

sagemaker-bot commented Mar 22, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 27, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 29, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 29, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 29, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 29, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 29, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 29, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 29, 2023

AWS CodeBuild CI Report

claytonparnell left a comment • edited Loading

Choose a reason for hiding this comment

sagemaker-bot commented Mar 29, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 30, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 31, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Mar 31, 2023

xiaoyi-cheng commented Mar 17, 2023 •

edited

Loading

codecov-commenter commented Mar 22, 2023 •

edited

Loading

claytonparnell left a comment •

edited

Loading