change: Enable Experiment integ test on beta clients #3590

qidewenwhen · 2023-01-13T21:24:50Z

Description of changes: Enable Experiment integ test on beta SageMaker and SageMaker Metrics clients so that we can include Experiment integ tests in beta stage in the service pipeline.

Testing done: Integ tests

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2023-01-13T22:07:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 0ca35ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-13T22:09:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 0ca35ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-01-13T22:10:32Z

Codecov Report

Merging #3590 (ce57b57) into master (483665a) will decrease coverage by 0.80%.
The diff coverage is 100.00%.

❗ Current head ce57b57 differs from pull request most recent head a381222. Consider uploading reports for the commit a381222 to get more accurate results

@@            Coverage Diff             @@
##           master    #3590      +/-   ##
==========================================
- Coverage   89.59%   88.80%   -0.80%     
==========================================
  Files         968      227     -741     
  Lines       89536    22140   -67396     
==========================================
- Hits        80219    19661   -60558     
+ Misses       9317     2479    -6838

Impacted Files	Coverage Δ
src/sagemaker/experiments/run.py	`99.13% <100.00%> (ø)`
...b/python3.8/site-packages/sagemaker/rl/__init__.py
...3.8/site-packages/sagemaker/jumpstart/artifacts.py
...ib/python3.10/site-packages/sagemaker/vpc_utils.py
...r/cli/compatibility/v2/modifiers/training_input.py
...on3.9/site-packages/sagemaker/debugger/debugger.py
...n3.10/site-packages/sagemaker/xgboost/estimator.py
...ackages/sagemaker/feature_store/dataset_builder.py
.../python3.7/site-packages/sagemaker/deprecations.py
...ackages/sagemaker/feature_store/dataset_builder.py
... and 1186 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2023-01-13T22:14:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 0ca35ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-13T22:19:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 0ca35ba
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-13T22:32:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 0ca35ba
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-13T23:30:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ce57b57
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-13T23:32:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ce57b57
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-13T23:42:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: ce57b57
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-13T23:55:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: ce57b57
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T01:42:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ce57b57
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T05:15:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: d5ad31d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T05:17:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: d5ad31d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T05:20:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: d5ad31d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T05:30:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: d5ad31d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T06:05:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: d5ad31d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-18T22:14:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: a381222
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-18T22:15:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: a381222
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-18T22:51:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: a381222
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-18T23:01:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: a381222
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-18T23:14:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: a381222
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T19:56:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 8d7f3ac
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T20:35:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 8d7f3ac
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T20:36:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 8d7f3ac
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T20:37:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 8d7f3ac
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T22:47:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 8d7f3ac
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

claytonparnell

/bot run slow-tests

sagemaker-bot · 2023-01-27T21:34:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 8d7f3ac
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

danabens · 2023-01-30T18:34:56Z

slow tests dont seem to be providing much value with their inconsistency

(11 durations < 0.005s hidden.  Use -vv to show these durations.)
=========================== short test summary info ============================
FAILED tests/integ/test_inference_pipeline.py::test_inference_pipeline_model_deploy_and_update_endpoint
FAILED tests/integ/test_clarify_model_monitor.py::test_run_explainability_monitor
FAILED tests/integ/test_model_quality_monitor.py::test_run_model_quality_monitor
FAILED tests/integ/test_model_quality_monitor.py::test_run_model_quality_monitor_baseline
FAILED tests/integ/test_clarify_model_monitor.py::test_run_bias_monitor - sto...
============ 5 failed, 11 passed, 1 warning in 10170.16s (2:49:30) =============
ERROR: InvocationError for command /codebuild/output/src448114084/src/github.com/aws/sagemaker-python-sdk/.tox/py310/bin/pytest --cov=sagemaker --cov-append tests/integ -m slow_test -n 16 --durations 0 (exited with code 1)
___________________________________ summary ____________________________________
ERROR:   py310: commands failed

sagemaker-bot · 2023-01-30T20:20:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 8d7f3ac
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

qidewenwhen requested a review from a team as a code owner January 13, 2023 21:24

qidewenwhen requested review from claytonparnell and removed request for a team January 13, 2023 21:24

qidewenwhen force-pushed the beta branch from 12dbd37 to 0ca35ba Compare January 13, 2023 21:27

qidewenwhen force-pushed the beta branch from 0ca35ba to ce57b57 Compare January 13, 2023 22:50

qidewenwhen force-pushed the beta branch from ce57b57 to d5ad31d Compare January 14, 2023 04:37

knikure assigned claytonparnell Jan 17, 2023

qidewenwhen requested a review from danabens January 18, 2023 18:35

qidewenwhen force-pushed the beta branch from d5ad31d to a381222 Compare January 18, 2023 22:08

change: Enable Experiment integ test on beta clients

8d7f3ac

qidewenwhen force-pushed the beta branch from a381222 to 8d7f3ac Compare January 24, 2023 19:54

claytonparnell reviewed Jan 27, 2023

View reviewed changes

danabens approved these changes Jan 30, 2023

View reviewed changes

claytonparnell approved these changes Jan 30, 2023

View reviewed changes

leoroy approved these changes Jan 31, 2023

View reviewed changes

mufaddal-rohawala merged commit bde191f into aws:master Feb 1, 2023

JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this pull request Mar 4, 2023

change: Enable Experiment integ test on beta clients (aws#3590)

8dea878

nmadan pushed a commit to nmadan/sagemaker-python-sdk that referenced this pull request Apr 18, 2023

change: Enable Experiment integ test on beta clients (aws#3590)

01ea241

qidewenwhen deleted the beta branch January 25, 2024 18:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

change: Enable Experiment integ test on beta clients #3590

change: Enable Experiment integ test on beta clients #3590

qidewenwhen commented Jan 13, 2023

sagemaker-bot commented Jan 13, 2023

sagemaker-bot commented Jan 13, 2023

codecov-commenter commented Jan 13, 2023 •

edited

Loading

sagemaker-bot commented Jan 13, 2023

sagemaker-bot commented Jan 13, 2023

sagemaker-bot commented Jan 13, 2023

sagemaker-bot commented Jan 13, 2023

sagemaker-bot commented Jan 13, 2023

sagemaker-bot commented Jan 13, 2023

sagemaker-bot commented Jan 13, 2023

sagemaker-bot commented Jan 14, 2023

sagemaker-bot commented Jan 14, 2023

sagemaker-bot commented Jan 14, 2023

sagemaker-bot commented Jan 14, 2023

sagemaker-bot commented Jan 14, 2023

sagemaker-bot commented Jan 14, 2023

sagemaker-bot commented Jan 18, 2023

sagemaker-bot commented Jan 18, 2023

sagemaker-bot commented Jan 18, 2023

sagemaker-bot commented Jan 18, 2023

sagemaker-bot commented Jan 18, 2023

sagemaker-bot commented Jan 24, 2023

sagemaker-bot commented Jan 24, 2023

sagemaker-bot commented Jan 24, 2023

sagemaker-bot commented Jan 24, 2023

sagemaker-bot commented Jan 24, 2023

claytonparnell left a comment

sagemaker-bot commented Jan 27, 2023

danabens commented Jan 30, 2023

sagemaker-bot commented Jan 30, 2023

change: Enable Experiment integ test on beta clients #3590

change: Enable Experiment integ test on beta clients #3590

Conversation

qidewenwhen commented Jan 13, 2023

Merge Checklist

General

Tests

sagemaker-bot commented Jan 13, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 13, 2023

AWS CodeBuild CI Report

codecov-commenter commented Jan 13, 2023 • edited Loading

Codecov Report

sagemaker-bot commented Jan 13, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 13, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 13, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 13, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 13, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 13, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 13, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 24, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 24, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 24, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 24, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 24, 2023

AWS CodeBuild CI Report

claytonparnell left a comment

Choose a reason for hiding this comment

sagemaker-bot commented Jan 27, 2023

AWS CodeBuild CI Report

danabens commented Jan 30, 2023

sagemaker-bot commented Jan 30, 2023

AWS CodeBuild CI Report

codecov-commenter commented Jan 13, 2023 •

edited

Loading