feat: added ANALYSIS_CONFIG_SCHEMA_V1_0 in clarify #3325

dosatos · 2022-08-23T14:04:53Z

Issue #, if available:

Description of changes:
When containers run, such schema validation is applied anyways.
So added ANALYSIS_CONFIG_SCHEMA_V1_0 to shorten feedback.

I also, added a flag for users to skip early validation. So they are not blocked in case we add a new feature in containers, but cannot manage deploying the update in SDK on time.

Testing done:

Added unit tests for
Used Schema containers use on the backend side.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2022-08-23T14:12:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e13997a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-23T14:47:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 37050d5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-23T14:48:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 37050d5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-23T14:49:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 37050d5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-23T14:50:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 37050d5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-23T15:17:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 4eeaee0
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-23T15:52:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 303e60b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-23T16:05:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 303e60b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-08-23T16:05:29Z

Codecov Report

Merging #3325 (60c42ae) into master (ace07d7) will decrease coverage by 0.84%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3325      +/-   ##
==========================================
- Coverage   90.02%   89.17%   -0.85%     
==========================================
  Files         868      204     -664     
  Lines       76852    18985   -57867     
==========================================
- Hits        69185    16930   -52255     
+ Misses       7667     2055    -5612

Impacted Files	Coverage Δ
src/sagemaker/clarify.py	`95.76% <100.00%> (ø)`
...b/python3.10/site-packages/sagemaker/amazon/lda.py
....9/site-packages/sagemaker/apiutils/_base_types.py
...n3.8/site-packages/sagemaker/tensorflow/serving.py
...thon3.9/site-packages/sagemaker/predictor_async.py
...hon3.7/site-packages/sagemaker/spark/processing.py
...ython3.10/site-packages/sagemaker/xgboost/model.py
...8/lib/python3.8/site-packages/sagemaker/clarify.py
...on3.10/site-packages/sagemaker/sparkml/__init__.py
...site-packages/sagemaker/debugger/metrics_config.py
... and 1063 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2022-08-23T16:14:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 303e60b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-23T16:19:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 303e60b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-23T16:32:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 303e60b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-24T06:35:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 19dbb0c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-24T06:41:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 19dbb0c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-24T06:55:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 19dbb0c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-24T07:13:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 19dbb0c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-24T07:20:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 19dbb0c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-24T10:32:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 0a69d86
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-24T10:40:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 0a69d86
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-24T10:53:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 0a69d86
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-24T11:10:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 0a69d86
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-24T11:16:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 0a69d86
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

larroy

I would leave the s3 validation for a separate change and get further feedback, and do the basic validation first. I think otherwise is good

sagemaker-bot · 2022-09-05T07:59:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: eb389fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-05T08:04:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: eb389fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-05T08:14:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: eb389fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-05T08:22:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: eb389fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-05T08:36:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: eb389fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

dosatos · 2022-09-05T09:26:13Z

not sure why, but the conflicts kept popping - I cleaned it again and re-based it from scratch (note: had to overwrite the commits, though). waiting for build stages to finish

sagemaker-bot · 2022-09-05T09:26:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 1a9b8a4
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-05T09:55:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 1a9b8a4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-05T10:08:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 1a9b8a4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-05T10:21:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 1a9b8a4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-05T10:29:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 1a9b8a4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-15T07:41:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 60c42ae
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-15T08:05:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 60c42ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-15T08:10:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 60c42ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-15T08:27:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 60c42ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-15T08:43:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 60c42ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-21T19:54:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 60c42ae
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-21T20:19:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 60c42ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-21T20:27:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 60c42ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-21T20:34:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 60c42ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-21T20:57:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 60c42ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-21T22:40:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 60c42ae
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-09-22T22:11:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 60c42ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Co-authored-by: Yeldos Balgabekov <[email protected]>

dosatos force-pushed the clarify/early_validation branch from 2d122b7 to e13997a Compare August 23, 2022 14:08

dosatos force-pushed the clarify/early_validation branch from e13997a to 37050d5 Compare August 23, 2022 14:45

dosatos force-pushed the clarify/early_validation branch from 37050d5 to 4eeaee0 Compare August 23, 2022 15:13

dosatos force-pushed the clarify/early_validation branch from 4eeaee0 to 303e60b Compare August 23, 2022 15:22

dosatos force-pushed the clarify/early_validation branch 2 times, most recently from e08e6ed to 19dbb0c Compare August 24, 2022 06:05

larroy reviewed Aug 24, 2022

View reviewed changes

dosatos force-pushed the clarify/early_validation branch from 0a69d86 to 4db22a0 Compare August 24, 2022 19:41

dosatos force-pushed the clarify/early_validation branch from 491b7a6 to eb389fe Compare September 5, 2022 07:29

dosatos force-pushed the clarify/early_validation branch from eb389fe to 1a9b8a4 Compare September 5, 2022 09:21

feat: added ANALYSIS_CONFIG_SCHEMA_V1_0 in clarify

60c42ae

dosatos force-pushed the clarify/early_validation branch from 1a9b8a4 to 60c42ae Compare September 15, 2022 07:36

shreyapandit approved these changes Sep 21, 2022

View reviewed changes

shreyapandit merged commit b4ea839 into aws:master Sep 22, 2022

JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this pull request Mar 4, 2023

feat: added ANALYSIS_CONFIG_SCHEMA_V1_0 in clarify (aws#3325)

91b764d

Co-authored-by: Yeldos Balgabekov <[email protected]>

feat: added ANALYSIS_CONFIG_SCHEMA_V1_0 in clarify #3325

feat: added ANALYSIS_CONFIG_SCHEMA_V1_0 in clarify #3325

Conversation

dosatos commented Aug 23, 2022 • edited Loading

Merge Checklist

General

Tests

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

codecov-commenter commented Aug 23, 2022 • edited Loading

Codecov Report

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 23, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 24, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 24, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 24, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 24, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 24, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 24, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 24, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 24, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 24, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Aug 24, 2022

AWS CodeBuild CI Report

larroy left a comment • edited Loading

Choose a reason for hiding this comment

sagemaker-bot commented Sep 5, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 5, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 5, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 5, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 5, 2022

AWS CodeBuild CI Report

dosatos commented Sep 5, 2022

sagemaker-bot commented Sep 5, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 5, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 5, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 5, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 5, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Sep 15, 2022

AWS CodeBuild CI Report

dosatos commented Aug 23, 2022 •

edited

Loading

codecov-commenter commented Aug 23, 2022 •

edited

Loading

larroy left a comment •

edited

Loading