change: Add PipelineVariable annotation in estimator, processing, tuner, transformer base classes #3182

qidewenwhen · 2022-06-20T17:55:20Z

Description of changes: This PR adds PipelineVariable annotation to base classes defined in estimator.py, processing.py, tuner.py, transformer.py. The PipelineVariable annotation will be leveraged by a test mechanism: #3163 to scan and check the incompatible issues between Pipeline steps and downstream classes caused by pipeline variables.

Testing done: Unit tests

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2022-06-20T18:02:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c7f3b0f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-20T18:19:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c7f3b0f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-20T18:31:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: c7f3b0f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-20T18:42:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: c7f3b0f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-06-20T19:00:07Z

Codecov Report

Merging #3182 (9c95260) into master (c70e30c) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3182      +/-   ##
==========================================
+ Coverage   88.78%   88.79%   +0.01%     
==========================================
  Files         202      202              
  Lines       17822    17833      +11     
==========================================
+ Hits        15823    15835      +12     
+ Misses       1999     1998       -1

Impacted Files	Coverage Δ
src/sagemaker/estimator.py	`89.56% <100.00%> (+0.01%)`	⬆️
src/sagemaker/network.py	`100.00% <100.00%> (ø)`
src/sagemaker/parameter.py	`88.37% <100.00%> (+0.56%)`	⬆️
src/sagemaker/processing.py	`94.32% <100.00%> (+0.03%)`	⬆️
src/sagemaker/transformer.py	`94.44% <100.00%> (+0.06%)`	⬆️
src/sagemaker/tuner.py	`94.62% <100.00%> (+0.02%)`	⬆️
src/sagemaker/session.py	`71.03% <0.00%> (+0.07%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c70e30c...9c95260. Read the comment docs.

sagemaker-bot · 2022-06-20T19:00:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: c7f3b0f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-20T20:09:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: c7f3b0f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-20T20:20:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c7f3b0f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-20T22:18:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: a3e860a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-20T22:44:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: a3e860a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-20T22:58:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: a3e860a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-20T23:09:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: a3e860a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-20T23:15:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: a3e860a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T01:07:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: a3e860a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T01:18:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: a3e860a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T01:55:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: d7b28c1
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T01:55:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: d7b28c1
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T01:55:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: d7b28c1
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T01:55:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e31c7e1
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T01:56:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: d7b28c1
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T02:16:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e31c7e1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T02:28:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e31c7e1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T02:33:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: e31c7e1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T02:59:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e31c7e1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-21T17:21:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e31c7e1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-28T17:50:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: d07c96c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-28T18:01:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: d07c96c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-28T18:09:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: d07c96c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-28T18:18:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: d07c96c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-28T18:31:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: d07c96c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-30T16:59:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 8c66831
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-30T17:07:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 8c66831
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-30T17:19:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 8c66831
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-30T17:20:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 8c66831
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-30T17:45:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 8c66831
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…ner, transformer base classes

sagemaker-bot · 2022-07-05T23:15:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 9c95260
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-05T23:24:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 9c95260
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-05T23:32:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 9c95260
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-05T23:54:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 9c95260
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-06T00:49:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 9c95260
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-06T01:42:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9c95260
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-06T04:35:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9c95260
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-06T16:22:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9c95260
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…ner, transformer base classes (aws#3182) Co-authored-by: Dewen Qi <[email protected]>

qidewenwhen requested a review from jerrypeng7773 June 20, 2022 18:31

qidewenwhen force-pushed the est_anno branch from c7f3b0f to a3e860a Compare June 20, 2022 20:50

qidewenwhen force-pushed the est_anno branch 2 times, most recently from d7b28c1 to e31c7e1 Compare June 21, 2022 01:50

qidewenwhen force-pushed the est_anno branch from e31c7e1 to 4ec6396 Compare June 21, 2022 21:24

qidewenwhen force-pushed the est_anno branch from eb6dc7e to d07c96c Compare June 28, 2022 17:23

navaj0 approved these changes Jun 28, 2022

View reviewed changes

qidewenwhen force-pushed the est_anno branch from d07c96c to 8c66831 Compare June 30, 2022 16:31

navinsoni approved these changes Jul 5, 2022

View reviewed changes

qidewenwhen removed the request for review from jerrypeng7773 July 5, 2022 21:52

change: Add PipelineVariable annotation in estimatory, processing, tu…

9c95260

…ner, transformer base classes

qidewenwhen force-pushed the est_anno branch from 8c66831 to 9c95260 Compare July 5, 2022 22:49

BasilBeirouti approved these changes Jul 6, 2022

View reviewed changes

BasilBeirouti merged commit 46c68a9 into aws:master Jul 6, 2022

navinsoni pushed a commit to navinsoni/sagemaker-python-sdk that referenced this pull request Jul 7, 2022

change: Add PipelineVariable annotation in estimatory, processing, tu…

14f11d8

…ner, transformer base classes (aws#3182) Co-authored-by: Dewen Qi <[email protected]>

jerrypeng7773 pushed a commit to jerrypeng7773/sagemaker-python-sdk that referenced this pull request Aug 8, 2022

change: Add PipelineVariable annotation in estimatory, processing, tu…

70c3e3e

…ner, transformer base classes (aws#3182) Co-authored-by: Dewen Qi <[email protected]>

JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this pull request Mar 4, 2023

change: Add PipelineVariable annotation in estimatory, processing, tu…

ae7b372

…ner, transformer base classes (aws#3182) Co-authored-by: Dewen Qi <[email protected]>

change: Add PipelineVariable annotation in estimator, processing, tuner, transformer base classes #3182

change: Add PipelineVariable annotation in estimator, processing, tuner, transformer base classes #3182

Conversation

qidewenwhen commented Jun 20, 2022

Merge Checklist

General

Tests

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

codecov-commenter commented Jun 20, 2022 • edited Loading

Codecov Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 21, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 28, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 28, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 28, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 28, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 28, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jun 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jul 5, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Jul 5, 2022

codecov-commenter commented Jun 20, 2022 •

edited

Loading