change: Add PipelineVariable annotation in model base class and tensorflow estimator #3190

qidewenwhen · 2022-06-22T18:05:32Z

Description of changes: This PR does the following:

Add PipelineVariable annotation in Model and TensorFlow. The PipelineVariable annotation will be leveraged by a test mechanism: change: Implement test mechanism for Pipeline variables #3163 to scan and check the incompatible issues between Pipeline steps and downstream classes caused by pipeline variables.
Fix issues figured out by the test mechanism.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2022-06-22T18:32:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 900befd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T18:54:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 900befd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-06-22T18:54:32Z

Codecov Report

Merging #3190 (e1d252b) into master (284ddbe) will decrease coverage by 0.79%.
The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master    #3190      +/-   ##
==========================================
- Coverage   89.82%   89.03%   -0.80%     
==========================================
  Files         645      203     -442     
  Lines       55518    18306   -37212     
==========================================
- Hits        49871    16298   -33573     
+ Misses       5647     2008    -3639

Impacted Files	Coverage Δ
...agemaker/serverless/serverless_inference_config.py	`100.00% <ø> (ø)`
src/sagemaker/estimator.py	`89.24% <57.14%> (ø)`
src/sagemaker/tensorflow/estimator.py	`92.18% <75.00%> (ø)`
src/sagemaker/amazon/amazon_estimator.py	`88.43% <100.00%> (ø)`
src/sagemaker/debugger/debugger.py	`80.76% <100.00%> (ø)`
src/sagemaker/debugger/profiler_config.py	`95.83% <100.00%> (ø)`
src/sagemaker/drift_check_baselines.py	`100.00% <100.00%> (ø)`
.../sagemaker/huggingface/training_compiler/config.py	`94.11% <100.00%> (ø)`
src/sagemaker/inputs.py	`100.00% <100.00%> (ø)`
src/sagemaker/metadata_properties.py	`100.00% <100.00%> (ø)`
... and 851 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2022-06-22T19:00:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 900befd
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T19:11:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 900befd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T19:22:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 900befd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T19:51:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 74ca63e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T20:09:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e7e28f7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T20:18:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e7e28f7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T20:30:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e7e28f7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T20:31:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 74ca63e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T20:50:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: e7e28f7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T20:53:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 74ca63e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T21:10:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e7e28f7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-06-22T22:40:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 74ca63e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-01T20:43:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 6c5fb56
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-01T20:57:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: dff3260
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-01T21:20:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: dff3260
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-01T21:22:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: dff3260
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-01T21:23:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 6c5fb56
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-01T21:25:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 6c5fb56
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-01T21:27:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 6c5fb56
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-01T21:43:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: dff3260
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-07-01T21:44:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 6c5fb56
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-05T03:58:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c5eec9f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-05T04:14:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: c5eec9f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-05T04:18:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: c5eec9f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-05T04:22:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c5eec9f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-05T04:32:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: c5eec9f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

navaj0

Let's apply the annotations to the doc string as well.

sagemaker-bot · 2022-08-05T17:58:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: c5eec9f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

qidewenwhen · 2022-08-05T22:06:26Z

Let's apply the annotations to the doc string as well.

Synced up offline. Will open a separate PR for the doc string update as it's a big change.

…timator Model annotate update change: Add PipelineVariable annotation to composite argument of training go with model base and tf

sagemaker-bot · 2022-08-08T17:05:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e1d252b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-08T17:20:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e1d252b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-08T17:23:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: e1d252b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-08T17:29:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e1d252b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-08T17:51:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e1d252b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-08T20:41:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e1d252b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-09T17:26:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e1d252b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-11T17:22:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e1d252b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-12T16:21:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e1d252b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-14T01:05:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e1d252b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-08-15T16:44:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e1d252b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…timator (aws#3190) Model annotate update change: Add PipelineVariable annotation to composite argument of training go with model base and tf Co-authored-by: Dewen Qi <[email protected]>

qidewenwhen marked this pull request as draft June 22, 2022 18:05

qidewenwhen force-pushed the model_anno branch from 900befd to 74ca63e Compare June 22, 2022 19:47

qidewenwhen force-pushed the model_anno branch from 74ca63e to e7e28f7 Compare June 22, 2022 20:03

qidewenwhen force-pushed the model_anno branch 2 times, most recently from dff3260 to 6c5fb56 Compare July 1, 2022 20:39

qidewenwhen force-pushed the model_anno branch from 6c5fb56 to 8e36074 Compare August 2, 2022 04:23

qidewenwhen force-pushed the model_anno branch 2 times, most recently from 2a810b2 to c5eec9f Compare August 5, 2022 03:29

qidewenwhen marked this pull request as ready for review August 5, 2022 03:29

navaj0 reviewed Aug 5, 2022

View reviewed changes

navaj0 approved these changes Aug 5, 2022

View reviewed changes

change: Add Pipeline annotation in model base class and tensorflow es…

e1d252b

…timator Model annotate update change: Add PipelineVariable annotation to composite argument of training go with model base and tf

qidewenwhen force-pushed the model_anno branch from c5eec9f to e1d252b Compare August 8, 2022 16:35

shreyapandit approved these changes Aug 8, 2022

View reviewed changes

qidewenwhen changed the title ~~change: Add Pipeline annotation in model base class and tensorflow estimator~~ change: Add PipelineVariable annotation in model base class and tensorflow estimator Aug 12, 2022

shreyapandit merged commit 8d7dd32 into aws:master Aug 15, 2022

qidewenwhen deleted the model_anno branch January 25, 2024 18:33

change: Add PipelineVariable annotation in model base class and tensorflow estimator #3190

change: Add PipelineVariable annotation in model base class and tensorflow estimator #3190

Uh oh!

Conversation

qidewenwhen commented Jun 22, 2022

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Jun 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 1, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 1, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 1, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 1, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 1, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 1, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 1, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 1, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 1, 2022

codecov-commenter commented Jun 22, 2022 •

edited

Loading