fix: jsonGet interpolation issue 2426 + allow step depends on pass in step instance #2477

jerrypeng7773 · 2021-06-18T20:43:03Z

Issue #, if available:

Description of changes:

fix git issue 2426 JsonGet interpolation issue
allow users to pass in step instances in step dependsOn list.

Testing done:

unit and integ tests

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

… step instance

sagemaker-bot · 2021-06-18T20:45:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ae85184
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T20:47:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ae85184
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T21:18:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: ae85184
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T21:19:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ae85184
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T21:22:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: ae85184
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…integ test

sagemaker-bot · 2021-06-18T21:49:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 0853932
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T22:05:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 8a17276
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T22:23:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 0853932
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T22:25:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 0853932
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T22:30:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 0853932
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T22:31:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 8a17276
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T22:36:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 8a17276
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T22:37:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 8a17276
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T22:38:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 8a17276
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T23:08:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 395d650
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T23:33:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 395d650
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T23:42:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 395d650
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T23:43:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 395d650
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-18T23:43:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 395d650
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

staubhp · 2021-06-21T15:38:41Z

src/sagemaker/workflow/steps.py

-    def add_depends_on(self, step_names: List[str]):
-        """Add step names to the current step depends on list"""
+    def add_depends_on(self, step_names: Union[List[str], List["Step"]]):
+        """Add step names or stpe instances to the current step depends on list"""


sagemaker-bot · 2021-06-22T18:09:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5635282
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-22T18:33:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5635282
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-22T18:37:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5635282
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-22T18:42:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5635282
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-22T19:27:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 5635282
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T17:17:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5e27a41
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T17:43:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5e27a41
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T17:43:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5e27a41
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T17:43:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: b0dd7a1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T17:50:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5e27a41
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T18:11:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: b0dd7a1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T18:12:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: b0dd7a1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T18:16:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b0dd7a1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T18:37:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: b0dd7a1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T19:33:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 71d16b3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T19:59:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 71d16b3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T20:05:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 71d16b3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T20:08:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 71d16b3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-06-23T20:08:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 71d16b3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…ds on pass in step instance (aws#2477)" This reverts commit 4c0d3cf.

…ds on pass in step instance (#2477)" (#2490) This reverts commit 4c0d3cf. Co-authored-by: Wang Napieralski <[email protected]>

jerrypeng7773 added 2 commits June 18, 2021 13:26

fix: jsonGet interpolation issue 2426 + allow step depends on pass in…

0a1ac2f

… step instance

fix: remove debug print and add JsonGet input type check

ae85184

fix: fix lint and unused import + verify depends on step instance in …

0853932

…integ test

fix doc after moving JsonGet to pipeline functions class

8a17276

jerrypeng7773 requested review from danabens and staubhp June 18, 2021 21:55

fix integ test for test_workflow_with_clarify

395d650

staubhp reviewed Jun 21, 2021

View reviewed changes

staubhp previously approved these changes Jun 21, 2021

View reviewed changes

fix: fix typo in docstring

2fcf491

jerrypeng7773 dismissed staubhp’s stale review via 2fcf491 June 21, 2021 16:00

Merge branch 'master' into fix-issue-2426

5635282

Merge branch 'master' into fix-issue-2426

5e27a41

Merge branch 'master' into fix-issue-2426

b0dd7a1

EthanShouhanCheng approved these changes Jun 23, 2021

View reviewed changes

Merge branch 'master' into fix-issue-2426

71d16b3

ahsan-z-khan approved these changes Jun 23, 2021

View reviewed changes

ahsan-z-khan changed the title ~~fix: jsonGet interpolation issue 2426 + allow step depends on pass in…~~ fix: jsonGet interpolation issue 2426 + allow step depends on pass in step instance Jun 23, 2021

ahsan-z-khan merged commit 4c0d3cf into aws:master Jun 23, 2021

icywang86rui pushed a commit to icywang86rui/sagemaker-python-sdk that referenced this pull request Jun 27, 2021

fix: revert "fix: jsonGet interpolation issue 2426 + allow step depen…

f1a3893

…ds on pass in step instance (aws#2477)" This reverts commit 4c0d3cf.

icywang86rui mentioned this pull request Jun 27, 2021

fix: revert "fix: jsonGet interpolation issue 2426" #2490

Merged

7 tasks

icywang86rui added a commit that referenced this pull request Jun 27, 2021

fix: revert "fix: jsonGet interpolation issue 2426 + allow step depen…

1e9a576

…ds on pass in step instance (#2477)" (#2490) This reverts commit 4c0d3cf. Co-authored-by: Wang Napieralski <[email protected]>

fix: jsonGet interpolation issue 2426 + allow step depends on pass in step instance #2477

fix: jsonGet interpolation issue 2426 + allow step depends on pass in step instance #2477

Conversation

jerrypeng7773 commented Jun 18, 2021 • edited by ahsan-z-khan Loading

Merge Checklist

General

Tests

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 18, 2021

AWS CodeBuild CI Report

staubhp Jun 21, 2021

Choose a reason for hiding this comment

jerrypeng7773 Jun 21, 2021

Choose a reason for hiding this comment

sagemaker-bot commented Jun 22, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 22, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 22, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 22, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 22, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

AWS CodeBuild CI Report

sagemaker-bot commented Jun 23, 2021

jerrypeng7773 commented Jun 18, 2021 •

edited by ahsan-z-khan

Loading