feature: add pipeline experiment config #2331

navaj0 · 2021-05-08T22:24:17Z

Issue #, if available:

Description of changes:

Add pipeline experiment config, which instructs pipeline to automatically create experiment and trial upon execution.

Testing done:

Unit tests
Integration tests

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2021-05-08T22:27:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 73e21a0
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-08T22:59:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 73e21a0
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-08T23:01:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 73e21a0
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-08T23:02:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 73e21a0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-08T23:08:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 73e21a0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-08T23:47:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: f5ebe8d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-09T00:12:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: f5ebe8d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-09T00:17:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: f5ebe8d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-09T00:19:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: f5ebe8d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-09T00:22:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: f5ebe8d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-10T21:31:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 078c30d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

dpatro · 2021-05-10T21:22:08Z

src/sagemaker/workflow/pipeline.py

+        pipeline_experiment_config (Optional[PipelineExperimentConfig]): If set,
+            the workflow will attempt to create an experiment and trial before
+            executing the steps. Creation will be skipped if an experiment or a trial with
+            the same name already exists.
+            If set to None, no experiment or trial will be created automatically.


nit: Should we include a comment for if not set? Something like If not set, by default the pipeline execution will pipeline-name as the experiment name and the execution-id as the trial name

Updated the doc.

sagemaker-bot · 2021-05-10T21:57:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 078c30d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-10T21:59:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 078c30d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-10T22:05:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 078c30d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-10T22:06:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 078c30d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-11T05:53:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 6a1ec81
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-11T06:19:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 6a1ec81
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-11T06:24:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 6a1ec81
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-11T06:28:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 6a1ec81
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-11T06:44:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 6a1ec81
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

danabens · 2021-05-11T13:06:49Z

src/sagemaker/workflow/pipeline.py

+            executing the steps. Creation will be skipped if an experiment or a trial with
+            the same name already exists. By default, pipeline name is used as


potentially the same trial could be used for multiple executions?

Yes. If users override the default config in that way.

danabens · 2021-05-11T13:09:36Z

src/sagemaker/workflow/pipeline.py

@@ -106,7 +119,7 @@ def _create_args(self, role_arn: str, description: str):

        Args:
            role_arn (str): The role arn that is assumed by pipelines to create step artifacts.
-            pipeline_description (str): A description of the pipeline.
+            description (str): A description of the pipeline.


nit - looks like this is just a doc change but ideally doc would be checked statically to avoid this

We do run docstyle, pylint. All passed with that error. Maybe some misconfiguration or docstyle doesn't check that at all.

sagemaker-bot · 2021-05-11T23:40:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: b8ef564
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-12T00:07:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: b8ef564
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-12T00:09:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: b8ef564
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-12T00:14:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b8ef564
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-12T00:25:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: b8ef564
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T15:01:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 331d24f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T15:27:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 331d24f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T15:33:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 331d24f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T15:37:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 331d24f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T15:58:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 331d24f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T16:46:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 331d24f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T17:47:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 331d24f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T20:29:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ba0e1da
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T20:55:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ba0e1da
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T20:58:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: ba0e1da
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T21:03:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ba0e1da
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-05-13T21:49:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: ba0e1da
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

navaj0 force-pushed the pipeline-experiment-config branch from 73e21a0 to f5ebe8d Compare May 8, 2021 23:37

navaj0 requested review from danabens and icywang86rui May 10, 2021 20:53

dpatro reviewed May 10, 2021

View reviewed changes

feature: add pipeline experiment config

6a1ec81

navaj0 force-pushed the pipeline-experiment-config branch from 078c30d to 6a1ec81 Compare May 11, 2021 05:43

danabens reviewed May 11, 2021

View reviewed changes

danabens approved these changes May 11, 2021

View reviewed changes

danabens reviewed May 11, 2021

View reviewed changes

dpatro approved these changes May 11, 2021

View reviewed changes

navaj0 requested a review from ajaykarpur May 11, 2021 23:05

Merge branch 'master' into pipeline-experiment-config

b8ef564

Merge branch 'master' into pipeline-experiment-config

331d24f

ajaykarpur approved these changes May 13, 2021

View reviewed changes

navaj0 removed the request for review from icywang86rui May 13, 2021 19:47

Merge branch 'master' into pipeline-experiment-config

ba0e1da

ajaykarpur merged commit cb16a1b into aws:master May 14, 2021

		executing the steps. Creation will be skipped if an experiment or a trial with
		the same name already exists. By default, pipeline name is used as

feature: add pipeline experiment config #2331

feature: add pipeline experiment config #2331

Uh oh!

Conversation

navaj0 commented May 8, 2021

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented May 8, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 8, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 8, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 8, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 8, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 8, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 9, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 9, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 9, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 9, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 10, 2021

AWS CodeBuild CI Report

Uh oh!

dpatro May 10, 2021

Choose a reason for hiding this comment

Uh oh!

navaj0 May 11, 2021

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented May 10, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 10, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 10, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 10, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 11, 2021

AWS CodeBuild CI Report

Uh oh!

danabens May 11, 2021

Choose a reason for hiding this comment

Uh oh!

navaj0 May 11, 2021

Choose a reason for hiding this comment