change: Enable load_run auto pass in experiment config #3586

qidewenwhen · 2023-01-12T03:00:43Z

Description of changes: Currently only the Run constructor support auto passing in experiment config to job env. This pr enables load_run auto pass in exp config

Testing done: Unit tests and integ tests

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2023-01-12T03:09:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 2b85225
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-12T03:42:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 2b85225
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-12T03:42:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2b85225
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-12T03:47:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 2b85225
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-01-12T04:00:59Z

Codecov Report

Merging #3586 (ee088f1) into master (5cb00a4) will decrease coverage by 0.79%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3586      +/-   ##
==========================================
- Coverage   89.59%   88.80%   -0.79%     
==========================================
  Files         968      227     -741     
  Lines       89584    22157   -67427     
==========================================
- Hits        80263    19677   -60586     
+ Misses       9321     2480    -6841

Impacted Files	Coverage Δ
src/sagemaker/experiments/run.py	`99.13% <100.00%> (ø)`
...ython3.10/site-packages/sagemaker/content_types.py
...on3.10/site-packages/sagemaker/mxnet/processing.py
...python3.7/site-packages/sagemaker/lambda_helper.py
...0/site-packages/sagemaker/huggingface/estimator.py
...ython3.7/site-packages/sagemaker/mxnet/defaults.py
...lib/python3.10/site-packages/sagemaker/pipeline.py
...3.10/site-packages/sagemaker/workflow/fail_step.py
....8/site-packages/sagemaker/huggingface/__init__.py
...10/site-packages/sagemaker/experiments/_metrics.py
... and 1186 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2023-01-12T04:01:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 2b85225
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-12T19:32:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 083f95c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-12T19:32:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 083f95c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-12T19:45:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 083f95c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-12T19:51:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 083f95c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-12T20:58:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 083f95c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-12T21:42:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 083f95c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T04:40:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 3f077dd
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T05:21:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 3f077dd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T05:24:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 3f077dd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T05:37:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 3f077dd
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T05:39:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 3f077dd
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-14T05:52:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 3f077dd
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-18T19:59:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 306bd57
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-18T20:00:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 306bd57
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-18T20:00:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 306bd57
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-18T20:22:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 306bd57
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-19T20:03:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 42d1f8d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-19T20:20:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 42d1f8d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-19T20:25:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 42d1f8d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-19T20:49:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 42d1f8d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

leoroy · 2023-01-23T18:52:11Z

tests/unit/sagemaker/experiments/test_run_context.py

+        assert _RunContext.get_current_run() == run_obj
+
+    expected_exp_config = run_obj.experiment_config
+    assert mock_start_job.call_args[0][9] == expected_exp_config


Does transform job arg use list instead of dict?

No, it does not use list (i.e. []) as job arg.
The method _Transform.start_new takes in *args, which means it takes arguments one by one according to their defined order in function signature (see here).

For any mock object (i.e. mock_start_job)'s call_args attribute, its index 0 is the *args and its index 1 is the **kwargs. Since _Transform.start_new takes in *args, we should verify mock_start_job.call_args[0]. Given that the 10th attribute experiment_config is what we care, we should assert mock_start_job.call_args[0][9] accordingly.

Ref:

Python args and kwargs: Demystified

sagemaker-bot · 2023-01-23T20:50:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ee088f1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-23T20:55:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ee088f1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-23T21:15:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: ee088f1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-23T21:20:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ee088f1
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-23T21:36:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: ee088f1
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-23T22:44:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ee088f1
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-23T23:32:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: ee088f1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T03:20:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ee088f1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T18:14:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: ee088f1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T18:14:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ee088f1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T18:15:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ee088f1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T18:15:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: ee088f1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T18:17:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ee088f1
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

qidewenwhen requested a review from a team as a code owner January 12, 2023 03:00

qidewenwhen requested review from navinsoni and removed request for a team January 12, 2023 03:00

qidewenwhen force-pushed the load branch from 2b85225 to 083f95c Compare January 12, 2023 18:50

qidewenwhen force-pushed the load branch from 083f95c to 3f077dd Compare January 14, 2023 04:37

knikure assigned navinsoni Jan 17, 2023

qidewenwhen requested a review from danabens January 18, 2023 18:35

qidewenwhen force-pushed the load branch from 3f077dd to 306bd57 Compare January 18, 2023 19:19

leoroy approved these changes Jan 23, 2023

View reviewed changes

change: Enable load_run auto pass in exp config

ee088f1

qidewenwhen force-pushed the load branch from 42d1f8d to ee088f1 Compare January 23, 2023 20:09

knikure approved these changes Jan 24, 2023

View reviewed changes

knikure changed the title ~~change: Enable load_run auto pass in exp config~~ change: Enable load_run auto pass in experiment config Jan 24, 2023

knikure merged commit cff14b8 into aws:master Jan 24, 2023

knikure assigned knikure and unassigned navinsoni and knikure Jan 24, 2023

JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this pull request Mar 4, 2023

change: Enable load_run auto pass in experiment config (aws#3586)

954b49d

nmadan pushed a commit to nmadan/sagemaker-python-sdk that referenced this pull request Apr 18, 2023

change: Enable load_run auto pass in experiment config (aws#3586)

081597f

qidewenwhen deleted the load branch January 25, 2024 18:33

change: Enable load_run auto pass in experiment config #3586

change: Enable load_run auto pass in experiment config #3586

Conversation

qidewenwhen commented Jan 12, 2023

Merge Checklist

General

Tests

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

codecov-commenter commented Jan 12, 2023 • edited Loading

Codecov Report

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 12, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 14, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 18, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 19, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 19, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 19, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 19, 2023

AWS CodeBuild CI Report

leoroy Jan 23, 2023

Choose a reason for hiding this comment

qidewenwhen Jan 23, 2023

Choose a reason for hiding this comment

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 24, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 24, 2023

codecov-commenter commented Jan 12, 2023 •

edited

Loading