fix(experiments/run.py): Stop duplication of RUN_TC_TAG on Consecutive Experiment Runs #3608

alex6499cat · 2023-01-24T15:27:25Z

Issue #, if available:

Description of changes:

Whenever you execute an experiment run, the RUN_TC_TAG is appended to your run tags via the _append_run_tc_label_to_tags function. If you pass the same variable for tags over more than one experiment run, the RUN_TC_TAG gets appended more than once to your tag variable. This will cause a duplicate tag exception unless you manually remove the RUN_TC_TAG after each run. This change is to make sure that the RUN_TC_TAG doesn't get duplicated if it already exists on your tags.

RUN_TC_TAG

 {
                "key": "sagemaker:trial-component-source",
                "value": "run"
}

Exception

ClientError: An error occurred (ValidationException) when calling the CreateTrialComponent operation: Duplicate tag key

Scenario to re-create the exception

tags = [
  {
    "Key": "cat",
    "Value": "cat"
  }
]
experiment_name = 'experiment-name-1'

timestamp = time.strftime("%Y-%m-%d-%H-%M-%S", time.gmtime())
run_name1 = f'{experiment_name}-{timestamp}-1'
run_name2 = f'{experiment_name}-{timestamp}-2'

with Run(experiment_name=experiment_name, run_name=run_name1, sagemaker_session=Session(),tags=tags) as run:
    run.log_parameters({"test":"test"})

with Run(experiment_name=experiment_name, run_name=run_name2, sagemaker_session=Session(),tags=tags) as run:
    run.log_parameters({"test":"test"})

Testing done:

Unit test updated for this scenario
Manually PIP installed the project after making my change and re-tested that the scenario above didn't fail after making my changes.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…to tags only if it doesn't exist in the tags in the _append_run_tc_label_to_tags function This change is to stop the RUN_TC_TAG from being duplicated if you run multiple experiments using the same variable for your tags

claytonparnell

/bot run all

sagemaker-bot · 2023-01-31T17:54:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 99b8c5d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

claytonparnell · 2023-01-31T18:13:13Z

@alex6499cat Thank you for this fix! Can you please remove the trailing whitespace in test_run.py:914

sagemaker-bot · 2023-01-31T18:32:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 99b8c5d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-31T18:43:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 99b8c5d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

alex6499cat · 2023-01-31T18:55:00Z

@alex6499cat Thank you for this fix! Can you please remove the trailing whitespace in test_run.py:914

You're welcome! I just made a commit to remove it. @claytonparnell

sagemaker-bot · 2023-01-31T18:57:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 99b8c5d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

claytonparnell

/bot run slow-tests

sagemaker-bot · 2023-01-31T19:59:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: f85af78
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-01-31T20:00:37Z

Codecov Report

Merging #3608 (f85af78) into master (5cb00a4) will decrease coverage by 0.80%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3608      +/-   ##
==========================================
- Coverage   89.59%   88.80%   -0.80%     
==========================================
  Files         968      227     -741     
  Lines       89584    22154   -67430     
==========================================
- Hits        80264    19674   -60590     
+ Misses       9320     2480    -6840

Impacted Files	Coverage Δ
src/sagemaker/experiments/run.py	`99.11% <100.00%> (ø)`
...3.9/site-packages/sagemaker/jumpstart/constants.py
...python3.8/site-packages/sagemaker/lineage/query.py
...thon3.9/site-packages/sagemaker/mxnet/estimator.py
...te-packages/sagemaker/workflow/pipeline_context.py
...s/sagemaker/tensorflow/training_compiler/config.py
.../sagemaker/cli/compatibility/v2/ast_transformer.py
...b/python3.7/site-packages/sagemaker/script_uris.py
.../lib/python3.9/site-packages/sagemaker/pipeline.py
.../lib/python3.8/site-packages/sagemaker/fw_utils.py
... and 1186 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2023-01-31T20:42:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 99b8c5d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala

/bot run all

sagemaker-bot · 2023-02-01T04:44:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: f85af78
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-02-01T04:45:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: f85af78
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-02-01T05:07:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: f85af78
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-02-01T06:55:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: f85af78
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-02-01T12:04:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: f85af78
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-02-01T20:12:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: f85af78
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-02-01T20:38:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: f85af78
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-02-01T22:23:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: f85af78
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…e Experiment Runs (aws#3608)

alex6499cat requested a review from a team as a code owner January 24, 2023 15:27

alex6499cat requested review from vikas0203 and removed request for a team January 24, 2023 15:27

knikure assigned vikas0203 Jan 26, 2023

claytonparnell reviewed Jan 31, 2023

View reviewed changes

refactor(test_run.py): remove white space

f85af78

claytonparnell reviewed Jan 31, 2023

View reviewed changes

mufaddal-rohawala reviewed Feb 1, 2023

View reviewed changes

mufaddal-rohawala approved these changes Feb 1, 2023

View reviewed changes

mufaddal-rohawala merged commit e5b47ea into aws:master Feb 1, 2023

JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this pull request Mar 4, 2023

fix(experiments/run.py): Stop duplication of RUN_TC_TAG on Consecutiv…

4dc5d51

…e Experiment Runs (aws#3608)

nmadan pushed a commit to nmadan/sagemaker-python-sdk that referenced this pull request Apr 18, 2023

fix(experiments/run.py): Stop duplication of RUN_TC_TAG on Consecutiv…

1d57e3d

…e Experiment Runs (aws#3608)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(experiments/run.py): Stop duplication of RUN_TC_TAG on Consecutive Experiment Runs #3608

fix(experiments/run.py): Stop duplication of RUN_TC_TAG on Consecutive Experiment Runs #3608

alex6499cat commented Jan 24, 2023

claytonparnell left a comment

sagemaker-bot commented Jan 31, 2023

claytonparnell commented Jan 31, 2023

sagemaker-bot commented Jan 31, 2023

sagemaker-bot commented Jan 31, 2023

alex6499cat commented Jan 31, 2023 •

edited

Loading

sagemaker-bot commented Jan 31, 2023

claytonparnell left a comment •

edited

Loading

sagemaker-bot commented Jan 31, 2023

codecov-commenter commented Jan 31, 2023 •

edited

Loading

sagemaker-bot commented Jan 31, 2023

mufaddal-rohawala left a comment

sagemaker-bot commented Feb 1, 2023

sagemaker-bot commented Feb 1, 2023

sagemaker-bot commented Feb 1, 2023

sagemaker-bot commented Feb 1, 2023

sagemaker-bot commented Feb 1, 2023

sagemaker-bot commented Feb 1, 2023

sagemaker-bot commented Feb 1, 2023

sagemaker-bot commented Feb 1, 2023

fix(experiments/run.py): Stop duplication of RUN_TC_TAG on Consecutive Experiment Runs #3608

fix(experiments/run.py): Stop duplication of RUN_TC_TAG on Consecutive Experiment Runs #3608

Conversation

alex6499cat commented Jan 24, 2023

Merge Checklist

General

Tests

claytonparnell left a comment

Choose a reason for hiding this comment

sagemaker-bot commented Jan 31, 2023

AWS CodeBuild CI Report

claytonparnell commented Jan 31, 2023

sagemaker-bot commented Jan 31, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Jan 31, 2023

AWS CodeBuild CI Report

alex6499cat commented Jan 31, 2023 • edited Loading

sagemaker-bot commented Jan 31, 2023

AWS CodeBuild CI Report

claytonparnell left a comment • edited Loading

Choose a reason for hiding this comment

sagemaker-bot commented Jan 31, 2023

AWS CodeBuild CI Report

codecov-commenter commented Jan 31, 2023 • edited Loading

Codecov Report

sagemaker-bot commented Jan 31, 2023

AWS CodeBuild CI Report

mufaddal-rohawala left a comment

Choose a reason for hiding this comment

sagemaker-bot commented Feb 1, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Feb 1, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Feb 1, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Feb 1, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Feb 1, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Feb 1, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Feb 1, 2023

AWS CodeBuild CI Report

sagemaker-bot commented Feb 1, 2023

AWS CodeBuild CI Report

alex6499cat commented Jan 31, 2023 •

edited

Loading

claytonparnell left a comment •

edited

Loading

codecov-commenter commented Jan 31, 2023 •

edited

Loading