fix: Fix bug forcing uploaded tar to be named sourcedir #3412

claytonparnell · 2022-10-13T00:23:40Z

Issue #, if available:

Description of changes: Bug was found which would upload bootstrapping code to override the source directory if it was named anything other than 'sourcedir.tar.gz'. Fix allows source directory to have any name.

EDIT 11/30 - It seems there are several places in the background which require the source directory to be named sourcedir.tar.gz. Rather than trying to refactor everything, I added documentation clarifying the requirement and validation which raises an error when misnamed. Also added testing of validation.

Testing done: Ran customer workflow using new update, confirmed that the uploaded source directory was not replaced, and bootstrapping code was separately uploaded as 'runproc.sh'. Also ran tox tests.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

claytonparnell

/bot run all

sagemaker-bot · 2022-10-13T00:28:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5ffe718
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-13T01:10:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5ffe718
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-13T01:19:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5ffe718
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-13T01:28:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 5ffe718
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-13T01:31:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5ffe718
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

claytonparnell

/bot run pr, local-mode-tests

sagemaker-bot · 2022-10-13T15:53:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5ffe718
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-13T16:36:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5ffe718
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-13T16:39:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 5ffe718
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-13T17:00:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5ffe718
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-13T18:42:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5ffe718
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-19T17:13:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 51f275a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-19T17:53:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 51f275a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-19T18:19:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 51f275a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-19T18:19:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 51f275a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-19T19:43:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 51f275a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

vikas0203

/bot run all

sagemaker-bot · 2022-10-19T20:02:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 51f275a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-19T20:03:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 51f275a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-19T20:43:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 51f275a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-19T21:00:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 51f275a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-19T22:18:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 8d91406
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-10-19T22:30:07Z

Codecov Report

Merging #3412 (fbb73ae) into master (1fa2377) will decrease coverage by 0.35%.
The diff coverage is 83.55%.

❗ Current head fbb73ae differs from pull request most recent head 77d6fe5. Consider uploading reports for the commit 77d6fe5 to get more accurate results

@@            Coverage Diff             @@
##           master    #3412      +/-   ##
==========================================
- Coverage   89.17%   88.81%   -0.36%     
==========================================
  Files         204      205       +1     
  Lines       18979    19461     +482     
==========================================
+ Hits        16924    17285     +361     
- Misses       2055     2176     +121

Impacted Files	Coverage Δ
src/sagemaker/automl/automl.py	`91.32% <ø> (ø)`
src/sagemaker/huggingface/model.py	`89.87% <ø> (ø)`
src/sagemaker/pipeline.py	`85.33% <ø> (ø)`
src/sagemaker/sklearn/model.py	`92.98% <ø> (-1.76%)`	⬇️
...sagemaker/workflow/monitor_batch_transform_step.py	`0.00% <0.00%> (ø)`
src/sagemaker/xgboost/model.py	`96.36% <ø> (-1.82%)`	⬇️
...agemaker/model_monitor/clarify_model_monitoring.py	`89.84% <27.58%> (-5.09%)`	⬇️
src/sagemaker/session.py	`71.48% <54.54%> (-0.19%)`	⬇️
src/sagemaker/training_compiler/config.py	`80.43% <60.00%> (-2.90%)`	⬇️
src/sagemaker/pytorch/estimator.py	`94.73% <63.63%> (-5.27%)`	⬇️
... and 38 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2022-10-19T22:33:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 8d91406
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-19T22:53:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 8d91406
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-20T00:41:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 8d91406
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-10-20T01:59:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 8d91406
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T19:24:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: fbb73ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T19:32:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: fbb73ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T19:41:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: fbb73ae
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T22:17:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: fbb73ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T22:22:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: fbb73ae
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T22:25:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: fbb73ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T22:35:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: fbb73ae
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T22:46:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: fbb73ae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-01T02:01:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 77d6fe5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-01T02:35:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 77d6fe5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-01T03:02:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 77d6fe5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-01T03:24:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 77d6fe5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-01T04:48:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 77d6fe5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-02T02:48:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 77d6fe5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

claytonparnell changed the title ~~Fix bug forcing uploaded tar to be named sourcedir~~ fix: Fix bug forcing uploaded tar to be named sourcedir Oct 13, 2022

claytonparnell commented Oct 13, 2022

View reviewed changes

claytonparnell force-pushed the sourcedir-bugfix branch from 5ffe718 to 51f275a Compare October 19, 2022 17:09

vikas0203 reviewed Oct 19, 2022

View reviewed changes

claytonparnell requested a review from navinsoni November 30, 2022 21:37

claytonparnell and others added 5 commits November 30, 2022 17:55

Fix bug forcing uploaded tar to be named sourcedir

1ab7aeb

Fix parenthesis causing black-check fail

b4859b0

Shift approach to requiring/validating name sourcedir

5d98320

add testing

2e699d7

Fix black formatting issue

77d6fe5

knikure force-pushed the sourcedir-bugfix branch from fbb73ae to 77d6fe5 Compare December 1, 2022 01:55

knikure approved these changes Dec 2, 2022

View reviewed changes

knikure merged commit 7f9f3b0 into aws:master Dec 2, 2022

mizanfiu pushed a commit to mizanfiu/sagemaker-python-sdk that referenced this pull request Dec 13, 2022

fix: Fix bug forcing uploaded tar to be named sourcedir (aws#3412)

a00e3ff

mufaddal-rohawala pushed a commit to mufaddal-rohawala/sagemaker-python-sdk that referenced this pull request Dec 19, 2022

fix: Fix bug forcing uploaded tar to be named sourcedir (aws#3412)

39dd6bb

mufaddal-rohawala pushed a commit that referenced this pull request Dec 20, 2022

fix: Fix bug forcing uploaded tar to be named sourcedir (#3412)

b716e67

JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this pull request Mar 4, 2023

fix: Fix bug forcing uploaded tar to be named sourcedir (aws#3412)

45e1f62

JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this pull request Mar 4, 2023

fix: Fix bug forcing uploaded tar to be named sourcedir (aws#3412)

930d28f

nmadan pushed a commit to nmadan/sagemaker-python-sdk that referenced this pull request Apr 18, 2023

fix: Fix bug forcing uploaded tar to be named sourcedir (aws#3412)

e655b20

nmadan pushed a commit to nmadan/sagemaker-python-sdk that referenced this pull request Apr 18, 2023

fix: Fix bug forcing uploaded tar to be named sourcedir (aws#3412)

0b64464

fix: Fix bug forcing uploaded tar to be named sourcedir #3412

fix: Fix bug forcing uploaded tar to be named sourcedir #3412

Conversation

claytonparnell commented Oct 13, 2022 • edited Loading

Merge Checklist

General

Tests

claytonparnell left a comment

Choose a reason for hiding this comment

sagemaker-bot commented Oct 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 13, 2022

AWS CodeBuild CI Report

claytonparnell left a comment

Choose a reason for hiding this comment

sagemaker-bot commented Oct 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

vikas0203 left a comment

Choose a reason for hiding this comment

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

codecov-commenter commented Oct 19, 2022 • edited Loading

Codecov Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 19, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Oct 20, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 1, 2022

claytonparnell commented Oct 13, 2022 •

edited

Loading

codecov-commenter commented Oct 19, 2022 •

edited

Loading