fix: FrameworkProcessor S3 uploads #3493

brockwade633 · 2022-11-30T00:57:50Z

Issue #, if available:
#3467

Description of changes:
These changes address an issue where the runproc.sh artifact generated by FrameworkProcessors was getting uploaded to the wrong directory and thus getting overwritten.

Note: Some file changes are actually duplicated from this PR: #3460, to fix failing tests. The actual files up for review are:

src/sagemaker/processing.py
src/sagemaker/workflow/utilities.py
tests/data/framework_processor_data/*
tests/integ/sagemaker/workflow/test_workflow.py

Testing done:
Add integ test that verifies runproc.sh artifact s3 paths are distinct from others and uploaded to separate directories, and that each runproc.sh script contains the right command

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2022-11-30T01:04:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 71adb25
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T01:37:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 71adb25
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-11-30T01:39:46Z

Codecov Report

Merging #3493 (585b8df) into master (1fa2377) will decrease coverage by 0.28%.
The diff coverage is 87.63%.

@@            Coverage Diff             @@
##           master    #3493      +/-   ##
==========================================
- Coverage   89.17%   88.88%   -0.29%     
==========================================
  Files         204      213       +9     
  Lines       18979    20557    +1578     
==========================================
+ Hits        16924    18272    +1348     
- Misses       2055     2285     +230

Impacted Files	Coverage Δ
src/sagemaker/feature_store/feature_group.py	`76.52% <ø> (ø)`
src/sagemaker/huggingface/model.py	`89.87% <ø> (ø)`
src/sagemaker/pipeline.py	`85.33% <ø> (ø)`
src/sagemaker/sklearn/model.py	`92.98% <ø> (-1.76%)`	⬇️
src/sagemaker/sparkml/model.py	`100.00% <ø> (ø)`
...sagemaker/workflow/monitor_batch_transform_step.py	`0.00% <0.00%> (ø)`
src/sagemaker/xgboost/model.py	`96.36% <ø> (-1.82%)`	⬇️
...agemaker/model_monitor/clarify_model_monitoring.py	`89.84% <27.58%> (-5.09%)`	⬇️
src/sagemaker/transformer.py	`85.41% <40.62%> (-9.07%)`	⬇️
.../sagemaker/huggingface/training_compiler/config.py	`90.47% <50.00%> (ø)`
... and 56 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2022-11-30T02:16:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 71adb25
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T02:21:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 71adb25
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T02:23:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 71adb25
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T02:40:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 71adb25
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T02:44:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 71adb25
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T07:15:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2531a1c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T07:21:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 2531a1c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T07:22:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 2531a1c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T07:33:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 2531a1c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T07:42:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 2531a1c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T09:08:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 92a8eea
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-30T09:20:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 92a8eea
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-07T04:43:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 89e66df
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-07T05:21:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 89e66df
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-07T05:22:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 89e66df
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-07T05:34:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 89e66df
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-07T05:43:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 89e66df
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-07T07:59:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 585b8df
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-07T08:04:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 585b8df
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-07T08:18:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 585b8df
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-07T08:23:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 585b8df
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-07T08:28:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 585b8df
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

brockwade633 · 2022-12-07T16:18:01Z

@brockwade633 the integ sagemaker-python-sdk-pr tests are failing on unconfigured AWS region in session

Fixed

mufaddal-rohawala

/bot run slow-tests

sagemaker-bot · 2022-12-08T03:51:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 585b8df
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-08T03:55:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 585b8df
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-08T04:09:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 585b8df
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-08T04:17:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 585b8df
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-12-08T04:19:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 585b8df
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Co-authored-by: Brock Wade <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]>

Brock Wade added 13 commits November 7, 2022 05:44

bug-fix: support idempotency for framework and spark processors

85d5be2

change: update test path

645c938

change: try using os path

864a90a

Merge remote-tracking branch 'upstream/master' into runproc-bug-fix

2a1274c

fix: frameworkprocessor side effects, testing

db69b5f

fix: unrelated test suite pollution, and update spark base class

8d7f82e

Merge branch 'master' into step-args-bug-fix

c2be61f

merge latest

59d9b42

fix: more testing

54dd242

retain same function return type

bb9a044

Merge branch 'step-args-bug-fix' into runproc-bug-fix

c8966c7

testing

623d85b

fix: formatting

71adb25

fix: extend testing

2531a1c

fix: add region_name to test

92a8eea

change: re-spawn tests

89e66df

Brock Wade added 2 commits December 6, 2022 23:21

fix: add pipeline session s3 downloader

7732b9e

Merge branch 'master' into runproc-bug-fix

585b8df

knikure approved these changes Dec 7, 2022

View reviewed changes

mufaddal-rohawala approved these changes Dec 8, 2022

View reviewed changes

mufaddal-rohawala reviewed Dec 8, 2022

View reviewed changes

knikure changed the title ~~Bug-fix: FrameworkProcessor S3 uploads~~ fix: FrameworkProcessor S3 uploads Dec 8, 2022

knikure merged commit a28d1dd into aws:master Dec 8, 2022

mizanfiu pushed a commit to mizanfiu/sagemaker-python-sdk that referenced this pull request Dec 13, 2022

fix: FrameworkProcessor S3 uploads (aws#3493)

98f9406

Co-authored-by: Brock Wade <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]>

mufaddal-rohawala added a commit to mufaddal-rohawala/sagemaker-python-sdk that referenced this pull request Dec 19, 2022

fix: FrameworkProcessor S3 uploads (aws#3493)

881c113

Co-authored-by: Brock Wade <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]>

mufaddal-rohawala added a commit that referenced this pull request Dec 20, 2022

fix: FrameworkProcessor S3 uploads (#3493)

50b0c5e

Co-authored-by: Brock Wade <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]>

JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this pull request Mar 4, 2023

fix: FrameworkProcessor S3 uploads (aws#3493)

0fc6c5d

Co-authored-by: Brock Wade <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]>

JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this pull request Mar 4, 2023

fix: FrameworkProcessor S3 uploads (aws#3493)

e04e2e8

Co-authored-by: Brock Wade <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]>

nmadan pushed a commit to nmadan/sagemaker-python-sdk that referenced this pull request Apr 18, 2023

fix: FrameworkProcessor S3 uploads (aws#3493)

5dc8468

Co-authored-by: Brock Wade <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: FrameworkProcessor S3 uploads #3493

fix: FrameworkProcessor S3 uploads #3493

brockwade633 commented Nov 30, 2022 •

edited

Loading

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

codecov-commenter commented Nov 30, 2022 •

edited

Loading

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Nov 30, 2022

sagemaker-bot commented Dec 7, 2022

sagemaker-bot commented Dec 7, 2022

sagemaker-bot commented Dec 7, 2022

sagemaker-bot commented Dec 7, 2022

sagemaker-bot commented Dec 7, 2022

sagemaker-bot commented Dec 7, 2022

sagemaker-bot commented Dec 7, 2022

sagemaker-bot commented Dec 7, 2022

sagemaker-bot commented Dec 7, 2022

sagemaker-bot commented Dec 7, 2022

brockwade633 commented Dec 7, 2022

mufaddal-rohawala left a comment

sagemaker-bot commented Dec 8, 2022

sagemaker-bot commented Dec 8, 2022

sagemaker-bot commented Dec 8, 2022

sagemaker-bot commented Dec 8, 2022

sagemaker-bot commented Dec 8, 2022

fix: FrameworkProcessor S3 uploads #3493

fix: FrameworkProcessor S3 uploads #3493

Conversation

brockwade633 commented Nov 30, 2022 • edited Loading

Merge Checklist

General

Tests

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

codecov-commenter commented Nov 30, 2022 • edited Loading

Codecov Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 30, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 7, 2022

AWS CodeBuild CI Report

brockwade633 commented Dec 7, 2022

mufaddal-rohawala left a comment

Choose a reason for hiding this comment

sagemaker-bot commented Dec 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Dec 8, 2022

AWS CodeBuild CI Report

brockwade633 commented Nov 30, 2022 •

edited

Loading

codecov-commenter commented Nov 30, 2022 •

edited

Loading