documentation: the SageMaker distributed data parallel v1.4.0 release #2980

mchoi8739 · 2022-03-08T19:55:32Z

Description of changes:

Doc renewal for the the SageMaker distributed data parallel library v1.4.0 release.
Added release note for SMDDP v1.4.0
Made doc updates to the framework estimator classes to present correct distribution config options
Archived the previous version of the doc
Made structural changes

Testing done:
Locally built.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…ion for lambda step (aws#2824)

Co-authored-by: Ahsan Khan <[email protected]>

Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Ahsan Khan <[email protected]> Co-authored-by: Shreya Pandit <[email protected]> Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Mohamed Ali Jamaoui <[email protected]> Co-authored-by: ci <ci> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: sreedes <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Miyoung <[email protected]> Co-authored-by: Ameen Khan <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Xiaoguang Chen <[email protected]> Co-authored-by: Jonathan Guinegagne <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Yifei Zhu <[email protected]> Co-authored-by: Qingzi-Lan <[email protected]>

…o dev

…o smddp-1.4.0-doc-sync-dev

Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: Dewen Qi <[email protected]>

Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]>

…ws#2950) Co-authored-by: Payton Staub <[email protected]>

…ws#2971)

Zha0q1

Thank you!

sagemaker-bot · 2022-03-08T21:34:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e1b77a7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T21:34:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 8095031
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

roywei

LGTM, reviewed staging website as well. Thank you!

sagemaker-bot · 2022-03-08T21:44:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: bc86c5a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T22:03:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 8095031
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T22:03:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: bc86c5a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T22:04:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e1b77a7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-08T22:09:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 4a885d7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-09T00:49:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ad068b6
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-09T01:44:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ad068b6
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-09T01:47:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ad068b6
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mchoi8739 · 2022-03-09T20:38:06Z

@ahsan-z-khan why did you change the base branch to master? Isn't the master branch deprecated?

ahsan-z-khan · 2022-03-09T22:46:54Z

@ahsan-z-khan why did you change the base branch to master? Isn't the master branch deprecated?

From today, we have started using master branch. as going forward all of the releases will be from master branch.

ahsan-z-khan

Can you rebase so it shows only your commits?

mchoi8739 · 2022-03-09T22:49:12Z

Can you rebase so it shows only your commits?

I think your rebasing worked fine and I don't need to do it from my side again. Having your commits does not affect anything, so it looks fine to me.

sagemaker-bot · 2022-03-09T23:08:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 43db88b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-09T23:47:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 43db88b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-10T00:54:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 43db88b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…ker-python-sdk into smddp-1.4.0-doc

sagemaker-bot · 2022-03-10T19:28:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: b85f772
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-10T20:20:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b85f772
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-10T21:06:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: b85f772
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…aws#2980) Co-authored-by: Ahsan Khan <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Shreya Pandit <[email protected]> Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Mohamed Ali Jamaoui <[email protected]> Co-authored-by: ci <ci> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: sreedes <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Miyoung <[email protected]> Co-authored-by: Ameen Khan <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Xiaoguang Chen <[email protected]> Co-authored-by: Jonathan Guinegagne <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Yifei Zhu <[email protected]> Co-authored-by: Qingzi-Lan <[email protected]> Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Dewen Qi <[email protected]> Co-authored-by: qidewenwhen <[email protected]> Co-authored-by: Xinghan Chen <[email protected]> Co-authored-by: Tulio Casagrande <[email protected]> Co-authored-by: HappyAmazonian <[email protected]>

xchen909 and others added 30 commits January 10, 2022 15:02

fix: fixes unnecessary session call while generating pipeline definit…

8210375

…ion for lambda step (aws#2824)

feature: Add models_v2 under lineage context (aws#2800)

972a6d2

feature: enable python 3.9 (aws#2802)

7206b9e

Co-authored-by: Ahsan Khan <[email protected]>

change: Update CHANGELOG.md (aws#2842)

127c964

doc: Document the available ExecutionVariables (aws#2807)

88e4d68

fix: Remove duplicate vertex/edge in query lineage (aws#2784)

b3c19d8

fix: Update Static Endpoint (aws#2931)

b591959

Add exception in test_action (aws#2938)

3c5ea3a

change: pin test dependencies (aws#2929)

9d84e2e

feature: Add FailStep Support for Sagemaker Pipeline (aws#2872)

cede5fa

change: use recommended inference image uri from Neo API (aws#2923)

a765512

Merge branch 'dev' of https://github.com/aws/sagemaker-python-sdk int…

4a1c4df

…o dev

archive previous doc

3833310

restructure the herring api doc

cb9d235

update pytorch page

5e4128e

polish doc style and structure

5159c85

fix: jumpstart model table (aws#2954)

6670e30

update distributed training doc

9a5e8bc

Merge branch 'dev' of https://github.com/aws/sagemaker-python-sdk int…

0026f6b

…o smddp-1.4.0-doc-sync-dev

change: update code to get commit_id in codepipeline (aws#2961)

f166b60

feature: Data Serializer (aws#2956)

086258d

change: reorganize test files for workflow (aws#2960)

a39b750

Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: Dewen Qi <[email protected]>

feature: TensorFlow 2.4 for Neo (aws#2861)

28fd737

Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]>

fix: Remove sagemaker_job_name from hyperparameters in TrainingStep (a…

20df3d7

…ws#2950) Co-authored-by: Payton Staub <[email protected]>

fix: Style update in DataSerializer (aws#2962)

b9f90dc

documentation: sync upstream

35863e2

documentation: add ref

ee0757d

documentation: smddp doc update (aws#2968)

6db3774

fix: container env generation for S3 URI and add test for the same (a…

d610bfb

…ws#2971)

Zha0q1 approved these changes Mar 8, 2022

View reviewed changes

roywei approved these changes Mar 8, 2022

View reviewed changes

Trigger Build

ad068b6

ahsan-z-khan changed the base branch from dev to master March 9, 2022 19:35

Merge branch 'master' into smddp-1.4.0-doc

43db88b

ahsan-z-khan reviewed Mar 9, 2022

View reviewed changes

mchoi8739 added 2 commits March 10, 2022 11:02

Trigger Build

4e19986

Merge branch 'smddp-1.4.0-doc' of https://github.com/mchoi8739/sagema…

b85f772

…ker-python-sdk into smddp-1.4.0-doc

ahsan-z-khan approved these changes Mar 10, 2022

View reviewed changes

ahsan-z-khan merged commit 34cb1c7 into aws:master Mar 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

documentation: the SageMaker distributed data parallel v1.4.0 release #2980

documentation: the SageMaker distributed data parallel v1.4.0 release #2980

mchoi8739 commented Mar 8, 2022 •

edited

Loading

Zha0q1 left a comment

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

roywei left a comment

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 8, 2022

sagemaker-bot commented Mar 9, 2022

sagemaker-bot commented Mar 9, 2022

sagemaker-bot commented Mar 9, 2022

mchoi8739 commented Mar 9, 2022

ahsan-z-khan commented Mar 9, 2022

ahsan-z-khan left a comment

mchoi8739 commented Mar 9, 2022 •

edited

Loading

sagemaker-bot commented Mar 9, 2022

sagemaker-bot commented Mar 9, 2022

sagemaker-bot commented Mar 10, 2022

sagemaker-bot commented Mar 10, 2022

sagemaker-bot commented Mar 10, 2022

sagemaker-bot commented Mar 10, 2022

documentation: the SageMaker distributed data parallel v1.4.0 release #2980

documentation: the SageMaker distributed data parallel v1.4.0 release #2980

Conversation

mchoi8739 commented Mar 8, 2022 • edited Loading

Merge Checklist

General

Tests

Zha0q1 left a comment

Choose a reason for hiding this comment

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

roywei left a comment

Choose a reason for hiding this comment

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 8, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 9, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 9, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 9, 2022

AWS CodeBuild CI Report

mchoi8739 commented Mar 9, 2022

ahsan-z-khan commented Mar 9, 2022

ahsan-z-khan left a comment

Choose a reason for hiding this comment

mchoi8739 commented Mar 9, 2022 • edited Loading

sagemaker-bot commented Mar 9, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 9, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 10, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 10, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 10, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Mar 10, 2022

AWS CodeBuild CI Report

mchoi8739 commented Mar 8, 2022 •

edited

Loading

mchoi8739 commented Mar 9, 2022 •

edited

Loading