feat: add Tensorflow and Pytorch version for SM Training Compiler and expand to regular regions #3045

access2rohit · 2022-04-04T19:10:06Z

Description of changes:

Update tensorflow version for Training Compiler to 2.6.3 and Pytorch to 1.10.2
Added unit test for Tensorflow.
Added all supported regions for integration tests

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

codecov-commenter · 2022-04-04T19:27:21Z

Codecov Report

Merging #3045 (43fdd48) into master (fd753c3) will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3045      +/-   ##
==========================================
+ Coverage   89.38%   89.39%   +0.01%     
==========================================
  Files         197      197              
  Lines       16670    16670              
==========================================
+ Hits        14900    14902       +2     
+ Misses       1770     1768       -2

Impacted Files	Coverage Δ
src/sagemaker/image_uris.py	`97.95% <0.00%> (+0.51%)`	⬆️
src/sagemaker/huggingface/estimator.py	`93.02% <0.00%> (+1.16%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fd753c3...43fdd48. Read the comment docs.

sagemaker-bot · 2022-04-04T19:35:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: f2e961f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-04T20:11:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: f2e961f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

access2rohit · 2022-04-13T18:24:46Z

@Lokiiiiii can you review Pytorch related changes.

sagemaker-bot · 2022-04-13T18:47:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 68eacf2
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T18:47:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: d4a8eeb
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T19:08:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: d4a8eeb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T19:10:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: d4a8eeb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T19:22:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 68eacf2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T19:27:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 565051f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T19:33:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 565051f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T19:34:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 68eacf2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T19:34:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 565051f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T19:41:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c5be9b5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T20:11:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: a547e7a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T20:38:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: a547e7a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-13T20:51:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: c5be9b5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

integ tests failing

sagemaker-bot · 2022-04-14T01:20:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 7c5c2d5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-14T04:50:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 49fb908
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…ll supported regions for integration tests

sagemaker-bot · 2022-04-14T23:39:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: aa202cd
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-15T00:43:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: aa202cd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-15T00:48:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: aa202cd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-15T00:59:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: b130ad4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-15T02:09:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b130ad4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-18T18:38:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 08058f9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-18T22:05:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 43fdd48
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-18T22:31:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 43fdd48
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-18T23:12:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 43fdd48
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

access2rohit · 2022-04-18T23:14:08Z

I have fixed all the CI test failures. Can I get 2 reviews on this?

access2rohit · 2022-04-19T03:57:35Z

Thank you @BasilBeirouti

… expand to regular regions (aws#3045) update tf version for training compiler and expand to regular regions

ahsan-z-khan marked this pull request as draft April 5, 2022 17:54

access2rohit changed the title ~~[WIP]update tf version for training compiler and expand to regular regions~~ Update tf version for training compiler and expand to regular regions Apr 13, 2022

access2rohit marked this pull request as ready for review April 13, 2022 18:20

access2rohit changed the title ~~Update tf version for training compiler and expand to regular regions~~ Update tensorflow version for SM Training Compiler and expand to regular regions Apr 13, 2022

access2rohit changed the title ~~Update tensorflow version for SM Training Compiler and expand to regular regions~~ Update Tensorflow and Pytorch version for SM Training Compiler and expand to regular regions Apr 13, 2022

ahsan-z-khan requested a review from navinsoni April 13, 2022 18:30

ahsan-z-khan previously approved these changes Apr 13, 2022

View reviewed changes

ahsan-z-khan changed the title ~~Update Tensorflow and Pytorch version for SM Training Compiler and expand to regular regions~~ feat: add Tensorflow and Pytorch version for SM Training Compiler and expand to regular regions Apr 13, 2022

access2rohit force-pushed the trcomp-tf-2.6.3 branch 2 times, most recently from 565051f to c5be9b5 Compare April 13, 2022 19:15

access2rohit force-pushed the trcomp-tf-2.6.3 branch from a547e7a to 7c5c2d5 Compare April 14, 2022 01:05

Rohit Kumar Srivastava added 5 commits April 14, 2022 23:23

update tf version for training compiler and expand to regular regions

4f31c86

removing gov cloud regions and making pytorch container cuda-11.3

796f2cd

add unit tests for tensorflow and update version for pytorch. Added a…

f34d2ac

…ll supported regions for integration tests

nit:fixing alignment

b176fdc

fixing tf and pt unit tests

86f3594

access2rohit force-pushed the trcomp-tf-2.6.3 branch from 49fb908 to aa202cd Compare April 14, 2022 23:23

fixing black-check

b130ad4

access2rohit force-pushed the trcomp-tf-2.6.3 branch from aa202cd to b130ad4 Compare April 15, 2022 00:34

fix integ tests for tensorflow

08058f9

Lokiiiiii approved these changes Apr 18, 2022

View reviewed changes

updating names to trcomp

43fdd48

navinsoni requested a review from ahsan-z-khan April 18, 2022 23:17

navinsoni approved these changes Apr 18, 2022

View reviewed changes

BasilBeirouti approved these changes Apr 19, 2022

View reviewed changes

BasilBeirouti merged commit 4b88ea8 into aws:master Apr 19, 2022

access2rohit deleted the trcomp-tf-2.6.3 branch April 19, 2022 03:57

mufaddal-rohawala mentioned this pull request Apr 19, 2022

fix: integs for training compiler in non-PDX regions #3073

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add Tensorflow and Pytorch version for SM Training Compiler and expand to regular regions #3045

feat: add Tensorflow and Pytorch version for SM Training Compiler and expand to regular regions #3045

access2rohit commented Apr 4, 2022 •

edited

Loading

codecov-commenter commented Apr 4, 2022 •

edited

Loading

sagemaker-bot commented Apr 4, 2022

sagemaker-bot commented Apr 4, 2022

access2rohit commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

sagemaker-bot commented Apr 14, 2022

sagemaker-bot commented Apr 14, 2022

sagemaker-bot commented Apr 14, 2022

sagemaker-bot commented Apr 15, 2022

sagemaker-bot commented Apr 15, 2022

sagemaker-bot commented Apr 15, 2022

sagemaker-bot commented Apr 15, 2022

sagemaker-bot commented Apr 18, 2022

sagemaker-bot commented Apr 18, 2022

sagemaker-bot commented Apr 18, 2022

sagemaker-bot commented Apr 18, 2022

access2rohit commented Apr 18, 2022 •

edited

Loading

access2rohit commented Apr 19, 2022

feat: add Tensorflow and Pytorch version for SM Training Compiler and expand to regular regions #3045

feat: add Tensorflow and Pytorch version for SM Training Compiler and expand to regular regions #3045

Conversation

access2rohit commented Apr 4, 2022 • edited Loading

Merge Checklist

General

Tests

codecov-commenter commented Apr 4, 2022 • edited Loading

Codecov Report

sagemaker-bot commented Apr 4, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 4, 2022

AWS CodeBuild CI Report

access2rohit commented Apr 13, 2022

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 13, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 14, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 14, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 14, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 15, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 18, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 18, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 18, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Apr 18, 2022

AWS CodeBuild CI Report

access2rohit commented Apr 18, 2022 • edited Loading

access2rohit commented Apr 19, 2022

access2rohit commented Apr 4, 2022 •

edited

Loading

codecov-commenter commented Apr 4, 2022 •

edited

Loading

access2rohit commented Apr 18, 2022 •

edited

Loading