Fixing Pytorch training python version in tests #5084

nargokul · 2025-03-11T22:55:32Z

Issue #, if available:
New Python Version support for Pytorch added in automated Commit yesterday .
30fe0ee

Description of changes:

Testing done:
Unit Test Build

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)
If adding any dependency in requirements.txt files, I have spell checked and ensured they exist in PyPi

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

* Fixing Pytorch training python version in tests * Updating Inference test handling

author Molly He <[email protected]> 1740529460 -0800 committer Molly He <[email protected]> 1744156034 -0700 parent fb22b91 author Molly He <[email protected]> 1740529460 -0800 committer Molly He <[email protected]> 1744155953 -0700 parent fb22b91 author Molly He <[email protected]> 1740529460 -0800 committer Molly He <[email protected]> 1744155703 -0700 parent fb22b91 author Molly He <[email protected]> 1740529460 -0800 committer Molly He <[email protected]> 1744155683 -0700 Test py312 ci support Test py312 ci support first commit test to point to personal stack changed tox.ini Revert "changed tox.ini" This reverts commit 696cb47. Revert "Revert "changed tox.ini"" This reverts commit 5d35bd0. Revert "Revert "Revert "changed tox.ini""" This reverts commit 2ab95fb. Revert "Revert "changed tox.ini"" This reverts commit 5d35bd0. Revert "changed tox.ini" This reverts commit 696cb47. Revert "test to point to personal stack" This reverts commit 49de844. Revert "first commit" This reverts commit 1374184. add pyproject.toml add py312 to ci bump numpy version numpy version change add py312 upgrade pip version add setuptools wheel to tox.ini add pyyaml version constraint, remove py312 from docstring because there is no py312 image yet update pyyaml version constraint update pyyaml version constraint deprecate py38 bump scipy bump tensorflow and tensorboard bump dill bump apache-airflow to ensure dill and greenlet version remove constraint for apache-airflow remove constraint for apache-airflow bump torch version bump torchvision version new tests try changing some tests update model trainer test fix test_pipeline add constraint to apache in tox Fix key error in _send_metrics() (aws#5068) Co-authored-by: pintaoz <[email protected]> Use sagemaker session's s3_resource in download_folder (aws#5064) Co-authored-by: pintaoz <[email protected]> Fix error when there is no session to call _create_model_request() (aws#5062) * Fix error when there is no session to call _create_model_request() * Fix codestyle --------- Co-authored-by: pintaoz <[email protected]> Ensure Model.is_repack() returns a boolean (aws#5060) * Ensure Model.is_repack() returns a boolean * update test --------- Co-authored-by: pintaoz <[email protected]> feat: Make DistributedConfig Extensible (aws#5039) * feat: Make DistributedConfig Extensible * pylint * Include none types when creating config jsons for safer reference * fix: update test to account for changes * format * Add integ test * pylint * prepare release v2.240.0 * update development version to v2.240.1.dev0 * Fix key error in _send_metrics() (aws#5068) Co-authored-by: pintaoz <[email protected]> * fix: Added check for the presence of model package group before creating one (aws#5063) Co-authored-by: Keshav Chandak <[email protected]> * Use sagemaker session's s3_resource in download_folder (aws#5064) Co-authored-by: pintaoz <[email protected]> * remove union * fix merge artifact * Change dir path to distributed_drivers * update paths --------- Co-authored-by: ci <ci> Co-authored-by: pintaoz-aws <[email protected]> Co-authored-by: pintaoz <[email protected]> Co-authored-by: Keshav Chandak <[email protected]> Co-authored-by: Keshav Chandak <[email protected]> Skip tests with deprecated instance type (aws#5077) Co-authored-by: pintaoz <[email protected]> pipeline definition function doc update (aws#5074) Co-authored-by: Rohan Gujarathi <[email protected]> feat: add integ tests for training JumpStart models in private hub (aws#5076) * feat: add integ tests for training JumpStart models in private hub * fixed formatting * remove unused imports * fix unused imports * fix unit test failure and fix bug around versioning * fix formatting * fix unit tests * fix model_uri usage issue * fix some formatting * separate private hub setup code * add try catch block * fix flake8 issue so except clause is not bare * black formatting fix: resolve infinite loop in _find_config on Windows systems (aws#4970) * fix: resolve Windows path handling in _find_config * Replace Path.match("/") with Path.anchor comparison * Fix infinite loop in _studio.py path traversal * test: Add tests for the new root path exploration * Fix formatting style * Fixed line to long * Fix docstyle by running black manually * Fix testcase with \\ when running on non-windows machines * Fix formatting style * cleanup unused import change: update image_uri_configs 03-11-2025 07:18:09 PST Fixing Pytorch training python version in tests (aws#5084) * Fixing Pytorch training python version in tests * Updating Inference test handling remove s3 output location requirement from hub class init (aws#5081) * remove s3 output location requirement from hub class init * fix integ test hub * lint * fix test --------- Co-authored-by: Gokul Anantha Narayanan <[email protected]> fix: Prevent RunContext overlap between test_run tests (aws#5083) Co-authored-by: Gokul Anantha Narayanan <[email protected]> remove py38 from unit testing fix integ test by bumping py38 to py39 for PyTorch change framework_version that supports py39 in integ tests remove py38 from unit testing Update estimator.py

* Fixing Pytorch training python version in tests * Updating Inference test handling

* prepare release v2.241.0 * update development version to v2.241.1.dev0 * pipeline definition function doc update (#5074) Co-authored-by: Rohan Gujarathi <[email protected]> * feat: add integ tests for training JumpStart models in private hub (#5076) * feat: add integ tests for training JumpStart models in private hub * fixed formatting * remove unused imports * fix unused imports * fix unit test failure and fix bug around versioning * fix formatting * fix unit tests * fix model_uri usage issue * fix some formatting * separate private hub setup code * add try catch block * fix flake8 issue so except clause is not bare * black formatting * fix: resolve infinite loop in _find_config on Windows systems (#4970) * fix: resolve Windows path handling in _find_config * Replace Path.match("/") with Path.anchor comparison * Fix infinite loop in _studio.py path traversal * test: Add tests for the new root path exploration * Fix formatting style * Fixed line to long * Fix docstyle by running black manually * Fix testcase with \\ when running on non-windows machines * Fix formatting style * cleanup unused import * change: update image_uri_configs 03-11-2025 07:18:09 PST * Fixing Pytorch training python version in tests (#5084) * Fixing Pytorch training python version in tests * Updating Inference test handling * remove s3 output location requirement from hub class init (#5081) * remove s3 output location requirement from hub class init * fix integ test hub * lint * fix test --------- Co-authored-by: Gokul Anantha Narayanan <[email protected]> * fix: Prevent RunContext overlap between test_run tests (#5083) Co-authored-by: Gokul Anantha Narayanan <[email protected]> * Torch upgrade (#5086) * Fix Flake8 Violations * UPDATE PYTORCH VERSION TO ADDRESS SECURITY RISK **Description** Currently used Pytorch version has a possible vulnerability . Internal - https://tiny.amazon.com/p5i4jla1 **Testing Done** Unit and Integration tests in the CodeBuild * REvert CPU Versions * Test Fix * Codestyle fixes * debug attempt * Fixes * Fix * Fix * prepare release v2.242.0 * update development version to v2.242.1.dev0 * add new regions to JUMPSTART_LAUNCHED_REGIONS (#5089) Co-authored-by: isha chidrawar <[email protected]> Co-authored-by: Gokul Anantha Narayanan <[email protected]> * ADD Documentation to ReadtheDocs for Upgrading torch versions (#5090) * ADD Documentation to ReadtheDocs for Upgrading torch versions **Description** **Testing Done** Only documentation updates * Fix for Codestyle * Remove unused import * Flake8 Fix * CodeStyle Fixes * feature: Enabled update_endpoint through model_builder (#5085) * feature: Enabled update_endpoint through model_builder * fix: fix unit test, black-check, pylint errors * fix: fix black-check, pylint errors --------- Co-authored-by: Roja Reddy Sareddy <[email protected]> * fix: factor in set instance type when building JumpStart models in ModelBuilder. (#5093) * Remove main function entrypoint in ModelBuilder dependency manager. * Remove main function entrypoint in ModelBuilder dependency manager. * fix: factor in set instance type when building JumpStart models in ModelBuilder. * Remove default instance type from ModelBuilder. * Restore default instance type. Tweak integ test. --------- Co-authored-by: Joseph Zhang <[email protected]> * change: update image_uri_configs 03-21-2025 07:17:55 PST * Skip tests failed due to deprecated instance type (#5097) Co-authored-by: pintaoz <[email protected]> * Feat: Added support for returing most recently created approved model package in a group (#5092) Co-authored-by: Keshav Chandak <[email protected]> * change: update image_uri_configs 03-25-2025 07:18:13 PST * chore: fix integ tests to use latest version of model (#5104) * change: update image_uri_configs 03-26-2025 07:18:16 PST * Update Jinja version (#5101) * Aligned disable_output_compression for @Remote with Estimator (#5094) * Update transformers version (#5102) * fix: use temp file in unit tests (#5106) * fix: fix flaky spark processor integ (#5109) * fix: fix flaky spark processor integ * format * fix: fix flaky clarify model monitor test (#5107) * chore: move jumpstart region definitions to json file (#5095) * chore: move jumpstart region definitions to json file * chore: address formatting issues * fix: neo regions not ga in 5 regions * chore: make variable private --------- Co-authored-by: Erick Benitez-Ramos <[email protected]> * change: Update for PT 2.5.1, SMP 2.8.0 (#5071) * Added the config utils - centralized module for managing config file utils * Add image_uris to sagemaker utils * Updated python version * Added other common utils in pysdk to this centralized module * Test Intelligent Defaults with new utils module --------- Co-authored-by: ci <ci> Co-authored-by: Rohan Gujarathi <[email protected]> Co-authored-by: Rohan Gujarathi <[email protected]> Co-authored-by: Rohan Narayan <[email protected]> Co-authored-by: Julian Grimm <[email protected]> Co-authored-by: sagemaker-bot <[email protected]> Co-authored-by: Gokul Anantha Narayanan <[email protected]> Co-authored-by: Ben Crabtree <[email protected]> Co-authored-by: rrrkharse <[email protected]> Co-authored-by: IshaChid76 <[email protected]> Co-authored-by: isha chidrawar <[email protected]> Co-authored-by: Roja Reddy Sareddy <[email protected]> Co-authored-by: cj-zhang <[email protected]> Co-authored-by: Joseph Zhang <[email protected]> Co-authored-by: pintaoz-aws <[email protected]> Co-authored-by: pintaoz <[email protected]> Co-authored-by: Keshav Chandak <[email protected]> Co-authored-by: Keshav Chandak <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> Co-authored-by: Bruno Pistone <[email protected]> Co-authored-by: evakravi <[email protected]> Co-authored-by: Victor Zhu <[email protected]>

Fixing Pytorch training python version in tests

444816e

nargokul requested a review from a team as a code owner March 11, 2025 22:55

nargokul requested a review from mufaddal-rohawala March 11, 2025 22:55

nargokul temporarily deployed to auto-approve March 11, 2025 22:55 — with GitHub Actions Inactive

Updating Inference test handling

d612480

nargokul temporarily deployed to auto-approve March 11, 2025 23:50 — with GitHub Actions Inactive

sage-maker approved these changes Mar 12, 2025

View reviewed changes

rsareddy0329 self-requested a review March 12, 2025 00:40

rsareddy0329 approved these changes Mar 12, 2025

View reviewed changes

nargokul merged commit b6bf8cf into aws:master Mar 12, 2025
14 checks passed

mollyheamazon pushed a commit to mollyheamazon/sagemaker-python-sdk that referenced this pull request Mar 14, 2025

Fixing Pytorch training python version in tests (aws#5084)

b872f3e

* Fixing Pytorch training python version in tests * Updating Inference test handling

evakravi pushed a commit to evakravi/sagemaker-python-sdk that referenced this pull request Mar 20, 2025

Fixing Pytorch training python version in tests (aws#5084)

1605cc8

* Fixing Pytorch training python version in tests * Updating Inference test handling

mollyheamazon pushed a commit to mollyheamazon/sagemaker-python-sdk that referenced this pull request Apr 8, 2025

Fixing Pytorch training python version in tests (aws#5084)

40e1a0f

* Fixing Pytorch training python version in tests * Updating Inference test handling

mollyheamazon mentioned this pull request Apr 9, 2025

Py312 upgrade step 2: Update dependencies, integ tests and unit tests #5123

Merged

10 tasks

mollyheamazon pushed a commit that referenced this pull request Apr 21, 2025

Fixing Pytorch training python version in tests (#5084)

06c0968

* Fixing Pytorch training python version in tests * Updating Inference test handling

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixing Pytorch training python version in tests #5084

Fixing Pytorch training python version in tests #5084

nargokul commented Mar 11, 2025

Fixing Pytorch training python version in tests #5084

Fixing Pytorch training python version in tests #5084

Conversation

nargokul commented Mar 11, 2025

Merge Checklist

General

Tests