Skip to content

Commit 0971c55

Browse files
makungaj1knikureCaptainiaJonathan Makungaevakravi
authored
feature: model optimization (#4775)
* feat: Benchmark feature initial commit (#1463) * Sync Master benchmark feature (#1461) * feat: support config_name in all JumpStart interfaces (#4583) (#4607) * add-config-name * address comments * updates for set config * docstyle * updates * fix * format * format * remove tests * Add ReadOnly APIs (#4606) * Add ReadOnly APIs * Resolving PR review comments * Resolve PR review comments * Refactoring * Refactoring * Add Caching * Refactore * Resolving conflicts * Add Unit Tests * Fix Unit Tests * Fix unit tests * Fix UT * Refactoring * Fix Integ tests * refactoring after Notebook testing * Fix code styles --------- Co-authored-by: Jonathan Makunga <[email protected]> * feat: tag JumpStart resource with config names (#4608) * tag config name * format * resolving comments * format * format * update * fix * format * updates inference component config name * fix: tests * ModelBuilder: Add functionalities to get and set deployment config. (#4614) * Add funtionalities to get and set deployment config * Resolve PR comments * ModelBuilder-JS * Add Unit tests * Refactoring * Testing with Notebook * Test backward compatibility * Remove Accelerated column if all not enabled * Fix docstring * Resolved PR Review comments * Docstring * increase code coverage --------- Co-authored-by: Jonathan Makunga <[email protected]> * Benchmark feature v2 (#4618) * Add funtionalities to get and set deployment config * Resolve PR comments * ModelBuilder-JS * Add Unit tests * Refactoring * Testing with Notebook * Test backward compatibility * Remove Accelerated column if all not enabled * Fix docstring * Resolved PR Review comments * Docstring * increase code coverage * Testing fix with Notebook * Only fetch instance rate metrics if not present * Increase code coverage --------- Co-authored-by: Jonathan Makunga <[email protected]> * fix: populate default config name to model (#4617) * fix: populate default config name to model * update condition * fix * format * flake8 * fix tests * fix coverage * temporarily skip integ test vulnerbility * fix tolerate attach method * format * fix predictor * format * Fix fetch instance rate bug (#4624) Co-authored-by: Jonathan Makunga <[email protected]> * chore: require config name and instance type in set_deployment_config (#4625) * require config_name and instance_type in set config * docstring * add supported instance types check * add more tests * format * fix tests * Deployment Configs - Follow-ups (#4626) * Init Deployment configs outside Model init. * Testing with NB * Testing with NB-V2 * Refactoring, NB testing * NB Testing and Refactoring * Testing * Refactoring * Testing with NB * Debug * Debug display API * Debug with NB * Testing with NB * Refactoring * Refactoring * Refactoring and NB testing * Testing with NB * Refactoring * Prefix instance type with ml * Fix unit tests --------- Co-authored-by: Jonathan Makunga <[email protected]> * fix: use different separator to flatten dict (#4629) * Use separate tags for inference and training configs (#4635) * Use separate tags for inference and training * format * format * format * format * Add supported inference and incremental training configs (#4637) * supported inference configs * add tests * format * tests * tests * address comments * format and address comments * updates * formt * format * Benchmark feature fixes (#4632) * Filter down Benchmark Metrics * Filter down Benchmark Metrics * Testing NB * Testing MB * Testing * Refactoring * Unit tests * Display instance type first, and instance rate last * Display unbalanced metrics * Testing with NB * Testing with NB * Debug * Debug * Testing with NB * Testing with NB * Testing with NB * Refactoring * Refactoring * Refactoring * Unit tests * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Refactoring * Debug * Config ranking * Debug * Debug * Debug * Debug * Debug * Ranking * Ranking-Debug * Ranking-Debug * Ranking-Debug * Ranking-Debug * Ranking-Debug * Ranking-Debug * Debug * Debug * Debug * Debug * Refactoring * Contact JumpStart team to fix flaky test. test_list_jumpstart_models_script_filter --------- Co-authored-by: Jonathan Makunga <[email protected]> * fix: typo and merge with master branch (#4649) * Merge master into benchmark feature (#4652) * Merge master into master-benchmark-feature (#4656) * Master benchmark feature (#4658) * Master benchmark feature merge master (#4661) * Master benchmark feature (#4672) * fix: mainline alt config parsing (#4602) * fix: parsing * fix: commit tests * fix: types * updated * fix * Add Triton v24.03 URI (#4605) Co-authored-by: Nikhil Kulkarni <[email protected]> * feature: support session tag chaining for training job (#4596) * feature: support session tag chaining for training job * fix: resolve typo * fix: resolve typo and build failure * fix: resolve typo and unit test failure --------- Co-authored-by: Jessica Zhu <jessicazhu3@[email protected]> * prepare release v2.217.0 * update development version to v2.217.1.dev0 * fix: properly close files in lineage queries and tests (#4587) Closes #4458 * feature: set default allow_pickle param to False (#4557) * breaking: set default allow_pickle param to False * breaking: fix unit tests and linting NumpyDeserializer will not allow deserialization unless allow_pickle flag is set to True explicitly * fix: black-check --------- Co-authored-by: Ashwin Krishna <[email protected]> * Fix:invalid component error with new metadata (#4634) * fix: invalid component name * tests * format * fix vulnerable model integ tests llama 2 * updated * fix: training dataset location * prepare release v2.218.0 * update development version to v2.218.1.dev0 * chore: update skipped flaky tests (#4644) * Update skipped flaky tests * flake8 * format * format * chore: release tgi 2.0.1 (#4642) * chore: release tgi 2.0.1 * minor fix --------- Co-authored-by: Zhaoqi <[email protected]> * fix: Fix UserAgent logging in Python SDK (#4647) * prepare release v2.218.1 * update development version to v2.218.2.dev0 * feature: allow choosing js payload by alias in private method * Updates for SMP v2.3.1 (#4660) Co-authored-by: Suhit Kodgule <[email protected]> * chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /doc (#4655) Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](https://github.com/pallets/jinja/compare/3.1.3...3.1.4) --- updated-dependencies: - dependency-name: jinja2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump tqdm from 4.66.2 to 4.66.3 in /tests/data/serve_resources/mlflow/pytorch (#4650) Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.2 to 4.66.3. - [Release notes](https://github.com/tqdm/tqdm/releases) - [Commits](https://github.com/tqdm/tqdm/compare/v4.66.2...v4.66.3) --- updated-dependencies: - dependency-name: tqdm dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /requirements/extras (#4654) Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](https://github.com/pallets/jinja/compare/3.1.3...3.1.4) --- updated-dependencies: - dependency-name: jinja2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * prepare release v2.219.0 * update development version to v2.219.1.dev0 * fix: skip flakey tests pending investigation (#4667) * change: update image_uri_configs 05-09-2024 07:17:41 PST * Add tensorflow_serving support for mlflow models and enable lineage tracking for mlflow models (#4662) * Initial commit for tensorflow_serving support of MLflow * Add integ tests for mlflow tf_serving * fix style issues * remove unused attributes from tf builder * Add deep ping for tf_serving local mode * Initial commit for lineage impl * Initial commit for tensorflow_serving support of MLflow * Add integ tests for mlflow tf_serving * fix style issues * remove unused attributes from tf builder * Add deep ping for tf_serving local mode * Add integ tests and uts * fix local mode for tf_serving * Allow lineage tracking only in sagemaker endpoint mode * fix regex pattern * fix style issues * fix regex pattern and hard coded py version in ut * fix missing session * Resolve pr comments and fix regex for mlflow registry and ids * fix: model builder race condition on sagemaker session (#4673) Co-authored-by: Jonathan Makunga <[email protected]> * feat: Add telemetry support for mlflow models (#4674) * Initial commit for telemetry support * Fix style issues and add more logger messages * fix value error messages in ut * feat: add new images for HF TGI release (#4677) * chore: add new images for HF TGI release * test * feature: AutoGluon 1.1.0 image_uris update (#4679) Co-authored-by: Ubuntu <[email protected]> * change: add debug logs to workflow container dist creation (#4682) * prepare release v2.220.0 * update development version to v2.220.1.dev0 * fix: Image URI should take precedence for HF models (#4684) * Fix: Image URI should take precedence for HF models * Fix formatting * Fix formatting * Fix formatting * Increase coverage - UT pass * feat: support config_name in all JumpStart interfaces (#4583) (#4607) * add-config-name * address comments * updates for set config * docstyle * updates * fix * format * format * remove tests * Add ReadOnly APIs (#4606) * Add ReadOnly APIs * Resolving PR review comments * Resolve PR review comments * Refactoring * Refactoring * Add Caching * Refactore * Resolving conflicts * Add Unit Tests * Fix Unit Tests * Fix unit tests * Fix UT * Refactoring * Fix Integ tests * refactoring after Notebook testing * Fix code styles --------- Co-authored-by: Jonathan Makunga <[email protected]> * feat: tag JumpStart resource with config names (#4608) * tag config name * format * resolving comments * format * format * update * fix * format * updates inference component config name * fix: tests * ModelBuilder: Add functionalities to get and set deployment config. (#4614) * Add funtionalities to get and set deployment config * Resolve PR comments * ModelBuilder-JS * Add Unit tests * Refactoring * Testing with Notebook * Test backward compatibility * Remove Accelerated column if all not enabled * Fix docstring * Resolved PR Review comments * Docstring * increase code coverage --------- Co-authored-by: Jonathan Makunga <[email protected]> * Benchmark feature v2 (#4618) * Add funtionalities to get and set deployment config * Resolve PR comments * ModelBuilder-JS * Add Unit tests * Refactoring * Testing with Notebook * Test backward compatibility * Remove Accelerated column if all not enabled * Fix docstring * Resolved PR Review comments * Docstring * increase code coverage * Testing fix with Notebook * Only fetch instance rate metrics if not present * Increase code coverage --------- Co-authored-by: Jonathan Makunga <[email protected]> * fix: populate default config name to model (#4617) * fix: populate default config name to model * update condition * fix * format * flake8 * fix tests * fix coverage * temporarily skip integ test vulnerbility * fix tolerate attach method * format * fix predictor * format * Fix fetch instance rate bug (#4624) Co-authored-by: Jonathan Makunga <[email protected]> * chore: require config name and instance type in set_deployment_config (#4625) * require config_name and instance_type in set config * docstring * add supported instance types check * add more tests * format * fix tests * Deployment Configs - Follow-ups (#4626) * Init Deployment configs outside Model init. * Testing with NB * Testing with NB-V2 * Refactoring, NB testing * NB Testing and Refactoring * Testing * Refactoring * Testing with NB * Debug * Debug display API * Debug with NB * Testing with NB * Refactoring * Refactoring * Refactoring and NB testing * Testing with NB * Refactoring * Prefix instance type with ml * Fix unit tests --------- Co-authored-by: Jonathan Makunga <[email protected]> * fix: use different separator to flatten dict (#4629) * Use separate tags for inference and training configs (#4635) * Use separate tags for inference and training * format * format * format * format * Add supported inference and incremental training configs (#4637) * supported inference configs * add tests * format * tests * tests * address comments * format and address comments * updates * formt * format * Benchmark feature fixes (#4632) * Filter down Benchmark Metrics * Filter down Benchmark Metrics * Testing NB * Testing MB * Testing * Refactoring * Unit tests * Display instance type first, and instance rate last * Display unbalanced metrics * Testing with NB * Testing with NB * Debug * Debug * Testing with NB * Testing with NB * Testing with NB * Refactoring * Refactoring * Refactoring * Unit tests * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Custom lru * Refactoring * Debug * Config ranking * Debug * Debug * Debug * Debug * Debug * Ranking * Ranking-Debug * Ranking-Debug * Ranking-Debug * Ranking-Debug * Ranking-Debug * Ranking-Debug * Debug * Debug * Debug * Debug * Refactoring * Contact JumpStart team to fix flaky test. test_list_jumpstart_models_script_filter --------- Co-authored-by: Jonathan Makunga <[email protected]> * fix: typo and merge with master branch (#4649) * Merge master into benchmark feature (#4652) * Merge master into master-benchmark-feature (#4656) * Master benchmark feature (#4658) * Remove duplicate line in types.py * Remove duplicate lines * Remove duplicate lines * Remove duplicate lines * Remove duplicate lines * fix unit test --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Haotian An <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: jessicazhu3 <[email protected]> Co-authored-by: Jessica Zhu <jessicazhu3@[email protected]> Co-authored-by: ci <ci> Co-authored-by: Justin <[email protected]> Co-authored-by: ASHWIN KRISHNA <[email protected]> Co-authored-by: Ashwin Krishna <[email protected]> Co-authored-by: Haixin Wang <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Kalyani Nikure <[email protected]> Co-authored-by: Keerthan Vasist <[email protected]> Co-authored-by: SuhitK <[email protected]> Co-authored-by: Suhit Kodgule <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: sagemaker-bot <[email protected]> Co-authored-by: jiapinw <[email protected]> Co-authored-by: Jonathan Makunga <[email protected]> Co-authored-by: Jonathan Makunga <[email protected]> Co-authored-by: Prateek M Desai <[email protected]> Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Samrudhi Sharma <[email protected]> Co-authored-by: evakravi <[email protected]> * fix benchmark feature read-only apis (#4675) * Rearrange benchmark metric table * Refactoring * Refactoring * Refactoring * Refactoring * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Refactoring * Refactoring * Refactoring * Refactoring * Refactoring * Add Unit tests * Refactoring * Refactoring * hide index from DataFrame --------- Co-authored-by: Jonathan Makunga <[email protected]> * feat: update alt config to work with model packages (#4706) * feat: update alt config to work with model packages * format * remove env vars for model package * fix tests * Update: ReadOnly APIs (#4707) * Model data arn * Refactoring * Refactoring * acceleration_configs * Refactoring * UT * Add Filter * UT * Revert "UT" * UT * UT --------- Co-authored-by: Jonathan Makunga <[email protected]> * ModelBuilder to support display with filter. (#4712) Co-authored-by: Jonathan Makunga <[email protected]> * Sync branch (#4718) * fix: mainline alt config parsing (#4602) * fix: parsing * fix: commit tests * fix: types * updated * fix * Add Triton v24.03 URI (#4605) Co-authored-by: Nikhil Kulkarni <[email protected]> * feature: support session tag chaining for training job (#4596) * feature: support session tag chaining for training job * fix: resolve typo * fix: resolve typo and build failure * fix: resolve typo and unit test failure --------- Co-authored-by: Jessica Zhu <jessicazhu3@[email protected]> * prepare release v2.217.0 * update development version to v2.217.1.dev0 * fix: properly close files in lineage queries and tests (#4587) Closes #4458 * feature: set default allow_pickle param to False (#4557) * breaking: set default allow_pickle param to False * breaking: fix unit tests and linting NumpyDeserializer will not allow deserialization unless allow_pickle flag is set to True explicitly * fix: black-check --------- Co-authored-by: Ashwin Krishna <[email protected]> * Fix:invalid component error with new metadata (#4634) * fix: invalid component name * tests * format * fix vulnerable model integ tests llama 2 * updated * fix: training dataset location * prepare release v2.218.0 * update development version to v2.218.1.dev0 * chore: update skipped flaky tests (#4644) * Update skipped flaky tests * flake8 * format * format * chore: release tgi 2.0.1 (#4642) * chore: release tgi 2.0.1 * minor fix --------- Co-authored-by: Zhaoqi <[email protected]> * fix: Fix UserAgent logging in Python SDK (#4647) * prepare release v2.218.1 * update development version to v2.218.2.dev0 * feature: allow choosing js payload by alias in private method * Updates for SMP v2.3.1 (#4660) Co-authored-by: Suhit Kodgule <[email protected]> * chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /doc (#4655) Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](https://github.com/pallets/jinja/compare/3.1.3...3.1.4) --- updated-dependencies: - dependency-name: jinja2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump tqdm from 4.66.2 to 4.66.3 in /tests/data/serve_resources/mlflow/pytorch (#4650) Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.2 to 4.66.3. - [Release notes](https://github.com/tqdm/tqdm/releases) - [Commits](https://github.com/tqdm/tqdm/compare/v4.66.2...v4.66.3) --- updated-dependencies: - dependency-name: tqdm dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /requirements/extras (#4654) Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](https://github.com/pallets/jinja/compare/3.1.3...3.1.4) --- updated-dependencies: - dependency-name: jinja2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * prepare release v2.219.0 * update development version to v2.219.1.dev0 * fix: skip flakey tests pending investigation (#4667) * change: update image_uri_configs 05-09-2024 07:17:41 PST * Add tensorflow_serving support for mlflow models and enable lineage tracking for mlflow models (#4662) * Initial commit for tensorflow_serving support of MLflow * Add integ tests for mlflow tf_serving * fix style issues * remove unused attributes from tf builder * Add deep ping for tf_serving local mode * Initial commit for lineage impl * Initial commit for tensorflow_serving support of MLflow * Add integ tests for mlflow tf_serving * fix style issues * remove unused attributes from tf builder * Add deep ping for tf_serving local mode * Add integ tests and uts * fix local mode for tf_serving * Allow lineage tracking only in sagemaker endpoint mode * fix regex pattern * fix style issues * fix regex pattern and hard coded py version in ut * fix missing session * Resolve pr comments and fix regex for mlflow registry and ids * fix: model builder race condition on sagemaker session (#4673) Co-authored-by: Jonathan Makunga <[email protected]> * feat: Add telemetry support for mlflow models (#4674) * Initial commit for telemetry support * Fix style issues and add more logger messages * fix value error messages in ut * feat: add new images for HF TGI release (#4677) * chore: add new images for HF TGI release * test * feature: AutoGluon 1.1.0 image_uris update (#4679) Co-authored-by: Ubuntu <[email protected]> * change: add debug logs to workflow container dist creation (#4682) * prepare release v2.220.0 * update development version to v2.220.1.dev0 * fix: Image URI should take precedence for HF models (#4684) * Fix: Image URI should take precedence for HF models * Fix formatting * Fix formatting * Fix formatting * Increase coverage - UT pass * feat: onboard tei image config to pysdk (#4681) * feat: onboard tei image config to pysdk * fix formatting issue * minor fix func name * fix unit tests --------- Co-authored-by: Mufaddal Rohawala <[email protected]> * fix: model builder limited container support for endpoint mode. (#4683) * Allow ModelBuilder's endpoint mode for Jumpstart models packaged with containers other than TGI and DJL * increase coverage * Add JS Support for MMS Serving * Add JS Support for MMS Serving * Unit tests * Refactoring * Refactoring * Refactoring --------- Co-authored-by: Jonathan Makunga <[email protected]> * change: Add more debuging (#4687) * change: cover tei with image_uris.retrieve API (#4689) * fix: JS Model with non-TGI/non-DJL deployment failure (#4688) * Debug * Debug * Debug * Debug * Debug * Debug * fix docstyle * Refactoring * Add Integ tests --------- Co-authored-by: Jonathan Makunga <[email protected]> * Feat: Pull latest tei container for sentence similiarity models on HuggingFace hub (#4686) * Update: Pull latest tei container for sentence similiarity models * Fix formatting * Address PR comments * Fix formatting * Fix check * Switch sentence similarity to be deployed on tgi * Fix formatting * Fix formatting * Fix formatting * Fix formatting * Introduce TEI builder with TGI server * Fix formmatting * Add integ test * Fix formatting * Add integ test * Add integ test * Add integ test * Add integ test * Add integ test * Fix formatting * Move to G5 for integ test * Fix formatting * Integ test updates * Integ test updates * Integ test updates * Fix formatting * Integ test updates * Move back to generate for ping * Integ test updates * Integ test updates * Fix: Add Image URI overrides for transformers models (#4693) * Fix: Add Image URI overrides for transformers models * Increase coverage * Fix formatting * prepare release v2.221.0 * update development version to v2.221.1.dev0 * Add tei cpu image (#4695) * Add tei cpu image * fix format issue * fix unit tests * fix typo * fix typo * Feat: Add TEI support for ModelBuilder (#4694) * Add TEI Serving * Add TEI Serving * Add TEI Serving * Add TEI Serving * Add TEI Serving * Add TEI Serving * Notebook testing * Notebook testing * Notebook testing * Refactoring * Refactoring * UT * UT * Refactoring * Test coverage * Refactoring * Refactoring --------- Co-authored-by: Jonathan Makunga <[email protected]> * Convert pytorchddp distribution to smdistributed distribution (#4698) * rewrite pytorchddp to smdistributed * remove instance type check * Update estimator.py * remove validate_pytorch_distribution * fix * fix unit tests * fix formatting * check instance type not None * prepare release v2.221.1 * update development version to v2.221.2.dev0 * Update: SM Endpoint Routing Strategy Support. (#4702) * RoutingConfig * Refactoring * Docstring * UT * Refactoring * Refactoring --------- Co-authored-by: Jonathan Makunga <[email protected]> * change: update image_uri_configs 05-29-2024 07:17:35 PST * Making project name in workflow files dynamic (#4708) * fix: Fix ci unit-tests (#4713) * chore(deps): bump requests from 2.31.0 to 2.32.2 in /tests/data/serve_resources/mlflow/pytorch (#4709) Bumps [requests](https://github.com/psf/requests) from 2.31.0 to 2.32.2. - [Release notes](https://github.com/psf/requests/releases) - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md) - [Commits](https://github.com/psf/requests/compare/v2.31.0...v2.32.2) --- updated-dependencies: - dependency-name: requests dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump apache-airflow from 2.9.0 to 2.9.1 in /requirements/extras (#4703) * chore(deps): bump apache-airflow in /requirements/extras Bumps [apache-airflow](https://github.com/apache/airflow) from 2.9.0 to 2.9.1. - [Release notes](https://github.com/apache/airflow/releases) - [Changelog](https://github.com/apache/airflow/blob/main/RELEASE_NOTES.rst) - [Commits](https://github.com/apache/airflow/compare/2.9.0...2.9.1) --- updated-dependencies: - dependency-name: apache-airflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update tox.ini to bump apache-airflow --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kalyani Nikure <[email protected]> * chore(deps): bump mlflow from 2.10.2 to 2.12.1 in /tests/data/serve_resources/mlflow/pytorch (#4690) Bumps [mlflow](https://github.com/mlflow/mlflow) from 2.10.2 to 2.12.1. - [Release notes](https://github.com/mlflow/mlflow/releases) - [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md) - [Commits](https://github.com/mlflow/mlflow/compare/v2.10.2...v2.12.1) --- updated-dependencies: - dependency-name: mlflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump mlflow from 2.11.1 to 2.12.1 in /tests/data/serve_resources/mlflow/xgboost (#4692) Bumps [mlflow](https://github.com/mlflow/mlflow) from 2.11.1 to 2.12.1. - [Release notes](https://github.com/mlflow/mlflow/releases) - [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md) - [Commits](https://github.com/mlflow/mlflow/compare/v2.11.1...v2.12.1) --- updated-dependencies: - dependency-name: mlflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump mlflow from 2.11.1 to 2.12.1 in /tests/data/serve_resources/mlflow/tensorflow (#4691) Bumps [mlflow](https://github.com/mlflow/mlflow) from 2.11.1 to 2.12.1. - [Release notes](https://github.com/mlflow/mlflow/releases) - [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md) - [Commits](https://github.com/mlflow/mlflow/compare/v2.11.1...v2.12.1) --- updated-dependencies: - dependency-name: mlflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * change: Updates for DJL 0.28.0 release (#4701) * Sync Branch --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Haotian An <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: jessicazhu3 <[email protected]> Co-authored-by: Jessica Zhu <jessicazhu3@[email protected]> Co-authored-by: ci <ci> Co-authored-by: Justin <[email protected]> Co-authored-by: ASHWIN KRISHNA <[email protected]> Co-authored-by: Ashwin Krishna <[email protected]> Co-authored-by: Haixin Wang <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Kalyani Nikure <[email protected]> Co-authored-by: Keerthan Vasist <[email protected]> Co-authored-by: SuhitK <[email protected]> Co-authored-by: Suhit Kodgule <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: sagemaker-bot <[email protected]> Co-authored-by: jiapinw <[email protected]> Co-authored-by: Jonathan Makunga <[email protected]> Co-authored-by: Prateek M Desai <[email protected]> Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Samrudhi Sharma <[email protected]> Co-authored-by: Tom Bousso <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Tyler Osterberg <[email protected]> * Merge --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Haotian An <[email protected]> Co-authored-by: Jonathan Makunga <[email protected]> Co-authored-by: evakravi <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: jessicazhu3 <[email protected]> Co-authored-by: Jessica Zhu <jessicazhu3@[email protected]> Co-authored-by: Justin <[email protected]> Co-authored-by: ASHWIN KRISHNA <[email protected]> Co-authored-by: Ashwin Krishna <[email protected]> Co-authored-by: Haixin Wang <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Kalyani Nikure <[email protected]> Co-authored-by: Keerthan Vasist <[email protected]> Co-authored-by: SuhitK <[email protected]> Co-authored-by: Suhit Kodgule <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: sagemaker-bot <[email protected]> Co-authored-by: jiapinw <[email protected]> Co-authored-by: Prateek M Desai <[email protected]> Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Samrudhi Sharma <[email protected]> Co-authored-by: Tom Bousso <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Tyler Osterberg <[email protected]> * Fix UT (#1465) Co-authored-by: Jonathan Makunga <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Jonathan Makunga <[email protected]> Co-authored-by: Haotian An <[email protected]> Co-authored-by: Jonathan Makunga <[email protected]> Co-authored-by: evakravi <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: jessicazhu3 <[email protected]> Co-authored-by: Jessica Zhu <jessicazhu3@[email protected]> Co-authored-by: Justin <[email protected]> Co-authored-by: ASHWIN KRISHNA <[email protected]> Co-authored-by: Ashwin Krishna <[email protected]> Co-authored-by: Haixin Wang <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Keerthan Vasist <[email protected]> Co-authored-by: SuhitK <[email protected]> Co-authored-by: Suhit Kodgule <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: sagemaker-bot <[email protected]> Co-authored-by: jiapinw <[email protected]> Co-authored-by: Prateek M Desai <[email protected]> Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Samrudhi Sharma <[email protected]> Co-authored-by: Tom Bousso <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Tyler Osterberg <[email protected]> * feat: Model class to support AdditionalModelDataSources (#1469) * Add support for AdditionalModelDataSources * Resolve PR comments * Resolve PR comments * Resolve PR comments * fix unit tests * Resolve PR comments --------- Co-authored-by: Jonathan Makunga <[email protected]> * feat: additional hosting model data source parsing (#1467) * feat: Additional Model Data source parsing * address comments * address comments * format * Add optimize to ModelBuilder (#1468) * Add optimize to ModelBuilder * Add polling for job completion * fix UTs --------- Co-authored-by: Jacky Lee <[email protected]> * feat: Added utils for extracting JS data sources (#1471) * added utils for accessing hosting data sources * added utils for accessing hosting data sources * removed other changes * fixed formatting issues * remove .keys() * updated JumpStartModelDataSource * fix slots * remove print * fix tests * update tests * fix: update passing additional model data sources to API (#1472) * feat: Added utils for extracting JS data sources (#1471) * added utils for accessing hosting data sources * added utils for accessing hosting data sources * removed other changes * fixed formatting issues * remove .keys() * updated JumpStartModelDataSource * fix slots * remove print * fix tests * update tests * fix: update passing additional model data sources to API * format * format * format * format and address comments * format * format * format --------- Co-authored-by: Adam Kozdrowicz <[email protected]> * fix: overriding instance specific fields in config components (#1478) * fix: instance specific variables override * format * Feat: Add optimize to ModelBuilder JS (#1474) * QS JS vanilla model * Use Alt config for Optimization * JS Optimize * Resolve config * inject additional tags * Inject tags * Refactoring * Refactoring * Filter Deployment config * Refactoring * Refactoring * Refactoring * Refactoring --------- Co-authored-by: Jonathan Makunga <[email protected]> * feat: use Neo bucket in speculative decoding data source (#1479) * Use Neo bucket in speculative decoding data source * address comments * format * address comments * add buckets to regional config * remove opt-in regions for neo buckets * feat: add build/deploy support for fine-tuned JS models (#1473) * feat: add support for fine-tuned JS models * Refactor * Refactor * Refactor * Refactor * pylint * pylint --------- Co-authored-by: Jacky Lee <[email protected]> * update: Add optimize to ModelBuilder JS (#1480) * Testing with Notebook * Refactoring * _poll_optimization_job refactoring * Resolve PR Comments * Refactoring * Refactoring * refactoring * Fix conflicts * Notebook testing * Refactoring * Refactoring --------- Co-authored-by: Jonathan Makunga <[email protected]> * update: Add optimize to ModelBuilder JS (#1485) * MB JS Optimize * UT * Refactore * UT * UT * refactore * refactore --------- Co-authored-by: Jonathan Makunga <[email protected]> * feat: add quicksilver telemetry (#1482) * feat: add quicksilver telemetry fields * pylint * add UTs * pylint * Refactor * add gated and fine-tuned to telemetry * fix: typo * fix: jumpstart var * refactor model_hub * pylint * update TEI/TGI to remove jumpstart field * reorder telemetry schema * refactor --------- Co-authored-by: Jacky Lee <[email protected]> * unit: tests for fine tuned JS model support (#1481) * UTs * flake8 --------- Co-authored-by: Jacky Lee <[email protected]> * fix: use current session and role when setting config (#1493) * fix: use current session and role when setting config * format * fix: training arn support (#1494) * fix: training job ARN * pylint --------- Co-authored-by: Jacky Lee <[email protected]> * Bug bash fixes (#1492) * HF Optimized * Revert "HF Optimized" * MB HF Optimize support * Refactoring * HF only s3 upload if optimize * reuse role if provided in MB * Refactoring * New requirements * Draft * Refactoring * Refactoring * Bug Bash fixes * UT * UT * Fix for parsing optimization output * Tag fix * UT * UT --------- Co-authored-by: Jonathan Makunga <[email protected]> * Bug fixes (#1496) * Bug fixes * refcatore * ENV update * Remove code duplication * Fix Integ tests * Fix MB EULA bug --------- Co-authored-by: Jonathan Makunga <[email protected]> * JS Optimize api ref * Refactoring * Refactoring * Fix issues * Channel name * Channel name * Optimization output * neuron model env * Merge master into master-benchmark-feature (#1502) * prepare release v2.222.0 * update development version to v2.222.1.dev0 * fix: estimator.deploy not respecting instance type (#4724) * fix: estimator.deploy not respecting instance type * chore: add inline comment about using user supplied instance type * First changes (#4723) Co-authored-by: Bryannah Hernandez <[email protected]> * prepare release v2.222.1 * update development version to v2.222.2.dev0 * change: update image_uri_configs 06-12-2024 07:17:03 PST * fix: Fix ci unit-tests (#4728) * Implement custom telemetry logging in SDK (#4721) * Fix Sniping bug fix (#4730) * Python SDK bucket sniping fix bug * Python SDK bucket sniping fix bug * Minor fixes to default bucket function and fixing unit tests * fix - Fixes from Pylint failures * fix - Fixes from Flake8 failures * fix - More Flake8 fixes * fix - Remove Whitespace from blankline * fix - Fix black recommendations * fix - Adjust tabbing --------- Co-authored-by: Jiao Liu <[email protected]> Co-authored-by: liujiaor <[email protected]> * feature: add 'ModelCard' property to Register step (#4726) * feature: add 'ModelCard' property to RegisterModel step * Updated ModelCard content type * fix: ModelCard Object integ Test fix --------- Co-authored-by: Gokul A <[email protected]> * prepare release v2.223.0 * update development version to v2.223.1.dev0 * Fix Dependabot Issues - MLFlow Version (#4731) * fix - Address Dependapot issues * fix -Update MLFLOW Version * Fix: AttributeError: 'NoneType' object has no attribute 'len' error in session.py (#4735) * fix - Address Dependapot issues * fix -Update MLFLOW Version * fix - Update fetching Length for NoneType Error * change: Enable telemetry logging for Remote function (#4729) * change: Enhance telemetry logging module and feature coverage * Fix default session issue * fix unit-tests * chore: use ml.g5.2xlarge for integ test (#4741) * feat: JumpStartModel attach (#4680) * feat: JumpStartModel attach * fix: unit tests * chore: change order of kwargs to pass unit tests * chore: update docstrings, add tests * fix: docstring * fix: integ tests * chore: address PR comments --------- Co-authored-by: Erick Benitez-Ramos <[email protected]> * Upgrading to PT 2.3 for release (#4732) * upgrading to PT 2.3 for release * reverting mistake in modifying dataparallel --------- Co-authored-by: Andrew Tian <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> * feat(sagemaker-mlflow): New features for SageMaker MLflow (#4744) * feat: add support for mlflow inputs (#1441) * feat: add support for mlflow inputs * fix: typo * fix: doc * fix: S3 regex * fix: refactor * fix: refactor typo * fix: pylint * fix: pylint * fix: black and pylint --------- Co-authored-by: Jacky Lee <[email protected]> * fix: lineage tracking bug (#1447) * fix: lineage bug * fix: lineage * fix: add validation for tracking ARN input with MLflow input type * fix: bug * fix: unit tests * fix: mock * fix: args --------- Co-authored-by: Jacky Lee <[email protected]> * [Fix] regex for RunId to handle empty artifact path and change mlflow plugin name (#1455) * [Fix] run id regex pattern such that empty artifact path is handled * Change mlflow plugin name as per legal team requirement * Update describe_mlflow_tracking_server call to align with api changes (#1466) * feat: (sagemaker-mlflow) Adding Presigned Url function to SDK (#1462) (#1477) * mlflow presigned url changes * addressing design feedback * test changes * change: mlflow plugin name (#1489) Co-authored-by: Jacky Lee <[email protected]> --------- Co-authored-by: Jacky Lee <[email protected]> Co-authored-by: Jacky Lee <[email protected]> Co-authored-by: jiapinw <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> * prepare release v2.224.0 * update development version to v2.224.1.dev0 * fix: Model server override logic (#4733) * fix: Model server override logic * Fix formatting --------- Co-authored-by: Erick Benitez-Ramos <[email protected]> * chore(deps): bump apache-airflow from 2.9.1 to 2.9.2 in /requirements/extras (#4740) Bumps [apache-airflow](https://github.com/apache/airflow) from 2.9.1 to 2.9.2. - [Release notes](https://github.com/apache/airflow/releases) - [Changelog](https://github.com/apache/airflow/blob/main/RELEASE_NOTES.rst) - [Commits](https://github.com/apache/airflow/compare/2.9.1...2.9.2) --- updated-dependencies: - dependency-name: apache-airflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <[email protected]> * fix: Update tox.ini (#4747) * change: Update README.rst to show conda-forge version of SageMaker SDK (#4749) * JumpStart CuratedHub Launch (#4748) * Implement CuratedHub APIs (#1449) * Implement CuratedHub Admin APIs * making some parameters optional in create_hub_content_reference as per the API design * add describe_hub and list_hubs APIs * implement delete_hub API * Implement list_hub_contents API * create CuratedHub class and supported utils * implement list_models and address comments * Add unit tests * add describe_model function * cache retrieval for describeHubContent changes * fix curated hub class unit tests * add utils needed for curatedHub * Cache retrieval * implement get_hub_model_reference() * cleanup HUB type datatype * cleanup constants * rename list_public_models to list_jumpstart_service_hub_models * implement describe_model_reference * Rename CuratedHub to Hub * address nit * address nits and fix failing tests --------- Co-authored-by: Malav Shastri <[email protected]> * feat: implement list_jumpstart_service_hub_models function to fetch JumpStart public hub models (#1456) * Implement CuratedHub Admin APIs * making some parameters optional in create_hub_content_reference as per the API design * add describe_hub and list_hubs APIs * implement delete_hub API * Implement list_hub_contents API * create CuratedHub class and supported utils * implement list_models and address comments * Add unit tests * add describe_model function * cache retrieval for describeHubContent changes * fix curated hub class unit tests * add utils needed for curatedHub * Cache retrieval * implement get_hub_model_reference() * cleanup HUB type datatype * cleanup constants * rename list_public_models to list_jumpstart_service_hub_models * implement describe_model_reference * Rename CuratedHub to Hub * address nit * address nits and fix failing tests * implement list_jumpstart_service_hub_models function --------- Co-authored-by: Malav Shastri <[email protected]> * Feat/Curated Hub hub_arn and hub_content_type support (#1453) * get_model_spec() changes to support hub_arn and hub_content_type * implement get_hub_model_reference() * support hub_arn and hub_content_type for specs retrieval * add support for hub_arn and hub_content_type for serializers, deserializers, estimators, models, predictors and various spec retrieval functionalities * address nits and test failures * remove hub_content_type support --------- Co-authored-by: Malav Shastri <[email protected]> * feat: implement curated hub parser and bug bash fixes (#1457) * implement HubContentDocument parser * modify the parser to remove aliases for hubcontent documents * bug fix * update boto3 * Bug Fix in the parser * Improve Hub Class and related functionalities * Bug Fix and parser updates * add missing hub_arn support * Add model reference deployment support and other minor bug fixes * fix: retrieve correct image_uri (parser update) * fix: retrieve correct model URI and model data path from HubContentDocument (parser update) * Add model reference deployment support * Model accessor and cache retrival bug fixes * fix: curated hub model training workflow * fix: pass sagemaker sessions object to retrieve model specs from describe_hub_content call * fix: fix payload retrieval for curated hub models * modify constants, enums * fix: update parser * Address nits in the parser * Add unit tests for parser * implement pagination for list_models utility * feat: support wildcard chars for model versions * Address nits and comments * Add Hub Content Arn Tag to training and hosting * Add Hub Content Arn Tag to training and hosting * fix: HubContentDocument schema version * fix broken unit tests * fix prepare_container_def unit tests to include ModelReferenceArn * fix unit tests for test_session.py * revert boto version changes * Fix unit tests * support wildcard model versions for training workflow * Add test cases for get_model_versions * Add/fix unit tests --------- Co-authored-by: Malav Shastri <[email protected]> * address unit tests failures in codebuild * change list_jumpstart_service_hub_models to list_sagemaker_public_hub_models() * fix: Changing list input output shapes * fix: gated model training bug * run black -l 100 * flake 8 * address formatting issues * black -l * DocStyle issues * address flake8, pylint * blake -l * pass model type down * disabling pylint for release * disable pylint --------- Co-authored-by: Malav Shastri <[email protected]> Co-authored-by: chrstfu <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> * prepare release v2.224.1 * update development version to v2.224.2.dev0 * fix: list_models() for python3.8 (#4756) * fix: list_models() for python3.8 * fix linting --------- Co-authored-by: Malav Shastri <[email protected]> * Update DJLModel class for latest container releases (#4754) * simplify and refactor djl model for latest container releases * update model builder for new DJLModel implementation * fix formatting/linting suggestions * update DJLModel documentation on docs site * address reviewer feedback * Feature: Update model card on model package request (#4739) * Feature: Update model card on model package request * Feature: Update model card on model package request * fix: update_model_card input types * Feature: register proprietary models from jumpstart (#4753) * Feature: register proprietary models from jumpstart Feature: register proprietary models from jumpstart * fix: register jumpstart models on model registry * fixed get_model_id_version_from_endpoint naming issue * fixed issues with model builder * cleanup types.py file * fixed jumpstart unit tests * fixed issue in model_builder --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: ci <ci> Co-authored-by: evakravi <[email protected]> Co-authored-by: bryannahm1 <[email protected]> Co-authored-by: Bryannah Hernandez <[email protected]> Co-authored-by: sagemaker-bot <[email protected]> Co-authored-by: Kalyani Nikure <[email protected]> Co-authored-by: Gokul A <[email protected]> Co-authored-by: Jiao Liu <[email protected]> Co-authored-by: liujiaor <[email protected]> Co-authored-by: selvask-aws <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> Co-authored-by: adtian2 <[email protected]> Co-authored-by: Andrew Tian <[email protected]> Co-authored-by: ananth102 <[email protected]> Co-authored-by: Jacky Lee <[email protected]> Co-authored-by: Jacky Lee <[email protected]> Co-authored-by: jiapinw <[email protected]> Co-authored-by: Samrudhi Sharma <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Malav Shastri <[email protected]> Co-authored-by: Malav Shastri <[email protected]> Co-authored-by: chrstfu <[email protected]> Co-authored-by: Siddharth Venkatesan <[email protected]> * feat: Support Alt Configs for Public & Curated Hub (#1505) * feat: add alt config support for public & curated hub * fix: make telemetry logger persist certain information (#1500) * refactor telemetry logger * refactor * refactor * pylint + UT * add tag * add remove tags * handle tags again * pylint --------- Co-authored-by: Jacky Lee <[email protected]> * Optimize support for hf models (#1499) * HF support * refactoring * Refactoring * Refactoing * HF Refactoring * Refactoring * UT * Fix UT * Resolving PR comments * HF Token * Resolving PR comments * Fix UT * Fix JS ModelServer deploy wrapper override * Fix tests * fix UT * Resolve PR comments * fix doc --------- Co-authored-by: Jonathan Makunga <[email protected]> * Fixing bugs (#1506) * Fixing bugs * Refactoring * Increase coverage * Fix UT * Fix UT * Increase coverage * Fix UT * Refactoring * Fix UT --------- Co-authored-by: Jonathan Makunga <[email protected]> * Fix public optimize api signature (#1507) * Fix public optimize api signature * JS Compilation fix * Refactoring * Refactoring --------- Co-authored-by: Jonathan Makunga <[email protected]> * Refactoring * Integration tests * Skip Alt Config integ tests as metadata aren't fully deployed. * Fix metric column name * Refactoring * Display API * Relax set deployment error handling * Override region for draft model data source * use latest boto3 * EBS Volue * model tags * UT * FIX UT --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Kalyani Nikure <[email protected]> Co-authored-by: Haotian An <[email protected]> Co-authored-by: Jonathan Makunga <[email protected]> Co-authored-by: evakravi <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: jessicazhu3 <[email protected]> Co-authored-by: Jessica Zhu <jessicazhu3@[email protected]> Co-authored-by: Justin <[email protected]> Co-authored-by: ASHWIN KRISHNA <[email protected]> Co-authored-by: Ashwin Krishna <[email protected]> Co-authored-by: Haixin Wang <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Keerthan Vasist <[email protected]> Co-authored-by: SuhitK <[email protected]> Co-authored-by: Suhit Kodgule <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: sagemaker-bot <[email protected]> Co-authored-by: jiapinw <[email protected]> Co-authored-by: Prateek M Desai <[email protected]> Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Samrudhi Sharma <[email protected]> Co-authored-by: Tom Bousso <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Tyler Osterberg <[email protected]> Co-authored-by: Jacky Lee <[email protected]> Co-authored-by: Jacky Lee <[email protected]> Co-authored-by: Adam Kozdrowicz <[email protected]> Co-authored-by: bryannahm1 <[email protected]> Co-authored-by: Bryannah Hernandez <[email protected]> Co-authored-by: Gokul A <[email protected]> Co-authored-by: Jiao Liu <[email protected]> Co-authored-by: liujiaor <[email protected]> Co-authored-by: selvask-aws <[email protected]> Co-authored-by: adtian2 <[email protected]> Co-authored-by: Andrew Tian <[email protected]> Co-authored-by: ananth102 <[email protected]> Co-authored-by: Malav Shastri <[email protected]> Co-authored-by: Malav Shastri <[email protected]> Co-authored-by: chrstfu <[email protected]> Co-authored-by: Siddharth Venkatesan <[email protected]>
1 parent 502b060 commit 0971c55

File tree

90 files changed

+7996
-817
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

90 files changed

+7996
-817
lines changed

requirements/extras/test_requirements.txt

+1
Original file line numberDiff line numberDiff line change
@@ -38,3 +38,4 @@ accelerate>=0.24.1,<=0.27.0
3838
schema==0.7.5
3939
tensorflow>=2.1,<=2.16
4040
mlflow>=2.12.2,<2.13
41+
huggingface_hub>=0.23.4

setup.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ def read_requirements(filename):
4949
# Declare minimal set for installation
5050
required_packages = [
5151
"attrs>=23.1.0,<24",
52-
"boto3>=1.33.3,<2.0",
52+
"boto3>=1.34.142,<2.0",
5353
"cloudpickle==2.2.1",
5454
"google-pasta",
5555
"numpy>=1.9.0,<2.0",

src/sagemaker/accept_types.py

+3
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ def retrieve_default(
8282
tolerate_deprecated_model: bool = False,
8383
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
8484
model_type: JumpStartModelType = JumpStartModelType.OPEN_WEIGHTS,
85+
config_name: Optional[str] = None,
8586
) -> str:
8687
"""Retrieves the default accept type for the model matching the given arguments.
8788
@@ -105,6 +106,7 @@ def retrieve_default(
105106
object, used for SageMaker interactions. If not
106107
specified, one is created using the default AWS configuration
107108
chain. (Default: sagemaker.jumpstart.constants.DEFAULT_JUMPSTART_SAGEMAKER_SESSION).
109+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
108110
Returns:
109111
str: The default accept type to use for the model.
110112
@@ -125,4 +127,5 @@ def retrieve_default(
125127
tolerate_deprecated_model=tolerate_deprecated_model,
126128
sagemaker_session=sagemaker_session,
127129
model_type=model_type,
130+
config_name=config_name,
128131
)

src/sagemaker/content_types.py

+3
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ def retrieve_default(
8282
tolerate_deprecated_model: bool = False,
8383
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
8484
model_type: JumpStartModelType = JumpStartModelType.OPEN_WEIGHTS,
85+
config_name: Optional[str] = None,
8586
) -> str:
8687
"""Retrieves the default content type for the model matching the given arguments.
8788
@@ -105,6 +106,7 @@ def retrieve_default(
105106
object, used for SageMaker interactions. If not
106107
specified, one is created using the default AWS configuration
107108
chain. (Default: sagemaker.jumpstart.constants.DEFAULT_JUMPSTART_SAGEMAKER_SESSION).
109+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
108110
Returns:
109111
str: The default content type to use for the model.
110112
@@ -125,6 +127,7 @@ def retrieve_default(
125127
tolerate_deprecated_model=tolerate_deprecated_model,
126128
sagemaker_session=sagemaker_session,
127129
model_type=model_type,
130+
config_name=config_name,
128131
)
129132

130133

src/sagemaker/deserializers.py

+3
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,7 @@ def retrieve_default(
102102
tolerate_deprecated_model: bool = False,
103103
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
104104
model_type: JumpStartModelType = JumpStartModelType.OPEN_WEIGHTS,
105+
config_name: Optional[str] = None,
105106
) -> BaseDeserializer:
106107
"""Retrieves the default deserializer for the model matching the given arguments.
107108
@@ -125,6 +126,7 @@ def retrieve_default(
125126
object, used for SageMaker interactions. If not
126127
specified, one is created using the default AWS configuration
127128
chain. (Default: sagemaker.jumpstart.constants.DEFAULT_JUMPSTART_SAGEMAKER_SESSION).
129+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
128130
Returns:
129131
BaseDeserializer: The default deserializer to use for the model.
130132
@@ -146,4 +148,5 @@ def retrieve_default(
146148
tolerate_deprecated_model=tolerate_deprecated_model,
147149
sagemaker_session=sagemaker_session,
148150
model_type=model_type,
151+
config_name=config_name,
149152
)

src/sagemaker/enums.py

+9
Original file line numberDiff line numberDiff line change
@@ -40,3 +40,12 @@ class RoutingStrategy(Enum):
4040
"""The endpoint routes requests to the specific instances that have
4141
more capacity to process them.
4242
"""
43+
44+
45+
class Tag(str, Enum):
46+
"""Enum class for tag keys to apply to models."""
47+
48+
OPTIMIZATION_JOB_NAME = "sagemaker-sdk:optimization-job-name"
49+
SPECULATIVE_DRAFT_MODEL_PROVIDER = "sagemaker-sdk:speculative-draft-model-provider"
50+
FINE_TUNING_MODEL_PATH = "sagemaker-sdk:fine-tuning-model-path"
51+
FINE_TUNING_JOB_NAME = "sagemaker-sdk:fine-tuning-job-name"

src/sagemaker/environment_variables.py

+3
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ def retrieve_default(
3737
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
3838
instance_type: Optional[str] = None,
3939
script: JumpStartScriptScope = JumpStartScriptScope.INFERENCE,
40+
config_name: Optional[str] = None,
4041
) -> Dict[str, str]:
4142
"""Retrieves the default container environment variables for the model matching the arguments.
4243
@@ -68,6 +69,7 @@ def retrieve_default(
6869
variables specific for the instance type.
6970
script (JumpStartScriptScope): The JumpStart script for which to retrieve environment
7071
variables.
72+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
7173
Returns:
7274
dict: The variables to use for the model.
7375
@@ -91,4 +93,5 @@ def retrieve_default(
9193
sagemaker_session=sagemaker_session,
9294
instance_type=instance_type,
9395
script=script,
96+
config_name=config_name,
9497
)

src/sagemaker/huggingface/llm_utils.py

+25
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,9 @@
1313
"""Functions for generating ECR image URIs for pre-built SageMaker Docker images."""
1414
from __future__ import absolute_import
1515

16+
import os
1617
from typing import Optional
18+
import importlib.util
1719

1820
import urllib.request
1921
from urllib.error import HTTPError, URLError
@@ -123,3 +125,26 @@ def get_huggingface_model_metadata(model_id: str, hf_hub_token: Optional[str] =
123125
"Did not find model metadata for the following HuggingFace Model ID %s" % model_id
124126
)
125127
return hf_model_metadata_json
128+
129+
130+
def download_huggingface_model_metadata(
131+
model_id: str, model_local_path: str, hf_hub_token: Optional[str] = None
132+
) -> None:
133+
"""Downloads the HuggingFace Model snapshot via HuggingFace API.
134+
135+
Args:
136+
model_id (str): The HuggingFace Model ID
137+
model_local_path (str): The local path to save the HuggingFace Model snapshot.
138+
hf_hub_token (str): The HuggingFace Hub Token
139+
140+
Raises:
141+
ImportError: If huggingface_hub is not installed.
142+
"""
143+
if not importlib.util.find_spec("huggingface_hub"):
144+
raise ImportError("Unable to import huggingface_hub, check if huggingface_hub is installed")
145+
146+
from huggingface_hub import snapshot_download
147+
148+
os.makedirs(model_local_path, exist_ok=True)
149+
logger.info("Downloading model %s from Hugging Face Hub to %s", model_id, model_local_path)
150+
snapshot_download(repo_id=model_id, local_dir=model_local_path, token=hf_hub_token)

src/sagemaker/hyperparameters.py

+3
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ def retrieve_default(
3737
tolerate_vulnerable_model: bool = False,
3838
tolerate_deprecated_model: bool = False,
3939
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
40+
config_name: Optional[str] = None,
4041
) -> Dict[str, str]:
4142
"""Retrieves the default training hyperparameters for the model matching the given arguments.
4243
@@ -69,6 +70,7 @@ def retrieve_default(
6970
object, used for SageMaker interactions. If not
7071
specified, one is created using the default AWS configuration
7172
chain. (Default: sagemaker.jumpstart.constants.DEFAULT_JUMPSTART_SAGEMAKER_SESSION).
73+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
7274
Returns:
7375
dict: The hyperparameters to use for the model.
7476
@@ -90,6 +92,7 @@ def retrieve_default(
9092
tolerate_vulnerable_model=tolerate_vulnerable_model,
9193
tolerate_deprecated_model=tolerate_deprecated_model,
9294
sagemaker_session=sagemaker_session,
95+
config_name=config_name,
9396
)
9497

9598

src/sagemaker/image_uris.py

+3
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,7 @@ def retrieve(
7171
inference_tool=None,
7272
serverless_inference_config=None,
7373
sagemaker_session=DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
74+
config_name=None,
7475
) -> str:
7576
"""Retrieves the ECR URI for the Docker image matching the given arguments.
7677
@@ -126,6 +127,7 @@ def retrieve(
126127
object, used for SageMaker interactions. If not
127128
specified, one is created using the default AWS configuration
128129
chain. (Default: sagemaker.jumpstart.constants.DEFAULT_JUMPSTART_SAGEMAKER_SESSION).
130+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
129131
130132
Returns:
131133
str: The ECR URI for the corresponding SageMaker Docker image.
@@ -166,6 +168,7 @@ def retrieve(
166168
tolerate_vulnerable_model,
167169
tolerate_deprecated_model,
168170
sagemaker_session=sagemaker_session,
171+
config_name=config_name,
169172
)
170173

171174
if training_compiler_config and (framework in [HUGGING_FACE_FRAMEWORK, "pytorch"]):

src/sagemaker/instance_types.py

+3
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ def retrieve_default(
3737
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
3838
training_instance_type: Optional[str] = None,
3939
model_type: JumpStartModelType = JumpStartModelType.OPEN_WEIGHTS,
40+
config_name: Optional[str] = None,
4041
) -> str:
4142
"""Retrieves the default instance type for the model matching the given arguments.
4243
@@ -67,6 +68,7 @@ def retrieve_default(
6768
Optionally supply this to get a inference instance type conditioned
6869
on the training instance, to ensure compatability of training artifact to inference
6970
instance. (Default: None).
71+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
7072
Returns:
7173
str: The default instance type to use for the model.
7274
@@ -92,6 +94,7 @@ def retrieve_default(
9294
sagemaker_session=sagemaker_session,
9395
training_instance_type=training_instance_type,
9496
model_type=model_type,
97+
config_name=config_name,
9598
)
9699

97100

src/sagemaker/jumpstart/artifacts/environment_variables.py

+7
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ def _retrieve_default_environment_variables(
4040
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
4141
instance_type: Optional[str] = None,
4242
script: JumpStartScriptScope = JumpStartScriptScope.INFERENCE,
43+
config_name: Optional[str] = None,
4344
) -> Dict[str, str]:
4445
"""Retrieves the inference environment variables for the model matching the given arguments.
4546
@@ -71,6 +72,7 @@ def _retrieve_default_environment_variables(
7172
environment variables specific for the instance type.
7273
script (JumpStartScriptScope): The JumpStart script for which to retrieve
7374
environment variables.
75+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
7476
Returns:
7577
dict: the inference environment variables to use for the model.
7678
"""
@@ -88,6 +90,7 @@ def _retrieve_default_environment_variables(
8890
tolerate_vulnerable_model=tolerate_vulnerable_model,
8991
tolerate_deprecated_model=tolerate_deprecated_model,
9092
sagemaker_session=sagemaker_session,
93+
config_name=config_name,
9194
)
9295

9396
default_environment_variables: Dict[str, str] = {}
@@ -126,6 +129,7 @@ def _retrieve_default_environment_variables(
126129
tolerate_deprecated_model=tolerate_deprecated_model,
127130
sagemaker_session=sagemaker_session,
128131
instance_type=instance_type,
132+
config_name=config_name,
129133
)
130134
)
131135

@@ -173,6 +177,7 @@ def _retrieve_gated_model_uri_env_var_value(
173177
tolerate_deprecated_model: bool = False,
174178
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
175179
instance_type: Optional[str] = None,
180+
config_name: Optional[str] = None,
176181
) -> Optional[str]:
177182
"""Retrieves the gated model env var URI matching the given arguments.
178183
@@ -198,6 +203,7 @@ def _retrieve_gated_model_uri_env_var_value(
198203
chain. (Default: sagemaker.jumpstart.constants.DEFAULT_JUMPSTART_SAGEMAKER_SESSION).
199204
instance_type (str): An instance type to optionally supply in order to get
200205
environment variables specific for the instance type.
206+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
201207
202208
Returns:
203209
Optional[str]: the s3 URI to use for the environment variable, or None if the model does not
@@ -220,6 +226,7 @@ def _retrieve_gated_model_uri_env_var_value(
220226
tolerate_vulnerable_model=tolerate_vulnerable_model,
221227
tolerate_deprecated_model=tolerate_deprecated_model,
222228
sagemaker_session=sagemaker_session,
229+
config_name=config_name,
223230
)
224231

225232
s3_key: Optional[str] = (

src/sagemaker/jumpstart/artifacts/hyperparameters.py

+3
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ def _retrieve_default_hyperparameters(
3737
tolerate_deprecated_model: bool = False,
3838
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
3939
instance_type: Optional[str] = None,
40+
config_name: Optional[str] = None,
4041
):
4142
"""Retrieves the training hyperparameters for the model matching the given arguments.
4243
@@ -69,6 +70,7 @@ def _retrieve_default_hyperparameters(
6970
chain. (Default: sagemaker.jumpstart.constants.DEFAULT_JUMPSTART_SAGEMAKER_SESSION).
7071
instance_type (str): An instance type to optionally supply in order to get hyperparameters
7172
specific for the instance type.
73+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
7274
Returns:
7375
dict: the hyperparameters to use for the model.
7476
"""
@@ -86,6 +88,7 @@ def _retrieve_default_hyperparameters(
8688
tolerate_vulnerable_model=tolerate_vulnerable_model,
8789
tolerate_deprecated_model=tolerate_deprecated_model,
8890
sagemaker_session=sagemaker_session,
91+
config_name=config_name,
8992
)
9093

9194
default_hyperparameters: Dict[str, str] = {}

src/sagemaker/jumpstart/artifacts/image_uris.py

+4
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ def _retrieve_image_uri(
4747
tolerate_vulnerable_model: bool = False,
4848
tolerate_deprecated_model: bool = False,
4949
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
50+
config_name: Optional[str] = None,
5051
):
5152
"""Retrieves the container image URI for JumpStart models.
5253
@@ -98,6 +99,7 @@ def _retrieve_image_uri(
9899
object, used for SageMaker interactions. If not
99100
specified, one is created using the default AWS configuration
100101
chain. (Default: sagemaker.jumpstart.constants.DEFAULT_JUMPSTART_SAGEMAKER_SESSION).
102+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
101103
Returns:
102104
str: the ECR URI for the corresponding SageMaker Docker image.
103105
@@ -120,6 +122,7 @@ def _retrieve_image_uri(
120122
tolerate_vulnerable_model=tolerate_vulnerable_model,
121123
tolerate_deprecated_model=tolerate_deprecated_model,
122124
sagemaker_session=sagemaker_session,
125+
config_name=config_name,
123126
)
124127

125128
if image_scope == JumpStartScriptScope.INFERENCE:
@@ -213,4 +216,5 @@ def _retrieve_image_uri(
213216
distribution=distribution,
214217
base_framework_version=base_framework_version_override or base_framework_version,
215218
training_compiler_config=training_compiler_config,
219+
config_name=config_name,
216220
)

src/sagemaker/jumpstart/artifacts/incremental_training.py

+3
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ def _model_supports_incremental_training(
3434
tolerate_vulnerable_model: bool = False,
3535
tolerate_deprecated_model: bool = False,
3636
sagemaker_session: Session = DEFAULT_JUMPSTART_SAGEMAKER_SESSION,
37+
config_name: Optional[str] = None,
3738
) -> bool:
3839
"""Returns True if the model supports incremental training.
3940
@@ -57,6 +58,7 @@ def _model_supports_incremental_training(
5758
object, used for SageMaker interactions. If not
5859
specified, one is created using the default AWS configuration
5960
chain. (Default: sagemaker.jumpstart.constants.DEFAULT_JUMPSTART_SAGEMAKER_SESSION).
61+
config_name (Optional[str]): Name of the JumpStart Model config to apply. (Default: None).
6062
Returns:
6163
bool: the support status for incremental training.
6264
"""
@@ -74,6 +76,7 @@ def _model_supports_incremental_training(
7476
tolerate_vulnerable_model=tolerate_vulnerable_model,
7577
tolerate_deprecated_model=tolerate_deprecated_model,
7678
sagemaker_session=sagemaker_session,
79+
config_name=config_name,
7780
)
7881

7982
return model_specs.supports_incremental_training()

0 commit comments

Comments
 (0)