From 2a34e41aed2df12f49a570c3fad8d30a469eef6c Mon Sep 17 00:00:00 2001 From: Lauren Yu <6631887+laurenyu@users.noreply.github.com> Date: Thu, 21 May 2020 14:10:56 -0700 Subject: [PATCH] doc: consolidate framework version and image information This changes the following: - for MXNet, TF, and PyTorch, the docs point to the DLC documentation - for RL, the docs point to the sagemaker-rl-container repository - for Chainer, src/sagemaker/chainer/README.rst has been removed in favor of doc/using_chainer.rst --- README.rst | 162 +++++----------------------- doc/using_chainer.rst | 57 +++++++++- doc/using_mxnet.rst | 14 +-- doc/using_pytorch.rst | 11 +- doc/using_rl.rst | 40 +------ doc/using_tf.rst | 7 +- src/sagemaker/chainer/README.rst | 74 ------------- src/sagemaker/mxnet/README.rst | 48 --------- src/sagemaker/pytorch/README.rst | 103 ------------------ src/sagemaker/rl/README.rst | 50 --------- src/sagemaker/tensorflow/README.rst | 81 -------------- 11 files changed, 100 insertions(+), 547 deletions(-) delete mode 100644 src/sagemaker/chainer/README.rst delete mode 100644 src/sagemaker/mxnet/README.rst delete mode 100644 src/sagemaker/pytorch/README.rst delete mode 100644 src/sagemaker/rl/README.rst delete mode 100644 src/sagemaker/tensorflow/README.rst diff --git a/README.rst b/README.rst index 96ea315c79..ccac78dc99 100644 --- a/README.rst +++ b/README.rst @@ -39,31 +39,31 @@ For detailed API reference please go to: `Read the Docs `__ -2. `Using the SageMaker Python SDK `__ -3. `MXNet SageMaker Estimators <#mxnet-sagemaker-estimators>`__ -4. `TensorFlow SageMaker Estimators <#tensorflow-sagemaker-estimators>`__ -5. `Chainer SageMaker Estimators <#chainer-sagemaker-estimators>`__ -6. `PyTorch SageMaker Estimators <#pytorch-sagemaker-estimators>`__ -7. `Scikit-learn SageMaker Estimators <#scikit-learn-sagemaker-estimators>`__ -8. `XGBoost SageMaker Estimators <#xgboost-sagemaker-estimators>`__ -9. `SageMaker Reinforcement Learning Estimators <#sagemaker-reinforcement-learning-estimators>`__ -10. `SageMaker SparkML Serving <#sagemaker-sparkml-serving>`__ -11. `AWS SageMaker Estimators <#aws-sagemaker-estimators>`__ -12. `Using SageMaker AlgorithmEstimators `__ -13. `Consuming SageMaker Model Packages `__ -14. `BYO Docker Containers with SageMaker Estimators `__ -15. `SageMaker Automatic Model Tuning `__ -16. `SageMaker Batch Transform `__ -17. `Secure Training and Inference with VPC `__ -18. `BYO Model `__ -19. `Inference Pipelines `__ -20. `Amazon SageMaker Operators for Kubernetes <#amazon-sagemaker-operators-for-kubernetes>`__ -21. `Amazon SageMaker Operators in Apache Airflow <#sagemaker-workflow>`__ -22. `SageMaker Autopilot <#sagemaker-autopilot>`__ -23. `Model Monitoring <#amazon-sagemaker-model-monitoring>`__ -24. `SageMaker Debugger <#amazon-sagemaker-debugger>`__ -25. `SageMaker Processing <#amazon-sagemaker-processing>`__ +#. `Installing SageMaker Python SDK <#installing-the-sagemaker-python-sdk>`__ +#. `Using the SageMaker Python SDK `__ +#. `Using MXNet `__ +#. `Using TensorFlow `__ +#. `Using Chainer `__ +#. `Using PyTorch `__ +#. `Scikit-learn SageMaker Estimators <#scikit-learn-sagemaker-estimators>`__ +#. `XGBoost SageMaker Estimators <#xgboost-sagemaker-estimators>`__ +#. `SageMaker Reinforcement Learning Estimators `__ +#. `SageMaker SparkML Serving <#sagemaker-sparkml-serving>`__ +#. `AWS SageMaker Estimators <#aws-sagemaker-estimators>`__ +#. `Using SageMaker AlgorithmEstimators `__ +#. `Consuming SageMaker Model Packages `__ +#. `BYO Docker Containers with SageMaker Estimators `__ +#. `SageMaker Automatic Model Tuning `__ +#. `SageMaker Batch Transform `__ +#. `Secure Training and Inference with VPC `__ +#. `BYO Model `__ +#. `Inference Pipelines `__ +#. `Amazon SageMaker Operators for Kubernetes <#amazon-sagemaker-operators-for-kubernetes>`__ +#. `Amazon SageMaker Operators in Apache Airflow <#sagemaker-workflow>`__ +#. `SageMaker Autopilot <#sagemaker-autopilot>`__ +#. `Model Monitoring `__ +#. `SageMaker Debugger `__ +#. `SageMaker Processing `__ Installing the SageMaker Python SDK @@ -197,73 +197,6 @@ Preview the site with a Python web server: View the website by visiting http://localhost:8000 - -MXNet SageMaker Estimators --------------------------- - -By using MXNet SageMaker Estimators, you can train and host MXNet models on Amazon SageMaker. - -Supported versions of MXNet: ``0.12.1``, ``1.0.0``, ``1.1.0``, ``1.2.1``, ``1.3.0``, ``1.4.0``, ``1.4.1``, ``1.6.0``. - -Supported versions of MXNet for Elastic Inference: ``1.3.0``, ``1.4.0``, ``1.4.1``, ``1.5.1``. - -We recommend that you use the latest supported version, because that's where we focus most of our development efforts. - -For more information, see `Using MXNet with the SageMaker Python SDK`_. - -.. _Using MXNet with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_mxnet.html - - -TensorFlow SageMaker Estimators -------------------------------- - -By using TensorFlow SageMaker Estimators, you can train and host TensorFlow models on Amazon SageMaker. - -Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``, ``1.9.0``, ``1.10.0``, ``1.11.0``, ``1.12.0``, ``1.13.1``, ``1.14.0``, ``1.15.0``, ``1.15.2``, ``2.0.0``, ``2.0.1``, ``2.1.0``. - -Supported versions of TensorFlow for Elastic Inference: ``1.11.0``, ``1.12.0``, ``1.13.1``, ``1.14.0``, ``1.15.0``, ``2.0.0``. - -We recommend that you use the latest supported version, because that's where we focus most of our development efforts. - -For more information, see `Using TensorFlow with the SageMaker Python SDK`_. - -.. _Using TensorFlow with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_tf.html - - -Chainer SageMaker Estimators ----------------------------- - -By using Chainer SageMaker Estimators, you can train and host Chainer models on Amazon SageMaker. - -Supported versions of Chainer: ``4.0.0``, ``4.1.0``, ``5.0.0``. - -We recommend that you use the latest supported version, because that's where we focus most of our development efforts. - -For more information about Chainer, see https://github.com/chainer/chainer. - -For more information about Chainer SageMaker Estimators, see `Using Chainer with the SageMaker Python SDK`_. - -.. _Using Chainer with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_chainer.html - - -PyTorch SageMaker Estimators ----------------------------- - -With PyTorch SageMaker Estimators, you can train and host PyTorch models on Amazon SageMaker. - -Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``, ``1.4.0``, ``1.5.0``. - -Supported versions of PyTorch for Elastic Inference: ``1.3.1``. - -We recommend that you use the latest supported version, because that's where we focus most of our development efforts. - -For more information about PyTorch, see https://github.com/pytorch/pytorch. - -For more information about PyTorch SageMaker Estimators, see `Using PyTorch with the SageMaker Python SDK`_. - -.. _Using PyTorch with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_pytorch.html - - Scikit-learn SageMaker Estimators --------------------------------- @@ -295,22 +228,6 @@ For more information about XGBoost SageMaker Estimators, see `Using XGBoost with .. _Using XGBoost with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_xgboost.html -SageMaker Reinforcement Learning Estimators -------------------------------------------- - -With Reinforcement Learning (RL) Estimators, you can use reinforcement learning to train models on Amazon SageMaker. - -Supported versions of Coach: ``0.10.1``, ``0.11.1`` with TensorFlow, ``0.11.0`` with TensorFlow or MXNet. -For more information about Coach, see https://github.com/NervanaSystems/coach - -Supported versions of Ray: ``0.5.3``, ``0.6.5`` with TensorFlow. -For more information about Ray, see https://github.com/ray-project/ray - -For more information about SageMaker RL Estimators, see `SageMaker Reinforcement Learning Estimators`_. - -.. _SageMaker Reinforcement Learning Estimators: src/sagemaker/rl/README.rst - - SageMaker SparkML Serving ------------------------- @@ -385,32 +302,3 @@ on your data, and hosts a series of models on an Inference Pipeline. For more information about SageMaker Autopilot, see `SageMaker Autopilot`_. .. _SageMaker Autopilot: src/sagemaker/automl/README.rst - -Amazon SageMaker Model Monitoring ---------------------------------- - -You can use Amazon SageMaker Model Monitoring to automatically detect concept drift by monitoring your machine learning models. - -For more information, see `Amazon SageMaker Model Monitoring`_. - -.. _Amazon SageMaker Model Monitoring: https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_model_monitoring.html - -Amazon SageMaker Debugger -------------------------- - -You can use Amazon SageMaker Debugger to automatically detect anomalies while training your machine learning models. - -For more information, see `Amazon SageMaker Debugger`_. - -.. _Amazon SageMaker Debugger: https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html - - -Amazon SageMaker Processing ---------------------------------- - -You can use Amazon SageMaker Processing to perform data processing tasks such as data pre- and post-processing, feature engineering, data validation, and model evaluation - - -For more information, see `Amazon SageMaker Processing`_. - -.. _Amazon SageMaker Processing: https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_processing.html diff --git a/doc/using_chainer.rst b/doc/using_chainer.rst index 1d1588ef01..64302cc005 100644 --- a/doc/using_chainer.rst +++ b/doc/using_chainer.rst @@ -4,7 +4,11 @@ Using Chainer with the SageMaker Python SDK With Chainer Estimators, you can train and host Chainer models on Amazon SageMaker. -For information about supported versions of Chainer, see the `Chainer README `__. +Supported versions of Chainer: ``4.0.0``, ``4.1.0``, ``5.0.0``. + +We recommend that you use the latest supported version, because that's where we focus most of our development efforts. + +For more information about Chainer, see https://github.com/chainer/chainer. For general information about using the SageMaker Python SDK, see :ref:`overview:Using the SageMaker Python SDK`. @@ -638,6 +642,51 @@ The following are optional arguments. When you create a ``Chainer`` object, you SageMaker Chainer Docker containers *********************************** -You can visit the SageMaker Chainer containers repository here: https://github.com/aws/sagemaker-chainer-container - -For information about SageMaker Chainer Docker containers and their dependencies, see `SageMaker Chainer Docker containers `_. +When training and deploying training scripts, SageMaker runs your Python script in a Docker container with several +libraries installed. When creating the Estimator and calling deploy to create the SageMaker Endpoint, you can control +the environment your script runs in. + +SageMaker runs Chainer Estimator scripts in either Python 2.7 or Python 3.5. You can select the Python version by +passing a py_version keyword arg to the Chainer Estimator constructor. Setting this to py3 (the default) will cause your +training script to be run on Python 3.5. Setting this to py2 will cause your training script to be run on Python 2.7 +This Python version applies to both the Training Job, created by fit, and the Endpoint, created by deploy. + +The Chainer Docker images have the following dependencies installed: + ++-----------------------------+-------------+-------------+-------------+ +| Dependencies | chainer 4.0 | chainer 4.1 | chainer 5.0 | ++-----------------------------+-------------+-------------+-------------+ +| chainer | 4.0.0 | 4.1.0 | 5.0.0 | ++-----------------------------+-------------+-------------+-------------+ +| chainercv | 0.9.0 | 0.10.0 | 0.10.0 | ++-----------------------------+-------------+-------------+-------------+ +| chainermn | 1.2.0 | 1.3.0 | N/A | ++-----------------------------+-------------+-------------+-------------+ +| CUDA (GPU image only) | 9.0 | 9.0 | 9.0 | ++-----------------------------+-------------+-------------+-------------+ +| cupy | 4.0.0 | 4.1.0 | 5.0.0 | ++-----------------------------+-------------+-------------+-------------+ +| matplotlib | 2.2.0 | 2.2.0 | 2.2.0 | ++-----------------------------+-------------+-------------+-------------+ +| mpi4py | 3.0.0 | 3.0.0 | 3.0.0 | ++-----------------------------+-------------+-------------+-------------+ +| numpy | 1.14.3 | 1.15.3 | 1.15.4 | ++-----------------------------+-------------+-------------+-------------+ +| opencv-python | 3.4.0.12 | 3.4.0.12 | 3.4.0.12 | ++-----------------------------+-------------+-------------+-------------+ +| Pillow | 5.1.0 | 5.3.0 | 5.3.0 | ++-----------------------------+-------------+-------------+-------------+ +| Python | 2.7 or 3.5 | 2.7 or 3.5 | 2.7 or 3.5 | ++-----------------------------+-------------+-------------+-------------+ + +The Docker images extend Ubuntu 16.04. + +You must select a version of Chainer by passing a ``framework_version`` keyword arg to the Chainer Estimator +constructor. Currently supported versions are listed in the above table. You can also set framework_version to only +specify major and minor version, which will cause your training script to be run on the latest supported patch +version of that minor version. + +Alternatively, you can build your own image by following the instructions in the SageMaker Chainer containers +repository, and passing ``image_name`` to the Chainer Estimator constructor. + +You can visit the SageMaker Chainer containers repository at https://github.com/aws/sagemaker-chainer-container diff --git a/doc/using_mxnet.rst b/doc/using_mxnet.rst index a4cd8391f0..50c98ca5b6 100644 --- a/doc/using_mxnet.rst +++ b/doc/using_mxnet.rst @@ -4,7 +4,9 @@ Use MXNet with the SageMaker Python SDK With the SageMaker Python SDK, you can train and host MXNet models on Amazon SageMaker. -For information about supported versions of MXNet, see the `MXNet README `__. +For information about supported versions of MXNet, see the `AWS documentation `__. + +We recommend that you use the latest supported version because that's where we focus our development efforts. For general information about using the SageMaker Python SDK, see :ref:`overview:Using the SageMaker Python SDK`. @@ -807,9 +809,9 @@ For information about the different MXNet-related classes in the SageMaker Pytho SageMaker MXNet Containers ************************** -For information about SageMaker MXNet containers, see the following topics: - -- training: https://github.com/aws/sagemaker-mxnet-container -- serving: https://github.com/aws/sagemaker-mxnet-serving-container +For information about the SageMaker MXNet containers, see: -For information about the dependencies installed in SageMaker MXNet containers, see https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/mxnet/README.rst#sagemaker-mxnet-containers. +- `SageMaker MXNet training toolkit `_ +- `SageMaker MXNet serving toolkit `_ +- `Deep Learning Container (DLC) Dockerfiles for MXNet `_ +- `Deep Learning Container (DLC) Images `_ and `release notes `_ diff --git a/doc/using_pytorch.rst b/doc/using_pytorch.rst index 995e030067..4c8cde6e98 100644 --- a/doc/using_pytorch.rst +++ b/doc/using_pytorch.rst @@ -4,9 +4,7 @@ Using PyTorch with the SageMaker Python SDK With PyTorch Estimators and Models, you can train and host PyTorch models on Amazon SageMaker. -Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``, ``1.4.0``, ``1.5.0``. - -Supported versions of PyTorch for Elastic Inference: ``1.3.1``. +For information about supported versions of PyTorch, see the `AWS documentation `__. We recommend that you use the latest supported version because that's where we focus our development efforts. @@ -758,6 +756,9 @@ The following are optional arguments. When you create a ``PyTorch`` object, you SageMaker PyTorch Docker Containers *********************************** -For information about SageMaker PyTorch containers, see `the SageMaker PyTorch container repository `_ and `SageMaker PyTorch Serving container repository `__. +For information about the SageMaker PyTorch containers, see: -For information about SageMaker PyTorch container dependencies, see `SageMaker PyTorch Containers `_. +- `SageMaker PyTorch training toolkit `_ +- `SageMaker PyTorch serving toolkit `_ +- `Deep Learning Container (DLC) Dockerfiles for PyTorch `_ +- `Deep Learning Container (DLC) Images `_ and `release notes `_ diff --git a/doc/using_rl.rst b/doc/using_rl.rst index 926613f15f..5bba4f3dc9 100644 --- a/doc/using_rl.rst +++ b/doc/using_rl.rst @@ -6,11 +6,7 @@ Using Reinforcement Learning with the SageMaker Python SDK With Reinforcement Learning (RL) Estimators, you can train reinforcement learning models on Amazon SageMaker. -Supported versions of Coach: ``0.11.1``, ``0.10.1`` with TensorFlow, ``0.11.0`` with TensorFlow or MXNet. -For more information about Coach, see https://github.com/NervanaSystems/coach - -Supported versions of Ray: ``0.5.3`` with TensorFlow. -For more information about Ray, see https://github.com/ray-project/ray +For supported RL toolkits and their versions, see https://github.com/aws/sagemaker-rl-container/#rl-images-provided-by-sagemaker RL Training ----------- @@ -287,36 +283,4 @@ These are also available in SageMaker Notebook Instance hosted Jupyter notebooks SageMaker RL Docker Containers ------------------------------ -When training and deploying training scripts, SageMaker runs your Python script in a Docker container with several -libraries installed. When creating the Estimator and calling deploy to create the SageMaker Endpoint, you can control -the environment your script runs in. - -SageMaker runs RL Estimator scripts in either Python 3.5 for MXNet or Python 3.6 for TensorFlow. - -The Docker images have the following dependencies installed: - -+-------------------------+-------------------+-------------------+-------------------+ -| Dependencies | Coach 0.10.1 | Coach 0.11.0 | Ray 0.5.3 | -+-------------------------+-------------------+-------------------+-------------------+ -| Python | 3.6 | 3.5(MXNet) or | 3.6 | -| | | 3.6(TensorFlow) | | -+-------------------------+-------------------+-------------------+-------------------+ -| CUDA (GPU image only) | 9.0 | 9.0 | 9.0 | -+-------------------------+-------------------+-------------------+-------------------+ -| DL Framework | TensorFlow-1.11.0 | MXNet-1.3.0 or | TensorFlow-1.11.0 | -| | | TensorFlow-1.11.0 | | -+-------------------------+-------------------+-------------------+-------------------+ -| gym | 0.10.5 | 0.10.5 | 0.10.5 | -+-------------------------+-------------------+-------------------+-------------------+ - -The Docker images extend Ubuntu 16.04. - -You can select version of by passing a ``framework_version`` keyword arg to the RL Estimator constructor. -Currently supported versions are listed in the above table. You can also set ``framework_version`` to only specify major and -minor version, which will cause your training script to be run on the latest supported patch version of that minor -version. - -Alternatively, you can build your own image by following the instructions in the SageMaker RL containers -repository, and passing ``image_name`` to the RL Estimator constructor. - -You can visit `the SageMaker RL containers repository `_. +For more about the Docker images themselves, visit `the SageMaker RL containers repository `_. diff --git a/doc/using_tf.rst b/doc/using_tf.rst index a222fb1160..0540734ee6 100644 --- a/doc/using_tf.rst +++ b/doc/using_tf.rst @@ -951,4 +951,9 @@ For information about the different TensorFlow-related classes in the SageMaker SageMaker TensorFlow Docker containers ************************************** -For information about SageMaker TensorFlow Docker containers and their dependencies, see `SageMaker TensorFlow Docker containers `_. +For information about the SageMaker TensorFlow containers, see: + +- `SageMaker TensorFlow training toolkit `_ +- `SageMaker TensorFlow serving toolkit `_ +- `Deep Learning Container (DLC) Dockerfiles for TensorFlow `_ +- `Deep Learning Container (DLC) Images `_ and `release notes `_ diff --git a/src/sagemaker/chainer/README.rst b/src/sagemaker/chainer/README.rst deleted file mode 100644 index c3a2aa9b2b..0000000000 --- a/src/sagemaker/chainer/README.rst +++ /dev/null @@ -1,74 +0,0 @@ -======================================= -Chainer SageMaker Estimators and Models -======================================= - -With Chainer Estimators, you can train and host Chainer models on Amazon SageMaker. - -Supported versions of Chainer: ``4.0.0``, ``4.1.0``, ``5.0.0`` - -You can visit the Chainer repository at https://github.com/chainer/chainer. - -For information about using Chainer with the SageMaker Python SDK, see https://sagemaker.readthedocs.io/en/stable/using_chainer.html. - -Chainer Training Examples -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Amazon provides several example Jupyter notebooks that demonstrate end-to-end training on Amazon SageMaker using Chainer. -Please refer to: - -https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk - -These are also available in SageMaker Notebook Instance hosted Jupyter notebooks under the "sample notebooks" folder. - - -SageMaker Chainer Docker containers -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -When training and deploying training scripts, SageMaker runs your Python script in a Docker container with several -libraries installed. When creating the Estimator and calling deploy to create the SageMaker Endpoint, you can control -the environment your script runs in. - -SageMaker runs Chainer Estimator scripts in either Python 2.7 or Python 3.5. You can select the Python version by -passing a py_version keyword arg to the Chainer Estimator constructor. Setting this to py3 (the default) will cause your -training script to be run on Python 3.5. Setting this to py2 will cause your training script to be run on Python 2.7 -This Python version applies to both the Training Job, created by fit, and the Endpoint, created by deploy. - -The Chainer Docker images have the following dependencies installed: - -+-----------------------------+-------------+-------------+-------------+ -| Dependencies | chainer 4.0 | chainer 4.1 | chainer 5.0 | -+-----------------------------+-------------+-------------+-------------+ -| chainer | 4.0.0 | 4.1.0 | 5.0.0 | -+-----------------------------+-------------+-------------+-------------+ -| chainercv | 0.9.0 | 0.10.0 | 0.10.0 | -+-----------------------------+-------------+-------------+-------------+ -| chainermn | 1.2.0 | 1.3.0 | N/A | -+-----------------------------+-------------+-------------+-------------+ -| CUDA (GPU image only) | 9.0 | 9.0 | 9.0 | -+-----------------------------+-------------+-------------+-------------+ -| cupy | 4.0.0 | 4.1.0 | 5.0.0 | -+-----------------------------+-------------+-------------+-------------+ -| matplotlib | 2.2.0 | 2.2.0 | 2.2.0 | -+-----------------------------+-------------+-------------+-------------+ -| mpi4py | 3.0.0 | 3.0.0 | 3.0.0 | -+-----------------------------+-------------+-------------+-------------+ -| numpy | 1.14.3 | 1.15.3 | 1.15.4 | -+-----------------------------+-------------+-------------+-------------+ -| opencv-python | 3.4.0.12 | 3.4.0.12 | 3.4.0.12 | -+-----------------------------+-------------+-------------+-------------+ -| Pillow | 5.1.0 | 5.3.0 | 5.3.0 | -+-----------------------------+-------------+-------------+-------------+ -| Python | 2.7 or 3.5 | 2.7 or 3.5 | 2.7 or 3.5 | -+-----------------------------+-------------+-------------+-------------+ - -The Docker images extend Ubuntu 16.04. - -You must select a version of Chainer by passing a ``framework_version`` keyword arg to the Chainer Estimator -constructor. Currently supported versions are listed in the above table. You can also set framework_version to only -specify major and minor version, which will cause your training script to be run on the latest supported patch -version of that minor version. - -Alternatively, you can build your own image by following the instructions in the SageMaker Chainer containers -repository, and passing ``image_name`` to the Chainer Estimator constructor. - -You can visit the SageMaker Chainer containers repository here: https://github.com/aws/sagemaker-chainer-containers/ diff --git a/src/sagemaker/mxnet/README.rst b/src/sagemaker/mxnet/README.rst deleted file mode 100644 index 2368dcad58..0000000000 --- a/src/sagemaker/mxnet/README.rst +++ /dev/null @@ -1,48 +0,0 @@ -========================================= -Using MXNet with the SageMaker Python SDK -========================================= - -With the SageMaker Python SDK, you can train and host MXNet models on Amazon SageMaker. - -Supported versions of MXNet: ``0.12.1``, ``1.0.0``, ``1.1.0``, ``1.2.1``, ``1.3.0``, ``1.4.0``, ``1.4.1``, ``1.6.0``. - -Supported versions of MXNet for Elastic Inference: ``1.3.0``, ``1.4.0``, ``1.4.1``, ``1.5.1``. - -Supported versions of MXNet for Inferentia: ``1.5.1``. - -For information about using MXNet with the SageMaker Python SDK, see https://sagemaker.readthedocs.io/en/stable/using_mxnet.html. - -SageMaker MXNet Containers --------------------------- - -When training and deploying training scripts, SageMaker runs your Python script in a Docker container with several libraries installed. When creating the Estimator and calling deploy to create the SageMaker Endpoint, you can control the environment your script runs in. - -SageMaker runs MXNet scripts in either Python 2.7 or Python 3.6. You can select the Python version by passing a ``py_version`` keyword arg to the MXNet Estimator constructor. Setting this to ``py2`` (the default) will cause your training script to be run on Python 2.7. Setting this to ``py3`` will cause your training script to be run on Python 3.6. This Python version applies to both the Training Job, created by fit, and the Endpoint, created by deploy. - -Your MXNet training script will be run on version 1.2.1 by default. (See below for how to choose a different version, and currently supported versions.) The decision to use the GPU or CPU version of MXNet is made by the ``train_instance_type``, set on the MXNet constructor. If you choose a GPU instance type, your training job will be run on a GPU version of MXNet. If you choose a CPU instance type, your training job will be run on a CPU version of MXNet. Similarly, when you call deploy, specifying a GPU or CPU deploy_instance_type, will control which MXNet build your Endpoint runs. - -The Docker images have the following dependencies installed: - -+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+ -| Dependencies | MXNet 0.12.1 | MXNet 1.0.0 | MXNet 1.1.0 | MXNet 1.2.1 | MXNet 1.3.0 | MXNet 1.4.0 | MXNet 1.4.1 | MXNet 1.6.0 | -+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+ -| Python | 2.7 or 3.5 | 2.7 or 3.5| 2.7 or 3.5| 2.7 or 3.5| 2.7 or 3.5| 2.7 or 3.6| 2.7 or 3.6| 2.7 or 3.6| -+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+ -| CUDA (GPU image only) | 9.0 | 9.0 | 9.0 | 9.0 | 9.0 | 9.2 | 10.0 | 10.1 | -+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+ -| numpy | 1.13.3 | 1.13.3 | 1.13.3 | 1.14.5 | 1.14.6 | 1.16.3 | 1.14.5 | 1.17.4 | -+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+ -| onnx | N/A | N/A | N/A | 1.2.1 | 1.2.1 | 1.4.1 | 1.4.1 | 1.4.1 | -+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+ -| keras-mxnet | N/A | N/A | N/A | N/A | 2.2.2 | 2.2.4.1 | 2.2.4.1 | 2.2.4.1 | -+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+ - -The Docker images extend Ubuntu 16.04. - -You can select version of MXNet by passing a ``framework_version`` keyword arg to the MXNet Estimator constructor. Currently supported versions are listed in the above table. You can also set ``framework_version`` to only specify major and minor version, e.g ``1.4``, which will cause your training script to be run on the latest supported patch version of that minor version, which in this example would be 1.4.1. -Alternatively, you can build your own image by following the instructions in the SageMaker MXNet containers repository, and passing ``image_name`` to the MXNet Estimator constructor. - -You can visit the SageMaker MXNet container repositories here: - -- training: https://github.com/aws/sagemaker-mxnet-container -- serving: https://github.com/aws/sagemaker-mxnet-serving-container diff --git a/src/sagemaker/pytorch/README.rst b/src/sagemaker/pytorch/README.rst deleted file mode 100644 index 245181976a..0000000000 --- a/src/sagemaker/pytorch/README.rst +++ /dev/null @@ -1,103 +0,0 @@ -======================================= -SageMaker PyTorch Estimators and Models -======================================= - -With PyTorch Estimators and Models, you can train and host PyTorch models on Amazon SageMaker. - -Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``, ``1.4.0``, ``1.5.0``. - -Supported versions of PyTorch for Elastic Inference: ``1.3.1``. - -We recommend that you use the latest supported version, because that's where we focus most of our development efforts. - -You can visit the PyTorch repository at https://github.com/pytorch/pytorch. - -For information about using PyTorch with the SageMaker Python SDK, see https://sagemaker.readthedocs.io/en/stable/using_pytorch.html. - -PyTorch Training Examples -------------------------- - -Amazon provides several example Jupyter notebooks that demonstrate end-to-end training on Amazon SageMaker using PyTorch. -Please refer to: - -https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk - -These are also available in SageMaker Notebook Instance hosted Jupyter notebooks under the sample notebooks folder. - - -SageMaker PyTorch Docker Containers ------------------------------------ - -When training and deploying training scripts, SageMaker runs your Python script in a Docker container with several -libraries installed. When creating the Estimator and calling deploy to create the SageMaker Endpoint, you can control -the environment your script runs in. - -SageMaker runs PyTorch Estimator scripts in either Python 2 or Python 3. You can select the Python version by -passing a ``py_version`` keyword arg to the PyTorch Estimator constructor. Setting this to ``py3`` (the default) will cause your -training script to be run on Python 3.5. Setting this to ``py2`` will cause your training script to be run on Python 2.7 -This Python version applies to both the Training Job, created by fit, and the Endpoint, created by deploy. - -The PyTorch Docker images have the following dependencies installed: - -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| Dependencies | pytorch 0.4.0 | pytorch 1.0.0 | pytorch 1.1.0 | pytorch 1.2.0 | pytorch 1.3.1 | pytorch 1.4.0 | pytorch 1.5.0 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| boto3 | >=1.7.35 | >=1.9.11 | 1.9.82 | 1.9.249 | 1.10.34 | 1.12.4 | 1.10.32 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| botocore | >=1.10.35 | >=1.12.11 | >= 1.12.11 | 1.12.249 | 1.13.34 | 1.15.4 | 1.16.3 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| CUDA (GPU image only) | 9.0 | 9.0 | 10.1 | 10.0 | 10.1 | 10.1 | 10.1 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| numpy | >=1.14.3 | >=1.15.2 | 1.16.4 | 1.16.4 | 1.16.4 | 1.16.4 | 1.16.4 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| Pillow | >=5.1.0 | >=5.2.0 | 6.0.0 | 5.4.1 | 6.2.1 | 6.2.0 | 7.1.0 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| pip | >=10.0.1 | >=18.0 | >=18.0 | 19.3 | 19.3.1 | 20.0.2 | 20.0.2 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| python-dateutil | >=2.7.3 | >=2.7.3 | >=2.7.3 | 2.8.0 | 2.8.0 | 2.8.1 | 2.8.1 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| retrying | >=1.3.3 | >=1.3.3 | 1.3.3 | 1.3.3 | 1.3.3 | 1.3.3 | 1.3.3 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| s3transfer | >=0.1.13 | >=0.1.13 | >=0.1.13 | 0.2.1 | 0.2.1 | 0.3.3 | 0.3.3 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| sagemaker-containers | >=2.1.0 | >=2.1.0 | 2.4.10.post0 | 2.5.4 | 2.5.4 | 2.5.4 | N/A | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| sagemaker-training (training only) | N/A | N/A | N/A | N/A | N/A | N/A | >=3.4.2 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| sagemaker-inference (inference only) | N/A | N/A | N/A | N/A | 1.1.2 | 1.1.2 | >=1.2.2 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| sagemaker-pytorch-container | 1.0 | 1.1 | 1.2 | 1.2 | 1.3 | N/A | N/A | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| sagemaker-pytorch-training | N/A | N/A | N/A | N/A | N/A | N/A | >=2.1.1 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| sagemaker-pytorch-inference | N/A | N/A | N/A | N/A | N/A | 1.1.2 | >=1.2.2 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| setuptools | >=39.2.0 | >=40.4.3 | >=40.4.3 | 41.4.0 | 42.0.2.post20191203 | N/A | 46.1.3 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| six | >=1.11.0 | >=1.11.0 | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 | >=1.12.0 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| torch | 0.4.0 | 1.0.0 | 1.1.0 | 1.2.0 | 1.3.1 | 1.4.0 | 1.5.0 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| torchvision | 0.2.1 | 0.2.1 | 0.3.0 | 0.4.0a0+9232c4a | 0.4.2 | 0.5.0 | 0.6.0 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ -| Python | 2.7 or 3.5 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 | 3.6 | -+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+---------------------+ - -The Docker images extend Ubuntu 16.04. - -With most versions of PyTorch, if you need to install other dependencies, you can put them into ``requirements.txt`` file and put it in the source directory -(``source_dir``) you provide to the `PyTorch Estimator <#pytorch-estimators>`__. -For more, including directions specific to each version of PyTorch, see `Using third-party libraries `_. - -You can select version of PyTorch by passing a ``framework_version`` keyword arg to the PyTorch Estimator constructor. -Currently supported versions are listed in the above table. You can also set ``framework_version`` to only specify major and -minor version, which will cause your training script to be run on the latest supported patch version of that minor -version. - -Alternatively, you can build your own image by following the instructions in the SageMaker PyTorch containers -repository, and passing ``image_name`` to the PyTorch Estimator constructor. - -You can visit the SageMaker PyTorch containers repositories: - -- training: https://github.com/aws/sagemaker-pytorch-container -- serving: https://github.com/aws/sagemaker-pytorch-serving-container diff --git a/src/sagemaker/rl/README.rst b/src/sagemaker/rl/README.rst deleted file mode 100644 index 9cc1292423..0000000000 --- a/src/sagemaker/rl/README.rst +++ /dev/null @@ -1,50 +0,0 @@ -=========================================== -SageMaker Reinforcement Learning Estimators -=========================================== - -With Reinforcement Learning (RL) Estimators, you can train reinforcement learning models on Amazon SageMaker. - -Supported versions of Coach: ``0.11.1``, ``0.10.1`` with TensorFlow, ``0.11.0`` with TensorFlow or MXNet. -For more information about Coach, see https://github.com/NervanaSystems/coach - -Supported versions of Ray: ``0.6.5``, ``0.5.3`` with TensorFlow. -For more information about Ray, see https://github.com/ray-project/ray - -For information about using RL with the SageMaker Python SDK, see https://sagemaker.readthedocs.io/en/stable/using_rl.html. - -SageMaker RL Docker Containers ------------------------------- - -When training and deploying training scripts, SageMaker runs your Python script in a Docker container with several -libraries installed. When creating the Estimator and calling deploy to create the SageMaker Endpoint, you can control -the environment your script runs in. - -SageMaker runs RL Estimator scripts in either Python 3.5 for MXNet or Python 3.6 for TensorFlow. - -The Docker images have the following dependencies installed: - -+-------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+ -| Dependencies | Coach 0.10.1 | Coach 0.11.0 | Coach 0.11.1 | Ray 0.5.3 | Ray 0.6.5 | -+-------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+ -| Python | 3.6 | 3.5 (MXNet) or | 3.6 | 3.6 | 3.6 | -| | | 3.6 (TensorFlow) | | | | -+-------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+ -| CUDA (GPU image only) | 9.0 | 9.0 | 9.0 | 9.0 | 9.0 | -+-------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+ -| DL Framework | TensorFlow-1.11.0 | MXNet-1.3.0 or | TensorFlow-1.12.0 | TensorFlow-1.11.0 | TensorFlow-1.12.0 | -| | | TensorFlow-1.11.0 | | | | -+-------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+ -| gym | 0.10.5 | 0.10.5 | 0.11.0 | 0.10.5 | 0.12.1 | -+-------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+ - -The Docker images extend Ubuntu 16.04. - -You can select version of by passing a ``framework_version`` keyword arg to the RL Estimator constructor. -Currently supported versions are listed in the above table. You can also set ``framework_version`` to only specify major and -minor version, which will cause your training script to be run on the latest supported patch version of that minor -version. - -Alternatively, you can build your own image by following the instructions in the SageMaker RL containers -repository, and passing ``image_name`` to the RL Estimator constructor. - -You can visit `the SageMaker RL containers repository `_. diff --git a/src/sagemaker/tensorflow/README.rst b/src/sagemaker/tensorflow/README.rst deleted file mode 100644 index 54781cf330..0000000000 --- a/src/sagemaker/tensorflow/README.rst +++ /dev/null @@ -1,81 +0,0 @@ -TensorFlow SageMaker Estimators and Models -========================================== - -TensorFlow SageMaker Estimators allow you to run your own TensorFlow -training algorithms on SageMaker Learner, and to host your own TensorFlow -models on SageMaker Hosting. - -Documentation of the previous Legacy Mode versions: `1.4.1 `_, `1.5.0 `_, `1.6.0 `_, `1.7.0 `_, `1.8.0 `_, `1.9.0 `_, `1.10.0 `_ - -+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| WARNING | -+=============================================================================================================================================================================+ -| We have added a new format of your TensorFlow training script with TensorFlow version 1.11. | -| This new way gives the user script more flexibility. | -| This new format is called Script Mode, as opposed to Legacy Mode, which is what we support with TensorFlow 1.11 and older versions. | -| In addition we are adding Python 3 support with Script Mode. | -| Last supported version of Legacy Mode will be TensorFlow 1.12. | -| Script Mode is available with TensorFlow version 1.11 and newer. | -| Make sure you refer to the correct version of this README when you prepare your script. | -| You can find the Legacy Mode README `here `_. | -+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -Supported versions of TensorFlow for Elastic Inference: ``1.11``, ``1.12``, ``1.13``, ``1.14``, ``1.15``, ``2.0``. - -Supported versions of TensorFlow for Inferentia: ``1.15.0``. - -For information about using TensorFlow with the SageMaker Python SDK, see https://sagemaker.readthedocs.io/en/stable/using_tf.html. - -SageMaker TensorFlow Docker containers -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The latest containers include the following Python packages: - -+--------------------------------+--------------------+---------------+ -| Dependencies | TF 1.15.2 | TF 2.1 | -+--------------------------------+--------------------+---------------+ -| awscli | 1.18.1 | 1.18.3 | -+--------------------------------+--------------------+---------------+ -| boto3 | 1.12.1 | 1.12.3 | -+--------------------------------+--------------------+---------------+ -| botocore | 1.15.1 | 1.15.3 | -+--------------------------------+--------------------+---------------+ -| h5py | 2.10.0 | 2.10.0 | -+--------------------------------+--------------------+---------------+ -| horovod | 0.18.2 | 0.18.2 | -+--------------------------------+--------------------+---------------+ -| keras | 2.3.1 | 2.3.1 | -+--------------------------------+--------------------+---------------+ -| mpi4py | 3.0.2 | 3.0.3 | -+--------------------------------+--------------------+---------------+ -| numpy | 1.18.1 | 1.18.1 | -+--------------------------------+--------------------+---------------+ -| pandas | 0.24.2 | 1.0.1 | -+--------------------------------+--------------------+---------------+ -| pip | 20.0.2 | 20.0.2 | -+--------------------------------+--------------------+---------------+ -| Pillow | 6.2.1 | 7.0.0 | -+--------------------------------+--------------------+---------------+ -| Python | 2.7, 3.6 or 3.7 | 2.7 or 3.6 | -+--------------------------------+--------------------+---------------+ -| requests | 2.22.0 | 2.22.0 | -+--------------------------------+--------------------+---------------+ -| sagemaker-containers | 2.7.0 | 2.8.0 | -+--------------------------------+--------------------+---------------+ -| sagemaker-tensorflow-container | 1.15.0.1.1.0 | 2.0.0.1.1.0 | -+--------------------------------+--------------------+---------------+ -| scipy | 1.2.2 | 1.4.1 | -+--------------------------------+--------------------+---------------+ -| tensorflow | 1.15.2 | 2.1.0 | -+--------------------------------+--------------------+---------------+ - -Script Mode TensorFlow Docker images support Python 2.7 and Python 3.6, Python 3.7 for TensorFlow version 1.15.2. The Docker images extend Ubuntu 16.04. - -You can select version of TensorFlow by passing a ``framework_version`` keyword arg to the TensorFlow Estimator constructor. Currently supported versions are listed in the table above. You can also set ``framework_version`` to only specify major and minor version, e.g ``'1.6'``, which will cause your training script to be run on the latest supported patch version of that minor version, which in this example would be 1.6.0. -Alternatively, you can build your own image by following the instructions in the SageMaker TensorFlow containers -repository, and passing ``image_name`` to the TensorFlow Estimator constructor. - -For more information on the contents of the images, see the SageMaker TensorFlow containers repositories here: - -- training: https://github.com/aws/sagemaker-tensorflow-container -- serving: https://github.com/aws/sagemaker-tensorflow-serving-container