doc: consolidate framework version and image information (#1522)

This changes the following:
- for MXNet, TF, and PyTorch, the docs point to the DLC documentation
- for RL, the docs point to the sagemaker-rl-container repository
- for Chainer, src/sagemaker/chainer/README.rst has been removed in favor of doc/using_chainer.rst

Author: laurenyu

README.rst

@@ -39,31 +39,31 @@ For detailed API reference please go to: `Read the Docs <https://sagemaker.readt
 Table of Contents
 -----------------
 
-1. `Installing SageMaker Python SDK <#installing-the-sagemaker-python-sdk>`__
-2. `Using the SageMaker Python SDK <https://sagemaker.readthedocs.io/en/stable/overview.html>`__
-3. `MXNet SageMaker Estimators <#mxnet-sagemaker-estimators>`__
-4. `TensorFlow SageMaker Estimators <#tensorflow-sagemaker-estimators>`__
-5. `Chainer SageMaker Estimators <#chainer-sagemaker-estimators>`__
-6. `PyTorch SageMaker Estimators <#pytorch-sagemaker-estimators>`__
-7. `Scikit-learn SageMaker Estimators <#scikit-learn-sagemaker-estimators>`__
-8. `XGBoost SageMaker Estimators <#xgboost-sagemaker-estimators>`__
-9. `SageMaker Reinforcement Learning Estimators <#sagemaker-reinforcement-learning-estimators>`__
-10. `SageMaker SparkML Serving <#sagemaker-sparkml-serving>`__
-11. `AWS SageMaker Estimators <#aws-sagemaker-estimators>`__
-12. `Using SageMaker AlgorithmEstimators <https://sagemaker.readthedocs.io/en/stable/overview.html#using-sagemaker-algorithmestimators>`__
-13. `Consuming SageMaker Model Packages <https://sagemaker.readthedocs.io/en/stable/overview.html#consuming-sagemaker-model-packages>`__
-14. `BYO Docker Containers with SageMaker Estimators <https://sagemaker.readthedocs.io/en/stable/overview.html#byo-docker-containers-with-sagemaker-estimators>`__
-15. `SageMaker Automatic Model Tuning <https://sagemaker.readthedocs.io/en/stable/overview.html#sagemaker-automatic-model-tuning>`__
-16. `SageMaker Batch Transform <https://sagemaker.readthedocs.io/en/stable/overview.html#sagemaker-batch-transform>`__
-17. `Secure Training and Inference with VPC <https://sagemaker.readthedocs.io/en/stable/overview.html#secure-training-and-inference-with-vpc>`__
-18. `BYO Model <https://sagemaker.readthedocs.io/en/stable/overview.html#byo-model>`__
-19. `Inference Pipelines <https://sagemaker.readthedocs.io/en/stable/overview.html#inference-pipelines>`__
-20. `Amazon SageMaker Operators for Kubernetes <#amazon-sagemaker-operators-for-kubernetes>`__
-21. `Amazon SageMaker Operators in Apache Airflow <#sagemaker-workflow>`__
-22. `SageMaker Autopilot <#sagemaker-autopilot>`__
-23. `Model Monitoring <#amazon-sagemaker-model-monitoring>`__
-24. `SageMaker Debugger <#amazon-sagemaker-debugger>`__
-25. `SageMaker Processing <#amazon-sagemaker-processing>`__
+#. `Installing SageMaker Python SDK <#installing-the-sagemaker-python-sdk>`__
+#. `Using the SageMaker Python SDK <https://sagemaker.readthedocs.io/en/stable/overview.html>`__
+#. `Using MXNet <https://sagemaker.readthedocs.io/en/stable/using_mxnet.html>`__
+#. `Using TensorFlow <https://sagemaker.readthedocs.io/en/stable/using_tf.html>`__
+#. `Using Chainer <https://sagemaker.readthedocs.io/en/stable/using_chainer.html>`__
+#. `Using PyTorch <https://sagemaker.readthedocs.io/en/stable/using_pytorch.html>`__
+#. `Scikit-learn SageMaker Estimators <#scikit-learn-sagemaker-estimators>`__
+#. `XGBoost SageMaker Estimators <#xgboost-sagemaker-estimators>`__
+#. `SageMaker Reinforcement Learning Estimators <https://sagemaker.readthedocs.io/en/stable/using_rl.html>`__
+#. `SageMaker SparkML Serving <#sagemaker-sparkml-serving>`__
+#. `AWS SageMaker Estimators <#aws-sagemaker-estimators>`__
+#. `Using SageMaker AlgorithmEstimators <https://sagemaker.readthedocs.io/en/stable/overview.html#using-sagemaker-algorithmestimators>`__
+#. `Consuming SageMaker Model Packages <https://sagemaker.readthedocs.io/en/stable/overview.html#consuming-sagemaker-model-packages>`__
+#. `BYO Docker Containers with SageMaker Estimators <https://sagemaker.readthedocs.io/en/stable/overview.html#byo-docker-containers-with-sagemaker-estimators>`__
+#. `SageMaker Automatic Model Tuning <https://sagemaker.readthedocs.io/en/stable/overview.html#sagemaker-automatic-model-tuning>`__
+#. `SageMaker Batch Transform <https://sagemaker.readthedocs.io/en/stable/overview.html#sagemaker-batch-transform>`__
+#. `Secure Training and Inference with VPC <https://sagemaker.readthedocs.io/en/stable/overview.html#secure-training-and-inference-with-vpc>`__
+#. `BYO Model <https://sagemaker.readthedocs.io/en/stable/overview.html#byo-model>`__
+#. `Inference Pipelines <https://sagemaker.readthedocs.io/en/stable/overview.html#inference-pipelines>`__
+#. `Amazon SageMaker Operators for Kubernetes <#amazon-sagemaker-operators-for-kubernetes>`__
+#. `Amazon SageMaker Operators in Apache Airflow <#sagemaker-workflow>`__
+#. `SageMaker Autopilot <#sagemaker-autopilot>`__
+#. `Model Monitoring <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_model_monitoring.html>`__
+#. `SageMaker Debugger <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html>`__
+#. `SageMaker Processing <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_processing.html>`__
 
 
 Installing the SageMaker Python SDK
@@ -197,73 +197,6 @@ Preview the site with a Python web server:
 
 View the website by visiting http://localhost:8000
 
-
-MXNet SageMaker Estimators
---------------------------
-
-By using MXNet SageMaker Estimators, you can train and host MXNet models on Amazon SageMaker.
-
-Supported versions of MXNet: ``0.12.1``, ``1.0.0``, ``1.1.0``, ``1.2.1``, ``1.3.0``, ``1.4.0``, ``1.4.1``, ``1.6.0``.
-
-Supported versions of MXNet for Elastic Inference: ``1.3.0``, ``1.4.0``, ``1.4.1``, ``1.5.1``.
-
-We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
-
-For more information, see `Using MXNet with the SageMaker Python SDK`_.
-
-.. _Using MXNet with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_mxnet.html
-
-
-TensorFlow SageMaker Estimators
--------------------------------
-
-By using TensorFlow SageMaker Estimators, you can train and host TensorFlow models on Amazon SageMaker.
-
-Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``, ``1.9.0``, ``1.10.0``, ``1.11.0``, ``1.12.0``, ``1.13.1``, ``1.14.0``, ``1.15.0``, ``1.15.2``, ``2.0.0``, ``2.0.1``, ``2.1.0``.
-
-Supported versions of TensorFlow for Elastic Inference: ``1.11.0``, ``1.12.0``, ``1.13.1``, ``1.14.0``, ``1.15.0``, ``2.0.0``.
-
-We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
-
-For more information, see `Using TensorFlow with the SageMaker Python SDK`_.
-
-.. _Using TensorFlow with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_tf.html
-
-
-Chainer SageMaker Estimators
-----------------------------
-
-By using Chainer SageMaker Estimators, you can train and host Chainer models on Amazon SageMaker.
-
-Supported versions of Chainer: ``4.0.0``, ``4.1.0``, ``5.0.0``.
-
-We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
-
-For more information about Chainer, see https://github.com/chainer/chainer.
-
-For more information about Chainer SageMaker Estimators, see `Using Chainer with the SageMaker Python SDK`_.
-
-.. _Using Chainer with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_chainer.html
-
-
-PyTorch SageMaker Estimators
-----------------------------
-
-With PyTorch SageMaker Estimators, you can train and host PyTorch models on Amazon SageMaker.
-
-Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``, ``1.4.0``, ``1.5.0``.
-
-Supported versions of PyTorch for Elastic Inference: ``1.3.1``.
-
-We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
-
-For more information about PyTorch, see https://github.com/pytorch/pytorch.
-
-For more information about PyTorch SageMaker Estimators, see `Using PyTorch with the SageMaker Python SDK`_.
-
-.. _Using PyTorch with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_pytorch.html
-
-
 Scikit-learn SageMaker Estimators
 ---------------------------------
 
@@ -295,22 +228,6 @@ For more information about XGBoost SageMaker Estimators, see `Using XGBoost with
 .. _Using XGBoost with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_xgboost.html
 
 
-SageMaker Reinforcement Learning Estimators
--------------------------------------------
-
-With Reinforcement Learning (RL) Estimators, you can use reinforcement learning to train models on Amazon SageMaker.
-
-Supported versions of Coach: ``0.10.1``, ``0.11.1`` with TensorFlow, ``0.11.0`` with TensorFlow or MXNet.
-For more information about Coach, see https://github.com/NervanaSystems/coach
-
-Supported versions of Ray: ``0.5.3``, ``0.6.5`` with TensorFlow.
-For more information about Ray, see https://github.com/ray-project/ray
-
-For more information about SageMaker RL Estimators, see `SageMaker Reinforcement Learning Estimators`_.
-
-.. _SageMaker Reinforcement Learning Estimators: src/sagemaker/rl/README.rst
-
-
 SageMaker SparkML Serving
 -------------------------
 
@@ -385,32 +302,3 @@ on your data, and hosts a series of models on an Inference Pipeline.
 For more information about SageMaker Autopilot, see `SageMaker Autopilot`_.
 
 .. _SageMaker Autopilot: src/sagemaker/automl/README.rst
-
-Amazon SageMaker Model Monitoring
----------------------------------
-
-You can use Amazon SageMaker Model Monitoring to automatically detect concept drift by monitoring your machine learning models.
-
-For more information, see `Amazon SageMaker Model Monitoring`_.
-
-.. _Amazon SageMaker Model Monitoring: https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_model_monitoring.html
-
-Amazon SageMaker Debugger
--------------------------
-
-You can use Amazon SageMaker Debugger to automatically detect anomalies while training your machine learning models.
-
-For more information, see `Amazon SageMaker Debugger`_.
-
-.. _Amazon SageMaker Debugger: https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html
-
-
-Amazon SageMaker Processing
----------------------------------
-
-You can use Amazon SageMaker Processing to perform data processing tasks such as data pre- and post-processing, feature engineering, data validation, and model evaluation
-
-
-For more information, see `Amazon SageMaker Processing`_.
-
-.. _Amazon SageMaker Processing: https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_processing.html

doc/using_chainer.rst

@@ -4,7 +4,11 @@ Using Chainer with the SageMaker Python SDK
 
 With Chainer Estimators, you can train and host Chainer models on Amazon SageMaker.
 
-For information about supported versions of Chainer, see the `Chainer README <https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/chainer/README.rst>`__.
+Supported versions of Chainer: ``4.0.0``, ``4.1.0``, ``5.0.0``.
+
+We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
+
+For more information about Chainer, see https://github.com/chainer/chainer.
 
 For general information about using the SageMaker Python SDK, see :ref:`overview:Using the SageMaker Python SDK`.
 
@@ -638,6 +642,51 @@ The following are optional arguments. When you create a ``Chainer`` object, you
 SageMaker Chainer Docker containers
 ***********************************
 
-You can visit the SageMaker Chainer containers repository here: https://github.com/aws/sagemaker-chainer-container
-
-For information about SageMaker Chainer Docker containers and their dependencies, see `SageMaker Chainer Docker containers <https://github.com/aws/sagemaker-python-sdk/tree/master/src/sagemaker/chainer#sagemaker-chainer-docker-containers>`_.
+When training and deploying training scripts, SageMaker runs your Python script in a Docker container with several
+libraries installed. When creating the Estimator and calling deploy to create the SageMaker Endpoint, you can control
+the environment your script runs in.
+
+SageMaker runs Chainer Estimator scripts in either Python 2.7 or Python 3.5. You can select the Python version by
+passing a py_version keyword arg to the Chainer Estimator constructor. Setting this to py3 (the default) will cause your
+training script to be run on Python 3.5. Setting this to py2 will cause your training script to be run on Python 2.7
+This Python version applies to both the Training Job, created by fit, and the Endpoint, created by deploy.
+
+The Chainer Docker images have the following dependencies installed:
+
++-----------------------------+-------------+-------------+-------------+
+| Dependencies                | chainer 4.0 | chainer 4.1 | chainer 5.0 |
++-----------------------------+-------------+-------------+-------------+
+| chainer                     | 4.0.0       | 4.1.0       | 5.0.0       |
++-----------------------------+-------------+-------------+-------------+
+| chainercv                   | 0.9.0       | 0.10.0      | 0.10.0      |
++-----------------------------+-------------+-------------+-------------+
+| chainermn                   | 1.2.0       | 1.3.0       | N/A         |
++-----------------------------+-------------+-------------+-------------+
+| CUDA (GPU image only)       | 9.0         | 9.0         | 9.0         |
++-----------------------------+-------------+-------------+-------------+
+| cupy                        | 4.0.0       | 4.1.0       | 5.0.0       |
++-----------------------------+-------------+-------------+-------------+
+| matplotlib                  | 2.2.0       | 2.2.0       | 2.2.0       |
++-----------------------------+-------------+-------------+-------------+
+| mpi4py                      | 3.0.0       | 3.0.0       | 3.0.0       |
++-----------------------------+-------------+-------------+-------------+
+| numpy                       | 1.14.3      | 1.15.3      | 1.15.4      |
++-----------------------------+-------------+-------------+-------------+
+| opencv-python               | 3.4.0.12    | 3.4.0.12    | 3.4.0.12    |
++-----------------------------+-------------+-------------+-------------+
+| Pillow                      | 5.1.0       | 5.3.0       | 5.3.0       |
++-----------------------------+-------------+-------------+-------------+
+| Python                      | 2.7 or 3.5  | 2.7 or 3.5  | 2.7 or 3.5  |
++-----------------------------+-------------+-------------+-------------+
+
+The Docker images extend Ubuntu 16.04.
+
+You must select a version of Chainer by passing a ``framework_version`` keyword arg to the Chainer Estimator
+constructor. Currently supported versions are listed in the above table. You can also set framework_version to only
+specify major and minor version, which will cause your training script to be run on the latest supported patch
+version of that minor version.
+
+Alternatively, you can build your own image by following the instructions in the SageMaker Chainer containers
+repository, and passing ``image_name`` to the Chainer Estimator constructor.
+
+You can visit the SageMaker Chainer containers repository at https://github.com/aws/sagemaker-chainer-container

doc/using_mxnet.rst

@@ -4,7 +4,9 @@ Use MXNet with the SageMaker Python SDK
 
 With the SageMaker Python SDK, you can train and host MXNet models on Amazon SageMaker.
 
-For information about supported versions of MXNet, see the `MXNet README <https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/mxnet/README.rst>`__.
+For information about supported versions of MXNet, see the `AWS documentation <https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/deep-learning-containers-images.html>`__.
+
+We recommend that you use the latest supported version because that's where we focus our development efforts.
 
 For general information about using the SageMaker Python SDK, see :ref:`overview:Using the SageMaker Python SDK`.
 
@@ -807,9 +809,9 @@ For information about the different MXNet-related classes in the SageMaker Pytho
 SageMaker MXNet Containers
 **************************
 
-For information about SageMaker MXNet containers, see the following topics:
-
-- training: https://github.com/aws/sagemaker-mxnet-container
-- serving: https://github.com/aws/sagemaker-mxnet-serving-container
+For information about the SageMaker MXNet containers, see:
 
-For information about the dependencies installed in SageMaker MXNet containers, see https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/mxnet/README.rst#sagemaker-mxnet-containers.
+- `SageMaker MXNet training toolkit <https://github.com/aws/sagemaker-mxnet-container>`_
+- `SageMaker MXNet serving toolkit <https://github.com/aws/sagemaker-mxnet-serving-container>`_
+- `Deep Learning Container (DLC) Dockerfiles for MXNet <https://github.com/aws/deep-learning-containers/tree/master/mxnet>`_
+- `Deep Learning Container (DLC) Images <https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/deep-learning-containers-images.html>`_ and `release notes <https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/dlc-release-notes.html>`_

doc/using_pytorch.rst

@@ -4,9 +4,7 @@ Using PyTorch with the SageMaker Python SDK
 
 With PyTorch Estimators and Models, you can train and host PyTorch models on Amazon SageMaker.
 
-Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``, ``1.4.0``, ``1.5.0``.
-
-Supported versions of PyTorch for Elastic Inference: ``1.3.1``.
+For information about supported versions of PyTorch, see the `AWS documentation <https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/deep-learning-containers-images.html>`__.
 
 We recommend that you use the latest supported version because that's where we focus our development efforts.
 
@@ -758,6 +756,9 @@ The following are optional arguments. When you create a ``PyTorch`` object, you
 SageMaker PyTorch Docker Containers
 ***********************************
 
-For information about SageMaker PyTorch containers, see  `the SageMaker PyTorch container repository <https://github.com/aws/sagemaker-pytorch-container>`_ and `SageMaker PyTorch Serving container repository <https://github.com/aws/sagemaker-pytorch-serving-container>`__.
+For information about the SageMaker PyTorch containers, see:
 
-For information about SageMaker PyTorch container dependencies, see `SageMaker PyTorch Containers <https://github.com/aws/sagemaker-python-sdk/tree/master/src/sagemaker/pytorch#sagemaker-pytorch-docker-containers>`_.
+- `SageMaker PyTorch training toolkit <https://github.com/aws/sagemaker-pytorch-container>`_
+- `SageMaker PyTorch serving toolkit <https://github.com/aws/sagemaker-pytorch-serving-container>`_
+- `Deep Learning Container (DLC) Dockerfiles for PyTorch <https://github.com/aws/deep-learning-containers/tree/master/pytorch>`_
+- `Deep Learning Container (DLC) Images <https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/deep-learning-containers-images.html>`_ and `release notes <https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/dlc-release-notes.html>`_