Update latest version of PyTorch to 1.0 (aws#547)

Author: nadiaya

CHANGELOG.rst

@@ -8,6 +8,7 @@ CHANGELOG
 * bug-fix: Local Mode: Allow support for SSH in local mode 
 * bug-fix: Append retry id to default Airflow job name to avoid name collisions in retry
 * bug-fix: Local Mode: No longer requires s3 permissions to run local entry point file
+* feature: Estimators: add support for PyTorch 1.0.0
 * bug-fix: Local Mode: Move dependency on sagemaker_s3_output from rl.estimator to model
 * doc-fix: Fix quotes in estimator.py and model.py
 

README.rst

@@ -406,13 +406,10 @@ PyTorch SageMaker Estimators
 
 With PyTorch SageMaker ``Estimators``, you can train and host PyTorch models on Amazon SageMaker.
 
-Supported versions of PyTorch: ``0.4.0``, ``1.0.0.dev`` ("Preview").
+Supported versions of PyTorch: ``0.4.0``, ``1.0.0``.
 
 We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
 
-You can try the "Preview" version of PyTorch by specifying ``'1.0.0.dev'`` for ``framework_version`` when creating your PyTorch estimator.
-This will ensure you're using the latest version of ``torch-nightly``.
-
 For more information about PyTorch, see https://github.com/pytorch/pytorch.
 
 For more information about PyTorch SageMaker ``Estimators``, see `PyTorch SageMaker Estimators and Models`_.

src/sagemaker/pytorch/README.rst

@@ -4,10 +4,9 @@ SageMaker PyTorch Estimators and Models
 
 With PyTorch Estimators and Models, you can train and host PyTorch models on Amazon SageMaker.
 
-Supported versions of PyTorch: ``0.4.0``, ``1.0.0.dev`` ("Preview").
+Supported versions of PyTorch: ``0.4.0``, ``1.0.0``.
 
-You can try the "Preview" version of PyTorch by specifying ``1.0.0.dev`` for ``framework_version`` when creating your PyTorch estimator.
-This will ensure you're using the latest version of ``torch-nightly``.
+We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
 
 You can visit the PyTorch repository at https://github.com/pytorch/pytorch.
 
@@ -49,7 +48,7 @@ You can then setup a ``PyTorch`` Estimator with keyword arguments to point to th
                                 role='SageMakerRole',
                                 train_instance_type='ml.p3.2xlarge',
                                 train_instance_count=1,
-                                framework_version='0.4.0')
+                                framework_version='1.0.0')
 
 After that, you simply tell the estimator to start a training job and provide an S3 URL
 that is the path to your training data within Amazon S3:
@@ -137,7 +136,7 @@ directories ('train' and 'test').
     pytorch_estimator = PyTorch('pytorch-train.py',
                                 train_instance_type='ml.p3.2xlarge',
                                 train_instance_count=1,
-                                framework_version='0.4.0',
+                                framework_version='1.0.0',
                                 hyperparameters = {'epochs': 20, 'batch-size': 64, 'learning-rate': 0.1})
     pytorch_estimator.fit({'train': 's3://my-data-bucket/path/to/my/training/data',
                            'test': 's3://my-data-bucket/path/to/my/test/data'})
@@ -338,7 +337,7 @@ operation.
     pytorch_estimator = PyTorch(entry_point='train_and_deploy.py',
                                 train_instance_type='ml.p3.2xlarge',
                                 train_instance_count=1,
-                                framework_version='0.4.0')
+                                framework_version='1.0.0')
     pytorch_estimator.fit('s3://my_bucket/my_training_data/')
 
     # Deploy my estimator to a SageMaker Endpoint and get a Predictor
@@ -675,21 +674,21 @@ When training and deploying training scripts, SageMaker runs your Python script
 libraries installed. When creating the Estimator and calling deploy to create the SageMaker Endpoint, you can control
 the environment your script runs in.
 
-SageMaker runs PyTorch Estimator scripts in either Python 2.7 or Python 3.5. You can select the Python version by
+SageMaker runs PyTorch Estimator scripts in either Python 2 or Python 3. You can select the Python version by
 passing a ``py_version`` keyword arg to the PyTorch Estimator constructor. Setting this to `py3` (the default) will cause your
 training script to be run on Python 3.5. Setting this to `py2` will cause your training script to be run on Python 2.7
 This Python version applies to both the Training Job, created by fit, and the Endpoint, created by deploy.
 
 The PyTorch Docker images have the following dependencies installed:
 
 +-----------------------------+---------------+-------------------+
-| Dependencies                | pytorch 0.4.0 | pytorch 1.0.0.dev |
+| Dependencies                | pytorch 0.4.0 | pytorch 1.0.0     |
 +-----------------------------+---------------+-------------------+
 | boto3                       | >=1.7.35      | >=1.9.11          |
 +-----------------------------+---------------+-------------------+
 | botocore                    | >=1.10.35     | >=1.12.11         |
 +-----------------------------+---------------+-------------------+
-| CUDA (GPU image only)       | 9.0           | 9.2               |
+| CUDA (GPU image only)       | 9.0           | 9.0               |
 +-----------------------------+---------------+-------------------+
 | numpy                       | >=1.14.3      | >=1.15.2          |
 +-----------------------------+---------------+-------------------+
@@ -711,11 +710,11 @@ The PyTorch Docker images have the following dependencies installed:
 +-----------------------------+---------------+-------------------+
 | six                         | >=1.11.0      | >=1.11.0          |
 +-----------------------------+---------------+-------------------+
-| torch (torch-nightly)       | 0.4.0         | 1.0.0.dev         |
+| torch                       | 0.4.0         | 1.0.0             |
 +-----------------------------+---------------+-------------------+
 | torchvision                 | 0.2.1         | 0.2.1             |
 +-----------------------------+---------------+-------------------+
-| Python                      | 2.7 or 3.5    | 2.7 or 3.5        |
+| Python                      | 2.7 or 3.5    | 2.7 or 3.6        |
 +-----------------------------+---------------+-------------------+
 
 The Docker images extend Ubuntu 16.04.

src/sagemaker/pytorch/estimator.py

@@ -29,6 +29,8 @@ class PyTorch(Framework):
 
     __framework_name__ = "pytorch"
 
+    LATEST_VERSION = '1.0'
+
     def __init__(self, entry_point, source_dir=None, hyperparameters=None, py_version=PYTHON_VERSION,
                  framework_version=None, image_name=None, **kwargs):
         """

tests/conftest.py

@@ -23,7 +23,7 @@
 from sagemaker.chainer import Chainer
 from sagemaker.local import LocalSession
 from sagemaker.mxnet import MXNet
-from sagemaker.pytorch.defaults import PYTORCH_VERSION
+from sagemaker.pytorch import PyTorch
 from sagemaker.rl import RLEstimator
 from sagemaker.sklearn.defaults import SKLEARN_VERSION
 from sagemaker.tensorflow.defaults import TF_VERSION
@@ -37,7 +37,7 @@ def pytest_addoption(parser):
     parser.addoption('--boto-config', action='store', default=None)
     parser.addoption('--chainer-full-version', action='store', default=Chainer.LATEST_VERSION)
     parser.addoption('--mxnet-full-version', action='store', default=MXNet.LATEST_VERSION)
-    parser.addoption('--pytorch-full-version', action='store', default=PYTORCH_VERSION)
+    parser.addoption('--pytorch-full-version', action='store', default=PyTorch.LATEST_VERSION)
     parser.addoption('--rl-coach-full-version', action='store',
                      default=RLEstimator.COACH_LATEST_VERSION)
     parser.addoption('--rl-ray-full-version', action='store',
@@ -114,7 +114,7 @@ def ei_mxnet_version(request):
     return request.param
 
 
-@pytest.fixture(scope='module', params=['0.4', '0.4.0'])
+@pytest.fixture(scope='module', params=['0.4', '0.4.0', '1.0', '1.0.0'])
 def pytorch_version(request):
     return request.param