Merge branch 'master' into keras_fn

Author: yangaws

CHANGELOG.rst

@@ -2,10 +2,17 @@
 CHANGELOG
 =========
 
-1.9.1dev
-========
+1.9.2
+=====
+
+* feature: add support for TensorFlow 1.9
+
+1.9.1
+=====
 
 * bug-fix: Estimators: Fix serialization of single records
+* bug-fix: deprecate enable_cloudwatch_metrics from Framework Estimators.
+* enhancement: Enable VPC config in training job creation
 
 1.9.0
 =====

README.rst

@@ -53,8 +53,8 @@ You can install from source by cloning this repository and running a pip install
 ::
 
     git clone https://github.com/aws/sagemaker-python-sdk.git
-    python setup.py sdist
-    pip install dist/sagemaker-1.9.0.tar.gz
+    cd sagemaker-python-sdk
+    pip install .
 
 Supported Operating Systems
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -244,6 +244,8 @@ By using MXNet SageMaker ``Estimators``, you can train and host MXNet models on
 
 Supported versions of MXNet: ``1.2.1``, ``1.1.0``, ``1.0.0``, ``0.12.1``.
 
+We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
+
 For more information, see `MXNet SageMaker Estimators and Models`_.
 
 .. _MXNet SageMaker Estimators and Models: src/sagemaker/mxnet/README.rst
@@ -254,7 +256,9 @@ TensorFlow SageMaker Estimators
 
 By using TensorFlow SageMaker ``Estimators``, you can train and host TensorFlow models on Amazon SageMaker.
 
-Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``.
+Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``, ``1.9.0``.
+
+We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
 
 For more information, see `TensorFlow SageMaker Estimators and Models`_.
 
@@ -268,6 +272,8 @@ By using Chainer SageMaker ``Estimators``, you can train and host Chainer models
 
 Supported versions of Chainer: ``4.0.0``, ``4.1.0``.
 
+We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
+
 For more information about Chainer, see https://github.com/chainer/chainer.
 
 For more information about  Chainer SageMaker ``Estimators``, see `Chainer SageMaker Estimators and Models`_.
@@ -280,7 +286,9 @@ PyTorch SageMaker Estimators
 
 With PyTorch SageMaker ``Estimators``, you can train and host PyTorch models on Amazon SageMaker.
 
-Supported versions of PyTorch: ``0.4.0``
+Supported versions of PyTorch: ``0.4.0``.
+
+We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
 
 For more information about PyTorch, see https://github.com/pytorch/pytorch.
 
@@ -326,7 +334,7 @@ SageMaker Automatic Model Tuning
 All of the estimators can be used with SageMaker Automatic Model Tuning, which performs hyperparameter tuning jobs.
 A hyperparameter tuning job finds the best version of a model by running many training jobs on your dataset using the algorithm with different values of hyperparameters within ranges
 that you specify. It then chooses the hyperparameter values that result in a model that performs the best, as measured by a metric that you choose.
-If you're not using an Amazon SageMaker built-in algorithm, then the metric is defined by a regular expression (regex) you provide. 
+If you're not using an Amazon SageMaker built-in algorithm, then the metric is defined by a regular expression (regex) you provide.
 The hyperparameter tuning job parses the training job's logs to find metrics that match the regex you defined.
 For more information about SageMaker Automatic Model Tuning, see `AWS documentation <https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html>`__.
 
@@ -377,7 +385,7 @@ In addition, the ``fit()`` call uses a list of ``RecordSet`` objects instead of
     # Start hyperparameter tuning job
     my_tuner.fit([train_records, test_records])
 
-To help attach a previously-started hyperparameter tuning job to a ``HyperparameterTuner`` instance, 
+To help attach a previously-started hyperparameter tuning job to a ``HyperparameterTuner`` instance,
 ``fit()`` adds the module path of the class used to create the tuner to the list of static hyperparameters by default.
 If the algorithm you are using cannot handle unknown hyperparameters
 (for example, an Amazon SageMaker built-in algorithm that does not have a custom estimator in the Python SDK),
@@ -521,4 +529,4 @@ After that, invoke the ``deploy()`` method on the ``Model``:
 
 This returns a predictor the same way an ``Estimator`` does when ``deploy()`` is called. You can now get inferences just like with any other model deployed on Amazon SageMaker.
 
-A full example is available in the `Amazon SageMaker examples repository <https://github.com/awslabs/amazon-sagemaker-examples/tree/master/advanced_functionality/mxnet_mnist_byom>`__.
+A full example is available in the `Amazon SageMaker examples repository <https://github.com/awslabs/amazon-sagemaker-examples/tree/master/advanced_functionality/mxnet_mnist_byom>`__.

setup.py

@@ -23,7 +23,7 @@ def read(fname):
 
 
 setup(name="sagemaker",
-      version="1.9.0",
+      version="1.9.2",
       description="Open source library for training and deploying models on Amazon SageMaker.",
       packages=find_packages('src'),
       package_dir={'': 'src'},

src/sagemaker/estimator.py

@@ -15,6 +15,7 @@
 import json
 import logging
 import os
+import warnings
 from abc import ABCMeta
 from abc import abstractmethod
 from six import with_metaclass
@@ -46,7 +47,8 @@ class EstimatorBase(with_metaclass(ABCMeta, object)):
 
     def __init__(self, role, train_instance_count, train_instance_type,
                  train_volume_size=30, train_max_run=24 * 60 * 60, input_mode='File',
-                 output_path=None, output_kms_key=None, base_job_name=None, sagemaker_session=None, tags=None):
+                 output_path=None, output_kms_key=None, base_job_name=None, sagemaker_session=None, tags=None,
+                 subnets=None, security_group_ids=None):
         """Initialize an ``EstimatorBase`` instance.
 
         Args:
@@ -77,6 +79,9 @@ def __init__(self, role, train_instance_count, train_instance_type,
                 using the default AWS configuration chain.
             tags (list[dict]): List of tags for labeling a training job. For more, see
                 https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
+            subnets (list[str]): List of subnet ids. If not specified training job will be created without VPC config.
+            security_group_ids (list[str]): List of security group ids. If not specified training job will be created
+                without VPC config.
         """
         self.role = role
         self.train_instance_count = train_instance_count
@@ -99,6 +104,10 @@ def __init__(self, role, train_instance_count, train_instance_type,
         self.output_kms_key = output_kms_key
         self.latest_training_job = None
 
+        # VPC configurations
+        self.subnets = subnets
+        self.security_group_ids = security_group_ids
+
     @abstractmethod
     def train_image(self):
         """Return the Docker image to use for training.
@@ -398,8 +407,9 @@ def start_new(cls, estimator, inputs):
         estimator.sagemaker_session.train(image=estimator.train_image(), input_mode=estimator.input_mode,
                                           input_config=config['input_config'], role=config['role'],
                                           job_name=estimator._current_job_name, output_config=config['output_config'],
-                                          resource_config=config['resource_config'], hyperparameters=hyperparameters,
-                                          stop_condition=config['stop_condition'], tags=estimator.tags)
+                                          resource_config=config['resource_config'], vpc_config=config['vpc_config'],
+                                          hyperparameters=hyperparameters, stop_condition=config['stop_condition'],
+                                          tags=estimator.tags)
 
         return cls(estimator.sagemaker_session, estimator._current_job_name)
 
@@ -550,8 +560,8 @@ def __init__(self, entry_point, source_dir=None, hyperparameters=None, enable_cl
                 The hyperparameters are made accessible as a dict[str, str] to the training code on SageMaker.
                 For convenience, this accepts other types for keys and values, but ``str()`` will be called
                 to convert them before training.
-            enable_cloudwatch_metrics (bool): Whether training and hosting containers will
-               generate CloudWatch metrics under the AWS/SageMakerContainer namespace (default: False).
+            enable_cloudwatch_metrics (bool): [DEPRECATED] Now there are cloudwatch metrics emitted by all SageMaker
+                training jobs. This will be ignored for now and removed in a further release.
             container_log_level (int): Log level to use within the container (default: logging.INFO).
                 Valid values are defined in the Python logging module.
             code_location (str): Name of the S3 bucket where custom code is uploaded (default: None).
@@ -564,7 +574,10 @@ def __init__(self, entry_point, source_dir=None, hyperparameters=None, enable_cl
         super(Framework, self).__init__(**kwargs)
         self.source_dir = source_dir
         self.entry_point = entry_point
-        self.enable_cloudwatch_metrics = enable_cloudwatch_metrics
+        if enable_cloudwatch_metrics:
+            warnings.warn('enable_cloudwatch_metrics is now deprecated and will be removed in the future.',
+                          DeprecationWarning)
+        self.enable_cloudwatch_metrics = False
         self.container_log_level = container_log_level
         self._hyperparameters = hyperparameters or {}
         self.code_location = code_location

src/sagemaker/job.py

@@ -59,12 +59,14 @@ def _load_config(inputs, estimator):
                                                         estimator.train_instance_type,
                                                         estimator.train_volume_size)
         stop_condition = _Job._prepare_stop_condition(estimator.train_max_run)
+        vpc_config = _Job._prepare_vpc_config(estimator.subnets, estimator.security_group_ids)
 
         return {'input_config': input_config,
                 'role': role,
                 'output_config': output_config,
                 'resource_config': resource_config,
-                'stop_condition': stop_condition}
+                'stop_condition': stop_condition,
+                'vpc_config': vpc_config}
 
     @staticmethod
     def _format_inputs_to_input_config(inputs):
@@ -143,6 +145,13 @@ def _prepare_resource_config(instance_count, instance_type, volume_size):
                 'InstanceType': instance_type,
                 'VolumeSizeInGB': volume_size}
 
+    @staticmethod
+    def _prepare_vpc_config(subnets, security_group_ids):
+        if subnets is None or security_group_ids is None:
+            return None
+        return {'Subnets': subnets,
+                'SecurityGroupIds': security_group_ids}
+
     @staticmethod
     def _prepare_stop_condition(max_run):
         return {'MaxRuntimeInSeconds': max_run}

src/sagemaker/mxnet/README.rst

@@ -543,9 +543,6 @@ The MXNetModel constructor takes the following arguments:
    directory with any other training source code dependencies including
    tne entry point file. Structure within this directory will be
    preserved when training on SageMaker.
--  ``enable_cloudwatch_metrics (boolean):`` Optional. If true, training
-   and hosting containers will generate Cloudwatch metrics under the
-   AWS/SageMakerContainer namespace.
 -  ``container_log_level (int):`` Log level to use within the container.
    Valid values are defined in the Python logging module.
 -  ``code_location (str):`` Optional. Name of the S3 bucket where your

src/sagemaker/pytorch/README.rst

@@ -1,4 +1,3 @@
-
 =======================================
 SageMaker PyTorch Estimators and Models
 =======================================
@@ -39,24 +38,21 @@ using to submit the script to SageMaker via a ``PyTorch`` Estimator. This will b
 Suppose that you already have a PyTorch training script called `pytorch-train.py`.
 You can then setup a ``PyTorch`` Estimator with keyword arguments to point to this script and define how SageMaker runs it:
 
-```python
+.. code:: python
 
     from sagemaker.pytorch import PyTorch
 
-    pytorch_estimator = PyTorch(entry_point="pytorch-train.py",
-                                role="SageMakerRole",
-                                train_instance_type="ml.p3.2xlarge",
+    pytorch_estimator = PyTorch(entry_point='pytorch-train.py',
+                                role='SageMakerRole',
+                                train_instance_type='ml.p3.2xlarge',
                                 train_instance_count=1)
-```
 
 After that, you simply tell the estimator to start a training job and provide an S3 URL
 that is the path to your training data within Amazon S3:
 
-```python
-
-    pytorch_estimator.fit("s3://bucket/path/to/training/data")
+.. code:: python
 
-```
+    pytorch_estimator.fit('s3://bucket/path/to/training/data')
 
 In the following sections, we'll discuss how to prepare a training script for execution on SageMaker,
 then how to run that script on SageMaker using a ``PyTorch`` Estimator.
@@ -443,7 +439,7 @@ the model server receives two pieces of information:
 -  The request data body, a byte array which is at most 5 MB (5 \* 1024
    \* 1024 bytes) in size.
 
-The SageMaker PyTorch model server will invoke an "input_fn" function in your hosting script,
+The SageMaker PyTorch model server will invoke an ``input_fn`` function in your hosting script,
 passing in this information. If you define an ``input_fn`` function definition,
 it should return an object that can be passed to ``predict_fn`` and have the following signature:
 
@@ -647,7 +643,7 @@ Please refer to:
 
 https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk
 
-These are also available in SageMaker Notebook Instance hosted Jupyter notebooks under the "sample notebooks" folder.
+These are also available in SageMaker Notebook Instance hosted Jupyter notebooks under the sample notebooks folder.
 
 
 SageMaker PyTorch Docker Containers

src/sagemaker/session.py

@@ -29,7 +29,6 @@
 from sagemaker.utils import name_from_image, secondary_training_status_message, secondary_training_status_changed
 import sagemaker.logs
 
-
 logging.basicConfig()
 LOGGER = logging.getLogger('sagemaker')
 LOGGER.setLevel(logging.INFO)
@@ -93,7 +92,12 @@ def _initialize(self, boto_session, sagemaker_client, sagemaker_runtime_client):
         self.sagemaker_client = sagemaker_client or self.boto_session.client('sagemaker')
         prepend_user_agent(self.sagemaker_client)
 
-        self.sagemaker_runtime_client = sagemaker_runtime_client or self.boto_session.client('runtime.sagemaker')
+        if sagemaker_runtime_client is not None:
+            self.sagemaker_runtime_client = sagemaker_runtime_client
+        else:
+            config = botocore.config.Config(read_timeout=80)
+            self.sagemaker_runtime_client = self.boto_session.client('runtime.sagemaker', config=config)
+
         prepend_user_agent(self.sagemaker_runtime_client)
 
         self.local_mode = False
@@ -202,7 +206,7 @@ def default_bucket(self):
         return self._default_bucket
 
     def train(self, image, input_mode, input_config, role, job_name, output_config,
-              resource_config, hyperparameters, stop_condition, tags):
+              resource_config, vpc_config, hyperparameters, stop_condition, tags):
         """Create an Amazon SageMaker training job.
 
         Args:
@@ -228,6 +232,13 @@ def train(self, image, input_mode, input_config, role, job_name, output_config,
                 * instance_type (str): Type of EC2 instance to use for training, for example, 'ml.c4.xlarge'.
                     The key in resource_config is 'InstanceType'.
 
+            vpc_config (dict): Contains values for VpcConfig:
+
+                * subnets (list[str]): List of subnet ids.
+                    The key in vpc_config is 'Subnets'.
+                * security_group_ids (list[str]): List of security group ids.
+                    The key in vpc_config is 'SecurityGroupIds'.
+
             hyperparameters (dict): Hyperparameters for model training. The hyperparameters are made accessible as
                 a dict[str, str] to the training code on SageMaker. For convenience, this accepts other types for
                 keys and values, but ``str()`` will be called to convert them before training.
@@ -259,6 +270,9 @@ def train(self, image, input_mode, input_config, role, job_name, output_config,
         if tags is not None:
             train_request['Tags'] = tags
 
+        if vpc_config is not None:
+            train_request['VpcConfig'] = vpc_config
+
         LOGGER.info('Creating training-job with name: {}'.format(job_name))
         LOGGER.debug('train request: {}'.format(json.dumps(train_request, indent=4)))
         self.sagemaker_client.create_training_job(**train_request)
@@ -1018,7 +1032,6 @@ def _deployment_entity_exists(describe_fn):
 
 
 def _train_done(sagemaker_client, job_name, last_desc):
-
     in_progress_statuses = ['InProgress', 'Created']
 
     desc = sagemaker_client.describe_training_job(TrainingJobName=job_name)

src/sagemaker/tensorflow/README.rst

@@ -6,7 +6,7 @@ TensorFlow SageMaker Estimators allow you to run your own TensorFlow
 training algorithms on SageMaker Learner, and to host your own TensorFlow
 models on SageMaker Hosting.
 
-Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``.
+Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``, ``1.9.0``.
 
 Training with TensorFlow
 ~~~~~~~~~~~~~~~~~~~~~~~~
@@ -833,7 +833,7 @@ SageMaker TensorFlow CPU images use TensorFlow built with Intel® MKL-DNN optimi
 In certain cases you might be able to get a better performance by disabling this optimization
 (`for example when using small models <https://github.com/awslabs/amazon-sagemaker-examples/blob/d88d1c19861fb7733941969f5a68821d9da2982e/sagemaker-python-sdk/tensorflow_iris_dnn_classifier_using_estimators/iris_dnn_classifier.py#L7-L9>`_)
 
-You can disable MKL-DNN optimization for TensorFlow ``1.8.0`` by setting two following environment variables:
+You can disable MKL-DNN optimization for TensorFlow ``1.8.0`` and above by setting two following environment variables:
 
 .. code:: python
 

src/sagemaker/tensorflow/defaults.py

@@ -12,4 +12,4 @@
 # language governing permissions and limitations under the License.
 from __future__ import absolute_import
 
-TF_VERSION = '1.8'
+TF_VERSION = '1.9'