Merge branch 'master' into doc-xgboost-byom

Author: laurenyu

CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## v1.70.2 (2020-07-22)
+
+### Bug Fixes and Other Changes
+
+ * convert network_config in processing_config to dict
+
+### Documentation Changes
+
+ * Add ECR URI Estimator example
+
+## v1.70.1 (2020-07-21)
+
+### Bug Fixes and Other Changes
+
+ * Nullable fields in processing_config
+
+## v1.70.0 (2020-07-20)
+
+### Features
+
+ * Add model monitor support for us-gov-west-1
+ * support TFS 2.2
+
+### Bug Fixes and Other Changes
+
+ * reshape Artifacts into data frame in ExperimentsAnalytics
+
+### Documentation Changes
+
+ * fix MXNet version info for requirements.txt support
+
 ## v1.69.0 (2020-07-09)
 
 ### Features

VERSION

@@ -1 +1 @@
-1.69.1.dev0
+1.70.3.dev0

doc/frameworks/mxnet/using_mxnet.rst

@@ -321,13 +321,14 @@ If there are other packages you want to use with your script, you can include a
 Both ``requirements.txt`` and your training script should be put in the same folder.
 You must specify this folder in ``source_dir`` argument when creating an MXNet estimator.
 
-The function of installing packages using ``requirements.txt`` is supported for all MXNet versions during training.
+The function of installing packages using ``requirements.txt`` is supported for MXNet versions 1.3.0 and higher during training.
+
 When serving an MXNet model, support for this function varies with MXNet versions.
 For MXNet 1.6.0 or newer, ``requirements.txt`` must be under folder ``code``.
 The SageMaker MXNet Estimator automatically saves ``code`` in ``model.tar.gz`` after training (assuming you set up your script and ``requirements.txt`` correctly as stipulated in the previous paragraph).
 In the case of bringing your own trained model for deployment, you must save ``requirements.txt`` under folder ``code`` in ``model.tar.gz`` yourself or specify it through ``dependencies``.
-For MXNet 1.4.1, ``requirements.txt`` is not supported for inference.
-For MXNet 0.12.1-1.3.0, ``requirements.txt`` must be in ``source_dir``.
+For MXNet 0.12.1-1.2.1, 1.4.0-1.4.1, ``requirements.txt`` is not supported for inference.
+For MXNet 1.3.0, ``requirements.txt`` must be in ``source_dir``.
 
 A ``requirements.txt`` file is a text file that contains a list of items that are installed by using ``pip install``.
 You can also specify the version of an item to install.

doc/frameworks/tensorflow/using_tf.rst

@@ -197,6 +197,28 @@ The following args are not permitted when using Script Mode:
 Where the S3 url is a path to your training data within Amazon S3.
 The constructor keyword arguments define how SageMaker runs your training script.
 
+Specify a Docker image using an Estimator
+-----------------------------------------
+
+There are use cases, such as extending an existing pre-built Amazon SageMaker images, that require specifing a Docker image when creating an Estimator by directly specifying the ECR URI instead of the Python and framework version. For a full list of available container URIs, see `Available Deep Learning Containers Images <https://github.com/aws/deep-learning-containers/blob/master/available_images.md>`__ For more information on using Docker containers, see `Use Your Own Algorithms or Models with Amazon SageMaker <https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms.html>`__.
+
+When specifying the image, you must use the ``image_name=''`` arg to replace the following arg:
+
+- ``py_version=''``
+
+You should still specify the ``framework_version=''`` arg because the SageMaker Python SDK accomodates for differences in the images based on the version.
+
+The following example uses the ``image_name=''`` arg to specify the container image, Python version, and framework version.
+
+.. code:: python
+
+   tf_estimator = TensorFlow(entry_point='tf-train.py',
+                             role='SageMakerRole',
+                             train_instance_count=1,
+                             train_instance_type='ml.p2.xlarge',
+                             image_name='763104351884.dkr.ecr.<region>.amazonaws.com/<framework>-<job type>:<framework version>-<cpu/gpu>-<python version>-ubuntu18.04',
+                             script_mode=True)
+
 For more information about the sagemaker.tensorflow.TensorFlow estimator, see `SageMaker TensorFlow Classes`_.
 
 Call the fit Method

src/sagemaker/analytics.py

@@ -431,6 +431,8 @@ def __init__(
         metric_names=None,
         parameter_names=None,
         sagemaker_session=None,
+        input_artifact_names=None,
+        output_artifact_names=None,
     ):
         """Initialize a ``ExperimentAnalytics`` instance.
 
@@ -450,6 +452,11 @@ def __init__(
             sagemaker_session (sagemaker.session.Session): Session object which manages interactions
                 with Amazon SageMaker APIs and any other AWS services needed. If not specified,
                 one is created using the default AWS configuration chain.
+            input_artifact_names(dict optional):The input artifacts for the experiment. Examples of
+                input artifacts are datasets, algorithms, hyperparameters, source code, and instance
+                types.
+            output_artifact_names(dict optional): The output artifacts for the experiment. Examples
+                of output artifacts are metrics, snapshots, logs, and images.
         """
         sagemaker_session = sagemaker_session or Session()
         self._sage_client = sagemaker_session.sagemaker_client
@@ -463,6 +470,8 @@ def __init__(
         self._sort_order = sort_order
         self._metric_names = metric_names
         self._parameter_names = parameter_names
+        self._input_artifact_names = input_artifact_names
+        self._output_artifact_names = output_artifact_names
         self._trial_components = None
         super(ExperimentAnalytics, self).__init__()
         self.clear_cache()
@@ -516,6 +525,21 @@ def _reshape_metrics(self, metrics):
                     out["{} - {}".format(metric_name, stat_type)] = stat_value
         return out
 
+    def _reshape_artifacts(self, artifacts, _artifact_names):
+        """Reshape trial component input/output artifacts to a pandas column
+        Args:
+            artifacts: trial component input/output artifacts
+        Returns:
+            dict: Key: artifacts name, Value: artifacts value
+        """
+        out = OrderedDict()
+        for name, value in sorted(artifacts.items()):
+            if _artifact_names and (name not in _artifact_names):
+                continue
+            out["{} - {}".format(name, "MediaType")] = value.get("MediaType")
+            out["{} - {}".format(name, "Value")] = value.get("Value")
+        return out
+
     def _reshape(self, trial_component):
         """Reshape trial component data to pandas columns
         Args:
@@ -533,6 +557,16 @@ def _reshape(self, trial_component):
 
         out.update(self._reshape_parameters(trial_component.get("Parameters", [])))
         out.update(self._reshape_metrics(trial_component.get("Metrics", [])))
+        out.update(
+            self._reshape_artifacts(
+                trial_component.get("InputArtifacts", []), self._input_artifact_names
+            )
+        )
+        out.update(
+            self._reshape_artifacts(
+                trial_component.get("OutputArtifacts", []), self._output_artifact_names
+            )
+        )
         return out
 
     def _fetch_dataframe(self):

src/sagemaker/model_monitor/model_monitoring.py

@@ -57,6 +57,7 @@
     "ca-central-1": "536280801234",
     "cn-north-1": "453000072557",
     "cn-northwest-1": "453252182341",
+    "us-gov-west-1": "362178532790",
 }
 
 STATISTICS_JSON_DEFAULT_FILE_NAME = "statistics.json"

src/sagemaker/workflow/airflow.py

@@ -1144,7 +1144,7 @@ def processing_config(
         config["Environment"] = processor.env
 
     if processor.network_config is not None:
-        config["NetworkConfig"] = processor.network_config
+        config["NetworkConfig"] = processor.network_config._to_request_dict()
 
     processing_resources = sagemaker.processing.ProcessingJob.prepare_processing_resources(
         instance_count=processor.instance_count,
@@ -1154,10 +1154,11 @@ def processing_config(
     )
     config["ProcessingResources"] = processing_resources
 
-    stopping_condition = sagemaker.processing.ProcessingJob.prepare_stopping_condition(
-        processor.max_runtime_in_seconds
-    )
-    config["StoppingCondition"] = stopping_condition
+    if processor.max_runtime_in_seconds is not None:
+        stopping_condition = sagemaker.processing.ProcessingJob.prepare_stopping_condition(
+            processor.max_runtime_in_seconds
+        )
+        config["StoppingCondition"] = stopping_condition
 
     if processor.tags is not None:
         config["Tags"] = processor.tags
@@ -1174,4 +1175,6 @@ def input_output_list_converter(object_list):
     Returns:
         List of dicts
     """
-    return [obj._to_request_dict() for obj in object_list]
+    if object_list:
+        return [obj._to_request_dict() for obj in object_list]
+    return object_list

tests/integ/test_experiments_analytics.py

@@ -43,6 +43,65 @@ def experiment(sagemaker_session):
         _delete_resources(sm, experiment_name, trials)
 
 
+@contextmanager
+def experiment_with_artifacts(sagemaker_session):
+    sm = sagemaker_session.sagemaker_client
+    trials = {}  # for resource cleanup
+
+    experiment_name = "experiment-" + str(uuid.uuid4())
+    try:
+        sm.create_experiment(ExperimentName=experiment_name)
+
+        # Search returns 10 results by default. Add 20 trials to verify pagination.
+        for i in range(20):
+            trial_name = "trial-" + str(uuid.uuid4())
+            sm.create_trial(TrialName=trial_name, ExperimentName=experiment_name)
+
+            trial_component_name = "tc-" + str(uuid.uuid4())
+            trials[trial_name] = trial_component_name
+
+            sm.create_trial_component(
+                TrialComponentName=trial_component_name, DisplayName="Training"
+            )
+            sm.update_trial_component(
+                TrialComponentName=trial_component_name,
+                Parameters={"hp1": {"NumberValue": i}},
+                InputArtifacts={
+                    "inputArtifacts1": {"MediaType": "text/csv", "Value": "s3:/foo/bar1"}
+                },
+                OutputArtifacts={
+                    "outputArtifacts1": {"MediaType": "text/plain", "Value": "s3:/foo/bar2"}
+                },
+            )
+            sm.associate_trial_component(
+                TrialComponentName=trial_component_name, TrialName=trial_name
+            )
+
+        time.sleep(15)  # wait for search to get updated
+
+        yield experiment_name
+    finally:
+        _delete_resources(sm, experiment_name, trials)
+
+
+@pytest.mark.canary_quick
+def test_experiment_analytics_artifacts(sagemaker_session):
+    with experiment_with_artifacts(sagemaker_session) as experiment_name:
+        analytics = ExperimentAnalytics(
+            experiment_name=experiment_name, sagemaker_session=sagemaker_session
+        )
+
+        assert list(analytics.dataframe().columns) == [
+            "TrialComponentName",
+            "DisplayName",
+            "hp1",
+            "inputArtifacts1 - MediaType",
+            "inputArtifacts1 - Value",
+            "outputArtifacts1 - MediaType",
+            "outputArtifacts1 - Value",
+        ]
+
+
 @pytest.mark.canary_quick
 def test_experiment_analytics(sagemaker_session):
     with experiment(sagemaker_session) as experiment_name:

tests/unit/test_airflow.py

@@ -16,6 +16,7 @@
 from mock import Mock, MagicMock, patch
 
 from sagemaker import chainer, estimator, model, mxnet, tensorflow, transformer, tuner, processing
+from sagemaker.network import NetworkConfig
 from sagemaker.processing import ProcessingInput, ProcessingOutput
 from sagemaker.workflow import airflow
 from sagemaker.amazon import amazon_estimator
@@ -1598,6 +1599,13 @@ def test_deploy_config_from_amazon_alg_estimator(sagemaker_session):
 @patch("sagemaker.utils.sagemaker_timestamp", MagicMock(return_value=TIME_STAMP))
 def test_processing_config(sagemaker_session):
 
+    network_config = NetworkConfig(
+        encrypt_inter_container_traffic=False,
+        enable_network_isolation=True,
+        security_group_ids=["sg1"],
+        subnets=["subnet1"],
+    )
+
     processor = processing.Processor(
         role="arn:aws:iam::0122345678910:role/SageMakerPowerUser",
         image_uri="{{ image_uri }}",
@@ -1612,6 +1620,7 @@ def test_processing_config(sagemaker_session):
         sagemaker_session=sagemaker_session,
         tags=[{"{{ key }}": "{{ value }}"}],
         env={"{{ key }}": "{{ value }}"},
+        network_config=network_config,
     )
 
     outputs = [
@@ -1699,5 +1708,10 @@ def test_processing_config(sagemaker_session):
         "RoleArn": "arn:aws:iam::0122345678910:role/SageMakerPowerUser",
         "StoppingCondition": {"MaxRuntimeInSeconds": 3600},
         "Tags": [{"{{ key }}": "{{ value }}"}],
+        "NetworkConfig": {
+            "EnableInterContainerTrafficEncryption": False,
+            "EnableNetworkIsolation": True,
+            "VpcConfig": {"SecurityGroupIds": ["sg1"], "Subnets": ["subnet1"]},
+        },
     }
     assert config == expected_config

tests/unit/test_experiments_analytics.py

@@ -40,6 +40,14 @@ def trial_component(trial_component_name):
                 "Count": 2.0,
             },
         ],
+        "InputArtifacts": {
+            "inputArtifacts1": {"MediaType": "text/plain", "Value": "s3:/foo/bar1"},
+            "inputArtifacts2": {"MediaType": "text/plain", "Value": "s3:/foo/bar2"},
+        },
+        "OutputArtifacts": {
+            "outputArtifacts1": {"MediaType": "text/csv", "Value": "s3:/sky/far1"},
+            "outputArtifacts2": {"MediaType": "text/csv", "Value": "s3:/sky/far2"},
+        },
     }
 
 
@@ -72,6 +80,14 @@ def test_trial_analytics_dataframe_all_metrics_hyperparams(mock_session):
                 ("metric2 - StdDev", [0.05, 0.05]),
                 ("metric2 - Last", [7.0, 7.0]),
                 ("metric2 - Count", [2.0, 2.0]),
+                ("inputArtifacts1 - MediaType", ["text/plain", "text/plain"]),
+                ("inputArtifacts1 - Value", ["s3:/foo/bar1", "s3:/foo/bar1"]),
+                ("inputArtifacts2 - MediaType", ["text/plain", "text/plain"]),
+                ("inputArtifacts2 - Value", ["s3:/foo/bar2", "s3:/foo/bar2"]),
+                ("outputArtifacts1 - MediaType", ["text/csv", "text/csv"]),
+                ("outputArtifacts1 - Value", ["s3:/sky/far1", "s3:/sky/far1"]),
+                ("outputArtifacts2 - MediaType", ["text/csv", "text/csv"]),
+                ("outputArtifacts2 - Value", ["s3:/sky/far2", "s3:/sky/far2"]),
             ]
         )
     )
@@ -117,6 +133,14 @@ def test_trial_analytics_dataframe_selected_hyperparams(mock_session):
                 ("metric2 - StdDev", [0.05, 0.05]),
                 ("metric2 - Last", [7.0, 7.0]),
                 ("metric2 - Count", [2.0, 2.0]),
+                ("inputArtifacts1 - MediaType", ["text/plain", "text/plain"]),
+                ("inputArtifacts1 - Value", ["s3:/foo/bar1", "s3:/foo/bar1"]),
+                ("inputArtifacts2 - MediaType", ["text/plain", "text/plain"]),
+                ("inputArtifacts2 - Value", ["s3:/foo/bar2", "s3:/foo/bar2"]),
+                ("outputArtifacts1 - MediaType", ["text/csv", "text/csv"]),
+                ("outputArtifacts1 - Value", ["s3:/sky/far1", "s3:/sky/far1"]),
+                ("outputArtifacts2 - MediaType", ["text/csv", "text/csv"]),
+                ("outputArtifacts2 - Value", ["s3:/sky/far2", "s3:/sky/far2"]),
             ]
         )
     )
@@ -157,6 +181,14 @@ def test_trial_analytics_dataframe_selected_metrics(mock_session):
                 ("metric1 - StdDev", [1.0, 1.0]),
                 ("metric1 - Last", [2.0, 2.0]),
                 ("metric1 - Count", [2.0, 2.0]),
+                ("inputArtifacts1 - MediaType", ["text/plain", "text/plain"]),
+                ("inputArtifacts1 - Value", ["s3:/foo/bar1", "s3:/foo/bar1"]),
+                ("inputArtifacts2 - MediaType", ["text/plain", "text/plain"]),
+                ("inputArtifacts2 - Value", ["s3:/foo/bar2", "s3:/foo/bar2"]),
+                ("outputArtifacts1 - MediaType", ["text/csv", "text/csv"]),
+                ("outputArtifacts1 - Value", ["s3:/sky/far1", "s3:/sky/far1"]),
+                ("outputArtifacts2 - MediaType", ["text/csv", "text/csv"]),
+                ("outputArtifacts2 - Value", ["s3:/sky/far2", "s3:/sky/far2"]),
             ]
         )
     )
@@ -203,6 +235,14 @@ def test_trial_analytics_dataframe_search_pagination(mock_session):
                 ("metric2 - StdDev", [0.05, 0.05]),
                 ("metric2 - Last", [7.0, 7.0]),
                 ("metric2 - Count", [2.0, 2.0]),
+                ("inputArtifacts1 - MediaType", ["text/plain", "text/plain"]),
+                ("inputArtifacts1 - Value", ["s3:/foo/bar1", "s3:/foo/bar1"]),
+                ("inputArtifacts2 - MediaType", ["text/plain", "text/plain"]),
+                ("inputArtifacts2 - Value", ["s3:/foo/bar2", "s3:/foo/bar2"]),
+                ("outputArtifacts1 - MediaType", ["text/csv", "text/csv"]),
+                ("outputArtifacts1 - Value", ["s3:/sky/far1", "s3:/sky/far1"]),
+                ("outputArtifacts2 - MediaType", ["text/csv", "text/csv"]),
+                ("outputArtifacts2 - Value", ["s3:/sky/far2", "s3:/sky/far2"]),
             ]
         )
     )