Skip to content

Commit d34d431

Browse files
author
Ivy Bazan
committed
Merge branch 'IvyBazan-master'
2 parents 191819d + e5b5984 commit d34d431

26 files changed

+389
-86
lines changed

‎CHANGELOG.md

+44
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,49 @@
11
# Changelog
22

3+
## v1.51.2 (2020-03-11)
4+
5+
### Bug Fixes and Other Changes
6+
7+
* handle empty inputs/outputs in ProcessingJob.from_processing_name()
8+
* use DLC images for GovCloud
9+
10+
### Testing and Release Infrastructure
11+
12+
* generate test job name at test start instead of module start
13+
14+
## v1.51.1 (2020-03-10)
15+
16+
### Bug Fixes and Other Changes
17+
18+
* skip pytorch ei test in unsupported regions
19+
20+
### Documentation Changes
21+
22+
* correct MultiString/MULTI_STRING docstring
23+
24+
## v1.51.0 (2020-03-09)
25+
26+
### Features
27+
28+
* pytorch 1.3.1 eia support
29+
30+
### Documentation Changes
31+
32+
* Update Kubernetes Operator default tag
33+
* improve docstring for tuner.best_estimator()
34+
35+
## v1.50.18.post0 (2020-03-05)
36+
37+
### Documentation Changes
38+
39+
* correct Estimator code_location default S3 path
40+
41+
## v1.50.18 (2020-03-04)
42+
43+
### Bug Fixes and Other Changes
44+
45+
* change default compile model max run to 15 mins
46+
347
## v1.50.17.post0 (2020-03-03)
448

549
### Testing and Release Infrastructure

‎README.rst

+4-2
Original file line numberDiff line numberDiff line change
@@ -176,7 +176,7 @@ TensorFlow SageMaker Estimators
176176

177177
By using TensorFlow SageMaker Estimators, you can train and host TensorFlow models on Amazon SageMaker.
178178

179-
Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``, ``1.9.0``, ``1.10.0``, ``1.11.0``, ``1.12.0``, ``1.13.1``, ``1.14.0``, ``1.15.0``, ``2.0.0``.
179+
Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``, ``1.9.0``, ``1.10.0``, ``1.11.0``, ``1.12.0``, ``1.13.1``, ``1.14.0``, ``1.15.0``, ``1.15.2``, ``2.0.0``, ``2.0.1``, ``2.1.0``.
180180

181181
Supported versions of TensorFlow for Elastic Inference: ``1.11.0``, ``1.12.0``, ``1.13.1``, ``1.14.0``.
182182

@@ -208,7 +208,9 @@ PyTorch SageMaker Estimators
208208

209209
With PyTorch SageMaker Estimators, you can train and host PyTorch models on Amazon SageMaker.
210210

211-
Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``.
211+
Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``, ``1.4.0``.
212+
213+
Supported versions of PyTorch for Elastic Inference: ``1.3.1``.
212214

213215
We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
214216

‎VERSION

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.50.18.dev0
1+
1.51.3.dev0

‎doc/amazon_sagemaker_operators_for_kubernetes.rst

+5-5
Original file line numberDiff line numberDiff line change
@@ -349,7 +349,7 @@ that directory to your \ ``PATH``.
349349

350350
export os="linux"
351351

352-
wget https://amazon-sagemaker-operator-for-k8s-us-east-1.s3.amazonaws.com/kubectl-smlogs-plugin/latest/${os}.amd64.tar.gz
352+
wget https://amazon-sagemaker-operator-for-k8s-us-east-1.s3.amazonaws.com/kubectl-smlogs-plugin/v1/${os}.amd64.tar.gz
353353
tar xvzf ${os}.amd64.tar.gz
354354

355355
# Move binaries to a directory in your homedir.
@@ -550,13 +550,13 @@ each region.
550550
+-------------+---------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------+
551551
| Region | Controller Image | Linux SMLogs |
552552
+=============+=============================================================================================+========================================================================================================================+
553-
| us-east-1 | ``957583890962.dkr.ecr.us-east-1.amazonaws.com/amazon-sagemaker-operator-for-k8s:latest`` | https://amazon-sagemaker-operator-for-k8s-us-east-1.s3.amazonaws.com/kubectl-smlogs-plugin/latest/linux.amd64.tar.gz |
553+
| us-east-1 | ``957583890962.dkr.ecr.us-east-1.amazonaws.com/amazon-sagemaker-operator-for-k8s:v1`` | https://amazon-sagemaker-operator-for-k8s-us-east-1.s3.amazonaws.com/kubectl-smlogs-plugin/v1/linux.amd64.tar.gz |
554554
+-------------+---------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------+
555-
| us-east-2 | ``922499468684.dkr.ecr.us-east-2.amazonaws.com/amazon-sagemaker-operator-for-k8s:latest`` | https://amazon-sagemaker-operator-for-k8s-us-east-2.s3.amazonaws.com/kubectl-smlogs-plugin/latest/linux.amd64.tar.gz |
555+
| us-east-2 | ``922499468684.dkr.ecr.us-east-2.amazonaws.com/amazon-sagemaker-operator-for-k8s:v1`` | https://amazon-sagemaker-operator-for-k8s-us-east-2.s3.amazonaws.com/kubectl-smlogs-plugin/v1/linux.amd64.tar.gz |
556556
+-------------+---------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------+
557-
| us-west-2 | ``640106867763.dkr.ecr.us-west-2.amazonaws.com/amazon-sagemaker-operator-for-k8s:latest`` | https://amazon-sagemaker-operator-for-k8s-us-west-2.s3.amazonaws.com/kubectl-smlogs-plugin/latest/linux.amd64.tar.gz |
557+
| us-west-2 | ``640106867763.dkr.ecr.us-west-2.amazonaws.com/amazon-sagemaker-operator-for-k8s:v1`` | https://amazon-sagemaker-operator-for-k8s-us-west-2.s3.amazonaws.com/kubectl-smlogs-plugin/v1/linux.amd64.tar.gz |
558558
+-------------+---------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------+
559-
| eu-west-1 | ``613661167059.dkr.ecr.eu-west-1.amazonaws.com/amazon-sagemaker-operator-for-k8s:latest`` | https://amazon-sagemaker-operator-for-k8s-eu-west-1.s3.amazonaws.com/kubectl-smlogs-plugin/latest/linux.amd64.tar.gz |
559+
| eu-west-1 | ``613661167059.dkr.ecr.eu-west-1.amazonaws.com/amazon-sagemaker-operator-for-k8s:v1`` | https://amazon-sagemaker-operator-for-k8s-eu-west-1.s3.amazonaws.com/kubectl-smlogs-plugin/v1/linux.amd64.tar.gz |
560560
+-------------+---------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------+
561561

562562

‎doc/using_pytorch.rst

+15
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ With PyTorch Estimators and Models, you can train and host PyTorch models on Ama
66

77
Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``.
88

9+
Supported versions of PyTorch for Elastic Inference: ``1.3.1``.
10+
911
We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
1012

1113
You can visit the PyTorch repository at https://github.com/pytorch/pytorch.
@@ -250,6 +252,14 @@ You use the SageMaker PyTorch model server to host your PyTorch model when you c
250252
Estimator. The model server runs inside a SageMaker Endpoint, which your call to ``deploy`` creates.
251253
You can access the name of the Endpoint by the ``name`` property on the returned ``Predictor``.
252254

255+
PyTorch on Amazon SageMaker has support for `Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html>`_, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance.
256+
In order to attach an Elastic Inference accelerator to your endpoint provide the accelerator type to ``accelerator_type`` to your ``deploy`` call.
257+
258+
.. code:: python
259+
260+
predictor = pytorch_estimator.deploy(instance_type='ml.m4.xlarge',
261+
initial_instance_count=1,
262+
accelerator_type='ml.eia2.medium')
253263
254264
The SageMaker PyTorch Model Server
255265
==================================
@@ -291,6 +301,11 @@ It loads the model parameters from a ``model.pth`` file in the SageMaker model d
291301
model.load_state_dict(torch.load(f))
292302
return model
293303
304+
However, if you are using PyTorch Elastic Inference, you do not have to provide a ``model_fn`` since the PyTorch serving
305+
container has a default one for you. But please note that if you are utilizing the default ``model_fn``, please save
306+
yor parameter file as ``model.pt`` instead of ``model.pth``. For more information on inference script, please refer to:
307+
`SageMaker PyTorch Default Inference Handler <https://github.com/aws/sagemaker-pytorch-serving-container/blob/master/src/sagemaker_pytorch_serving_container/default_inference_handler.py>`_.
308+
294309
Serve a PyTorch Model
295310
---------------------
296311

‎src/sagemaker/algorithm.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -328,8 +328,8 @@ def transformer(
328328
instance_type (str): Type of EC2 instance to use, for example,
329329
'ml.c4.xlarge'.
330330
strategy (str): The strategy used to decide how to batch records in
331-
a single request (default: None). Valid values: 'MULTI_RECORD'
332-
and 'SINGLE_RECORD'.
331+
a single request (default: None). Valid values: 'MultiRecord'
332+
and 'SingleRecord'.
333333
assemble_with (str): How the output is assembled (default: None).
334334
Valid values: 'Line' or 'None'.
335335
output_path (str): S3 location for saving the transform result. If

‎src/sagemaker/estimator.py

+5-5
Original file line numberDiff line numberDiff line change
@@ -823,8 +823,8 @@ def transformer(
823823
instance_type (str): Type of EC2 instance to use, for example,
824824
'ml.c4.xlarge'.
825825
strategy (str): The strategy used to decide how to batch records in
826-
a single request (default: None). Valid values: 'MULTI_RECORD'
827-
and 'SINGLE_RECORD'.
826+
a single request (default: None). Valid values: 'MultiRecord'
827+
and 'SingleRecord'.
828828
assemble_with (str): How the output is assembled (default: None).
829829
Valid values: 'Line' or 'None'.
830830
output_path (str): S3 location for saving the transform result. If
@@ -1470,7 +1470,7 @@ def __init__(
14701470
uploaded (default: None) - don't include a trailing slash since
14711471
a string prepended with a "/" is appended to ``code_location``. The code
14721472
file uploaded to S3 is 'code_location/job-name/source/sourcedir.tar.gz'.
1473-
If not specified, the default ``code location`` is s3://default_bucket/job-name/.
1473+
If not specified, the default ``code location`` is s3://output_bucket/job-name/.
14741474
image_name (str): An alternate image name to use instead of the
14751475
official Sagemaker image for the framework. This is useful to
14761476
run one of the Sagemaker supported frameworks with an image
@@ -1895,8 +1895,8 @@ def transformer(
18951895
instance_type (str): Type of EC2 instance to use, for example,
18961896
'ml.c4.xlarge'.
18971897
strategy (str): The strategy used to decide how to batch records in
1898-
a single request (default: None). Valid values: 'MULTI_RECORD'
1899-
and 'SINGLE_RECORD'.
1898+
a single request (default: None). Valid values: 'MultiRecord'
1899+
and 'SingleRecord'.
19001900
assemble_with (str): How the output is assembled (default: None).
19011901
Valid values: 'Line' or 'None'.
19021902
output_path (str): S3 location for saving the transform result. If

‎src/sagemaker/fw_utils.py

+22-25
Original file line numberDiff line numberDiff line change
@@ -53,9 +53,16 @@
5353
)
5454

5555
VALID_PY_VERSIONS = ["py2", "py3"]
56-
VALID_EIA_FRAMEWORKS = ["tensorflow", "tensorflow-serving", "mxnet", "mxnet-serving"]
56+
VALID_EIA_FRAMEWORKS = [
57+
"tensorflow",
58+
"tensorflow-serving",
59+
"mxnet",
60+
"mxnet-serving",
61+
"pytorch-serving",
62+
]
63+
PY2_RESTRICTED_EIA_FRAMEWORKS = ["pytorch-serving"]
5764
VALID_ACCOUNTS_BY_REGION = {"us-gov-west-1": "246785580436", "us-iso-east-1": "744548109606"}
58-
ASIMOV_VALID_ACCOUNTS_BY_REGION = {"us-iso-east-1": "886529160074"}
65+
ASIMOV_VALID_ACCOUNTS_BY_REGION = {"us-gov-west-1": "442386744353", "us-iso-east-1": "886529160074"}
5966
OPT_IN_ACCOUNTS_BY_REGION = {"ap-east-1": "057415533634", "me-south-1": "724002660598"}
6067
ASIMOV_OPT_IN_ACCOUNTS_BY_REGION = {"ap-east-1": "871362719292", "me-south-1": "217643126080"}
6168
DEFAULT_ACCOUNT = "520713654638"
@@ -71,6 +78,7 @@
7178
"mxnet-serving-eia": "mxnet-inference-eia",
7279
"pytorch": "pytorch-training",
7380
"pytorch-serving": "pytorch-inference",
81+
"pytorch-serving-eia": "pytorch-inference-eia",
7482
}
7583

7684
MERGED_FRAMEWORKS_LOWEST_VERSIONS = {
@@ -82,6 +90,7 @@
8290
"mxnet-serving-eia": [1, 4, 1],
8391
"pytorch": [1, 2, 0],
8492
"pytorch-serving": [1, 2, 0],
93+
"pytorch-serving-eia": [1, 3, 1],
8594
}
8695

8796
DEBUGGER_UNSUPPORTED_REGIONS = ["us-gov-west-1", "us-iso-east-1"]
@@ -124,25 +133,6 @@ def _is_dlc_version(framework, framework_version, py_version):
124133
return False
125134

126135

127-
def _use_dlc_image(region, framework, py_version, framework_version):
128-
"""Return if the DLC image should be used for the given framework,
129-
framework version, Python version, and region.
130-
131-
Args:
132-
region (str): The AWS region.
133-
framework (str): The framework name, e.g. "tensorflow-scriptmode".
134-
py_version (str): The Python version, e.g. "py3".
135-
framework_version (str): The framework version.
136-
137-
Returns:
138-
bool: Whether or not to use the corresponding DLC image.
139-
"""
140-
is_gov_region = region in VALID_ACCOUNTS_BY_REGION
141-
is_dlc_version = _is_dlc_version(framework, framework_version, py_version)
142-
143-
return ((not is_gov_region) or region in ASIMOV_VALID_ACCOUNTS_BY_REGION) and is_dlc_version
144-
145-
146136
def _registry_id(region, framework, py_version, account, framework_version):
147137
"""Return the Amazon ECR registry number (or AWS account ID) for
148138
the given framework, framework version, Python version, and region.
@@ -159,7 +149,7 @@ def _registry_id(region, framework, py_version, account, framework_version):
159149
specific one for the framework, framework version, Python version,
160150
and region, then ``account`` is returned.
161151
"""
162-
if _use_dlc_image(region, framework, py_version, framework_version):
152+
if _is_dlc_version(framework, framework_version, py_version):
163153
if region in ASIMOV_OPT_IN_ACCOUNTS_BY_REGION:
164154
return ASIMOV_OPT_IN_ACCOUNTS_BY_REGION.get(region)
165155
if region in ASIMOV_VALID_ACCOUNTS_BY_REGION:
@@ -207,6 +197,7 @@ def create_image_uri(
207197

208198
if _accelerator_type_valid_for_framework(
209199
framework=framework,
200+
py_version=py_version,
210201
accelerator_type=accelerator_type,
211202
optimized_families=optimized_families,
212203
):
@@ -243,7 +234,7 @@ def create_image_uri(
243234
else:
244235
device_type = "cpu"
245236

246-
use_dlc_image = _use_dlc_image(region, framework, py_version, framework_version)
237+
use_dlc_image = _is_dlc_version(framework, framework_version, py_version)
247238

248239
if not py_version or (use_dlc_image and framework == "tensorflow-serving-eia"):
249240
tag = "{}-{}".format(framework_version, device_type)
@@ -259,21 +250,27 @@ def create_image_uri(
259250

260251

261252
def _accelerator_type_valid_for_framework(
262-
framework, accelerator_type=None, optimized_families=None
253+
framework, py_version, accelerator_type=None, optimized_families=None
263254
):
264255
"""
265256
Args:
266257
framework:
258+
py_version:
267259
accelerator_type:
268260
optimized_families:
269261
"""
270262
if accelerator_type is None:
271263
return False
272264

265+
if py_version == "py2" and framework in PY2_RESTRICTED_EIA_FRAMEWORKS:
266+
raise ValueError(
267+
"{} is not supported with Amazon Elastic Inference in Python 2.".format(framework)
268+
)
269+
273270
if framework not in VALID_EIA_FRAMEWORKS:
274271
raise ValueError(
275272
"{} is not supported with Amazon Elastic Inference. Currently only "
276-
"Python-based TensorFlow and MXNet are supported.".format(framework)
273+
"Python-based TensorFlow, MXNet, PyTorch are supported.".format(framework)
277274
)
278275

279276
if optimized_families:

‎src/sagemaker/model.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -504,8 +504,8 @@ def transformer(
504504
instance_type (str): Type of EC2 instance to use, for example,
505505
'ml.c4.xlarge'.
506506
strategy (str): The strategy used to decide how to batch records in
507-
a single request (default: None). Valid values: 'MULTI_RECORD'
508-
and 'SINGLE_RECORD'.
507+
a single request (default: None). Valid values: 'MultiRecord'
508+
and 'SingleRecord'.
509509
assemble_with (str): How the output is assembled (default: None).
510510
Valid values: 'Line' or 'None'.
511511
output_path (str): S3 location for saving the transform result. If

‎src/sagemaker/pipeline.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -215,8 +215,8 @@ def transformer(
215215
instance_type (str): Type of EC2 instance to use, for example,
216216
'ml.c4.xlarge'.
217217
strategy (str): The strategy used to decide how to batch records in
218-
a single request (default: None). Valid values: 'MULTI_RECORD'
219-
and 'SINGLE_RECORD'.
218+
a single request (default: None). Valid values: 'MultiRecord'
219+
and 'SingleRecord'.
220220
assemble_with (str): How the output is assembled (default: None).
221221
Valid values: 'Line' or 'None'.
222222
output_path (str): S3 location for saving the transform result. If

‎src/sagemaker/processing.py

+26-15
Original file line numberDiff line numberDiff line change
@@ -648,10 +648,9 @@ def from_processing_name(cls, sagemaker_session, processing_job_name):
648648
"""
649649
job_desc = sagemaker_session.describe_processing_job(job_name=processing_job_name)
650650

651-
return cls(
652-
sagemaker_session=sagemaker_session,
653-
job_name=processing_job_name,
654-
inputs=[
651+
inputs = None
652+
if job_desc.get("ProcessingInputs"):
653+
inputs = [
655654
ProcessingInput(
656655
source=processing_input["S3Input"]["S3Uri"],
657656
destination=processing_input["S3Input"]["LocalPath"],
@@ -664,19 +663,31 @@ def from_processing_name(cls, sagemaker_session, processing_job_name):
664663
s3_compression_type=processing_input["S3Input"].get("S3CompressionType"),
665664
)
666665
for processing_input in job_desc["ProcessingInputs"]
667-
],
668-
outputs=[
666+
]
667+
668+
outputs = None
669+
if job_desc.get("ProcessingOutputConfig") and job_desc["ProcessingOutputConfig"].get(
670+
"Outputs"
671+
):
672+
outputs = [
669673
ProcessingOutput(
670-
source=job_desc["ProcessingOutputConfig"]["Outputs"][0]["S3Output"][
671-
"LocalPath"
672-
],
673-
destination=job_desc["ProcessingOutputConfig"]["Outputs"][0]["S3Output"][
674-
"S3Uri"
675-
],
676-
output_name=job_desc["ProcessingOutputConfig"]["Outputs"][0]["OutputName"],
674+
source=processing_output["S3Output"]["LocalPath"],
675+
destination=processing_output["S3Output"]["S3Uri"],
676+
output_name=processing_output["OutputName"],
677677
)
678-
],
679-
output_kms_key=job_desc["ProcessingOutputConfig"].get("KmsKeyId"),
678+
for processing_output in job_desc["ProcessingOutputConfig"]["Outputs"]
679+
]
680+
681+
output_kms_key = None
682+
if job_desc.get("ProcessingOutputConfig"):
683+
output_kms_key = job_desc["ProcessingOutputConfig"].get("KmsKeyId")
684+
685+
return cls(
686+
sagemaker_session=sagemaker_session,
687+
job_name=processing_job_name,
688+
inputs=inputs,
689+
outputs=outputs,
690+
output_kms_key=output_kms_key,
680691
)
681692

682693
@classmethod

‎src/sagemaker/pytorch/README.rst

+2
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ With PyTorch Estimators and Models, you can train and host PyTorch models on Ama
66

77
Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``, ``1.4.0``.
88

9+
Supported versions of PyTorch for Elastic Inference: ``1.3.1``.
10+
911
We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
1012

1113
You can visit the PyTorch repository at https://github.com/pytorch/pytorch.

0 commit comments

Comments
 (0)