Skip to content

Commit 1af767f

Browse files
laurenyulaurenyu
committed
doc: update TF documentation to reflect breaking changes and how to upgrade
1 parent 0f5c9b5 commit 1af767f

File tree

5 files changed

+311
-102
lines changed

5 files changed

+311
-102
lines changed

‎doc/frameworks/rl/using_rl.rst

+2-2
Original file line numberDiff line numberDiff line change
@@ -217,7 +217,7 @@ which was run when you called ``fit``. This was the model you saved to ``model_d
217217
In case if ``image_name`` was specified it would use provided image for the deployment.
218218

219219
``deploy`` returns a ``sagemaker.mxnet.MXNetPredictor`` for MXNet or
220-
``sagemaker.tensorflow.serving.Predictor`` for TensorFlow.
220+
``sagemaker.tensorflow.TensorFlowPredictor`` for TensorFlow.
221221

222222
``predict`` returns the result of inference against your model.
223223

@@ -241,7 +241,7 @@ In case if ``image_name`` was specified it would use provided image for the depl
241241
response = predictor.predict(data)
242242
243243
For more information please see `The SageMaker MXNet Model Server <https://sagemaker.readthedocs.io/en/stable/using_mxnet.html#the-sagemaker-mxnet-model-server>`_
244-
and `Deploying to TensorFlow Serving Endpoints <https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/tensorflow/deploying_tensorflow_serving.rst>`_ documentation.
244+
and `Deploying to TensorFlow Serving Endpoints <deploying_tensorflow_serving.html>`_ documentation.
245245

246246

247247
Working with Existing Training Jobs

‎src/sagemaker/tensorflow/deploying_tensorflow_serving.rst renamed to ‎doc/frameworks/tensorflow/deploying_tensorflow_serving.rst

+8-8
Original file line numberDiff line numberDiff line change
@@ -58,19 +58,19 @@ If you already have existing model artifacts in S3, you can skip training and de
5858

5959
.. code:: python
6060
61-
from sagemaker.tensorflow.serving import Model
61+
from sagemaker.tensorflow import TensorFlowModel
6262
63-
model = Model(model_data='s3://mybucket/model.tar.gz', role='MySageMakerRole')
63+
model = TensorFlowModel(model_data='s3://mybucket/model.tar.gz', role='MySageMakerRole')
6464
6565
predictor = model.deploy(initial_instance_count=1, instance_type='ml.c5.xlarge')
6666
6767
Python-based TensorFlow serving on SageMaker has support for `Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html>`__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to attach an Elastic Inference accelerator to your endpoint provide the accelerator type to accelerator_type to your deploy call.
6868

6969
.. code:: python
7070
71-
from sagemaker.tensorflow.serving import Model
71+
from sagemaker.tensorflow import TensorFlowModel
7272
73-
model = Model(model_data='s3://mybucket/model.tar.gz', role='MySageMakerRole')
73+
model = TensorFlowModel(model_data='s3://mybucket/model.tar.gz', role='MySageMakerRole')
7474
7575
predictor = model.deploy(initial_instance_count=1, instance_type='ml.c5.xlarge', accelerator_type='ml.eia1.medium')
7676
@@ -276,7 +276,7 @@ This customized Python code must be named ``inference.py`` and specified through
276276

277277
.. code::
278278
279-
from sagemaker.tensorflow.serving import Model
279+
from sagemaker.tensorflow import TensorFlowModel
280280
281281
model = Model(entry_point='inference.py',
282282
model_data='s3://mybucket/model.tar.gz',
@@ -429,7 +429,7 @@ processing. There are 2 ways to do this:
429429

430430
.. code::
431431
432-
from sagemaker.tensorflow.serving import Model
432+
from sagemaker.tensorflow import TensorFlowModel
433433
434434
model = Model(entry_point='inference.py',
435435
source_dir='source/directory',
@@ -447,7 +447,7 @@ processing. There are 2 ways to do this:
447447

448448
.. code::
449449
450-
from sagemaker.tensorflow.serving import Model
450+
from sagemaker.tensorflow import TensorFlowModel
451451
452452
model = Model(entry_point='inference.py',
453453
dependencies=['/path/to/folder/named/lib'],
@@ -546,7 +546,7 @@ For the remaining steps, let's return to python code using the SageMaker Python
546546

547547
.. code:: python
548548
549-
from sagemaker.tensorflow.serving import Model, Predictor
549+
from sagemaker.tensorflow import TensorFlowModel, TensorFlowPredictor
550550
551551
# change this to the name or ARN of your SageMaker execution role
552552
role = 'SageMakerRole'

‎doc/frameworks/tensorflow/index.rst

+4-2
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,15 @@
1-
#############################
1+
##########
22
TensorFlow
3-
#############################
3+
##########
44

55
A managed environment for TensorFlow training and hosting on Amazon SageMaker
66

77
.. toctree::
88
:maxdepth: 1
99

1010
using_tf
11+
deploying_tensorflow_serving
12+
upgrade_from_legacy
1113

1214
.. toctree::
1315
:maxdepth: 2

‎doc/frameworks/tensorflow/upgrade_from_legacy.rst

+254
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
######################################
2+
Upgrade from Legacy TensorFlow Support
3+
######################################
4+
5+
With v2 of the SageMaker Python SDK, support for legacy SageMaker TensorFlow images has been deprecated.
6+
This guide explains how to upgrade your SageMaker Python SDK usage.
7+
8+
For more information about using TensorFlow with the SageMaker Python SDK, see `Use TensorFlow with the SageMaker Python SDK <using_tf.html>`_.
9+
10+
.. contents::
11+
12+
********************************************
13+
What Constitutes "Legacy TensorFlow Support"
14+
********************************************
15+
16+
This guide is relevant if one of the following applies:
17+
18+
#. You are using TensorFlow versions 1.4-1.10
19+
#. You are using TensorFlow versions 1.11-1.12 with Python 2, and
20+
21+
- you do *not* have ``script_mode=True`` when creating your estimator
22+
- you are using ``sagemaker.tensorflow.model.TensorFlowModel`` and/or ``sagemaker.tensorflow.model.TensorFlowPredictor``
23+
24+
#. You are using a pre-built SageMaker image whose URI looks like ``520713654638.dkr.ecr.<region>.amazonaws.com/sagemaker-tensorflow:<tag>``
25+
26+
If one of the above applies, then keep reading.
27+
28+
**************
29+
How to Upgrade
30+
**************
31+
32+
We recommend that you use the latest supported version of TensorFlow because that's where we focus our development efforts.
33+
For information about supported versions of TensorFlow, see the `AWS documentation <https://aws.amazon.com/releasenotes/available-deep-learning-containers-images>`_.
34+
35+
For general information about using TensorFlow with the SageMaker Python SDK, see `Use TensorFlow with the SageMaker Python SDK <using_tf.html>`_.
36+
37+
Training Script
38+
===============
39+
40+
Newer versions of TensorFlow require your training script to be runnable as a command-line script, similar to what you might run outside of SageMaker. For more information, including how to adapt a locally-runnable script, see `Prepare a Training Script <using_tf.html#id1>`_.
41+
42+
In addition, your training script needs to save your model. If you have your own ``serving_input_fn`` implementation, then that can be passed to an exporter:
43+
44+
.. code:: python
45+
46+
import tensorflow as tf
47+
48+
exporter = tf.estimator.LatestExporter("Servo", serving_input_receiver_fn=serving_input_fn)
49+
50+
For an example of how to repackage your legacy TensorFlow training script for use with a newer version of TensorFlow,
51+
see `this example notebook <https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-sdk/tensorflow_moving_from_framework_mode_to_script_mode/tensorflow_moving_from_framework_mode_to_script_mode.ipynb>`_.
52+
53+
Inference Script
54+
================
55+
56+
Newer versions of TensorFlow Serving require a different format for the inference script. Some key differences:
57+
58+
- The script must be named ``inference.py``.
59+
- ``input_fn`` has been replaced by ``input_handler``.
60+
- ``output_fn`` has been replaced by ``output_handler``.
61+
62+
Like with the legacy versions, the pre-built SageMaker TensorFlow Serving images do have default implementations for pre- and post-processing.
63+
64+
For more information about implementing your own handlers, see `How to implement the pre- and/or post-processing handler(s) <using_tf.html#how-to-implement-the-pre-and-or-post-processing-handler-s>`_.
65+
66+
*****************************
67+
Continue with Legacy Versions
68+
*****************************
69+
70+
While not recommended, you can still use a legacy TensorFlow version with v2 of the SageMaker Python SDK.
71+
In order to do so, you need to change how a few parameters are defined.
72+
73+
Training
74+
========
75+
76+
When creating an estimator, v2 requires the following changes:
77+
78+
#. Explicitly specify the ECR image URI via ``image_name``.
79+
To determine the URI, you can use :func:`sagemaker.fw_utils.create_image_uri`.
80+
#. Specify ``model_dir=False``.
81+
#. Use hyperparameters for ``training_steps``, ``evaluation_steps``, ``checkpoint_path``, and ``requirements_file``.
82+
83+
For example, if using TF 1.10.0 with an ml.m4.xlarge instance in us-west-2,
84+
the difference in code would be as follows:
85+
86+
.. code:: python
87+
88+
from sagemaker.tensorflow import TensorFlow
89+
90+
# v1
91+
estimator = TensorFlow(
92+
...
93+
source_dir="code",
94+
framework_version="1.10.0",
95+
train_instance_type="ml.m4.xlarge",
96+
training_steps=100,
97+
evaluation_steps=10,
98+
checkpoint_path="s3://bucket/path",
99+
requirements_file="requirements.txt",
100+
)
101+
102+
# v2
103+
estimator = TensorFlow(
104+
...
105+
source_dir="code",
106+
framework_version="1.10.0",
107+
train_instance_type="ml.m4.xlarge",
108+
image_name="520713654638.dkr.ecr.us-west-2.amazonaws.com/sagemaker-tensorflow:1.10.0-cpu-py2",
109+
hyperparameters={
110+
"training_steps": 100,
111+
"evaluation_steps": 10,
112+
"checkpoint_path": "s3://bucket/path",
113+
"sagemaker_requirements": "requirements.txt",
114+
},
115+
model_dir=False,
116+
)
117+
118+
Requirements File with Training
119+
-------------------------------
120+
121+
To provide a requirements file, define a hyperparameter named "sagemaker_requirements" that contains the relative path to the requirements file from ``source_dir``.
122+
123+
Inference
124+
=========
125+
126+
Using a legacy TensorFlow version for endpoints and batch transform can be achieved with v2 of the SageMaker Python SDK with some minor changes to your code.
127+
128+
From an Estimator
129+
-----------------
130+
131+
If you are starting with a training job, you can call :func:`sagemaker.estimator.EstimatorBase.deploy` or :func:`sagemaker.tensorflow.estimator.Estimator.transformer` from your estimator for inference.
132+
133+
To specify the number of model server workers, you need to set it through an environment variable named ``MODEL_SERVER_WORKERS``:
134+
135+
.. code:: python
136+
137+
# v1
138+
estimator.deploy(..., model_server_workers=4)
139+
140+
# v2
141+
estimator.deploy(..., env={"MODEL_SERVER_WORKERS": 4})
142+
143+
From a Model
144+
------------
145+
146+
If you are starting with a model, v2 requires the following changes:
147+
148+
#. Use the the :class:`sagemaker.model.FrameworkModel` class.
149+
#. Explicitly specify the ECR image URI via ``image``.
150+
To determine the URI, you can use :func:`sagemaker.fw_utils.create_image_uri`.
151+
#. Use an environment variable for ``model_server_workers``.
152+
153+
For example, if using TF 1.10.0 with a CPU instance in us-west-2,
154+
the difference in code would be as follows:
155+
156+
.. code:: python
157+
158+
# v1
159+
from sagemaker.tensorflow import TensorFlowModel
160+
161+
model = TensorFlowModel(
162+
...
163+
py_version="py2",
164+
framework_version="1.10.0",
165+
model_server_workers=4,
166+
)
167+
168+
# v2
169+
from sagemaker.model import FrameworkModel
170+
171+
model = FrameworkModel(
172+
...
173+
image="520713654638.dkr.ecr.us-west-2.amazonaws.com/sagemaker-tensorflow:1.10.0-cpu-py2",
174+
env={"MODEL_SERVER_WORKERS": 4},
175+
)
176+
177+
Requirements File with Inference
178+
--------------------------------
179+
180+
To provide a requirements file, define an environment variable named ``SAGEMAKER_REQUIREMENTS`` that contains the relative path to the requirements file from ``source_dir``.
181+
182+
From an estimator:
183+
184+
.. code:: python
185+
186+
# for an endpoint
187+
estimator.deploy(..., env={"SAGEMAKER_REQUIREMENTS": "requirements.txt"})
188+
189+
# for batch transform
190+
estimator.transformer(..., env={"SAGEMAKER_REQUIREMENTS": "requirements.txt"})
191+
192+
From a model:
193+
194+
.. code:: python
195+
196+
from sagemaker.model import FrameworkModel
197+
198+
model = FrameworkModel(
199+
...
200+
source_dir="code",
201+
env={"SAGEMAKER_REQUIREMENTS": "requirements.txt"},
202+
)
203+
204+
205+
Predictors
206+
----------
207+
208+
If you want to use your model for endpoints, then you can use the :class:`sagemaker.predictor.RealTimePredictor` class instead of the legacy ``sagemaker.tensorflow.TensorFlowPredictor`` class:
209+
210+
.. code:: python
211+
212+
from sagemaker.model import FrameworkModel
213+
from sagemaker.predictor import RealTimePredictor
214+
215+
model = FrameworkModel(
216+
...
217+
predictor_cls=RealTimePredictor,
218+
)
219+
220+
predictor = model.deploy(...)
221+
222+
If you are using protobuf prediction data, then you need to serialize and deserialize the data yourself.
223+
224+
For example:
225+
226+
.. code:: python
227+
228+
from google.protobuf import json_format
229+
from protobuf_to_dict import protobuf_to_dict
230+
from tensorflow.core.framework import tensor_pb2
231+
232+
# Serialize the prediction data
233+
json_format.MessageToJson(data)
234+
235+
# Get the prediction result
236+
result = predictor.predict(data)
237+
238+
# Deserialize the prediction result
239+
protobuf_to_dict(json_format.Parse(result, tensor_pb2.TensorProto()))
240+
241+
Otherwise, you can use the serializers and deserialzers available in the SageMaker Python SDK or write your own.
242+
243+
For example, if you want to use JSON serialization and deserialization:
244+
245+
.. code:: python
246+
247+
from sagemaker.predictor import json_deserializer, json_serializer
248+
249+
predictor.content_type = "application/json"
250+
predictor.serializer = json_serializer
251+
predictor.accept = "application/json"
252+
predictor.deserializer = json_deserializer
253+
254+
predictor.predict(data)

0 commit comments

Comments
 (0)