Skip to content

Commit c20a4a2

Browse files
authored
doc: update README with pytorch eia section (#55)
* doc: pytorch eia
1 parent 470418d commit c20a4a2

File tree

1 file changed

+31
-0
lines changed

1 file changed

+31
-0
lines changed

README.rst

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ Table of Contents
2525

2626
#. `Getting Started <#getting-started>`__
2727
#. `Building your Image <#building-your-image>`__
28+
#. `Amazon Elastic Inference with PyTorch in SageMaker <#amazon-elastic-inference-with-pytorch-in-sagemaker>`__
2829
#. `Running the tests <#running-the-tests>`__
2930

3031
Getting Started
@@ -142,6 +143,36 @@ If you want to build "final" Docker images, then use:
142143
# GPU
143144
docker build -t preprod-pytorch:1.0.0-gpu-py3 -f docker/1.0.0/final/Dockerfile.gpu --build-arg py_version=3 .
144145

146+
Amazon Elastic Inference with PyTorch in SageMaker
147+
--------------------------------------------------
148+
`Amazon Elastic Inference <https://aws.amazon.com/machine-learning/elastic-inference/>`__ allows you to to attach
149+
low-cost GPU-powered acceleration to Amazon EC2 and Amazon SageMaker instances to reduce the cost of running deep
150+
learning inference by up to 75%. Currently, Amazon Elastic Inference supports TensorFlow, Apache MXNet, PyTorch,
151+
and ONNX models.
152+
153+
Support for using PyTorch with Amazon Elastic Inference in SageMaker is supported in the public SageMaker PyTorch serving containers.
154+
155+
* For information on how to use the Python SDK to create an endpoint with Amazon Elastic Inference and PyTorch in SageMaker, see `Deploying PyTorch Models <https://sagemaker.readthedocs.io/en/stable/using_pytorch.html#deploy-pytorch-models>`__.
156+
* For information on how Amazon Elastic Inference works, see `How EI Works <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html#ei-how-it-works>`__.
157+
* For more information in regards to using Amazon Elastic Inference in SageMaker, see `Amazon SageMaker Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html>`__.
158+
159+
Building the SageMaker Elastic Inference PyTorch Serving container
160+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
161+
Amazon Elastic Inference is designed to be used with AWS enhanced versions of TensorFlow serving, Apache MXNet or PyTorch serving.
162+
The SageMaker PyTorch containers with Amazon Elastic Inference support were built utilizing the
163+
same instructions listed `above <#building-your-image>`__ with the
164+
EIA Dockerfiles, which are all named ``Dockerfile.eia``, and can be found in the same ``docker/`` directory.
165+
166+
Example:
167+
168+
::
169+
170+
# PyTorch 1.3.1, Python 3, EI
171+
$ cp dist/sagemaker_pytorch_inference-*.tar.gz dist/sagemaker_pytorch_inference.tar.gz
172+
$ docker build -t preprod-pytorch-serving-eia:1.3.1-cpu-py3 -f docker/1.3.1/py3/Dockerfile.eia .
173+
174+
175+
* Currently, only PyTorch serving 1.3.1 is supported for Elastic Inference.
145176

146177
Running the tests
147178
-----------------

0 commit comments

Comments
 (0)