Skip to content

doc: update README with pytorch eia section #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 11, 2020
Merged

doc: update README with pytorch eia section #55

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fce039a
doc: pytorch eia
Mar 10, 2020
930c046
fix twine error
Mar 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Table of Contents

#. `Getting Started <#getting-started>`__
#. `Building your Image <#building-your-image>`__
#. `Amazon Elastic Inference with PyTorch in SageMaker <#amazon-elastic-inference-with-pytorch-in-sagemaker>`__
#. `Running the tests <#running-the-tests>`__

Getting Started
Expand Down Expand Up @@ -142,6 +143,36 @@ If you want to build "final" Docker images, then use:
# GPU
docker build -t preprod-pytorch:1.0.0-gpu-py3 -f docker/1.0.0/final/Dockerfile.gpu --build-arg py_version=3 .

Amazon Elastic Inference with PyTorch in SageMaker
--------------------------------------------------
`Amazon Elastic Inference <https://aws.amazon.com/machine-learning/elastic-inference/>`__ allows you to to attach
low-cost GPU-powered acceleration to Amazon EC2 and Amazon SageMaker instances to reduce the cost of running deep
learning inference by up to 75%. Currently, Amazon Elastic Inference supports TensorFlow, Apache MXNet, PyTorch,
and ONNX models.

Support for using PyTorch with Amazon Elastic Inference in SageMaker is supported in the public SageMaker PyTorch serving containers.
Copy link
Contributor

@ajaykarpur ajaykarpur Mar 11, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to rephrase this sentence to simply: "Amazon Elastic Inference is supported in the public SageMaker PyTorch serving containers."


* For information on how to use the Python SDK to create an endpoint with Amazon Elastic Inference and PyTorch in SageMaker, see `Deploying PyTorch Models <https://sagemaker.readthedocs.io/en/stable/using_pytorch.html#deploy-pytorch-models>`__.
* For information on how Amazon Elastic Inference works, see `How EI Works <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html#ei-how-it-works>`__.
* For more information in regards to using Amazon Elastic Inference in SageMaker, see `Amazon SageMaker Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html>`__.

Building the SageMaker Elastic Inference PyTorch Serving container
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Amazon Elastic Inference is designed to be used with AWS enhanced versions of TensorFlow serving, Apache MXNet or PyTorch serving.
The SageMaker PyTorch containers with Amazon Elastic Inference support were built utilizing the
same instructions listed `above <#building-your-image>`__ with the
EIA Dockerfiles, which are all named ``Dockerfile.eia``, and can be found in the same ``docker/`` directory.

Example:

::

# PyTorch 1.3.1, Python 3, EI
$ cp dist/sagemaker_pytorch_inference-*.tar.gz dist/sagemaker_pytorch_inference.tar.gz
$ docker build -t preprod-pytorch-serving-eia:1.3.1-cpu-py3 -f docker/1.3.1/py3/Dockerfile.eia .


* Currently, only PyTorch serving 1.3.1 is supported for Elastic Inference.

Running the tests
-----------------
Expand Down