Skip to content

doc: clarify support for requirements.txt in Tensorflow docs #1512

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 19, 2020
Merged
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions doc/using_tf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,34 @@ In your training script the channels will be stored in environment variables ``S
``output_path``.


Use third-party libraries
-------------------------

If there are other packages you want to use with your script, you can use a ``requirements.txt`` to install other dependencies at runtime.

For training, support for installing packages using ``requirements.txt`` varies by TensorFlow version as follows:

- For TensorFlow 1.11 or newer using Script Mode without Horovod, TensorFlow 1.15.2 with Python 3.7 or newer, and TensorFlow 2.2 or newer:
- Include a ``requirements.txt`` file in the same directory as your training script.
- You must specify this directory using the ``source_dir`` argument when creating a TensorFlow estimator.
- For older versions of TensorFlow using Script Mode with Horovod:
- Write a shell script for your entry point that first calls ``pip install -r requirements.txt``, then runs your training script.
- For an example of using shell scripts, see `this example notebook <https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-sdk/tensorflow_script_mode_using_shell_commands/tensorflow_script_mode_using_shell_commands.ipynb>`__.
- For older versions of TensorFlow using Legacy Mode:
- Specify the path to your ``requirements.txt`` file using the ``requirements_file`` argument.

For serving, support for installing packages using ``requirements.txt`` varies by TensorFlow version as follows:

- For TensorFlow 1.11 or newer:
- Include a ``requirements.txt`` file in the ``code`` directory.
- For older versions of TensorFlow:
- Specify the path to your ``requirements.txt`` file using the ``SAGEMAKER_REQUIREMENTS`` environment variable.

A ``requirements.txt`` file is a text file that contains a list of items that are installed by using ``pip install``.
You can also specify the version of an item to install.
For information about the format of a ``requirements.txt`` file, see `Requirements Files <https://pip.pypa.io/en/stable/user_guide/#requirements-files>`__ in the pip documentation.


Create an Estimator
===================

Expand Down Expand Up @@ -215,7 +243,7 @@ Calling ``fit`` starts a SageMaker training job. The training job will execute t
- starts asynchronous training

If the ``wait=False`` flag is passed to ``fit``, then it returns immediately. The training job continues running
asynchronously. Later, a Tensorflow estimator can be obtained by attaching to the existing training job.
asynchronously. Later, a TensorFlow estimator can be obtained by attaching to the existing training job.
If the training job is not finished, it starts showing the standard output of training and wait until it completes.
After attaching, the estimator can be deployed as usual.

Expand Down Expand Up @@ -882,8 +910,7 @@ in the following code:
You can also bring in external dependencies to help with your data
processing. There are 2 ways to do this:

1. If you included ``requirements.txt`` in your ``source_dir`` or in
your dependencies, the container installs the Python dependencies at runtime using ``pip install -r``:
1. If your model archive contains ``code/requirements.txt``, the container will install the Python dependencies at runtime using ``pip install -r``.

.. code::

Expand Down