Skip to content

Document upcoming MXNet training script format #390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Sep 18, 2018
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions src/sagemaker/mxnet/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -599,13 +599,16 @@ The code executed from your main guard needs to:
3. Save the model

Hyperparameters will be passed as command-line arguments to your training script.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest replacing "will be" with "are".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest replacing "will be" with "are"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used future tense because these instructions are going to be live awhile before the changes themselves are released - I'm afraid present tense might be too confusing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough

In addition, the locations for finding input data and saving the model and output data will need to be defined.
In addition, the locations for finding input data and saving the model and output data will be provided as environment variables rather than as arguments to a function.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion:

In addition, you specify the locations of input data and where to save the model artifacts and output data as environment variables in the container, rather than as arguments to a function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the container provides this info to the user, not the other way around. Would this be clearer?

In addition, the container will define the locations of input data and where to save the model artifacts and output data as environment variables rather than passing that information through train.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor change:

...rather than passing that information as arguments to the train function.

You can find the full list of available environment variables in the `SageMaker Containers README <https://github.com/aws/sagemaker-containers#list-of-provided-environment-variables-by-sagemaker-containers>`__.

We recommend using `an argument parser <https://docs.python.org/3.5/howto/argparse.html>`__ for this part.
Using the ``argparse`` library as an example, the code would look something like this:

.. code:: python

import argparse
import os

if __name__ == '__main__':
parser = argparse.ArgumentParser()
Expand All @@ -616,9 +619,9 @@ Using the ``argparse`` library as an example, the code would look something like
parser.add_argument('--learning-rate', type=float, default=0.1)

# input data and model directories
parser.add_argument('--model-dir', type=str, default='opt/ml/model')
parser.add_argument('--train', type=str, default='opt/ml/input/data/train')
parser.add_argument('--test', type=str, default='opt/ml/input/data/test')
parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR'])
parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAIN'])
parser.add_argument('--test', type=str, default=os.environ['SM_CHANNEL_TEST'])

args, _ = parser.parse_known_args()

Expand Down