You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Convert file to ASCII from UTF-8 to avoid compile failure on Windows (#15)
* Convert file to ascii from utf-8 to avoid compile (and install) failure on windows
* Replace all ansi 93 and 94 'smart' quote characters with regular ansi 22 octal quote character
* Replace characters 92 hex
* Replace character 91
Copy file name to clipboardExpand all lines: README.rst
+31-31
Original file line number
Diff line number
Diff line change
@@ -78,7 +78,7 @@ Building Sphinx docs
78
78
79
79
make html
80
80
81
-
You can edit the templates for any of the pages in the docs by editing the .rst files in the “doc” directory and then running “``make html``” again.
81
+
You can edit the templates for any of the pages in the docs by editing the .rst files in the "doc" directory and then running "``make html``" again.
82
82
83
83
84
84
SageMaker Python SDK Overview
@@ -109,7 +109,7 @@ With MXNet Estimators, you can train and host MXNet models on Amazon SageMaker.
109
109
Training with MXNet
110
110
~~~~~~~~~~~~~~~~~~~
111
111
112
-
Training MXNet models using ``MXNet`` Estimators is a two-step process. First, you prepare your training script, then second, you run this on SageMaker via an ``MXNet`` Estimator. You should prepare your script in a separate source file than the notebook, terminal session, or source file you’re using to submit the script to SageMaker via an ``MXNet`` Estimator.
112
+
Training MXNet models using ``MXNet`` Estimators is a two-step process. First, you prepare your training script, then second, you run this on SageMaker via an ``MXNet`` Estimator. You should prepare your script in a separate source file than the notebook, terminal session, or source file you're using to submit the script to SageMaker via an ``MXNet`` Estimator.
113
113
114
114
Suppose that you already have an MXNet training script called
115
115
``mxnet-train.py``. You can run this script in SageMaker as follows:
@@ -122,7 +122,7 @@ Suppose that you already have an MXNet training script called
122
122
123
123
Where the s3 url is a path to your training data, within Amazon S3. The constructor keyword arguments define how SageMaker runs your training script and are discussed, in detail, in a later section.
124
124
125
-
In the following sections, we’ll discuss how to prepare a training script for execution on SageMaker, then how to run that script on SageMaker using an ``MXNet`` Estimator.
125
+
In the following sections, we'll discuss how to prepare a training script for execution on SageMaker, then how to run that script on SageMaker using an ``MXNet`` Estimator.
126
126
127
127
Preparing the MXNet training script
128
128
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -135,13 +135,13 @@ When you run your script on SageMaker via the ``MXNet`` Estimator, SageMaker inj
135
135
to SageMaker TrainingJob that runs your MXNet training script. You
136
136
can use this to pass hyperparameters to your training script.
137
137
- ``input_data_config (dict[string,dict])``: The SageMaker TrainingJob
138
-
InputDataConfig object, that’s set when the SageMaker TrainingJob is
138
+
InputDataConfig object, that's set when the SageMaker TrainingJob is
139
139
created. This is discussed in more detail below.
140
140
- ``channel_input_dirs (dict[string,string])``: A collection of
141
141
directories containing training data. When you run training, you can
142
-
partition your training data into different logical “channels”.
143
-
Depending on your problem, some common channel ideas are: “train”,
144
-
“test”, “evaluation” or “images’,”labels“.
142
+
partition your training data into different logical "channels".
143
+
Depending on your problem, some common channel ideas are: "train",
144
+
"test", "evaluation" or "images',"labels".
145
145
- ``output_data_dir (str)``: A directory where your training script can
146
146
write data that will be moved to s3 after training is complete.
147
147
- ``num_gpus (int)``: The number of GPU devices available on your
@@ -161,7 +161,7 @@ A training script that takes advantage of all arguments would have the following
161
161
num_gpus, num_cpus, hosts, current_host):
162
162
pass
163
163
164
-
You don’t have to use all the arguments, arguments you don’t care about can be ignored by including ``**kwargs``.
164
+
You don't have to use all the arguments, arguments you don't care about can be ignored by including ``**kwargs``.
165
165
166
166
.. code:: python
167
167
@@ -170,7 +170,7 @@ You don’t have to use all the arguments, arguments you don’t care about can
170
170
pass
171
171
172
172
**Note: Writing a training script that imports correctly**
173
-
When SageMaker runs your training script, it imports it as a Python module and then invokes ``train`` on the imported module. Consequently, you should not include any statements that won’t execute successfully in SageMaker when your module is imported. For example, don’t attempt to open any local files in top-level statements in your training script.
173
+
When SageMaker runs your training script, it imports it as a Python module and then invokes ``train`` on the imported module. Consequently, you should not include any statements that won't execute successfully in SageMaker when your module is imported. For example, don't attempt to open any local files in top-level statements in your training script.
174
174
175
175
If you want to run your training script locally via the Python interpreter, look at using a ``___name__ == '__main__'`` guard, discussed in more detail here: https://stackoverflow.com/questions/419163/what-does-if-name-main-do .
176
176
@@ -182,7 +182,7 @@ You can import both ``mxnet`` and ``numpy`` in your training script. When your s
182
182
Running an MXNet training script in SageMaker
183
183
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
184
184
185
-
You run MXNet training scripts on SageMaker by creating ``MXNet`` Estimators. SageMaker training of your script is invoked when you call ``fit`` on an ``MXNet`` Estimator. The following code sample shows how you train a custom MXNet script “train.py”.
185
+
You run MXNet training scripts on SageMaker by creating ``MXNet`` Estimators. SageMaker training of your script is invoked when you call ``fit`` on an ``MXNet`` Estimator. The following code sample shows how you train a custom MXNet script "train.py".
186
186
187
187
.. code:: python
188
188
@@ -211,7 +211,7 @@ The following are required arguments to the ``MXNet`` constructor. When you crea
211
211
- ``train_instance_count`` Number of Amazon EC2 instances to use for
212
212
training.
213
213
- ``train_instance_type`` Type of EC2 instance to use for training, for
214
-
example, ‘ml.c4.xlarge’.
214
+
example, 'ml.c4.xlarge'.
215
215
216
216
Optional arguments
217
217
''''''''''''''''''
@@ -231,12 +231,12 @@ The following are optional arguments. When you create an ``MXNet`` object, you c
231
231
model training code.
232
232
- ``train_volume_size`` Size in GB of the EBS volume to use for storing
233
233
input data during training. Must be large enough to store training
234
-
data if input_mode=‘File’ is used (which is the default).
234
+
data if input_mode='File' is used (which is the default).
235
235
- ``train_max_run`` Timeout in hours for training, after which Amazon
236
236
SageMaker terminates the job regardless of its current status.
237
237
- ``input_mode`` The input mode that the algorithm supports. Valid
238
-
modes: ‘File’ - Amazon SageMaker copies the training dataset from the
239
-
s3 location to a directory in the Docker container. ‘Pipe’ - Amazon
238
+
modes: 'File' - Amazon SageMaker copies the training dataset from the
239
+
s3 location to a directory in the Docker container. 'Pipe' - Amazon
240
240
SageMaker streams data directly from s3 to the container via a Unix
241
241
named pipe.
242
242
- ``output_path`` s3 location where you want the training result (model
@@ -292,7 +292,7 @@ Just as you enable training by defining a ``train`` function in your training sc
292
292
293
293
SageMaker provides a default implementation of ``save`` that works with MXNet Module API ``Module`` objects. If your training script does not define a ``save`` function, then the default ``save`` function will be invoked on the return-value of your ``train`` function.
294
294
295
-
The following script demonstrates how to return a model from train, that’s compatible with the default ``save`` function.
295
+
The following script demonstrates how to return a model from train, that's compatible with the default ``save`` function.
296
296
297
297
.. code:: python
298
298
@@ -325,7 +325,7 @@ After your training job is complete, your model data will available in the s3 ``
325
325
MXNet Module serialization in SageMaker
326
326
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
327
327
328
-
If you train function returns a ``Module`` object, it will be serialized by the default Module serialization system, unless you’ve specified a custom ``save`` function.
328
+
If you train function returns a ``Module`` object, it will be serialized by the default Module serialization system, unless you've specified a custom ``save`` function.
329
329
330
330
The default serialization system generates three files:
331
331
@@ -369,7 +369,7 @@ After your ``train`` function completes, SageMaker will invoke ``save`` with the
369
369
370
370
**Note: How to save Gluon models with SageMaker**
371
371
372
-
If your train function returns a Gluon API ``net`` object as its model, you’ll need to write your own ``save`` function. You will want to serialize the ``net`` parameters. Saving ``net`` parameters is covered in the `Serialization section <http://gluon.mxnet.io/chapter03_deep-neural-networks/serialization.html>`__ of the collaborative Gluon deep-learning book `“The Straight Dope”<http://gluon.mxnet.io/index.html>`__.
372
+
If your train function returns a Gluon API ``net`` object as its model, you'll need to write your own ``save`` function. You will want to serialize the ``net`` parameters. Saving ``net`` parameters is covered in the `Serialization section <http://gluon.mxnet.io/chapter03_deep-neural-networks/serialization.html>`__ of the collaborative Gluon deep-learning book `"The Straight Dope"<http://gluon.mxnet.io/index.html>`__.
373
373
374
374
Deploying MXNet models
375
375
~~~~~~~~~~~~~~~~~~~~~~
@@ -408,9 +408,9 @@ As with MXNet training, you configure the MXNet model server by defining functio
408
408
Model loading
409
409
^^^^^^^^^^^^^
410
410
411
-
Before a model can be served, it must be loaded. The SageMaker model server loads your model by invoking a ``model_fn`` function on your training script. If you don’t provide a ``model_fn`` function, SageMaker will use a default ``model_fn`` function. The default function works with MXNet Module model objects, saved via the default ``save`` function.
411
+
Before a model can be served, it must be loaded. The SageMaker model server loads your model by invoking a ``model_fn`` function on your training script. If you don't provide a ``model_fn`` function, SageMaker will use a default ``model_fn`` function. The default function works with MXNet Module model objects, saved via the default ``save`` function.
412
412
413
-
If you wrote a custom ``save`` function then you may need to write a custom ``model_fn`` function. If your save function serializes ``Module`` objects under the same formatas the default ``save`` function, then you won’t need to write a custom model_fn function. If you do write a ``model_fn`` function must have the following signature:
413
+
If you wrote a custom ``save`` function then you may need to write a custom ``model_fn`` function. If your save function serializes ``Module`` objects under the same formatas the default ``save`` function, then you won't need to write a custom model_fn function. If you do write a ``model_fn`` function must have the following signature:
414
414
415
415
.. code:: python
416
416
@@ -482,11 +482,11 @@ Input processing
482
482
483
483
When an InvokeEndpoint operation is made against an Endpoint running a SageMaker MXNet model server, the model server receives two pieces of information:
484
484
485
-
- The request Content-Type, for example “application/json”
485
+
- The request Content-Type, for example "application/json"
486
486
- The request data body, a byte array which is at most 5 MB (5 \* 1024
487
487
\* 1024 bytes) in size.
488
488
489
-
The SageMaker MXNet model server will invoke an “input_fn” function in your training script, passing in this information. If you define an ``input_fn`` function definition, it should return an object that can be passed to ``predict_fn`` and have the following signature:
489
+
The SageMaker MXNet model server will invoke an "input_fn" function in your training script, passing in this information. If you define an ``input_fn`` function definition, it should return an object that can be passed to ``predict_fn`` and have the following signature:
490
490
491
491
.. code:: python
492
492
@@ -496,7 +496,7 @@ Where ``request_body`` is a byte buffer, ``request_content_type`` is a Python st
496
496
497
497
The SageMaker MXNet model server provides a default implementation of ``input_fn``. This function deserializes JSON or CSV encoded data into an MXNet ``NDArrayIter`` `(external API docs) <https://mxnet.incubator.apache.org/api/python/io.html#mxnet.io.NDArrayIter>`__ multi-dimensional array iterator. This works with the default ``predict_fn`` implementation, which expects an ``NDArrayIter`` as input.
498
498
499
-
Default json deserialization requires ``request_body`` contain a single json list. Sending multiple json objects within the same ``request_body`` is not supported. The list must have a dimensionality compatible with the MXNet ``net`` or ``Module`` object. Specifically, after the list is loaded, it’s either padded or split to fit the first dimension of the model input shape. The list’s shape must be identical to the model’s input shape, for all dimensions after the first.
499
+
Default json deserialization requires ``request_body`` contain a single json list. Sending multiple json objects within the same ``request_body`` is not supported. The list must have a dimensionality compatible with the MXNet ``net`` or ``Module`` object. Specifically, after the list is loaded, it's either padded or split to fit the first dimension of the model input shape. The list's shape must be identical to the model's input shape, for all dimensions after the first.
500
500
501
501
Default csv deserialization requires ``request_body`` contain one or more lines of CSV numerical data. The data is loaded into a two-dimensional array, where each line break defines the boundaries of the first dimension. This two-dimensional array is then re-shaped to be compatible with the shape expected by the model object. Specifically, the first dimension is kept unchanged, but the second dimension is reshaped to be consistent with the shape of all dimensions in the model, following the first dimension.
502
502
@@ -566,7 +566,7 @@ The ``output_fn`` has the following signature:
566
566
Where ``prediction`` is the result of invoking ``predict_fn`` and
567
567
``content_type`` is the InvokeEndpoint requested response content-type. The function should return a byte array of data serialized to content_type.
568
568
569
-
The default implementation expects ``prediction`` to be an ``NDArray`` and can serialize the result to either JSON or CSV. It accepts response content types of “application/json” and “text/csv”.
569
+
The default implementation expects ``prediction`` to be an ``NDArray`` and can serialize the result to either JSON or CSV. It accepts response content types of "application/json" and "text/csv".
570
570
571
571
Distributed MXNet training
572
572
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -642,20 +642,20 @@ The MXNetModel constructor takes the following arguments:
642
642
custom code will be uploaded to. If not specified, will use the
643
643
SageMaker default bucket created by sagemaker.Session.
644
644
- ``sagemaker_session (sagemaker.Session):`` The SageMaker Session
645
-
object, used for SageMaker interaction“”"
645
+
object, used for SageMaker interaction"""
646
646
647
647
Your model data must be a .tar.gz file in S3. SageMaker Training Job model data is saved to .tar.gz files in S3, however if you have local data you want to deploy, you can prepare the data yourself.
648
648
649
-
Assuming you have a local directory containg your model data named “my_model” you can tar and gzip compress the file and upload to S3 using the following commands:
649
+
Assuming you have a local directory containg your model data named "my_model" you can tar and gzip compress the file and upload to S3 using the following commands:
You’ll need to upload the data to S3 before training. You can use the AWS Command Line Tool (the aws cli) to achieve this.
1444
+
You'll need to upload the data to S3 before training. You can use the AWS Command Line Tool (the aws cli) to achieve this.
1445
1445
1446
-
If you don’t have the aws cli, you can install it using pip:
1446
+
If you don't have the aws cli, you can install it using pip:
1447
1447
1448
1448
::
1449
1449
1450
1450
pip install awscli --upgrade --user
1451
1451
1452
-
If you don’t have pip or want to learn more about installing the aws cli, please refer to the official `Amazon aws cli installation guide <http://docs.aws.amazon.com/cli/latest/userguide/installing.html>`__.
1452
+
If you don't have pip or want to learn more about installing the aws cli, please refer to the official `Amazon aws cli installation guide <http://docs.aws.amazon.com/cli/latest/userguide/installing.html>`__.
1453
1453
1454
1454
Once you have the aws cli installed, you can upload a directory of files to S3 with the following command:
0 commit comments