Skip to content

Commit 855552c

Browse files
stacichonavinsoni
authored andcommitted
documentation: New content for Pipelines local mode
1 parent 4e8bcab commit 855552c

File tree

7 files changed

+2229
-8
lines changed

7 files changed

+2229
-8
lines changed

doc/Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33

44
# You can set these variables from the command line.
55
SPHINXOPTS = -W
6-
SPHINXBUILD = python -msphinx
6+
SPHINXBUILD = python3 -msphinx
77
SPHINXPROJ = sagemaker
88
SOURCEDIR = .
99
BUILDDIR = _build

doc/amazon_sagemaker_model_building_pipeline.rst

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,54 @@ When you use :class:`sagemaker.workflow.pipeline_context.PipelineSession` rather
9797
.. warning::
9898
A :class:`sagemaker.workflow.pipeline_context.PipelineSession` must be given in order to start the job during pipeline execution time. Otherwise, a training job will get started immediately.
9999

100+
Local Pipeline Session
101+
======================
102+
103+
Like Pipeline Session, Local Pipeline Session provides a convenient way to capture input job arguments without starting the job. These input arguments can be provided in the :code:`step_args` parameter to their corresponding `Pipelines step type <https://sagemaker.readthedocs.io/en/stable/workflows/pipelines/sagemaker.workflow.pipelines.html#sagemaker.workflow.steps.Step>`__. The difference between :class:`sagemaker.workflow.pipeline_context.PipelineSession` and :class:`sagemaker.workflow.pipeline_context.LocalPipelineSession` is that :class:`sagemaker.workflow.pipeline_context.LocalPipelineSession` is used to run SageMaker pipelines locally (in local mode) whereas using :class:`sagemaker.workflow.pipeline_context.PipelineSession` runs the job on the managed service.
104+
105+
.. code-block:: python
106+
107+
from sagemaker.workflow.pipeline_context import LocalPipelineSession
108+
109+
local_pipeline_session = LocalPipelineSession()
110+
111+
pytorch_estimator = PyTorch(
112+
sagemaker_session=local_pipeline_session,
113+
role=sagemaker.get_execution_role(),
114+
instance_type="ml.c5.xlarge",
115+
instance_count=1,
116+
framework_version="1.8.0",
117+
py_version="py36",
118+
entry_point="./entry_point.py",
119+
)
120+
121+
step = TrainingStep(
122+
name="MyTrainingStep",
123+
step_args=pytorch_estimator.fit(
124+
inputs=TrainingInput(s3_data="s3://my-bucket/my-data/train"),
125+
)
126+
)
127+
128+
pipeline = Pipeline(
129+
name="MyPipeline",
130+
steps=[step],
131+
sagemaker_session=local_pipeline_session
132+
)
133+
134+
pipeline.create(
135+
role_arn=sagemaker.get_execution_role(),
136+
description="local pipeline example"
137+
)
138+
139+
// pipeline will execute locally
140+
pipeline.start()
141+
142+
steps = pipeline.list_steps()
143+
144+
training_job_name = steps['PipelineExecutionSteps'][0]['Metadata']['TrainingJob']['Arn']
145+
146+
step_outputs = pipeline_session.sagemaker_client.describe_training_job(TrainingJobName = training_job_name)
147+
100148
101149
Pipeline Parameters
102150
======================

0 commit comments

Comments
 (0)