From 44afc9269ad6a29796f07dfcb199206fc06e452b Mon Sep 17 00:00:00 2001 From: Balaji Veeramani Date: Thu, 5 Aug 2021 16:02:37 -0500 Subject: [PATCH 1/4] Add `LambdaModel` and `LambdaPredictor` documentation --- doc/api/inference/model.rst | 5 ++++ doc/api/inference/predictors.rst | 5 ++++ doc/overview.rst | 42 ++++++++++++++++++++++++++++++++ 3 files changed, 52 insertions(+) diff --git a/doc/api/inference/model.rst b/doc/api/inference/model.rst index 2846c58d0b..a975086162 100644 --- a/doc/api/inference/model.rst +++ b/doc/api/inference/model.rst @@ -15,3 +15,8 @@ Model :members: :undoc-members: :show-inheritance: + +.. autoclass:: sagemaker.serverless.model.LambdaModel + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/api/inference/predictors.rst b/doc/api/inference/predictors.rst index 6a9243f329..66ffcf45be 100644 --- a/doc/api/inference/predictors.rst +++ b/doc/api/inference/predictors.rst @@ -7,3 +7,8 @@ Make real-time predictions against SageMaker endpoints with Python objects :members: :undoc-members: :show-inheritance: + +.. autoclass:: sagemaker.serverless.predictor.LambdaPredictor + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/overview.rst b/doc/overview.rst index 02290ff94c..d16058eec4 100644 --- a/doc/overview.rst +++ b/doc/overview.rst @@ -1063,6 +1063,48 @@ You can also find these notebooks in the **Advanced Functionality** section of t For information about using sample notebooks in a SageMaker notebook instance, see `Use Example Notebooks `__ in the AWS documentation. +******************* +Serverless Inference +******************* + +You can use the SageMaker Python SDK to perform serverless inference on Lambda. + +To deploy models to Lambda, you must complete the following prerequisites: +1. `Package your model and inference code as a container image. `_ +2. `Create a role that lists Lambda as a trusted entity. `_ + +After completing the prerequisites, you can deploy your model to Lambda with +the ``LambdaModel`` class. + +.. code:: python + + from sagemaker.serverless import LambdaModel + + image_uri = f"{account}.dkr.ecr.{region}.amazonaws.com/{repository}:latest" + role = f"arn:aws:iam::{account}:role/{role}" + + model = LambdaModel(image_uri=image_uri, role=role) + predictor = model.deploy("my-lambda-function", timeout=20, memory_size=4092) + +The ``LambdaModel.deploy`` method returns a ``LambdaPredictor`` instance. Use +the ``LambdaPredictor`` instance to perform inference on Lambda. + +.. code:: python + + url = "https://c.files.bbci.co.uk/12A9B/production/_111434467_gettyimages-1143489763.jpg" + predictor.predict({"url": url}) # {'class': 'tabby'} + +Once you are done performing inference on Lambda, delete the ``LambdaModel`` +and ``LambdaPredictor`` instances. + +.. code:: python + + model.delete_model() + predictor.delete_predictor() + +For more details, see the API reference for `LambdaModel `_ +and `LambdaPredictor `_. + ****************** SageMaker Workflow ****************** From 7e4fd1d14cbd1f2ce534ffefe8ac1b635266905d Mon Sep 17 00:00:00 2001 From: Balaji Veeramani Date: Thu, 5 Aug 2021 18:56:30 -0500 Subject: [PATCH 2/4] Update overview.rst --- doc/overview.rst | 40 +++++++++++++++++++++------------------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/doc/overview.rst b/doc/overview.rst index d16058eec4..2485385940 100644 --- a/doc/overview.rst +++ b/doc/overview.rst @@ -523,7 +523,7 @@ That is, you can bring your own model: First, package the files for the trained model into a ``.tar.gz`` file, and upload the archive to S3. -Next, create a ``Model`` object that corresponds to the framework that you are using: `MXNetModel `__ or `TensorFlowModel `__. +Next, create a ``Model`` object that corresponds to the framework that you are using: `MXNetModel `__ or `TensorFlowModel `__. Example code using ``MXNetModel``: @@ -681,8 +681,8 @@ in the AWS documentation. For more detailed explanations of the classes that this library provides for automatic model tuning, see: -- `API docs for HyperparameterTuner and parameter range classes `__ -- `API docs for analytics classes `__ +- `API docs for HyperparameterTuner and parameter range classes `__ +- `API docs for analytics classes `__ ************************* SageMaker Batch Transform @@ -709,7 +709,7 @@ Alternatively, if you already have a SageMaker model, you can create an instance instance_count=1, instance_type='ml.m4.xlarge') -For a full list of the possible options to configure by using either of these methods, see the API docs for `Estimator `__ or `Transformer `__. +For a full list of the possible options to configure by using either of these methods, see the API docs for `Estimator `__ or `Transformer `__. After you create a ``Transformer`` object, you can invoke ``transform()`` to start a batch transform job with the S3 location of your data. You can also specify other attributes of your data, such as the content type. @@ -718,7 +718,7 @@ You can also specify other attributes of your data, such as the content type. transformer.transform('s3://my-bucket/batch-transform-input') -For more details about what can be specified here, see `API docs `__. +For more details about what can be specified here, see `API docs `__. ********** Local Mode @@ -1063,47 +1063,49 @@ You can also find these notebooks in the **Advanced Functionality** section of t For information about using sample notebooks in a SageMaker notebook instance, see `Use Example Notebooks `__ in the AWS documentation. -******************* +******************** Serverless Inference -******************* +******************** You can use the SageMaker Python SDK to perform serverless inference on Lambda. To deploy models to Lambda, you must complete the following prerequisites: -1. `Package your model and inference code as a container image. `_ -2. `Create a role that lists Lambda as a trusted entity. `_ -After completing the prerequisites, you can deploy your model to Lambda with -the ``LambdaModel`` class. +- `Package your model and inference code as a container image. `_ +- `Create a role that lists Lambda as a trusted entity. `_ + +After completing the prerequisites, you can deploy your model to Lambda using +the `LambdaModel`_ class. .. code:: python from sagemaker.serverless import LambdaModel - image_uri = f"{account}.dkr.ecr.{region}.amazonaws.com/{repository}:latest" - role = f"arn:aws:iam::{account}:role/{role}" + image_uri = "123456789012.dkr.ecr.us-west-2.amazonaws.com/my-lambda-repository:latest" + role = "arn:aws:iam::123456789012:role/MyLambdaExecutionRole" model = LambdaModel(image_uri=image_uri, role=role) predictor = model.deploy("my-lambda-function", timeout=20, memory_size=4092) -The ``LambdaModel.deploy`` method returns a ``LambdaPredictor`` instance. Use -the ``LambdaPredictor`` instance to perform inference on Lambda. +The ``deploy`` method returns a `LambdaPredictor`_ instance. Use the +`LambdaPredictor`_ ``predict`` method to perform inference on Lambda. .. code:: python url = "https://c.files.bbci.co.uk/12A9B/production/_111434467_gettyimages-1143489763.jpg" predictor.predict({"url": url}) # {'class': 'tabby'} -Once you are done performing inference on Lambda, delete the ``LambdaModel`` -and ``LambdaPredictor`` instances. +Once you are done performing inference on Lambda, free the `LambdaModel`_ and +`LambdaPredictor`_ resources using the ``delete_model`` and ``delete_predictor`` +methods. .. code:: python model.delete_model() predictor.delete_predictor() -For more details, see the API reference for `LambdaModel `_ -and `LambdaPredictor `_. +.. _LambdaModel : file:///Users/bvveeram/Documents/GitHub/sagemaker-python-sdk/doc/_build/html/api/inference/model.html#sagemaker.serverless.model.LambdaModel +.. _LambdaPredictor : file:///Users/bvveeram/Documents/GitHub/sagemaker-python-sdk/doc/_build/html/api/inference/predictors.html#sagemaker.serverless.predictor.LambdaPredictor ****************** SageMaker Workflow From 476c956be12ee0ab94188fcc9733bcc4610a5f43 Mon Sep 17 00:00:00 2001 From: Balaji Veeramani Date: Thu, 5 Aug 2021 19:02:04 -0500 Subject: [PATCH 3/4] Update overview.rst --- doc/overview.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/overview.rst b/doc/overview.rst index 2485385940..cdf506994f 100644 --- a/doc/overview.rst +++ b/doc/overview.rst @@ -523,7 +523,7 @@ That is, you can bring your own model: First, package the files for the trained model into a ``.tar.gz`` file, and upload the archive to S3. -Next, create a ``Model`` object that corresponds to the framework that you are using: `MXNetModel `__ or `TensorFlowModel `__. +Next, create a ``Model`` object that corresponds to the framework that you are using: `MXNetModel `__ or `TensorFlowModel `__. Example code using ``MXNetModel``: @@ -681,8 +681,8 @@ in the AWS documentation. For more detailed explanations of the classes that this library provides for automatic model tuning, see: -- `API docs for HyperparameterTuner and parameter range classes `__ -- `API docs for analytics classes `__ +- `API docs for HyperparameterTuner and parameter range classes `__ +- `API docs for analytics classes `__ ************************* SageMaker Batch Transform @@ -709,7 +709,7 @@ Alternatively, if you already have a SageMaker model, you can create an instance instance_count=1, instance_type='ml.m4.xlarge') -For a full list of the possible options to configure by using either of these methods, see the API docs for `Estimator `__ or `Transformer `__. +For a full list of the possible options to configure by using either of these methods, see the API docs for `Estimator `__ or `Transformer `__. After you create a ``Transformer`` object, you can invoke ``transform()`` to start a batch transform job with the S3 location of your data. You can also specify other attributes of your data, such as the content type. @@ -718,7 +718,7 @@ You can also specify other attributes of your data, such as the content type. transformer.transform('s3://my-bucket/batch-transform-input') -For more details about what can be specified here, see `API docs `__. +For more details about what can be specified here, see `API docs `__. ********** Local Mode @@ -1104,8 +1104,8 @@ methods. model.delete_model() predictor.delete_predictor() -.. _LambdaModel : file:///Users/bvveeram/Documents/GitHub/sagemaker-python-sdk/doc/_build/html/api/inference/model.html#sagemaker.serverless.model.LambdaModel -.. _LambdaPredictor : file:///Users/bvveeram/Documents/GitHub/sagemaker-python-sdk/doc/_build/html/api/inference/predictors.html#sagemaker.serverless.predictor.LambdaPredictor +.. _LambdaModel : https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.serverless.model.LambdaModel +.. _LambdaPredictor : https://sagemaker.readthedocs.io/en/stable/api/inference/predictors.html#sagemaker.serverless.predictor.LambdaPredictor ****************** SageMaker Workflow From 1b54bf0eeb8fda85b6ef03f381b81253fc338e53 Mon Sep 17 00:00:00 2001 From: Balaji Veeramani Date: Fri, 6 Aug 2021 11:59:11 -0500 Subject: [PATCH 4/4] Update overview.rst --- doc/overview.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/overview.rst b/doc/overview.rst index cdf506994f..2085abd24d 100644 --- a/doc/overview.rst +++ b/doc/overview.rst @@ -1092,7 +1092,7 @@ The ``deploy`` method returns a `LambdaPredictor`_ instance. Use the .. code:: python - url = "https://c.files.bbci.co.uk/12A9B/production/_111434467_gettyimages-1143489763.jpg" + url = "https://example.com/cat.jpeg" predictor.predict({"url": url}) # {'class': 'tabby'} Once you are done performing inference on Lambda, free the `LambdaModel`_ and