Skip to content

deprecation: deprecate Serverless Lambda model-predictor #2733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions doc/api/inference/model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,3 @@ Model
:undoc-members:
:show-inheritance:

.. autoclass:: sagemaker.serverless.model.LambdaModel
:members:
:undoc-members:
:show-inheritance:
5 changes: 0 additions & 5 deletions doc/api/inference/predictors.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,3 @@ Make real-time predictions against SageMaker endpoints with Python objects
:members:
:undoc-members:
:show-inheritance:

.. autoclass:: sagemaker.serverless.predictor.LambdaPredictor
:members:
:undoc-members:
:show-inheritance:
44 changes: 0 additions & 44 deletions doc/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1063,50 +1063,6 @@ You can also find these notebooks in the **Advanced Functionality** section of t
For information about using sample notebooks in a SageMaker notebook instance, see `Use Example Notebooks <https://docs.aws.amazon.com/sagemaker/latest/dg/howitworks-nbexamples.html>`__
in the AWS documentation.

********************
Serverless Inference
********************

You can use the SageMaker Python SDK to perform serverless inference on Lambda.

To deploy models to Lambda, you must complete the following prerequisites:

- `Package your model and inference code as a container image. <https://docs.aws.amazon.com/lambda/latest/dg/images-create.html>`_
- `Create a role that lists Lambda as a trusted entity. <https://docs.aws.amazon.com/lambda/latest/dg/lambda-intro-execution-role.html#permissions-executionrole-console>`_

After completing the prerequisites, you can deploy your model to Lambda using
the `LambdaModel`_ class.

.. code:: python

from sagemaker.serverless import LambdaModel

image_uri = "123456789012.dkr.ecr.us-west-2.amazonaws.com/my-lambda-repository:latest"
role = "arn:aws:iam::123456789012:role/MyLambdaExecutionRole"

model = LambdaModel(image_uri=image_uri, role=role)
predictor = model.deploy("my-lambda-function", timeout=20, memory_size=4092)

The ``deploy`` method returns a `LambdaPredictor`_ instance. Use the
`LambdaPredictor`_ ``predict`` method to perform inference on Lambda.

.. code:: python

url = "https://example.com/cat.jpeg"
predictor.predict({"url": url}) # {'class': 'tabby'}

Once you are done performing inference on Lambda, free the `LambdaModel`_ and
`LambdaPredictor`_ resources using the ``delete_model`` and ``delete_predictor``
methods.

.. code:: python

model.delete_model()
predictor.delete_predictor()

.. _LambdaModel : https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.serverless.model.LambdaModel
.. _LambdaPredictor : https://sagemaker.readthedocs.io/en/stable/api/inference/predictors.html#sagemaker.serverless.predictor.LambdaPredictor

******************
SageMaker Workflow
******************
Expand Down
84 changes: 6 additions & 78 deletions src/sagemaker/serverless/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,84 +12,12 @@
# language governing permissions and limitations under the License.
"""Models that can be deployed to serverless compute."""
from __future__ import absolute_import
from sagemaker.deprecations import deprecated

import time
from typing import Optional

import boto3
import botocore
@deprecated
class LambdaModel:
"""A model that can be deployed to Lambda.

from sagemaker.model import ModelBase
from sagemaker.deprecations import deprecation_warning
from .predictor import LambdaPredictor


@deprecation_warning(
msg="Based on customer experience and feedback an"
" alternative support will be added in near future",
date="10/27/2021",
)
class LambdaModel(ModelBase):
"""A model that can be deployed to Lambda."""

def __init__(
self, image_uri: str, role: str, client: Optional[botocore.client.BaseClient] = None
) -> None:
"""Initialize instance attributes.

Arguments:
image_uri: URI of a container image in the Amazon ECR registry. The image
should contain a handler that performs inference.
role: The Amazon Resource Name (ARN) of the IAM role that Lambda will assume
when it performs inference
client: The Lambda client used to interact with Lambda.
"""
self._client = client or boto3.client("lambda")
self._image_uri = image_uri
self._role = role

def deploy(
self, function_name: str, timeout: int, memory_size: int, wait: bool = True
) -> LambdaPredictor:
"""Create a Lambda function using the image specified in the constructor.

Arguments:
function_name: The name of the function.
timeout: The number of seconds that the function can run for before being terminated.
memory_size: The amount of memory in MB that the function has access to.
wait: If true, wait until the deployment completes (default: True).

Returns:
A LambdaPredictor instance that performs inference using the specified image.
"""
response = self._client.create_function(
FunctionName=function_name,
PackageType="Image",
Role=self._role,
Code={
"ImageUri": self._image_uri,
},
Timeout=timeout,
MemorySize=memory_size,
)

if not wait:
return LambdaPredictor(function_name, client=self._client)

# Poll function state.
polling_interval = 5
while response["State"] == "Pending":
time.sleep(polling_interval)
response = self._client.get_function_configuration(FunctionName=function_name)

if response["State"] != "Active":
raise RuntimeError("Failed to deploy model to Lambda: %s" % response["StateReason"])

return LambdaPredictor(function_name, client=self._client)

def delete_model(self) -> None:
"""Destroy resources associated with this model.

This method does not delete the image specified in the constructor. As
a result, this method is a no-op.
"""
note:: Deprecated in versions > v2.66.0. An alternative support will be added in near future.
"""
76 changes: 6 additions & 70 deletions src/sagemaker/serverless/predictor.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,76 +12,12 @@
# language governing permissions and limitations under the License.
"""Predictors that are hosted on serverless compute."""
from __future__ import absolute_import
from sagemaker.deprecations import deprecated

from typing import Optional, Tuple

import boto3
import botocore
@deprecated
class LambdaPredictor:
"""A deployed model hosted on Lambda.

from sagemaker import deserializers, serializers
from sagemaker.predictor import PredictorBase
from sagemaker.deprecations import deprecation_warning


@deprecation_warning(
msg="Based on customer experience and feedback an"
" alternative support will be added in near future",
date="10/27/2021",
)
class LambdaPredictor(PredictorBase):
"""A deployed model hosted on Lambda."""

def __init__(
self, function_name: str, client: Optional[botocore.client.BaseClient] = None
) -> None:
"""Initialize instance attributes.

Arguments:
function_name: The name of the function.
client: The Lambda client used to interact with Lambda.
"""
self._client = client or boto3.client("lambda")
self._function_name = function_name
self._serializer = serializers.JSONSerializer()
self._deserializer = deserializers.JSONDeserializer()

def predict(self, data: dict) -> dict:
"""Invoke the Lambda function specified in the constructor.

This function is synchronous. It will only return after the function
has produced a prediction.

Arguments:
data: The data sent to the Lambda function as input.

Returns:
The data returned by the Lambda function.
"""
response = self._client.invoke(
FunctionName=self._function_name,
InvocationType="RequestResponse",
Payload=self._serializer.serialize(data),
)
return self._deserializer.deserialize(
response["Payload"],
response["ResponseMetadata"]["HTTPHeaders"]["content-type"],
)

def delete_predictor(self) -> None:
"""Destroy the Lambda function specified in the constructor."""
self._client.delete_function(FunctionName=self._function_name)

@property
def content_type(self) -> str:
"""The MIME type of the data sent to the Lambda function."""
return self._serializer.CONTENT_TYPE

@property
def accept(self) -> Tuple[str]:
"""The content type(s) that are expected from the Lambda function."""
return self._deserializer.ACCEPT

@property
def function_name(self) -> str:
"""The name of the Lambda function this predictor invokes."""
return self._function_name
note:: Deprecated in versions > v2.66.0. An alternative support will be added in near future.
"""
12 changes: 0 additions & 12 deletions tests/data/serverless/Dockerfile

This file was deleted.

6 changes: 0 additions & 6 deletions tests/data/serverless/README.md

This file was deleted.

34 changes: 0 additions & 34 deletions tests/data/serverless/app.py

This file was deleted.

Loading