Skip to content

Add ARM64 architecture support #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 29, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 131 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,135 @@
## AWS Lambda Python Runtime Interface Client

<<<<<<< HEAD
The Lambda Runtime Interface Client helps with packaging functions using your own or community provided images.
It allows your runtime to receive requests from and send requests to the Lambda service.
The Runtime client starts the runtime and communicates with the Lambda [Runtime API](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html) i.e., it calls the API for invocation events, starts the function code, calls the API to return the response.
The Lambda Python Runtime Interface Client is vended through [pip](https://pypi.org/project/awslambdaric).
You can include this package in your preferred base image to make that base image Lambda compatible.

## Requirements
The Python Runtime Interface Client package currently works on Python versions:
- Python 3.6.x
- Python 3.7.x
- Python 3.8.x
- Python 3.9.x

## Usage

### Creating a Docker Image for Lambda with the Runtime Interface Client
First step is to choose the base image to be used. The supported Linux OS distributions are:

- Amazon Linux 2
- Alpine
- CentOS
- Debian
- Ubuntu


Then, the Runtime Interface Client needs to be installed. We provide both wheel and source distribution. If the OS/pip version used does not support manylinux2014 wheels, you will also need to install the required build dependencies. Also, the Lambda function code needs to be copied into the image.

```Docker
# Include global arg in this stage of the build
ARG FUNCTION_DIR

# Install aws-lambda-cpp build dependencies
RUN apt-get update && \
apt-get install -y \
g++ \
make \
cmake \
unzip \
libcurl4-openssl-dev


# Create function directory
RUN mkdir -p ${FUNCTION_DIR}

# Copy handler function
COPY app/* ${FUNCTION_DIR}

# Install the function's dependencies
RUN pip install \
--target ${FUNCTION_DIR} \
awslambdaruntimeclient
```

The next step would be to set the `ENTRYPOINT` property of the Docker image to invoke the Runtime Interface Client and then set the `CMD` argument to specify the desired handler.

Example Dockerfile (to keep the image light we use a multi-stage build):
```Docker
# Define custom function directory
ARG FUNCTION_DIR="/function"

FROM python:buster as build-image

# Include global arg in this stage of the build
ARG FUNCTION_DIR

# Install aws-lambda-cpp build dependencies
RUN apt-get update && \
apt-get install -y \
g++ \
make \
cmake \
unzip \
libcurl4-openssl-dev


# Create function directory
RUN mkdir -p ${FUNCTION_DIR}

# Copy handler function
COPY app/* ${FUNCTION_DIR}

# Install the function's dependencies
RUN pip install \
--target ${FUNCTION_DIR} \
awslambdaric


FROM python:buster

# Include global arg in this stage of the build
ARG FUNCTION_DIR

# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}

# Copy in the built dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}

ENTRYPOINT [ "/usr/local/bin/python", "-m", "awslambdaric" ]
CMD [ "app.handler" ]

```

### Local Testing

For testing locally you will need to set up a local Runtime Interface Emulator against which the Runtime Interface Client will make API calls. You will need to post data to the endpoint it creates in order to invoke your function. For more information check out the [Runtime Interface Emulator](https://github.com/aws/aws-lambda-runtime-interface-emulator).


## Development

### Building the package
Clone this repository and run:

```bash
make init
make build
```

### Running tests

Make sure the project is built:
```bash
make init build
```
Then,
* to run unit tests: `make test`
* to run integration tests: `make test-integ`
* to run examples: `make test-example`
=======
We have open-sourced a set of software packages, Runtime Interface Clients (RIC), that implement the Lambda
[Runtime API](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html), allowing you to seamlessly extend your preferred
base images to be Lambda compatible.
Expand All @@ -11,6 +141,7 @@ You can include this package in your preferred base image to make that base imag
## Requirements
The Python Runtime Interface Client package currently supports Python versions:
- 3.6.x up to and including 3.9.x
>>>>>>> origin/main

## Usage

Expand Down
2 changes: 1 addition & 1 deletion awslambdaric/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
"""

__version__ = "1.2.2"
__version__ = "2.0.0"
1 change: 1 addition & 0 deletions scripts/preinstall.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ else
--prefix "$ARTIFACTS_DIR" \
--disable-shared \
--without-ssl \
--with-pic \
--without-zlib && \
make && \
make install
Expand Down
39 changes: 24 additions & 15 deletions tests/integration/codebuild/buildspec.os.alpine.1.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ batch:
static:
ignore-failure: false
env:
type: LINUX_CONTAINER
privileged-mode: true
dynamic:
env:
Expand All @@ -27,15 +26,25 @@ phases:
- echo "Extracting and including the Runtime Interface Emulator"
- SCRATCH_DIR=".scratch"
- mkdir "${SCRATCH_DIR}"
- tar -xvf tests/integration/resources/aws-lambda-rie.tar.gz --directory "${SCRATCH_DIR}"
- ARCHITECTURE=$(arch)
- >
if [[ "$ARCHITECTURE" == "x86_64" ]]; then
RIE="aws-lambda-rie"
elif [[ "$ARCHITECTURE" == "aarch64" ]]; then
RIE="aws-lambda-rie-arm64"
else
echo "Architecture $ARCHITECTURE is not currently supported."
exit 1
fi
- tar -xvf tests/integration/resources/${RIE}.tar.gz --directory "${SCRATCH_DIR}"
- >
cp "tests/integration/docker/Dockerfile.echo.${OS_DISTRIBUTION}" \
"${SCRATCH_DIR}/Dockerfile.echo.${OS_DISTRIBUTION}.tmp"
- >
echo "RUN apk add curl" >> \
"${SCRATCH_DIR}/Dockerfile.echo.${OS_DISTRIBUTION}.tmp"
- >
echo "COPY ${SCRATCH_DIR}/aws-lambda-rie /usr/bin/aws-lambda-rie" >> \
echo "COPY ${SCRATCH_DIR}/${RIE} /usr/bin/${RIE}" >> \
"${SCRATCH_DIR}/Dockerfile.echo.${OS_DISTRIBUTION}.tmp"
- >
if [[ -z "${DOCKERHUB_USERNAME}" && -z "${DOCKERHUB_PASSWORD}" ]];
Expand All @@ -60,12 +69,11 @@ phases:
- >
docker run \
--detach \
-e "PYTHON_LOCATION=${PYTHON_LOCATION}" \
--name "${TEST_NAME}-app" \
--network "${TEST_NAME}-network" \
--entrypoint="" \
"${IMAGE_TAG}" \
sh -c '/usr/bin/aws-lambda-rie ${PYTHON_LOCATION} -m awslambdaric app.handler'
sh -c "/usr/bin/${RIE} ${PYTHON_LOCATION} -m awslambdaric app.handler"
- sleep 2
- >
docker run \
Expand All @@ -81,19 +89,20 @@ phases:
echo "Response: ${actual}"
if [[ "$actual" != "$expected" ]]; then
echo "fail! runtime: $RUNTIME - expected output $expected - got $actual"
echo "---------Container Logs: ${TEST_NAME}-app----------"
echo
docker logs "${TEST_NAME}-app"
echo
echo "---------------------------------------------------"
echo "--------Container Logs: ${TEST_NAME}-tester--------"
echo
docker logs "${TEST_NAME}-tester"
echo
echo "---------------------------------------------------"
exit -1
fi
finally:
- |
echo "---------Container Logs: ${TEST_NAME}-app----------"
echo
docker logs "${TEST_NAME}-app" || true
echo
echo "---------------------------------------------------"
echo "--------Container Logs: ${TEST_NAME}-tester--------"
echo
docker logs "${TEST_NAME}-tester" || true
echo
echo "---------------------------------------------------"
- echo "Cleaning up..."
- docker stop "${TEST_NAME}-app" || true
- docker rm --force "${TEST_NAME}-app" || true
Expand Down
39 changes: 24 additions & 15 deletions tests/integration/codebuild/buildspec.os.alpine.2.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ batch:
static:
ignore-failure: false
env:
type: LINUX_CONTAINER
privileged-mode: true
dynamic:
env:
Expand All @@ -29,15 +28,25 @@ phases:
- echo "Extracting and including the Runtime Interface Emulator"
- SCRATCH_DIR=".scratch"
- mkdir "${SCRATCH_DIR}"
- tar -xvf tests/integration/resources/aws-lambda-rie.tar.gz --directory "${SCRATCH_DIR}"
- ARCHITECTURE=$(arch)
- >
if [[ "$ARCHITECTURE" == "x86_64" ]]; then
RIE="aws-lambda-rie"
elif [[ "$ARCHITECTURE" == "aarch64" ]]; then
RIE="aws-lambda-rie-arm64"
else
echo "Architecture $ARCHITECTURE is not currently supported."
exit 1
fi
- tar -xvf tests/integration/resources/${RIE}.tar.gz --directory "${SCRATCH_DIR}"
- >
cp "tests/integration/docker/Dockerfile.echo.${OS_DISTRIBUTION}" \
"${SCRATCH_DIR}/Dockerfile.echo.${OS_DISTRIBUTION}.tmp"
- >
echo "RUN apk add curl" >> \
"${SCRATCH_DIR}/Dockerfile.echo.${OS_DISTRIBUTION}.tmp"
- >
echo "COPY ${SCRATCH_DIR}/aws-lambda-rie /usr/bin/aws-lambda-rie" >> \
echo "COPY ${SCRATCH_DIR}/${RIE} /usr/bin/${RIE}" >> \
"${SCRATCH_DIR}/Dockerfile.echo.${OS_DISTRIBUTION}.tmp"
- >
if [[ -z "${DOCKERHUB_USERNAME}" && -z "${DOCKERHUB_PASSWORD}" ]];
Expand All @@ -62,12 +71,11 @@ phases:
- >
docker run \
--detach \
-e "PYTHON_LOCATION=${PYTHON_LOCATION}" \
--name "${TEST_NAME}-app" \
--network "${TEST_NAME}-network" \
--entrypoint="" \
"${IMAGE_TAG}" \
sh -c '/usr/bin/aws-lambda-rie ${PYTHON_LOCATION} -m awslambdaric app.handler'
sh -c "/usr/bin/${RIE} ${PYTHON_LOCATION} -m awslambdaric app.handler"
- sleep 2
- >
docker run \
Expand All @@ -83,19 +91,20 @@ phases:
echo "Response: ${actual}"
if [[ "$actual" != "$expected" ]]; then
echo "fail! runtime: $RUNTIME - expected output $expected - got $actual"
echo "---------Container Logs: ${TEST_NAME}-app----------"
echo
docker logs "${TEST_NAME}-app"
echo
echo "---------------------------------------------------"
echo "--------Container Logs: ${TEST_NAME}-tester--------"
echo
docker logs "${TEST_NAME}-tester"
echo
echo "---------------------------------------------------"
exit -1
fi
finally:
- |
echo "---------Container Logs: ${TEST_NAME}-app----------"
echo
docker logs "${TEST_NAME}-app" || true
echo
echo "---------------------------------------------------"
echo "--------Container Logs: ${TEST_NAME}-tester--------"
echo
docker logs "${TEST_NAME}-tester" || true
echo
echo "---------------------------------------------------"
- echo "Cleaning up..."
- docker stop "${TEST_NAME}-app" || true
- docker rm --force "${TEST_NAME}-app" || true
Expand Down
Loading