Skip to content

Commit 2e5d0cb

Browse files
rubenfonsecaheitorlessa
rubenfonseca
and
heitorlessa
authored
feat(layers): add support for publishing v2 layer (#1558)
Co-authored-by: Heitor Lessa <[email protected]>
1 parent 4099d42 commit 2e5d0cb

15 files changed

+550
-221
lines changed

Diff for: ‎.github/workflows/publish_v2_layer.yml

+92
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
name: Deploy v2 layer to all regions
2+
3+
permissions:
4+
id-token: write
5+
contents: read
6+
7+
on:
8+
workflow_dispatch:
9+
inputs:
10+
latest_published_version:
11+
description: "Latest PyPi published version to rebuild latest docs for, e.g. v1.22.0"
12+
default: "v2.0.0"
13+
required: true
14+
# workflow_run:
15+
# workflows: ["Publish to PyPi"]
16+
# types:
17+
# - completed
18+
19+
jobs:
20+
build-layer:
21+
runs-on: ubuntu-latest
22+
if: ${{ (github.event.workflow_run.conclusion == 'success') || (github.event_name == 'workflow_dispatch') }}
23+
defaults:
24+
run:
25+
working-directory: ./layer
26+
steps:
27+
- name: checkout
28+
uses: actions/checkout@v3
29+
with:
30+
fetch-depth: 0
31+
- name: Install poetry
32+
run: pipx install poetry
33+
- name: Setup Node.js
34+
uses: actions/setup-node@v3
35+
with:
36+
node-version: "16.12"
37+
- name: Setup python
38+
uses: actions/setup-python@v4
39+
with:
40+
python-version: "3.9"
41+
cache: "pip"
42+
- name: Resolve and install project dependencies
43+
# CDK spawns system python when compiling stack
44+
# therefore it ignores both activated virtual env and cached interpreter by GH
45+
run: |
46+
poetry export --format requirements.txt --output requirements.txt
47+
pip install -r requirements.txt
48+
- name: Set release notes tag
49+
run: |
50+
RELEASE_INPUT=${{ inputs.latest_published_version }}
51+
LATEST_TAG=$(git describe --tag --abbrev=0)
52+
RELEASE_TAG_VERSION=${RELEASE_INPUT:-$LATEST_TAG}
53+
echo RELEASE_TAG_VERSION="${RELEASE_TAG_VERSION:1}" >> "$GITHUB_ENV"
54+
- name: Set up QEMU
55+
uses: docker/setup-qemu-action@8b122486cedac8393e77aa9734c3528886e4a1a8 # v2.0.0
56+
# NOTE: we need QEMU to build Layer against a different architecture (e.g., ARM)
57+
- name: Set up Docker Buildx
58+
id: builder
59+
uses: docker/setup-buildx-action@dc7b9719a96d48369863986a06765841d7ea23f6 # v2.0.0
60+
- name: install cdk and deps
61+
run: |
62+
npm install -g [email protected]
63+
cdk --version
64+
- name: CDK build
65+
run: cdk synth --context version="$RELEASE_TAG_VERSION" -o cdk.out
66+
- name: zip output
67+
run: zip -r cdk.out.zip cdk.out
68+
- name: Archive CDK artifacts
69+
uses: actions/upload-artifact@v3
70+
with:
71+
name: cdk-layer-artefact
72+
path: layer/cdk.out.zip
73+
74+
deploy-beta:
75+
needs:
76+
- build-layer
77+
uses: ./.github/workflows/reusable_deploy_v2_layer_stack.yml
78+
secrets: inherit
79+
with:
80+
stage: "BETA"
81+
artefact-name: "cdk-layer-artefact"
82+
environment: "layer-beta"
83+
84+
# deploy-prod:
85+
# needs:
86+
# - deploy-beta
87+
# uses: ./.github/workflows/reusable_deploy_layer_stack.yml
88+
# secrets: inherit
89+
# with:
90+
# stage: "PROD"
91+
# artefact-name: "cdk-layer-artefact"
92+
# environment: "layer-prod"

Diff for: ‎.github/workflows/reusable_deploy_v2_layer_stack.yml

+102
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
name: Deploy CDK Layer v2 stack
2+
3+
permissions:
4+
id-token: write
5+
contents: read
6+
7+
env:
8+
CDK_VERSION: 2.44.0
9+
10+
on:
11+
workflow_call:
12+
inputs:
13+
stage:
14+
description: "Deployment stage (BETA, PROD)"
15+
required: true
16+
type: string
17+
artefact-name:
18+
description: "CDK Layer Artefact name to download"
19+
required: true
20+
type: string
21+
environment:
22+
description: "GitHub Environment to use for encrypted secrets"
23+
required: true
24+
type: string
25+
26+
jobs:
27+
deploy-cdk-stack:
28+
runs-on: ubuntu-latest
29+
environment: ${{ inputs.environment }}
30+
defaults:
31+
run:
32+
working-directory: ./layer
33+
strategy:
34+
fail-fast: false
35+
matrix:
36+
region:
37+
[
38+
"af-south-1",
39+
"eu-central-1",
40+
"us-east-1",
41+
"us-east-2",
42+
"us-west-1",
43+
"us-west-2",
44+
"ap-east-1",
45+
"ap-south-1",
46+
"ap-northeast-1",
47+
"ap-northeast-2",
48+
"ap-southeast-1",
49+
"ap-southeast-2",
50+
"ca-central-1",
51+
"eu-west-1",
52+
"eu-west-2",
53+
"eu-west-3",
54+
"eu-south-1",
55+
"eu-north-1",
56+
"sa-east-1",
57+
"ap-southeast-3",
58+
"ap-northeast-3",
59+
"me-south-1",
60+
]
61+
steps:
62+
- name: checkout
63+
uses: actions/checkout@v3
64+
- name: Install poetry
65+
run: pipx install poetry
66+
- name: aws credentials
67+
uses: aws-actions/configure-aws-credentials@v1
68+
with:
69+
aws-region: ${{ matrix.region }}
70+
role-to-assume: ${{ secrets.AWS_LAYERS_ROLE_ARN }}
71+
- name: Setup Node.js
72+
uses: actions/setup-node@v3
73+
with:
74+
node-version: "16.12"
75+
- name: Setup python
76+
uses: actions/setup-python@v4
77+
with:
78+
python-version: "3.9"
79+
cache: "pip"
80+
- name: Resolve and install project dependencies
81+
# CDK spawns system python when compiling stack
82+
# therefore it ignores both activated virtual env and cached interpreter by GH
83+
run: |
84+
poetry export --format requirements.txt --output requirements.txt
85+
pip install -r requirements.txt
86+
- name: install cdk and deps
87+
run: |
88+
npm install -g "aws-cdk@$CDK_VERSION"
89+
cdk --version
90+
- name: install deps
91+
run: poetry install
92+
- name: Download artifact
93+
uses: actions/download-artifact@v3
94+
with:
95+
name: ${{ inputs.artefact-name }}
96+
path: layer
97+
- name: unzip artefact
98+
run: unzip cdk.out.zip
99+
- name: CDK Deploy Layer
100+
run: cdk deploy --app cdk.out --context region=${{ matrix.region }} 'LayerStack' --require-approval never --verbose
101+
- name: CDK Deploy Canary
102+
run: cdk deploy --app cdk.out --context region=${{ matrix.region}} --parameters DeployStage="${{ inputs.stage }}" 'CanaryStack' --require-approval never --verbose

Diff for: ‎Makefile

+2-2
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,12 @@ target:
66

77
dev:
88
pip install --upgrade pip pre-commit poetry
9-
poetry install --extras "pydantic"
9+
poetry install --extras "all"
1010
pre-commit install
1111

1212
dev-gitpod:
1313
pip install --upgrade pip poetry
14-
poetry install --extras "pydantic"
14+
poetry install --extras "all"
1515
pre-commit install
1616

1717
format:

Diff for: ‎layer/app.py

+4-1
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@
88
app = cdk.App()
99

1010
POWERTOOLS_VERSION: str = app.node.try_get_context("version")
11-
SSM_PARAM_LAYER_ARN: str = "/layers/powertools-layer-arn"
11+
SSM_PARAM_LAYER_ARN: str = "/layers/powertools-layer-v2-arn"
12+
SSM_PARAM_LAYER_ARM64_ARN: str = "/layers/powertools-layer-v2-arm64-arn"
1213

1314
if not POWERTOOLS_VERSION:
1415
raise ValueError(
@@ -21,13 +22,15 @@
2122
"LayerStack",
2223
powertools_version=POWERTOOLS_VERSION,
2324
ssm_paramter_layer_arn=SSM_PARAM_LAYER_ARN,
25+
ssm_parameter_layer_arm64_arn=SSM_PARAM_LAYER_ARM64_ARN,
2426
)
2527

2628
CanaryStack(
2729
app,
2830
"CanaryStack",
2931
powertools_version=POWERTOOLS_VERSION,
3032
ssm_paramter_layer_arn=SSM_PARAM_LAYER_ARN,
33+
ssm_parameter_layer_arm64_arn=SSM_PARAM_LAYER_ARM64_ARN,
3134
)
3235

3336
app.synth()

Diff for: ‎layer/layer/canary/app.py

+37-9
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,19 @@
11
import datetime
22
import json
33
import os
4+
import platform
45
from importlib.metadata import version
56

67
import boto3
8+
from pydantic import EmailStr
79

810
from aws_lambda_powertools import Logger, Metrics, Tracer
11+
from aws_lambda_powertools.utilities.parser import BaseModel, envelopes, event_parser
12+
from aws_lambda_powertools.utilities.typing import LambdaContext
13+
from aws_lambda_powertools.utilities.validation import validator
914

1015
logger = Logger(service="version-track")
11-
tracer = Tracer()
16+
tracer = Tracer() # this checks for aws-xray-sdk presence
1217
metrics = Metrics(namespace="powertools-layer-canary", service="PowertoolsLayerCanary")
1318

1419
layer_arn = os.getenv("POWERTOOLS_LAYER_ARN")
@@ -17,6 +22,26 @@
1722
event_bus_arn = os.getenv("VERSION_TRACKING_EVENT_BUS_ARN")
1823

1924

25+
# Model to check parser imports correctly, tests for pydantic and email-validator
26+
class OrderItem(BaseModel):
27+
order_id: int
28+
quantity: int
29+
description: str
30+
email: EmailStr
31+
32+
33+
# Tests for jmespath presence
34+
@event_parser(model=OrderItem, envelope=envelopes.EventBridgeEnvelope)
35+
def envelope_handler(event: OrderItem, context: LambdaContext):
36+
assert event.order_id != 1
37+
38+
39+
# Tests for fastjsonschema presence
40+
@validator(inbound_schema={}, envelope="detail")
41+
def validator_handler(event, context: LambdaContext):
42+
pass
43+
44+
2045
def handler(event):
2146
logger.info("Running checks")
2247
check_envs()
@@ -42,9 +67,7 @@ def on_create(event):
4267

4368

4469
def check_envs():
45-
logger.info(
46-
'Checking required envs ["POWERTOOLS_LAYER_ARN", "AWS_REGION", "STAGE"]'
47-
)
70+
logger.info('Checking required envs ["POWERTOOLS_LAYER_ARN", "AWS_REGION", "STAGE"]')
4871
if not layer_arn:
4972
raise ValueError("POWERTOOLS_LAYER_ARN is not set. Aborting...")
5073
if not powertools_version:
@@ -66,20 +89,19 @@ def verify_powertools_version() -> None:
6689
current_version = version("aws_lambda_powertools")
6790
if powertools_version != current_version:
6891
raise ValueError(
69-
f'Expected powertoosl version is "{powertools_version}", but layer contains version "{current_version}"'
92+
f'Expected Powertools version is "{powertools_version}", but layer contains version "{current_version}"'
7093
)
71-
logger.info(f"Current Powertools version is: {current_version}")
94+
logger.info(f"Current Powertools version is: {current_version} [{_get_architecture()}]")
7295

7396

7497
def send_notification():
7598
"""
7699
sends an event to version tracking event bridge
77100
"""
78101
if stage != "PROD":
79-
logger.info(
80-
"Not sending notification to event bus, because this is not the PROD stage"
81-
)
102+
logger.info("Not sending notification to event bus, because this is not the PROD stage")
82103
return
104+
83105
event = {
84106
"Time": datetime.datetime.now(),
85107
"Source": "powertools.layer.canary",
@@ -90,6 +112,7 @@ def send_notification():
90112
"version": powertools_version,
91113
"region": os.environ["AWS_REGION"],
92114
"layerArn": layer_arn,
115+
"architecture": _get_architecture(),
93116
}
94117
),
95118
}
@@ -102,3 +125,8 @@ def send_notification():
102125
if resp["FailedEntryCount"] != 0:
103126
logger.error(resp)
104127
raise ValueError("Failed to send deployment notification to version tracking")
128+
129+
130+
def _get_architecture() -> str:
131+
"""Returns aarch64, x86_64"""
132+
return platform.uname()[4]

0 commit comments

Comments
 (0)