feat(build): publish lib as a Lambda Layer #884

flochaz · 2022-05-19T15:58:55Z

Description of your changes

This PR aims to add the capability to publish safely AWS Lambda PowerTools for TypeScript.

TODO list:

Create the publisher code
Make feat: Generalize the construct to be usable by other PowerTools supported languages aws-samples/cdk-lambda-powertools-python-layer#52 merged
Add e2e for all runtimes
decide what we do for Node12 : Bug: lib not working for node12 if not bundled with tools like esbuild #899
Add publication flow to release github workflow
Figure out the public access of the layer
Update the doc
Add https://pypi.org/project/mkdocs-simple-hooks/ to update automatically the doc ... (python should do this soon so we can wait for their code :) )
Add SAM and CDK examples

How to verify this change

Test layer publisher

Build layer manually in your account:

cd layer-publisher
npm ci
npm run cdk deploy

Then you can use the layer shown as cfn output.

Or you can use the e2e tests that will:

Create a public layer in your account called e2e-tests-layer-nodejs
Create a Lambda function consuming the previously created layer
Check the outputs logs of the function checking no error is thrown

Test in github action

check https://github.com/flochaz/aws-lambda-powertools-typescript/actions/runs/2718610199

Diff with Powertools for Python version

The only difference with Python version for publishing layer is the way we do e2e tests:

Python is launching a Canary stack launching a Lambda Function (as a CustomResource) which consume the published layer in the same CDK App. This canary function deployed in their beta and prod account will use the layer and throw exception if any issue found making the stack deployment fail.
TypeScript version is doing E2E testing the same way we were doing for npm: layer-publisher cdk app contains a Jest test that will
1. Create a public layer in the test account called e2e-tests-layer-nodejs
2. Create a Lambda function consuming the previously created layer
3. Check the outputs logs of the function checking no error is thrown

Related issues, RFCs

Issue number: #826

PR status

Is this ready for review?: YES
Is it a breaking change?: NO

Checklist

Breaking change checklist

I have documented the migration process
I have added, implemented necessary warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

docs/index.md

github-actions · 2022-07-22T16:15:50Z

📊 Package size report `-50.7%↓`

File	Before	After
`commons-bundle.zip`	`6.8 kB`	—
`logger-bundle.zip`	`24.8 kB`	—
`metrics-bundle.zip`	`18.2 kB`	—
`tracer-bundle.zip`	`22.1 kB`	—
Total _{(Includes all files)}	`142.0 kB`	^-50.7%↓`70.0 kB`
Tarball size	`140.8 kB`	^-50.2%↓`70.2 kB`

Unchanged files

File	Size
`aws-lambda-powertools-commons-1.0.2.tgz`	`6.3 kB`
`aws-lambda-powertools-logger-1.0.2.tgz`	`24.3 kB`
`aws-lambda-powertools-metrics-1.0.2.tgz`	`17.7 kB`
`aws-lambda-powertools-tracer-1.0.2.tgz`	`21.6 kB`

_{🤖 This report was automatically generated by pkg-size-action}
(options hash: e17332db0705db7a856b60c057b7d255)

layer-publisher/tests/e2e/utils/e2eUtils.ts

github-actions · 2022-07-22T16:37:11Z

📊 Package size report `-50.7%↓`

File	Before	After
`commons-bundle.zip`	`6.8 kB`	—
`logger-bundle.zip`	`24.8 kB`	—
`metrics-bundle.zip`	`18.2 kB`	—
`tracer-bundle.zip`	`22.1 kB`	—
Total _{(Includes all files)}	`142.0 kB`	^-50.7%↓`70.0 kB`
Tarball size	`140.7 kB`	^-50.2%↓`70.2 kB`

Unchanged files

File	Size
`aws-lambda-powertools-commons-1.0.2.tgz`	`6.3 kB`
`aws-lambda-powertools-logger-1.0.2.tgz`	`24.3 kB`
`aws-lambda-powertools-metrics-1.0.2.tgz`	`17.7 kB`
`aws-lambda-powertools-tracer-1.0.2.tgz`	`21.6 kB`

_{🤖 This report was automatically generated by pkg-size-action}
(options hash: f15ee5cc09f53757f6faed42cd6f802b)

layer-publisher/tests/e2e/utils/e2eUtils.ts

dreamorosi · 2022-07-22T16:40:06Z

layer-publisher/tests/e2e/happy-case.test.ts

+
+const runtime = lambda.Runtime.ALL.find((r) => r.name === process.env.RUNTIME) ?? lambda.Runtime.NODEJS_14_X;
+
+const powerToolsPackageVersion = '1.0.1';


Is this synced to the actual version published on NPM? If yes, how do we increment it when we release?

no it's only to test and therefore on purpose set as a lower version than latest

in release workflow it takes latest from latest tag https://github.com/awslabs/aws-lambda-powertools-typescript/pull/884/files#diff-39e8ac1723228023ce8c44a0a27a9ce7db36f9ddaf8404cbf2e16c648ad42705R42

Perfect, thanks for the explanation.

Leaving the thread open so others can see the answer in case they have the same question

github-actions · 2022-07-22T16:41:24Z

📊 Package size report `-50.7%↓`

File	Before	After
`commons-bundle.zip`	`6.8 kB`	—
`logger-bundle.zip`	`24.8 kB`	—
`metrics-bundle.zip`	`18.2 kB`	—
`tracer-bundle.zip`	`22.1 kB`	—
Total _{(Includes all files)}	`142.0 kB`	^-50.7%↓`70.0 kB`
Tarball size	`140.7 kB`	^-50.2%↓`70.2 kB`

Unchanged files

File	Size
`aws-lambda-powertools-commons-1.0.2.tgz`	`6.3 kB`
`aws-lambda-powertools-logger-1.0.2.tgz`	`24.3 kB`
`aws-lambda-powertools-metrics-1.0.2.tgz`	`17.7 kB`
`aws-lambda-powertools-tracer-1.0.2.tgz`	`21.6 kB`

_{🤖 This report was automatically generated by pkg-size-action}
(options hash: a51bc9e9f4f835e32a61e7b391cf15a9)

docs/index.md

layer-publisher/tests/unit/layer-publisher.test.ts

dreamorosi · 2022-07-22T16:45:07Z

layer-publisher/cdk.json

@@ -0,0 +1,32 @@
+{


Putting this comment here because the file above called is a binary and doesn't allow comments:

Do we need this layer-publisher/[email protected]???

for now yes since my PR there is not merged yet : aws-samples/cdk-lambda-powertools-python-layer#52

Gotcha, thanks for the explanation

.github/workflows/pr_lint_and_test.yml

github-actions · 2022-07-22T16:59:01Z

📊 Package size report `-50.7%↓`

File	Before	After
`commons-bundle.zip`	`6.8 kB`	—
`logger-bundle.zip`	`24.8 kB`	—
`metrics-bundle.zip`	`18.2 kB`	—
`tracer-bundle.zip`	`22.1 kB`	—
Total _{(Includes all files)}	`142.0 kB`	^-50.7%↓`70.0 kB`
Tarball size	`140.8 kB`	^-50.2%↓`70.2 kB`

Unchanged files

File	Size
`aws-lambda-powertools-commons-1.0.2.tgz`	`6.3 kB`
`aws-lambda-powertools-logger-1.0.2.tgz`	`24.3 kB`
`aws-lambda-powertools-metrics-1.0.2.tgz`	`17.7 kB`
`aws-lambda-powertools-tracer-1.0.2.tgz`	`21.6 kB`

_{🤖 This report was automatically generated by pkg-size-action}
(options hash: 31921c0fee6bce29e62a7b9f4747f9cc)

github-actions · 2022-07-22T17:03:25Z

📊 Package size report `-50.7%↓`

File	Before	After
`commons-bundle.zip`	`6.8 kB`	—
`logger-bundle.zip`	`24.8 kB`	—
`metrics-bundle.zip`	`18.2 kB`	—
`tracer-bundle.zip`	`22.1 kB`	—
Total _{(Includes all files)}	`142.0 kB`	^-50.7%↓`70.0 kB`
Tarball size	`140.7 kB`	^-50.1%↓`70.2 kB`

Unchanged files

File	Size
`aws-lambda-powertools-commons-1.0.2.tgz`	`6.3 kB`
`aws-lambda-powertools-logger-1.0.2.tgz`	`24.3 kB`
`aws-lambda-powertools-metrics-1.0.2.tgz`	`17.7 kB`
`aws-lambda-powertools-tracer-1.0.2.tgz`	`21.6 kB`

_{🤖 This report was automatically generated by pkg-size-action}
(options hash: 015a1eeebe540ca3ec0dc6305e69f386)

dreamorosi

Looking great, I have left a series of comments which you have already addressed.

Thanks a lot for the work you've done on this. We all know having Layers is very important for the long term sustainability of the project so I just want to acknowledge your contribution.

From my side the last thing to agree is the naming of the Layer between:

AWSLambdaPowertoolsNodeJS (current name in the PR)
AWSLambdaPowertoolsTypeScript

On one hand I understand the current choice since that's the name of the AWS Lambda runtime. If I were looking at the Layer in isolation (i.e. from the console etc.) this name would not leave any doubt about the fact that it's compatible with that runtime. Also this name makes the JS support unambiguous.

On the other hand I personally prefer the AWSLambdaPowertoolsTypeScript option for the following reasons:

In the context of a Lambda Layer compatibility is already expressed by the Compatible runtimes field
This name is in line with the rest of the branding we are using here and elsewhere
We should clarify that this project can be unambiguously used with both JS & TS Lambda functions elsewhere, not in the name of the layer.

Would like however to hear the opinions of the rest of the maintainers as well as @heitorlessa before moving forward.

github-actions · 2022-08-04T10:12:42Z

📊 Package size report `-50.7%↓`

File	Before	After
`commons-bundle.zip`	`6.8 kB`	—
`logger-bundle.zip`	`24.8 kB`	—
`metrics-bundle.zip`	`18.2 kB`	—
`tracer-bundle.zip`	`22.1 kB`	—
Total _{(Includes all files)}	`142.0 kB`	^-50.7%↓`70.0 kB`
Tarball size	`140.8 kB`	^-50.2%↓`70.2 kB`

Unchanged files

File	Size
`aws-lambda-powertools-commons-1.0.2.tgz`	`6.3 kB`
`aws-lambda-powertools-logger-1.0.2.tgz`	`24.3 kB`
`aws-lambda-powertools-metrics-1.0.2.tgz`	`17.7 kB`
`aws-lambda-powertools-tracer-1.0.2.tgz`	`21.6 kB`

_{🤖 This report was automatically generated by pkg-size-action}
(options hash: b7ccdb16df8d9a213af05756706f4dac)

github-actions · 2022-08-04T15:57:53Z

📊 Package size report `-50.7%↓`

File	Before	After
`commons-bundle.zip`	`6.8 kB`	—
`logger-bundle.zip`	`24.8 kB`	—
`metrics-bundle.zip`	`18.2 kB`	—
`tracer-bundle.zip`	`22.1 kB`	—
Total _{(Includes all files)}	`142.0 kB`	^-50.7%↓`70.0 kB`
Tarball size	`140.8 kB`	^-50.2%↓`70.2 kB`

Unchanged files

File	Size
`aws-lambda-powertools-commons-1.0.2.tgz`	`6.3 kB`
`aws-lambda-powertools-logger-1.0.2.tgz`	`24.3 kB`
`aws-lambda-powertools-metrics-1.0.2.tgz`	`17.7 kB`
`aws-lambda-powertools-tracer-1.0.2.tgz`	`21.6 kB`

_{🤖 This report was automatically generated by pkg-size-action}
(options hash: 731be7d63b86b6ad43259050be54a2bb)

github-actions · 2022-08-04T16:01:37Z

📊 Package size report `-50.7%↓`

File	Before	After
`commons-bundle.zip`	`6.8 kB`	—
`logger-bundle.zip`	`24.8 kB`	—
`metrics-bundle.zip`	`18.2 kB`	—
`tracer-bundle.zip`	`22.1 kB`	—
Total _{(Includes all files)}	`142.0 kB`	^-50.7%↓`70.0 kB`
Tarball size	`140.8 kB`	^-50.2%↓`70.2 kB`

Unchanged files

File	Size
`aws-lambda-powertools-commons-1.0.2.tgz`	`6.3 kB`
`aws-lambda-powertools-logger-1.0.2.tgz`	`24.3 kB`
`aws-lambda-powertools-metrics-1.0.2.tgz`	`17.7 kB`
`aws-lambda-powertools-tracer-1.0.2.tgz`	`21.6 kB`

_{🤖 This report was automatically generated by pkg-size-action}
(options hash: 80ee68abe74bf056022dfdd9bf901dbd)

dreamorosi

LGTM

* chore: broke up pr workflow & measure package size * chore: explicitly set packages where to run cmds * fix: added missing dependency to examples/sam * added cache in workflow * chore: updated workflow + removed env var from command * chore: updated action version * fix: removed redundant env variable

…owertools-typescript into feat/layerPublisher

dreamorosi

Fixed merge conflicts

github-actions bot added the feature label May 19, 2022

flochaz force-pushed the feat/layerPublisher branch 4 times, most recently from 275f79f to e3fd639 Compare May 20, 2022 07:44

dreamorosi assigned flochaz May 20, 2022

dreamorosi added the internal PRs that introduce changes in governance, tech debt and chores (linting setup, baseline, etc.) label May 20, 2022

dreamorosi added this to the production-ready-release milestone May 20, 2022

flochaz mentioned this pull request May 20, 2022

Bug: lib not working for node12 if not bundled with tools like esbuild #899

Closed

saragerion removed this from the production-ready-release milestone May 27, 2022

saragerion added this to the Lambda layer milestone Jul 19, 2022