Skip to content

Commit e9741bb

Browse files
authored
docs(v2): document optional dependencies and local dev (aws-powertools#1574)
1 parent 2e5d0cb commit e9741bb

File tree

5 files changed

+63
-73
lines changed

5 files changed

+63
-73
lines changed

README.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,12 @@
1+
<!-- markdownlint-disable MD013 MD041 MD043 -->
12
# AWS Lambda Powertools for Python
23

34
[![Build](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/python_build.yml/badge.svg)](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/python_build.yml)
45
[![codecov.io](https://codecov.io/github/awslabs/aws-lambda-powertools-python/branch/develop/graphs/badge.svg)](https://app.codecov.io/gh/awslabs/aws-lambda-powertools-python)
56
![PythonSupport](https://img.shields.io/static/v1?label=python&message=3.6%20|%203.7|%203.8|%203.9&color=blue?style=flat-square&logo=python) ![PyPI version](https://badge.fury.io/py/aws-lambda-powertools.svg) ![PyPi monthly downloads](https://img.shields.io/pypi/dm/aws-lambda-powertools)
67
[![Join our Discord](https://dcbadge.vercel.app/api/server/B8zZKbbyET)](https://discord.gg/B8zZKbbyET)
78

8-
A suite of Python utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, and more. (AWS Lambda Powertools [Java](https://github.com/awslabs/aws-lambda-powertools-java) and [Typescript](https://github.com/awslabs/aws-lambda-powertools-typescript) is also available).
9+
A suite of Python utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, and more. (AWS Lambda Powertools for [Java](https://github.com/awslabs/aws-lambda-powertools-java), [Typescript](https://github.com/awslabs/aws-lambda-powertools-typescript), and [.NET](https://awslabs.github.io/aws-lambda-powertools-dotnet/){target="_blank"} are also available).
910

1011
**[📜Documentation](https://awslabs.github.io/aws-lambda-powertools-python/)** | **[🐍PyPi](https://pypi.org/project/aws-lambda-powertools/)** | **[Roadmap](https://awslabs.github.io/aws-lambda-powertools-python/latest/roadmap/)** | **[Detailed blog post](https://aws.amazon.com/blogs/opensource/simplifying-serverless-best-practices-with-lambda-powertools/)**
1112

docs/core/tracer.md

+8
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,14 @@ Tracer is an opinionated thin wrapper for [AWS X-Ray Python SDK](https://github.
1919
???+ tip
2020
All examples shared in this documentation are available within the [project repository](https://github.com/awslabs/aws-lambda-powertools-python/tree/develop/examples){target="_blank"}.
2121

22+
### Install
23+
24+
!!! info "This is not necessary if you're installing Powertools via [Lambda Layer](../index.md#lambda-layer){target="_blank"}"
25+
26+
Add `aws-lambda-powertools[tracer]` as a dependency in your preferred tool: _e.g._, _requirements.txt_, _pyproject.toml_.
27+
28+
This will ensure you have the required dependencies before using Tracer.
29+
2230
### Permissions
2331

2432
Before your use this utility, your AWS Lambda function [must have permissions](https://docs.aws.amazon.com/lambda/latest/dg/services-xray.html#services-xray-permissions) to send traces to AWS X-Ray.

docs/index.md

+19-1
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ description: AWS Lambda Powertools for Python
1111
A suite of utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, idempotency, batching, and more.
1212

1313
???+ note
14-
Lambda Powertools is also available for [Java](https://awslabs.github.io/aws-lambda-powertools-java/){target="_blank"} and [TypeScript](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/){target="_blank"}.
14+
Powertools is also available for [Java](https://awslabs.github.io/aws-lambda-powertools-java/){target="_blank"}, [TypeScript](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/){target="_blank"}, and [.NET](https://awslabs.github.io/aws-lambda-powertools-dotnet/){target="_blank"}
1515

1616
## Install
1717

@@ -20,11 +20,29 @@ Powertools is available in the following formats:
2020
* **Lambda Layer**: [**arn:aws:lambda:{region}:017000801446:layer:AWSLambdaPowertoolsPython:33**](#){: .copyMe}:clipboard:
2121
* **PyPi**: **`pip install aws-lambda-powertools`**
2222

23+
???+ info "Some utilities require additional dependencies"
24+
You can stop reading if you're using Lambda Layer.
25+
26+
[Tracer](./core/tracer.md){target="_blank"}, [Validation](./utilities/validation.md){target="_blank"} and [Parser](./utilities/parser.md){target="_blank"} require additional dependencies. If you prefer to install all of them, use `pip install aws-lambda-powertools[all]`.
27+
2328
???+ hint "Support this project by using Lambda Layers :heart:"
2429
Lambda Layers allow us to understand who uses this library in a non-intrusive way. This helps us justify and gain future investments for other Lambda Powertools languages.
2530

2631
When using Layers, you can add Lambda Powertools as a dev dependency (or as part of your virtual env) to not impact the development process.
2732

33+
### Local development
34+
35+
Powertools relies on the AWS SDK bundled in the Lambda runtime. This helps us achieve an optimal package size and initialization.
36+
37+
This means you need to add AWS SDK as a development dependency (not as a production dependency).
38+
39+
* **Pip**: `pip install aws-lambda-powertools[aws-sdk]`
40+
* **Poetry**: `poetry add aws-lambda-powertools[aws-sdk] --dev`
41+
* **Pipenv**: `pipenv install --dev "aws-lambda-powertools[aws-sdk]"`
42+
43+
???+ note "Local emulation"
44+
If you're running your code locally with [AWS SAM CLI](https://github.com/aws/aws-sam-cli){target="_blank"}, and not with your Python/IDE interpreter directly, this is not necessary. SAM CLI already brings the AWS SDK in its emulation image.
45+
2846
### Lambda Layer
2947

3048
[Lambda Layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html){target="_blank"} is a .zip file archive that can contain additional code, pre-packaged dependencies, data, or configuration files. Layers promote code sharing and separation of responsibilities so that you can iterate faster on writing business logic.

docs/utilities/parser.md

+26-71
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Parser
2+
title: Parser (Pydantic)
33
description: Utility
44
---
55
<!-- markdownlint-disable MD043 -->
@@ -12,20 +12,25 @@ This utility provides data parsing and deep validation using [Pydantic](https://
1212
* Built-in envelopes to unwrap, extend, and validate popular event sources payloads
1313
* Enforces type hints at runtime with user-friendly errors
1414

15-
**Extra dependency**
15+
## Getting started
1616

17-
???+ warning
17+
### Install
18+
19+
!!! info "This is not necessary if you're installing Powertools via [Lambda Layer](../index.md#lambda-layer){target="_blank"}"
20+
21+
Add `aws-lambda-powertools[parser]` as a dependency in your preferred tool: _e.g._, _requirements.txt_, _pyproject.toml_.
1822

23+
This will ensure you have the required dependencies before using Parser.
24+
25+
???+ warning
1926
This will increase the compressed package size by >10MB due to the Pydantic dependency.
2027

2128
To reduce the impact on the package size at the expense of 30%-50% of its performance [Pydantic can also be
2229
installed without binary files](https://pydantic-docs.helpmanual.io/install/#performance-vs-package-size-trade-off):
2330

24-
`SKIP_CYTHON=1 pip install --no-binary pydantic aws-lambda-powertools[pydantic]`
31+
Pip example: `SKIP_CYTHON=1 pip install --no-binary pydantic aws-lambda-powertools[parser]`
2532

26-
Install parser's extra dependencies using **`pip install aws-lambda-powertools[pydantic]`**.
27-
28-
## Defining models
33+
### Defining models
2934

3035
You can define models to parse incoming events by inheriting from `BaseModel`.
3136

@@ -47,11 +52,11 @@ class Order(BaseModel):
4752

4853
These are simply Python classes that inherit from BaseModel. **Parser** enforces type hints declared in your model at runtime.
4954

50-
## Parsing events
55+
### Parsing events
5156

5257
You can parse inbound events using **event_parser** decorator, or the standalone `parse` function. Both are also able to parse either dictionary or JSON string as an input.
5358

54-
### event_parser decorator
59+
#### event_parser decorator
5560

5661
Use the decorator for fail fast scenarios where you want your Lambda function to raise an exception in the event of a malformed payload.
5762

@@ -104,7 +109,7 @@ handler(event=payload, context=LambdaContext())
104109
handler(event=json.dumps(payload), context=LambdaContext()) # also works if event is a JSON string
105110
```
106111

107-
### parse function
112+
#### parse function
108113

109114
Use this standalone function when you want more control over the data validation process, for example returning a 400 error for malformed payloads.
110115

@@ -149,7 +154,7 @@ def my_function():
149154
}
150155
```
151156

152-
## Built-in models
157+
### Built-in models
153158

154159
Parser comes with the following built-in models:
155160

@@ -171,7 +176,7 @@ Parser comes with the following built-in models:
171176
| **KafkaSelfManagedEventModel** | Lambda Event Source payload for self managed Kafka payload |
172177
| **KafkaMskEventModel** | Lambda Event Source payload for AWS MSK payload |
173178

174-
### extending built-in models
179+
#### extending built-in models
175180

176181
You can extend them to include your own models, and yet have all other known fields parsed along the way.
177182

@@ -236,7 +241,9 @@ for order_item in ret.detail.items:
236241
3. Defined how part of our EventBridge event should look like by overriding `detail` key within our `OrderEventModel`
237242
4. Parser parsed the original event against `OrderEventModel`
238243

239-
## Envelopes
244+
## Advanced
245+
246+
### Envelopes
240247

241248
When trying to parse your payloads wrapped in a known structure, you might encounter the following situations:
242249

@@ -294,7 +301,7 @@ def handler(event: UserModel, context: LambdaContext):
294301
3. Parser parsed the original event against the EventBridge model
295302
4. Parser then parsed the `detail` key using `UserModel`
296303

297-
### Built-in envelopes
304+
#### Built-in envelopes
298305

299306
Parser comes with the following built-in envelopes, where `Model` in the return section is your given model.
300307

@@ -312,7 +319,7 @@ Parser comes with the following built-in envelopes, where `Model` in the return
312319
| **LambdaFunctionUrlEnvelope** | 1. Parses data using `LambdaFunctionUrlModel`. <br/> 2. Parses `body` key using your model and returns it. | `Model` |
313320
| **KafkaEnvelope** | 1. Parses data using `KafkaRecordModel`. <br/> 2. Parses `value` key using your model and returns it. | `Model` |
314321

315-
### Bringing your own envelope
322+
#### Bringing your own envelope
316323

317324
You can create your own Envelope model and logic by inheriting from `BaseEnvelope`, and implementing the `parse` method.
318325

@@ -377,7 +384,7 @@ Here's a snippet of how the EventBridge envelope we demonstrated previously is i
377384
3. Then, we parsed the incoming data with our envelope to confirm it matches EventBridge's structure defined in `EventBridgeModel`
378385
4. Lastly, we call `_parse` from `BaseEnvelope` to parse the data in our envelope (.detail) using the customer model
379386

380-
## Data model validation
387+
### Data model validation
381388

382389
???+ warning
383390
This is radically different from the **Validator utility** which validates events against JSON Schema.
@@ -394,7 +401,7 @@ Keep the following in mind regardless of which decorator you end up using it:
394401
* You must raise either `ValueError`, `TypeError`, or `AssertionError` when value is not compliant
395402
* You must return the value(s) itself if compliant
396403

397-
### validating fields
404+
#### validating fields
398405

399406
Quick validation to verify whether the field `message` has the value of `hello world`.
400407

@@ -439,7 +446,7 @@ class HelloWorldModel(BaseModel):
439446
parse(model=HelloWorldModel, event={"message": "hello universe", "sender": "universe"})
440447
```
441448

442-
### validating entire model
449+
#### validating entire model
443450

444451
`root_validator` can help when you have a complex validation mechanism. For example finding whether data has been omitted, comparing field values, etc.
445452

@@ -470,7 +477,7 @@ parse(model=UserModel, event=payload)
470477
???+ info
471478
You can read more about validating list items, reusing validators, validating raw inputs, and a lot more in <a href="https://pydantic-docs.helpmanual.io/usage/validators/">Pydantic's documentation</a>.
472479

473-
## Advanced use cases
480+
### Advanced use cases
474481

475482
???+ tip "Tip: Looking to auto-generate models from JSON, YAML, JSON Schemas, OpenApi, etc?"
476483
Use Koudai Aono's [data model code generation tool for Pydantic](https://github.com/koxudaxi/datamodel-code-generator)
@@ -535,55 +542,3 @@ If what you're trying to use isn't available as part of the high level import sy
535542
```python title="Pydantic import escape hatch"
536543
from aws_lambda_powertools.utilities.parser.pydantic import <what you'd like to import'>
537544
```
538-
539-
**What is the cold start impact in bringing this additional dependency?**
540-
541-
No significant cold start impact. It does increase the final uncompressed package by **71M**, when you bring the additional dependency that parser requires.
542-
543-
Artillery load test sample against a [hello world sample](https://github.com/aws-samples/cookiecutter-aws-sam-python) using Tracer, Metrics, and Logger with and without parser.
544-
545-
**No parser**
546-
547-
???+ info
548-
**Uncompressed package size**: 55M, **p99**: 180.3ms
549-
550-
```javascript
551-
Summary report @ 14:36:07(+0200) 2020-10-23
552-
Scenarios launched: 10
553-
Scenarios completed: 10
554-
Requests completed: 2000
555-
Mean response/sec: 114.81
556-
Response time (msec):
557-
min: 54.9
558-
max: 1684.9
559-
median: 68
560-
p95: 109.1
561-
p99: 180.3
562-
Scenario counts:
563-
0: 10 (100%)
564-
Codes:
565-
200: 2000
566-
```
567-
568-
**With parser**
569-
570-
???+ info
571-
**Uncompressed package size**: 128M, **p99**: 193.1ms
572-
573-
```javascript
574-
Summary report @ 14:29:23(+0200) 2020-10-23
575-
Scenarios launched: 10
576-
Scenarios completed: 10
577-
Requests completed: 2000
578-
Mean response/sec: 111.67
579-
Response time (msec):
580-
min: 54.3
581-
max: 1887.2
582-
median: 66.1
583-
p95: 113.3
584-
p99: 193.1
585-
Scenario counts:
586-
0: 10 (100%)
587-
Codes:
588-
200: 2000
589-
```

docs/utilities/validation.md

+8
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,14 @@ This utility provides JSON Schema validation for events and responses, including
1818
???+ tip
1919
All examples shared in this documentation are available within the [project repository](https://github.com/awslabs/aws-lambda-powertools-python/tree/develop/examples){target="_blank"}.
2020

21+
### Install
22+
23+
!!! info "This is not necessary if you're installing Powertools via [Lambda Layer](../index.md#lambda-layer){target="_blank"}"
24+
25+
Add `aws-lambda-powertools[validation]` as a dependency in your preferred tool: _e.g._, _requirements.txt_, _pyproject.toml_.
26+
27+
This will ensure you have the required dependencies before using Validation.
28+
2129
You can validate inbound and outbound events using [`validator` decorator](#validator-decorator).
2230

2331
You can also use the standalone `validate` function, if you want more control over the validation process such as handling a validation error.

0 commit comments

Comments
 (0)