diff --git a/docs/core/event_handler/api_gateway.md b/docs/core/event_handler/api_gateway.md index 4f984ecea6f..cfc6f4125d2 100644 --- a/docs/core/event_handler/api_gateway.md +++ b/docs/core/event_handler/api_gateway.md @@ -102,7 +102,7 @@ When using Amazon Application Load Balancer (ALB) to front your Lambda functions #### Lambda Function URL -When using [AWS Lambda Function URL](https://docs.aws.amazon.com/lambda/latest/dg/urls-configuration.html), you can use `LambdaFunctionUrlResolver`. +When using [AWS Lambda Function URL](https://docs.aws.amazon.com/lambda/latest/dg/urls-configuration.html){target="_blank"}, you can use `LambdaFunctionUrlResolver`. === "getting_started_lambda_function_url_resolver.py" @@ -253,7 +253,7 @@ When using [Custom Domain API Mappings feature](https://docs.aws.amazon.com/apig **Scenario**: You have a custom domain `api.mydomain.dev`. Then you set `/payment` API Mapping to forward any payment requests to your Payments API. -**Challenge**: This means your `path` value for any API requests will always contain `/payment/`, leading to HTTP 404 as Event Handler is trying to match what's after `payment/`. This gets further complicated with an [arbitrary level of nesting](https://github.com/awslabs/aws-lambda-powertools-roadmap/issues/34). +**Challenge**: This means your `path` value for any API requests will always contain `/payment/`, leading to HTTP 404 as Event Handler is trying to match what's after `payment/`. This gets further complicated with an [arbitrary level of nesting](https://github.com/awslabs/aws-lambda-powertools-roadmap/issues/34){target="_blank"}. To address this API Gateway behavior, we use `strip_prefixes` parameter to account for these prefixes that are now injected into the path regardless of which type of API Gateway you're using. @@ -344,7 +344,7 @@ You can use the `Response` class to have full control over the response. For exa Some event sources require headers and cookies to be encoded as `multiValueHeaders`. ???+ warning "Using multiple values for HTTP headers in ALB?" - Make sure you [enable the multi value headers feature](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/lambda-functions.html#multi-value-headers) to serialize response headers correctly. + Make sure you [enable the multi value headers feature](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/lambda-functions.html#multi-value-headers){target="_blank"} to serialize response headers correctly. === "fine_grained_responses.py" @@ -420,7 +420,7 @@ Like `compress` feature, the client must send the `Accept` header with the corre ### Debug mode -You can enable debug mode via `debug` param, or via `POWERTOOLS_DEV` [environment variable](../../index.md#environment-variables). +You can enable debug mode via `debug` param, or via `POWERTOOLS_DEV` [environment variable](../../index.md#environment-variables){target="_blank"}. This will enable full tracebacks errors in the response, print request and responses, and set CORS in development mode. diff --git a/docs/core/logger.md b/docs/core/logger.md index 305a7cef0f3..16f81e8b2ba 100644 --- a/docs/core/logger.md +++ b/docs/core/logger.md @@ -108,7 +108,7 @@ You can set a Correlation ID using `correlation_id_path` param by passing a [JME #### set_correlation_id method -You can also use `set_correlation_id` method to inject it anywhere else in your code. Example below uses [Event Source Data Classes utility](../utilities/data_classes.md) to easily access events properties. +You can also use `set_correlation_id` method to inject it anywhere else in your code. Example below uses [Event Source Data Classes utility](../utilities/data_classes.md){target="_blank"} to easily access events properties. === "set_correlation_id_method.py" @@ -242,7 +242,7 @@ You can remove any additional key from Logger state using `remove_keys`. #### Clearing all state -Logger is commonly initialized in the global scope. Due to [Lambda Execution Context reuse](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html), this means that custom keys can be persisted across invocations. If you want all custom keys to be deleted, you can use `clear_state=True` param in `inject_lambda_context` decorator. +Logger is commonly initialized in the global scope. Due to [Lambda Execution Context reuse](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html){target="_blank"}, this means that custom keys can be persisted across invocations. If you want all custom keys to be deleted, you can use `clear_state=True` param in `inject_lambda_context` decorator. ???+ tip "Tip: When is this useful?" It is useful when you add multiple custom keys conditionally, instead of setting a default `None` value if not present. Any key with `None` value is automatically removed by Logger. @@ -360,7 +360,7 @@ You can use any of the following built-in JMESPath expressions as part of [injec ### Reusing Logger across your code -Similar to [Tracer](./tracer.md#reusing-tracer-across-your-code), a new instance that uses the same `service` name - env var or explicit parameter - will reuse a previous Logger instance. Just like `logging.getLogger("logger_name")` would in the standard library if called with the same logger name. +Similar to [Tracer](./tracer.md#reusing-tracer-across-your-code){target="_blank"}, a new instance that uses the same `service` name - env var or explicit parameter - will reuse a previous Logger instance. Just like `logging.getLogger("logger_name")` would in the standard library if called with the same logger name. Notice in the CloudWatch Logs output how `payment_id` appeared as expected when logging in `collect.py`. @@ -407,7 +407,7 @@ You can use values ranging from `0.0` to `1` (100%) when setting `POWERTOOLS_LOG Sampling decision happens at the Logger initialization. This means sampling may happen significantly more or less than depending on your traffic patterns, for example a steady low number of invocations and thus few cold starts. ???+ note - Open a [feature request](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=feature-request%2C+triage&template=feature_request.md&title=) if you want Logger to calculate sampling for every invocation + Open a [feature request](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=feature-request%2C+triage&template=feature_request.md&title=){target="_blank"} if you want Logger to calculate sampling for every invocation === "sampling_debug_logs.py" @@ -744,4 +744,4 @@ Here's an example where we persist `payment_id` not `request_id`. Note that `pay ### How do I aggregate and search Powertools for AWS Lambda (Python) logs across accounts? -As of now, ElasticSearch (ELK) or 3rd party solutions are best suited to this task. Please refer to this [discussion for more details](https://github.com/awslabs/aws-lambda-powertools-python/issues/460) +As of now, ElasticSearch (ELK) or 3rd party solutions are best suited to this task. Please refer to this [discussion for more details](https://github.com/awslabs/aws-lambda-powertools-python/issues/460){target="_blank"} diff --git a/docs/core/metrics.md b/docs/core/metrics.md index ba9f746e867..6948a3d4c8d 100644 --- a/docs/core/metrics.md +++ b/docs/core/metrics.md @@ -3,9 +3,9 @@ title: Metrics description: Core utility --- -Metrics creates custom metrics asynchronously by logging metrics to standard output following [Amazon CloudWatch Embedded Metric Format (EMF)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html). +Metrics creates custom metrics asynchronously by logging metrics to standard output following [Amazon CloudWatch Embedded Metric Format (EMF)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html){target="_blank"}. -These metrics can be visualized through [Amazon CloudWatch Console](https://console.aws.amazon.com/cloudwatch/). +These metrics can be visualized through [Amazon CloudWatch Console](https://console.aws.amazon.com/cloudwatch/){target="_blank"}. ## Key features @@ -22,7 +22,7 @@ If you're new to Amazon CloudWatch, there are two terminologies you must be awar * **Dimensions**. Metrics metadata in key-value format. They help you slice and dice metrics visualization, for example `ColdStart` metric by Payment `service`. * **Metric**. It's the name of the metric, for example: `SuccessfulBooking` or `UpdatedBooking`. * **Unit**. It's a value representing the unit of measure for the corresponding metric, for example: `Count` or `Seconds`. -* **Resolution**. It's a value representing the storage resolution for the corresponding metric. Metrics can be either Standard or High resolution. Read more [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html#high-resolution-metrics). +* **Resolution**. It's a value representing the storage resolution for the corresponding metric. Metrics can be either Standard or High resolution. Read more [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html#high-resolution-metrics){target="_blank"}.
@@ -83,7 +83,7 @@ You can create metrics using `add_metric`, and you can create dimensions for all ### Adding high-resolution metrics -You can create [high-resolution metrics](https://aws.amazon.com/about-aws/whats-new/2023/02/amazon-cloudwatch-high-resolution-metric-extraction-structured-logs/) passing `resolution` parameter to `add_metric`. +You can create [high-resolution metrics](https://aws.amazon.com/about-aws/whats-new/2023/02/amazon-cloudwatch-high-resolution-metric-extraction-structured-logs/){target="_blank"} passing `resolution` parameter to `add_metric`. ???+ tip "When is it useful?" High-resolution metrics are data with a granularity of one second and are very useful in several situations such as telemetry, time series, real-time incident management, and others. @@ -154,7 +154,7 @@ This decorator also **validates**, **serializes**, and **flushes** all your metr * Maximum of 29 user-defined dimensions * Namespace is set, and no more than one - * Metric units must be [supported by CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html) + * Metric units must be [supported by CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html){target="_blank"} #### Raising SchemaValidationError on empty metrics @@ -191,7 +191,7 @@ If it's a cold start invocation, this feature will: This has the advantage of keeping cold start metric separate from your application metrics, where you might have unrelated dimensions. ???+ info - We do not emit 0 as a value for ColdStart metric for cost reasons. [Let us know](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=feature-request%2C+triage&template=feature_request.md&title=) if you'd prefer a flag to override it. + We do not emit 0 as a value for ColdStart metric for cost reasons. [Let us know](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=feature-request%2C+triage&template=feature_request.md&title=){target="_blank"} if you'd prefer a flag to override it. ## Advanced @@ -219,7 +219,7 @@ You can add high-cardinality data as part of your Metrics log with `add_metadata CloudWatch EMF uses the same dimensions across all your metrics. Use `single_metric` if you have a metric that should have different dimensions. ???+ info - Generally, this would be an edge case since you [pay for unique metric](https://aws.amazon.com/cloudwatch/pricing). Keep the following formula in mind: + Generally, this would be an edge case since you [pay for unique metric](https://aws.amazon.com/cloudwatch/pricing){target="_blank"}. Keep the following formula in mind: **unique metric = (metric_name + dimension_name + dimension_value)** @@ -292,7 +292,7 @@ The former creates metrics asynchronously via CloudWatch Logs, and the latter us !!! important "Key concept" CloudWatch [considers a metric unique](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Metric){target="_blank"} by a combination of metric **name**, metric **namespace**, and zero or more metric **dimensions**. -With EMF, metric dimensions are shared with any metrics you define. With `PutMetricData` API, you can set a [list](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html) defining one or more metrics with distinct dimensions. +With EMF, metric dimensions are shared with any metrics you define. With `PutMetricData` API, you can set a [list](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html){target="_blank"} defining one or more metrics with distinct dimensions. This is a subtle yet important distinction. Imagine you had the following metrics to emit: @@ -329,7 +329,7 @@ That is why `Metrics` shares data across instances by default, as that covers 80 For example, `Metrics(namespace="ServerlessAirline", service="booking")` -Make sure to set `POWERTOOLS_METRICS_NAMESPACE` and `POWERTOOLS_SERVICE_NAME` before running your tests to prevent failing on `SchemaValidation` exception. You can set it before you run tests or via pytest plugins like [dotenv](https://pypi.org/project/pytest-dotenv/). +Make sure to set `POWERTOOLS_METRICS_NAMESPACE` and `POWERTOOLS_SERVICE_NAME` before running your tests to prevent failing on `SchemaValidation` exception. You can set it before you run tests or via pytest plugins like [dotenv](https://pypi.org/project/pytest-dotenv/){target="_blank"}. ```bash title="Injecting dummy Metric Namespace before running tests" --8<-- "examples/metrics/src/run_tests_env_var.sh" @@ -374,4 +374,4 @@ You can read standard output and assert whether metrics have been flushed. Here' ``` ???+ tip - For more elaborate assertions and comparisons, check out [our functional testing for Metrics utility.](https://github.com/awslabs/aws-lambda-powertools-python/blob/develop/tests/functional/test_metrics.py) + For more elaborate assertions and comparisons, check out [our functional testing for Metrics utility.](https://github.com/awslabs/aws-lambda-powertools-python/blob/develop/tests/functional/test_metrics.py){target="_blank"} diff --git a/docs/core/tracer.md b/docs/core/tracer.md index 0b701928d10..b65a7c0107f 100644 --- a/docs/core/tracer.md +++ b/docs/core/tracer.md @@ -3,7 +3,7 @@ title: Tracer description: Core utility --- -Tracer is an opinionated thin wrapper for [AWS X-Ray Python SDK](https://github.com/aws/aws-xray-sdk-python/). +Tracer is an opinionated thin wrapper for [AWS X-Ray Python SDK](https://github.com/aws/aws-xray-sdk-python/){target="_blank"}. ![Tracer showcase](../media/tracer_utility_showcase.png) @@ -29,7 +29,7 @@ Add `aws-lambda-powertools[tracer]` as a dependency in your preferred tool: _e.g ### Permissions -Before your use this utility, your AWS Lambda function [must have permissions](https://docs.aws.amazon.com/lambda/latest/dg/services-xray.html#services-xray-permissions) to send traces to AWS X-Ray. +Before your use this utility, your AWS Lambda function [must have permissions](https://docs.aws.amazon.com/lambda/latest/dg/services-xray.html#services-xray-permissions){target="_blank"} to send traces to AWS X-Ray. ```yaml hl_lines="9 12" title="AWS Serverless Application Model (SAM) example" --8<-- "examples/tracer/sam/template.yaml" @@ -51,7 +51,7 @@ You can quickly start by initializing `Tracer` and use `capture_lambda_handler` ### Annotations & Metadata -**Annotations** are key-values associated with traces and indexed by AWS X-Ray. You can use them to filter traces and to create [Trace Groups](https://aws.amazon.com/about-aws/whats-new/2018/11/aws-xray-adds-the-ability-to-group-traces/) to slice and dice your transactions. +**Annotations** are key-values associated with traces and indexed by AWS X-Ray. You can use them to filter traces and to create [Trace Groups](https://aws.amazon.com/about-aws/whats-new/2018/11/aws-xray-adds-the-ability-to-group-traces/){target="_blank"} to slice and dice your transactions. ```python hl_lines="8" title="Adding annotations with put_annotation method" --8<-- "examples/tracer/src/put_trace_annotations.py" @@ -107,7 +107,7 @@ You can trace asynchronous functions and generator functions (including context ### Patching modules -Tracer automatically patches all [supported libraries by X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-python-patching.html) during initialization, by default. Underneath, AWS X-Ray SDK checks whether a supported library has been imported before patching. +Tracer automatically patches all [supported libraries by X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-python-patching.html){target="_blank"} during initialization, by default. Underneath, AWS X-Ray SDK checks whether a supported library has been imported before patching. If you're looking to shave a few microseconds, or milliseconds depending on your function memory configuration, you can patch specific modules using `patch_modules` param: @@ -172,7 +172,7 @@ You can use `aiohttp_trace_config` function to create a valid [aiohttp trace_con You can use `tracer.provider` attribute to access all methods provided by AWS X-Ray `xray_recorder` object. -This is useful when you need a feature available in X-Ray that is not available in the Tracer utility, for example [thread-safe](https://github.com/aws/aws-xray-sdk-python/#user-content-trace-threadpoolexecutor), or [context managers](https://github.com/aws/aws-xray-sdk-python/#user-content-start-a-custom-segmentsubsegment). +This is useful when you need a feature available in X-Ray that is not available in the Tracer utility, for example [thread-safe](https://github.com/aws/aws-xray-sdk-python/#user-content-trace-threadpoolexecutor){target="_blank"}, or [context managers](https://github.com/aws/aws-xray-sdk-python/#user-content-start-a-custom-segmentsubsegment){target="_blank"}. ```python hl_lines="14" title="Tracing a code block with in_subsegment escape hatch" --8<-- "examples/tracer/src/sdk_escape_hatch.py" @@ -181,7 +181,7 @@ This is useful when you need a feature available in X-Ray that is not available ### Concurrent asynchronous functions ???+ warning - [X-Ray SDK will raise an exception](https://github.com/aws/aws-xray-sdk-python/issues/164) when async functions are run and traced concurrently + [X-Ray SDK will raise an exception](https://github.com/aws/aws-xray-sdk-python/issues/164){target="_blank"} when async functions are run and traced concurrently A safe workaround mechanism is to use `in_subsegment_async` available via Tracer escape hatch (`tracer.provider`). @@ -221,4 +221,4 @@ Tracer is disabled by default when not running in the AWS Lambda environment - T * Use annotations on key operations to slice and dice traces, create unique views, and create metrics from it via Trace Groups * Use a namespace when adding metadata to group data more easily -* Annotations and metadata are added to the current subsegment opened. If you want them in a specific subsegment, use a [context manager](https://github.com/aws/aws-xray-sdk-python/#start-a-custom-segmentsubsegment) via the escape hatch mechanism +* Annotations and metadata are added to the current subsegment opened. If you want them in a specific subsegment, use a [context manager](https://github.com/aws/aws-xray-sdk-python/#start-a-custom-segmentsubsegment){target="_blank"} via the escape hatch mechanism diff --git a/docs/index.md b/docs/index.md index a1590aaf564..aa7c1323a94 100644 --- a/docs/index.md +++ b/docs/index.md @@ -14,9 +14,9 @@ Powertools for AWS Lambda (Python) is a developer toolkit to implement Serverles You can choose to support us in three ways: - 1) [**Become a reference customer**](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=customer-reference&template=support_powertools.yml&title=%5BSupport+Lambda+Powertools%5D%3A+%3Cyour+organization+name%3E). This gives us permission to list your company in our documentation. + 1) [**Become a reference customer**](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=customer-reference&template=support_powertools.yml&title=%5BSupport+Lambda+Powertools%5D%3A+%3Cyour+organization+name%3E){target="_blank"}. This gives us permission to list your company in our documentation. - 2) [**Share your work**](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=community-content&template=share_your_work.yml&title=%5BI+Made+This%5D%3A+%3CTITLE%3E). Blog posts, video, sample projects you used Powertools! + 2) [**Share your work**](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=community-content&template=share_your_work.yml&title=%5BI+Made+This%5D%3A+%3CTITLE%3E){target="_blank"}. Blog posts, video, sample projects you used Powertools! 3) Use [**Lambda Layers**](#lambda-layer) or [**SAR**](#sar), if possible. This helps us understand who uses Powertools for AWS Lambda (Python) in a non-intrusive way, and helps us gain future investments for other Powertools for AWS Lambda languages. @@ -462,8 +462,8 @@ Compared with the [public Layer ARN](#lambda-layer) option, SAR allows you to ch | App | ARN | Description | | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------- | -| [aws-lambda-powertools-python-layer](https://serverlessrepo.aws.amazon.com/applications/eu-west-1/057560766410/aws-lambda-powertools-python-layer) | [arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer](#){: .copyMe}:clipboard: | Contains all extra dependencies (e.g: pydantic). | -| [aws-lambda-powertools-python-layer-arm64](https://serverlessrepo.aws.amazon.com/applications/eu-west-1/057560766410/aws-lambda-powertools-python-layer-arm64) | [arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer-arm64](#){: .copyMe}:clipboard: | Contains all extra dependencies (e.g: pydantic). For arm64 functions. | +| [aws-lambda-powertools-python-layer](https://serverlessrepo.aws.amazon.com/applications/eu-west-1/057560766410/aws-lambda-powertools-python-layer){target="_blank"} | [arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer](#){: .copyMe}:clipboard: | Contains all extra dependencies (e.g: pydantic). | +| [aws-lambda-powertools-python-layer-arm64](https://serverlessrepo.aws.amazon.com/applications/eu-west-1/057560766410/aws-lambda-powertools-python-layer-arm64){target="_blank"} | [arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer-arm64](#){: .copyMe}:clipboard: | Contains all extra dependencies (e.g: pydantic). For arm64 functions. | ??? note "Click to expand and copy SAR code snippets for popular frameworks" @@ -547,7 +547,7 @@ Compared with the [public Layer ARN](#lambda-layer) option, SAR allows you to ch === "Terraform" - > Credits to [Dani Comnea](https://github.com/DanyC97) for providing the Terraform equivalent. + > Credits to [Dani Comnea](https://github.com/DanyC97){target="_blank"} for providing the Terraform equivalent. ```terraform hl_lines="12-13 15-20 23-25 40" terraform { @@ -595,7 +595,7 @@ Compared with the [public Layer ARN](#lambda-layer) option, SAR allows you to ch ??? example "Example: Least-privileged IAM permissions to deploy Layer" - > Credits to [mwarkentin](https://github.com/mwarkentin) for providing the scoped down IAM permissions. + > Credits to [mwarkentin](https://github.com/mwarkentin){target="_blank"} for providing the scoped down IAM permissions. The region and the account id for `CloudFormationTransform` and `GetCfnTemplate` are fixed. @@ -679,21 +679,21 @@ Core utilities such as Tracing, Logging, Metrics, and Event Handler will be avai | Utility | Description | | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [**Tracing**](./core/tracer.md) | Decorators and utilities to trace Lambda function handlers, and both synchronous and asynchronous functions | -| [**Logger**](./core/logger.md) | Structured logging made easier, and decorator to enrich structured logging with key Lambda context details | -| [**Metrics**](./core/metrics.md) | Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF) | -| [**Event handler: AppSync**](./core/event_handler/appsync.md) | AppSync event handler for Lambda Direct Resolver and Amplify GraphQL Transformer function | +| [**Tracing**](./core/tracer.md){target="_blank"} | Decorators and utilities to trace Lambda function handlers, and both synchronous and asynchronous functions | +| [**Logger**](./core/logger.md){target="_blank"} | Structured logging made easier, and decorator to enrich structured logging with key Lambda context details | +| [**Metrics**](./core/metrics.md){target="_blank"} | Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF) | +| [**Event handler: AppSync**](./core/event_handler/appsync.md){target="_blank"} | AppSync event handler for Lambda Direct Resolver and Amplify GraphQL Transformer function | | [**Event handler: API Gateway, ALB and Lambda Function URL**](https://awslabs.github.io/aws-lambda-powertools-python/latest/core/event_handler/api_gateway/) | Amazon API Gateway REST/HTTP API and ALB event handler for Lambda functions invoked using Proxy integration, and Lambda Function URL | -| [**Middleware factory**](./utilities/middleware_factory.md) | Decorator factory to create your own middleware to run logic before, and after each Lambda invocation | -| [**Parameters**](./utilities/parameters.md) | Retrieve parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager, or Amazon DynamoDB, and cache them for a specific amount of time | -| [**Batch processing**](./utilities/batch.md) | Handle partial failures for AWS SQS batch processing | -| [**Typing**](./utilities/typing.md) | Static typing classes to speedup development in your IDE | -| [**Validation**](./utilities/validation.md) | JSON Schema validator for inbound events and responses | -| [**Event source data classes**](./utilities/data_classes.md) | Data classes describing the schema of common Lambda event triggers | -| [**Parser**](./utilities/parser.md) | Data parsing and deep validation using Pydantic | -| [**Idempotency**](./utilities/idempotency.md) | Idempotent Lambda handler | -| [**Feature Flags**](./utilities/feature_flags.md) | A simple rule engine to evaluate when one or multiple features should be enabled depending on the input | -| [**Streaming**](./utilities/streaming.md) | Streams datasets larger than the available memory as streaming data. | +| [**Middleware factory**](./utilities/middleware_factory.md){target="_blank"} | Decorator factory to create your own middleware to run logic before, and after each Lambda invocation | +| [**Parameters**](./utilities/parameters.md){target="_blank"} | Retrieve parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager, or Amazon DynamoDB, and cache them for a specific amount of time | +| [**Batch processing**](./utilities/batch.md){target="_blank"} | Handle partial failures for AWS SQS batch processing | +| [**Typing**](./utilities/typing.md){target="_blank"} | Static typing classes to speedup development in your IDE | +| [**Validation**](./utilities/validation.md){target="_blank"} | JSON Schema validator for inbound events and responses | +| [**Event source data classes**](./utilities/data_classes.md){target="_blank"} | Data classes describing the schema of common Lambda event triggers | +| [**Parser**](./utilities/parser.md){target="_blank"} | Data parsing and deep validation using Pydantic | +| [**Idempotency**](./utilities/idempotency.md){target="_blank"} | Idempotent Lambda handler | +| [**Feature Flags**](./utilities/feature_flags.md){target="_blank"} | A simple rule engine to evaluate when one or multiple features should be enabled depending on the input | +| [**Streaming**](./utilities/streaming.md){target="_blank"} | Streams datasets larger than the available memory as streaming data. | ## Environment variables @@ -703,18 +703,18 @@ Core utilities such as Tracing, Logging, Metrics, and Event Handler will be avai | Environment variable | Description | Utility | Default | | ----------------------------------------- | -------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | --------------------- | | **POWERTOOLS_SERVICE_NAME** | Sets service name used for tracing namespace, metrics dimension and structured logging | All | `"service_undefined"` | -| **POWERTOOLS_METRICS_NAMESPACE** | Sets namespace used for metrics | [Metrics](./core/metrics) | `None` | -| **POWERTOOLS_TRACE_DISABLED** | Explicitly disables tracing | [Tracing](./core/tracer) | `false` | -| **POWERTOOLS_TRACER_CAPTURE_RESPONSE** | Captures Lambda or method return as metadata. | [Tracing](./core/tracer) | `true` | -| **POWERTOOLS_TRACER_CAPTURE_ERROR** | Captures Lambda or method exception as metadata. | [Tracing](./core/tracer) | `true` | -| **POWERTOOLS_TRACE_MIDDLEWARES** | Creates sub-segment for each custom middleware | [Middleware factory](./utilities/middleware_factory) | `false` | -| **POWERTOOLS_LOGGER_LOG_EVENT** | Logs incoming event | [Logging](./core/logger) | `false` | -| **POWERTOOLS_LOGGER_SAMPLE_RATE** | Debug log sampling | [Logging](./core/logger) | `0` | -| **POWERTOOLS_LOG_DEDUPLICATION_DISABLED** | Disables log deduplication filter protection to use Pytest Live Log feature | [Logging](./core/logger) | `false` | -| **POWERTOOLS_PARAMETERS_MAX_AGE** | Adjust how long values are kept in cache (in seconds) | [Parameters](./utilities/parameters/#adjusting-cache-ttl) | `5` | -| **POWERTOOLS_PARAMETERS_SSM_DECRYPT** | Sets whether to decrypt or not values retrieved from AWS SSM Parameters Store | [Parameters](./utilities/parameters/#ssmprovider) | `false` | +| **POWERTOOLS_METRICS_NAMESPACE** | Sets namespace used for metrics | [Metrics](./core/metrics){target="_blank"} | `None` | +| **POWERTOOLS_TRACE_DISABLED** | Explicitly disables tracing | [Tracing](./core/tracer){target="_blank"} | `false` | +| **POWERTOOLS_TRACER_CAPTURE_RESPONSE** | Captures Lambda or method return as metadata. | [Tracing](./core/tracer){target="_blank"} | `true` | +| **POWERTOOLS_TRACER_CAPTURE_ERROR** | Captures Lambda or method exception as metadata. | [Tracing](./core/tracer){target="_blank"} | `true` | +| **POWERTOOLS_TRACE_MIDDLEWARES** | Creates sub-segment for each custom middleware | [Middleware factory](./utilities/middleware_factory){target="_blank"} | `false` | +| **POWERTOOLS_LOGGER_LOG_EVENT** | Logs incoming event | [Logging](./core/logger){target="_blank"} | `false` | +| **POWERTOOLS_LOGGER_SAMPLE_RATE** | Debug log sampling | [Logging](./core/logger){target="_blank"} | `0` | +| **POWERTOOLS_LOG_DEDUPLICATION_DISABLED** | Disables log deduplication filter protection to use Pytest Live Log feature | [Logging](./core/logger){target="_blank"} | `false` | +| **POWERTOOLS_PARAMETERS_MAX_AGE** | Adjust how long values are kept in cache (in seconds) | [Parameters](./utilities/parameters/#adjusting-cache-ttl){target="_blank"} | `5` | +| **POWERTOOLS_PARAMETERS_SSM_DECRYPT** | Sets whether to decrypt or not values retrieved from AWS SSM Parameters Store | [Parameters](./utilities/parameters/#ssmprovider){target="_blank"} | `false` | | **POWERTOOLS_DEV** | Increases verbosity across utilities | Multiple; see [POWERTOOLS_DEV effect below](#increasing-verbosity-across-utilities) | `false` | -| **LOG_LEVEL** | Sets logging level | [Logging](./core/logger) | `INFO` | +| **LOG_LEVEL** | Sets logging level | [Logging](./core/logger){target="_blank"} | `INFO` | ### Optimizing for non-production environments @@ -757,7 +757,7 @@ The following companies, among others, use Powertools: ### Sharing your work -Share what you did with Powertools for AWS Lambda (Python) 💞💞. Blog post, workshops, presentation, sample apps and others. Check out what the community has already shared about Powertools for AWS Lambda (Python) [here](https://awslabs.github.io/aws-lambda-powertools-python/latest/we_made_this/). +Share what you did with Powertools for AWS Lambda (Python) 💞💞. Blog post, workshops, presentation, sample apps and others. Check out what the community has already shared about Powertools for AWS Lambda (Python) [here](https://awslabs.github.io/aws-lambda-powertools-python/latest/we_made_this/){target="_blank"}. ### Using Lambda Layer or SAR diff --git a/docs/roadmap.md b/docs/roadmap.md index bd082bc8ef7..960003e198e 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -20,7 +20,7 @@ Some Lambda Event Sources require clusters (e.g., MSK) leading to additional del ### Observability providers -We want to extend Tracer, Metrics, and Logger to support any [observability provider](https://github.com/awslabs/aws-lambda-powertools-python/issues/1433). We need a RFC to define a contract and to identify two most requested observability providers that we can work with as an initial step. +We want to extend Tracer, Metrics, and Logger to support any [observability provider](https://github.com/awslabs/aws-lambda-powertools-python/issues/1433){target="_blank"}. We need a RFC to define a contract and to identify two most requested observability providers that we can work with as an initial step. ### Lambda Layer in release notes @@ -34,15 +34,15 @@ This means we have room to include a JSON map for Lambda Layers and facilitate a We want to enable MyPy strict mode against the code base. We need a RFC to identify most critical areas to start, and do so gradually as to not impact new features and enhancements in parallel. -This also means bringing `typing-extensions` as a runtime dependency to ensure complete coverage across all Python versions. Future wise, we might be able to experiment with [MyPyC](https://github.com/mypyc/mypyc) to compile less performing parts of the code base as a C-Extension. +This also means bringing `typing-extensions` as a runtime dependency to ensure complete coverage across all Python versions. Future wise, we might be able to experiment with [MyPyC](https://github.com/mypyc/mypyc){target="_blank"} to compile less performing parts of the code base as a C-Extension. ### New utilities - -With V2 launched, we want to resume working on new utilities, specifically but not limited to the most commonly asked: **(1)** [Sensitive Data Masking](https://github.com/awslabs/aws-lambda-powertools-python/issues/1173), **(2)** [Integration/End-to-end Testing](https://github.com/awslabs/aws-lambda-powertools-python/issues/1169), and **(3)** [Event Bridge](https://github.com/awslabs/aws-lambda-powertools-python/issues/1168). + +With V2 launched, we want to resume working on new utilities, specifically but not limited to the most commonly asked: **(1)** [Sensitive Data Masking](https://github.com/awslabs/aws-lambda-powertools-python/issues/1173){target="_blank"}, **(2)** [Integration/End-to-end Testing](https://github.com/awslabs/aws-lambda-powertools-python/issues/1169){target="_blank"}, and **(3)** [Event Bridge](https://github.com/awslabs/aws-lambda-powertools-python/issues/1168){target="_blank"}. ### Open iteration planning -We want to experiment running a bi-weekly audio channel on [Discord](https://discord.gg/B8zZKbbyET) to help us prioritize backlog in real-time. Depending on attendance, we might switch to run an office hours instead. +We want to experiment running a bi-weekly audio channel on [Discord](https://discord.gg/B8zZKbbyET){target="_blank"} to help us prioritize backlog in real-time. Depending on attendance, we might switch to run an office hours instead. ## Roadmap status definition @@ -89,7 +89,7 @@ Our end-to-end mechanism follows four major steps: * **Decision**. After carefully reviewing and discussing them, maintainers make a final decision on whether to start implementation, defer or reject it, and update everyone with the next steps. * **Implementation**. For approved features, maintainers give priority to the original authors for implementation unless it is a sensitive task that is best handled by maintainers. -???+ info "See [Maintainers](https://github.com/awslabs/aws-lambda-powertools-python/blob/develop/MAINTAINERS.md) document to understand how we triage issues and pull requests, labels and governance." +???+ info "See [Maintainers](https://github.com/awslabs/aws-lambda-powertools-python/blob/develop/MAINTAINERS.md){target="_blank"} document to understand how we triage issues and pull requests, labels and governance." ## Disclaimer diff --git a/docs/tutorial/index.md b/docs/tutorial/index.md index 29ec6ebadd2..c02f9cd93e9 100644 --- a/docs/tutorial/index.md +++ b/docs/tutorial/index.md @@ -111,7 +111,7 @@ As a result, a local API endpoint will be exposed and you can invoke it using yo ``` ???+ info - To learn more about local testing, please visit the [AWS SAM CLI local testing](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local-start-api.html) documentation. + To learn more about local testing, please visit the [AWS SAM CLI local testing](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local-start-api.html){target="_blank"} documentation. #### Live test @@ -147,7 +147,7 @@ At the end of the deployment, you will find the API endpoint URL within `Outputs ``` ???+ Info - For more details on AWS SAM deployment mechanism, see [SAM Deploy reference docs](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-deploy.html). + For more details on AWS SAM deployment mechanism, see [SAM Deploy reference docs](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-deploy.html){target="_blank"}. ## Routing @@ -364,13 +364,13 @@ Lastly, we used `return app.resolve(event, context)` so Event Handler can resolv From here, we could handle [404 routes](../core/event_handler/api_gateway.md#handling-not-found-routes){target="_blank"}, [error handling](../core/event_handler/api_gateway.md#exception-handling){target="_blank"}, [access query strings, payload](../core/event_handler/api_gateway.md#accessing-request-details){target="_blank"}, etc. ???+ tip - If you'd like to learn how python decorators work under the hood, you can follow [Real Python](https://realpython.com/primer-on-python-decorators/)'s article. + If you'd like to learn how python decorators work under the hood, you can follow [Real Python](https://realpython.com/primer-on-python-decorators/){target="_blank"}'s article. ## Structured Logging Over time, you realize that searching logs as text results in poor observability, it's hard to create metrics from, enumerate common exceptions, etc. -Then, you decided to propose production quality logging capabilities to your Lambda code. You found out that by having logs as `JSON` you can [structure them](https://docs.aws.amazon.com/lambda/latest/operatorguide/parse-logs.html), so that you can use any Log Analytics tool out there to quickly analyze them. +Then, you decided to propose production quality logging capabilities to your Lambda code. You found out that by having logs as `JSON` you can [structure them](https://docs.aws.amazon.com/lambda/latest/operatorguide/parse-logs.html){target="_blank"}, so that you can use any Log Analytics tool out there to quickly analyze them. This helps not only in searching, but produces consistent logs containing enough context and data to ask arbitrary questions on the status of your system. We can take advantage of CloudWatch Logs and Cloudwatch Insight for this purpose. @@ -487,7 +487,7 @@ Let's break this down: * **L5**: We add Powertools for AWS Lambda (Python) Logger; the boilerplate is now done for you. By default, we set `INFO` as the logging level if `LOG_LEVEL` env var isn't set. * **L22**: We use `logger.inject_lambda_context` decorator to inject key information from Lambda context into every log. -* **L22**: We also instruct Logger to use the incoming API Gateway Request ID as a [correlation id](../core/logger.md##set_correlation_id-method) automatically. +* **L22**: We also instruct Logger to use the incoming API Gateway Request ID as a [correlation id](../core/logger.md##set_correlation_id-method){target="_blank"} automatically. * **L22**: Since we're in dev, we also use `log_event=True` to automatically log each incoming request for debugging. This can be also set via [environment variables](./index.md#environment-variables){target="_blank"}. This is how the logs would look like now: @@ -709,7 +709,7 @@ Let's break it down: * **L45**: We include the final response under `response` key as part of the `handler` subsegment. ???+ info - If you want to understand how the Lambda execution environment (sandbox) works and why cold starts can occur, see this [blog series on Lambda performance](https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-1/). + If you want to understand how the Lambda execution environment (sandbox) works and why cold starts can occur, see this [blog series on Lambda performance](https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-1/){target="_blank"}. Repeat the process of building, deploying, and invoking your application via the API endpoint. @@ -788,7 +788,7 @@ From here, you can browse to specific logs in CloudWatch Logs Insight, Metrics D ![CloudWatch ServiceLens View](../media/tracer_utility_showcase_3.png) ???+ info - For more information on Amazon CloudWatch ServiceLens, please visit [link](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ServiceLens.html). + For more information on Amazon CloudWatch ServiceLens, please visit [link](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ServiceLens.html){target="_blank"}. ## Custom Metrics @@ -796,7 +796,7 @@ From here, you can browse to specific logs in CloudWatch Logs Insight, Metrics D Let's add custom metrics to better understand our application and business behavior (e.g. number of reservations, etc.). -By default, AWS Lambda adds [invocation and performance metrics](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-metrics.html#monitoring-metrics-types){target="_blank"}, and Amazon API Gateway adds [latency and some HTTP metrics](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-metrics-and-dimensions.html#api-gateway-metrics). +By default, AWS Lambda adds [invocation and performance metrics](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-metrics.html#monitoring-metrics-types){target="_blank"}, and Amazon API Gateway adds [latency and some HTTP metrics](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-metrics-and-dimensions.html#api-gateway-metrics){target="_blank"}. ???+ tip You can [optionally enable detailed metrics](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-metrics-and-dimensions.html#api-gateway-metricdimensions){target="_blank"} per each API route, stage, and method in API Gateway. @@ -936,7 +936,7 @@ Within `template.yaml`, we add [CloudWatchPutMetricPolicy](https://docs.aws.amaz ### Simplifying with Metrics -[Powertools for AWS Lambda (Python) Metrics](../core/metrics.md){target="_blank} uses [Amazon CloudWatch Embedded Metric Format (EMF)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html) to create custom metrics **asynchronously** via a native integration with Lambda. +[Powertools for AWS Lambda (Python) Metrics](../core/metrics.md){target="_blank} uses [Amazon CloudWatch Embedded Metric Format (EMF)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html){target="_blank"} to create custom metrics **asynchronously** via a native integration with Lambda. In general terms, EMF is a specification that expects metrics in a JSON payload within CloudWatch Logs. Lambda ingests all logs emitted by a given function into CloudWatch Logs. CloudWatch automatically looks up for log entries that follow the EMF format and transforms them into a CloudWatch metric. @@ -1041,7 +1041,7 @@ When it comes to the observability features ([Tracer](../core/tracer.md){target= This requires a change in mindset to ensure operational excellence is part of the software development lifecycle. ???+ tip - You can find more details on other leading practices described in the [Well-Architected Serverless Lens](https://aws.amazon.com/blogs/aws/new-serverless-lens-in-aws-well-architected-tool/). + You can find more details on other leading practices described in the [Well-Architected Serverless Lens](https://aws.amazon.com/blogs/aws/new-serverless-lens-in-aws-well-architected-tool/){target="_blank"}. Powertools for AWS Lambda (Python) is largely designed to make some of these practices easier to adopt from day 1. diff --git a/docs/upgrade.md b/docs/upgrade.md index 097c2d35b2a..fb7aef94712 100644 --- a/docs/upgrade.md +++ b/docs/upgrade.md @@ -9,7 +9,7 @@ description: Guide to update between major Powertools for AWS Lambda (Python) ve !!! warning "On March 31st, 2023, Powertools for AWS Lambda (Python) v1 reached end of support and will no longer receive updates or releases. If you are still using v1, we strongly recommend you to read our upgrade guide and update to the latest version." -Given our commitment to all of our customers using Powertools for AWS Lambda (Python), we will keep [Pypi](https://pypi.org/project/aws-lambda-powertools/) v1 releases and documentation 1.x versions to prevent any disruption. +Given our commitment to all of our customers using Powertools for AWS Lambda (Python), we will keep [Pypi](https://pypi.org/project/aws-lambda-powertools/){target="_blank"} v1 releases and documentation 1.x versions to prevent any disruption. ## Migrate to v2 from v1 diff --git a/docs/utilities/data_classes.md b/docs/utilities/data_classes.md index 04779ccf0f5..cbf8a5e9a6f 100644 --- a/docs/utilities/data_classes.md +++ b/docs/utilities/data_classes.md @@ -538,7 +538,7 @@ decompress and parse json data from the event. return "nothing to be processed" ``` -Alternatively, you can use `extract_cloudwatch_logs_from_record` to seamless integrate with the [Batch utility](./batch.md) for more robust log processing. +Alternatively, you can use `extract_cloudwatch_logs_from_record` to seamless integrate with the [Batch utility](./batch.md){target="_blank"} for more robust log processing. === "app.py" @@ -614,7 +614,7 @@ Data classes and utility functions to help create continuous delivery pipelines ### Cognito User Pool -Cognito User Pools have several [different Lambda trigger sources](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools-working-with-aws-lambda-triggers.html#cognito-user-identity-pools-working-with-aws-lambda-trigger-sources), all of which map to a different data class, which +Cognito User Pools have several [different Lambda trigger sources](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools-working-with-aws-lambda-triggers.html#cognito-user-identity-pools-working-with-aws-lambda-trigger-sources){target="_blank"}, all of which map to a different data class, which can be imported from `aws_lambda_powertools.data_classes.cognito_user_pool_event`: | Trigger/Event Source | Data Class | diff --git a/docs/utilities/feature_flags.md b/docs/utilities/feature_flags.md index 79657c436f6..c8a1faf4ab4 100644 --- a/docs/utilities/feature_flags.md +++ b/docs/utilities/feature_flags.md @@ -17,7 +17,7 @@ Feature flags are used to modify behaviour without changing the application's co **Dynamic flags**. Indicates something can have varying states, for example enable a list of premium features for customer X not Y. ???+ tip - You can use [Parameters utility](parameters.md) for static flags while this utility can do both static and dynamic feature flags. + You can use [Parameters utility](parameters.md){target="_blank"} for static flags while this utility can do both static and dynamic feature flags. ???+ warning Be mindful that feature flags can increase the complexity of your application over time; use them sparingly. @@ -453,7 +453,7 @@ These are the available options for further customization. | **max_age** | `5` | Number of seconds to cache feature flags configuration fetched from AWS AppConfig | | **sdk_config** | `None` | [Botocore Config object](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html){target="_blank"} | | **jmespath_options** | `None` | For advanced use cases when you want to bring your own [JMESPath functions](https://github.com/jmespath/jmespath.py#custom-functions){target="_blank"} | -| **logger** | `logging.Logger` | Logger to use for debug. You can optionally supply an instance of Powertools for AWS Lambda (Python) Logger. | +| **logger** | `logging.Logger` | Logger to use for debug. You can optionally supply an instance of Powertools for AWS Lambda (Python) Logger. | === "appconfig_provider_options.py" @@ -529,5 +529,5 @@ You can unit test your feature flags locally and independently without setting u | Method | When to use | Requires new deployment on changes | Supported services | | --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ---------------------------------- | ----------------------------------------------------- | | **[Environment variables](https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html){target="_blank"}** | Simple configuration that will rarely if ever change, because changing it requires a Lambda function deployment. | Yes | Lambda | -| **[Parameters utility](parameters.md)** | Access to secrets, or fetch parameters in different formats from AWS System Manager Parameter Store or Amazon DynamoDB. | No | Parameter Store, DynamoDB, Secrets Manager, AppConfig | +| **[Parameters utility](parameters.md){target="_blank"}** | Access to secrets, or fetch parameters in different formats from AWS System Manager Parameter Store or Amazon DynamoDB. | No | Parameter Store, DynamoDB, Secrets Manager, AppConfig | | **Feature flags utility** | Rule engine to define when one or multiple features should be enabled depending on the input. | No | AppConfig | diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md index f43dc68487f..37b34bf9750 100644 --- a/docs/utilities/idempotency.md +++ b/docs/utilities/idempotency.md @@ -75,7 +75,7 @@ If you're not [changing the default configuration for the DynamoDB persistence l | TTL attribute name | `expiration` | This can only be configured after your table is created if you're using AWS Console | ???+ tip "Tip: You can share a single state table for all functions" - You can reuse the same DynamoDB table to store idempotency state. We add `module_name` and [qualified name for classes and functions](https://peps.python.org/pep-3155/) in addition to the idempotency key as a hash key. + You can reuse the same DynamoDB table to store idempotency state. We add `module_name` and [qualified name for classes and functions](https://peps.python.org/pep-3155/){target="_blank"} in addition to the idempotency key as a hash key. ```yaml hl_lines="5-13 21-23" title="AWS Serverless Application Model (SAM) example" Resources: @@ -104,14 +104,14 @@ Resources: ``` ???+ warning "Warning: Large responses with DynamoDB persistence layer" - When using this utility with DynamoDB, your function's responses must be [smaller than 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items). + When using this utility with DynamoDB, your function's responses must be [smaller than 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items){target="_blank"}. Larger items cannot be written to DynamoDB and will cause exceptions. ???+ info "Info: DynamoDB" Each function invocation will generally make 2 requests to DynamoDB. If the result returned by your Lambda is less than 1kb, you can expect 2 WCUs per invocation. For retried invocations, you will - see 1WCU and 1RCU. Review the [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/) to + see 1WCU and 1RCU. Review the [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/){target="_blank"} to estimate the cost. ### Idempotent decorator @@ -247,7 +247,7 @@ When using `idempotent_function`, you must tell us which keyword parameter in yo #### Batch integration -You can can easily integrate with [Batch utility](batch.md) via context manager. This ensures that you process each record in an idempotent manner, and guard against a [Lambda timeout](#lambda-timeouts) idempotent situation. +You can can easily integrate with [Batch utility](batch.md){target="_blank"} via context manager. This ensures that you process each record in an idempotent manner, and guard against a [Lambda timeout](#lambda-timeouts) idempotent situation. ???+ "Choosing an unique batch record attribute" In this example, we choose `messageId` as our idempotency key since we know it'll be unique. @@ -339,7 +339,7 @@ Imagine the function executes successfully, but the client never receives the re ???+ tip "Deserializing JSON strings in payloads for increased accuracy." The payload extracted by the `event_key_jmespath` is treated as a string by default. This means there could be differences in whitespace even when the JSON payload itself is identical. - To alter this behaviour, we can use the [JMESPath built-in function](jmespath_functions.md#powertools_json-function) `powertools_json()` to treat the payload as a JSON object (dict) rather than a string. + To alter this behaviour, we can use the [JMESPath built-in function](jmespath_functions.md#powertools_json-function){target="_blank"} `powertools_json()` to treat the payload as a JSON object (dict) rather than a string. === "payment.py" @@ -410,7 +410,7 @@ Imagine the function executes successfully, but the client never receives the re ???+ note This is automatically done when you decorate your Lambda handler with [@idempotent decorator](#idempotent-decorator). -To prevent against extended failed retries when a [Lambda function times out](https://aws.amazon.com/premiumsupport/knowledge-center/lambda-verify-invocation-timeouts/), Powertools for AWS Lambda (Python) calculates and includes the remaining invocation available time as part of the idempotency record. +To prevent against extended failed retries when a [Lambda function times out](https://aws.amazon.com/premiumsupport/knowledge-center/lambda-verify-invocation-timeouts/){target="_blank"}, Powertools for AWS Lambda (Python) calculates and includes the remaining invocation available time as part of the idempotency record. ???+ example If a second invocation happens **after** this timestamp, and the record is marked as `INPROGRESS`, we will execute the invocation again as if it was in the `EXPIRED` state (e.g, `expire_seconds` field elapsed). @@ -698,15 +698,15 @@ When using DynamoDB as a persistence layer, you can alter the attribute names by Idempotent decorator can be further configured with **`IdempotencyConfig`** as seen in the previous example. These are the available options for further configuration -| Parameter | Default | Description | -| ------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------- | -| **event_key_jmespath** | `""` | JMESPath expression to extract the idempotency key from the event record using [built-in functions](/utilities/jmespath_functions) | -| **payload_validation_jmespath** | `""` | JMESPath expression to validate whether certain parameters have changed in the event while the event payload | -| **raise_on_no_idempotency_key** | `False` | Raise exception if no idempotency key was found in the request | -| **expires_after_seconds** | 3600 | The number of seconds to wait before a record is expired | -| **use_local_cache** | `False` | Whether to locally cache idempotency results | -| **local_cache_max_items** | 256 | Max number of items to store in local cache | -| **hash_function** | `md5` | Function to use for calculating hashes, as provided by [hashlib](https://docs.python.org/3/library/hashlib.html) in the standard library. | +| Parameter | Default | Description | +| ------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | +| **event_key_jmespath** | `""` | JMESPath expression to extract the idempotency key from the event record using [built-in functions](/utilities/jmespath_functions){target="_blank"} | +| **payload_validation_jmespath** | `""` | JMESPath expression to validate whether certain parameters have changed in the event while the event payload | +| **raise_on_no_idempotency_key** | `False` | Raise exception if no idempotency key was found in the request | +| **expires_after_seconds** | 3600 | The number of seconds to wait before a record is expired | +| **use_local_cache** | `False` | Whether to locally cache idempotency results | +| **local_cache_max_items** | 256 | Max number of items to store in local cache | +| **hash_function** | `md5` | Function to use for calculating hashes, as provided by [hashlib](https://docs.python.org/3/library/hashlib.html){target="_blank"} in the standard library. | ### Handling concurrent executions with the same payload @@ -912,7 +912,7 @@ This means that we will raise **`IdempotencyKeyError`** if the evaluation of **` ### Customizing boto configuration -The **`boto_config`** and **`boto3_session`** parameters enable you to pass in a custom [botocore config object](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html) or a custom [boto3 session](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html) when constructing the persistence store. +The **`boto_config`** and **`boto3_session`** parameters enable you to pass in a custom [botocore config object](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html){target="_blank"} or a custom [boto3 session](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html){target="_blank"} when constructing the persistence store. === "Custom session" @@ -1166,7 +1166,7 @@ The idempotency utility provides several routes to test your code. ### Disabling the idempotency utility When testing your code, you may wish to disable the idempotency logic altogether and focus on testing your business logic. To do this, you can set the environment variable `POWERTOOLS_IDEMPOTENCY_DISABLED` -with a truthy value. If you prefer setting this for specific tests, and are using Pytest, you can use [monkeypatch](https://docs.pytest.org/en/latest/monkeypatch.html) fixture: +with a truthy value. If you prefer setting this for specific tests, and are using Pytest, you can use [monkeypatch](https://docs.pytest.org/en/latest/monkeypatch.html){target="_blank"} fixture: === "tests.py" @@ -1221,7 +1221,7 @@ with a truthy value. If you prefer setting this for specific tests, and are usin ### Testing with DynamoDB Local -To test with [DynamoDB Local](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html), you can replace the `DynamoDB client` used by the persistence layer with one you create inside your tests. This allows you to set the endpoint_url. +To test with [DynamoDB Local](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html){target="_blank"}, you can replace the `DynamoDB client` used by the persistence layer with one you create inside your tests. This allows you to set the endpoint_url. === "tests.py" @@ -1281,7 +1281,7 @@ To test with [DynamoDB Local](https://docs.aws.amazon.com/amazondynamodb/latest/ ### How do I mock all DynamoDB I/O operations -The idempotency utility lazily creates the dynamodb [Table](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#table) which it uses to access DynamoDB. +The idempotency utility lazily creates the dynamodb [Table](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#table){target="_blank"} which it uses to access DynamoDB. This means it is possible to pass a mocked Table resource, or stub various methods. === "tests.py" @@ -1341,4 +1341,4 @@ This means it is possible to pass a mocked Table resource, or stub various metho ## Extra resources If you're interested in a deep dive on how Amazon uses idempotency when building our APIs, check out -[this article](https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/). +[this article](https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/){target="_blank"}. diff --git a/docs/utilities/jmespath_functions.md b/docs/utilities/jmespath_functions.md index 5550cdc507e..9313c702d6f 100644 --- a/docs/utilities/jmespath_functions.md +++ b/docs/utilities/jmespath_functions.md @@ -23,7 +23,7 @@ Built-in [JMESPath](https://jmespath.org/){target="_blank"} Functions to easily You might have events that contains encoded JSON payloads as string, base64, or even in compressed format. It is a common use case to decode and extract them partially or fully as part of your Lambda function invocation. -Powertools for AWS Lambda (Python) also have utilities like [validation](validation.md), [idempotency](idempotency.md), or [feature flags](feature_flags.md) where you might need to extract a portion of your data before using them. +Powertools for AWS Lambda (Python) also have utilities like [validation](validation.md){target="_blank"}, [idempotency](idempotency.md){target="_blank"}, or [feature flags](feature_flags.md){target="_blank"} where you might need to extract a portion of your data before using them. ???+ info "Terminology" **Envelope** is the terminology we use for the **JMESPath expression** to extract your JSON object from your data input. We might use those two terms interchangeably. @@ -81,7 +81,7 @@ These are all built-in envelopes you can use along with their expression as a re | **`SQS`** | `Records[*].powertools_json(body)` | ???+ tip "Using SNS?" - If you don't require SNS metadata, enable [raw message delivery](https://docs.aws.amazon.com/sns/latest/dg/sns-large-payload-raw-message-delivery.html){target="_blank"}. It will reduce multiple payload layers and size, when using SNS in combination with other services (_e.g., SQS, S3, etc_). + If you don't require SNS metadata, enable [raw message delivery](https://docs.aws.amazon.com/sns/latest/dg/sns-large-payload-raw-message-delivery.html). It will reduce multiple payload layers and size, when using SNS in combination with other services (_e.g., SQS, S3, etc_). ## Advanced diff --git a/docs/utilities/middleware_factory.md b/docs/utilities/middleware_factory.md index 1552311ea17..e855a3f451c 100644 --- a/docs/utilities/middleware_factory.md +++ b/docs/utilities/middleware_factory.md @@ -70,7 +70,7 @@ You can also have your own keyword arguments after the mandatory arguments. ## Advanced -For advanced use cases, you can instantiate [Tracer](../core/tracer.md) inside your middleware, and add annotations as well as metadata for additional operational insights. +For advanced use cases, you can instantiate [Tracer](../core/tracer.md){target="_blank"} inside your middleware, and add annotations as well as metadata for additional operational insights. === "advanced_middleware_tracer_function.py" ```python hl_lines="7 9 12 16 17 19 25 42" @@ -87,12 +87,12 @@ For advanced use cases, you can instantiate [Tracer](../core/tracer.md) inside y ### Tracing middleware **execution** -If you are making use of [Tracer](../core/tracer.md), you can trace the execution of your middleware to ease operations. +If you are making use of [Tracer](../core/tracer.md){target="_blank"}, you can trace the execution of your middleware to ease operations. This makes use of an existing Tracer instance that you may have initialized anywhere in your code. ???+ warning - You must [enable Active Tracing](../core/tracer/#permissions) in your Lambda function when using this feature, otherwise Lambda cannot send traces to XRay. + You must [enable Active Tracing](../core/tracer/#permissions){target="_blank"} in your Lambda function when using this feature, otherwise Lambda cannot send traces to XRay. === "getting_started_middleware_tracer_function.py" ```python hl_lines="8 14 15 36" @@ -105,13 +105,14 @@ This makes use of an existing Tracer instance that you may have initialized anyw --8<-- "examples/middleware_factory/src/getting_started_middleware_tracer_payload.json" ``` -When executed, your middleware name will [appear in AWS X-Ray Trace details as](../core/tracer.md) `## middleware_name`, in this example the middleware name is `## middleware_with_tracing`. +When executed, your middleware name will [appear in AWS X-Ray Trace details as](../core/tracer.md){target="_blank"} `## middleware_name`, in this example the middleware name is `## middleware_with_tracing`. ![Middleware simple Tracer](../media/middleware_factory_tracer_1.png) ### Combining Powertools for AWS Lambda (Python) utilities -You can create your own middleware and combine many features of Powertools for AWS Lambda (Python) such as [trace](../core/logger.md), [logs](../core/logger.md), [feature flags](feature_flags.md), [validation](validation.md), [jmespath_functions](jmespath_functions.md) and others to abstract non-functional code. + +You can create your own middleware and combine many features of Powertools for AWS Lambda (Python) such as [trace](../core/logger.md){target="_blank"}, [logs](../core/logger.md){target="_blank"}, [feature flags](feature_flags.md){target="_blank"}, [validation](validation.md){target="_blank"}, [jmespath_functions](jmespath_functions.md){target="_blank"} and others to abstract non-functional code. In the example below, we create a Middleware with the following features: diff --git a/docs/utilities/parameters.md b/docs/utilities/parameters.md index 7c77a976983..a7fe0278511 100644 --- a/docs/utilities/parameters.md +++ b/docs/utilities/parameters.md @@ -296,7 +296,7 @@ You can create your own custom parameter store provider by inheriting the `BaseP All transformation and caching logic is handled by the `get()` and `get_multiple()` methods from the base provider class. -Here are two examples of implementing a custom parameter store. One using an external service like [Hashicorp Vault](https://www.vaultproject.io/), a widely popular key-value and secret storage and the other one using [Amazon S3](https://aws.amazon.com/s3/?nc1=h_ls), a popular object storage. +Here are two examples of implementing a custom parameter store. One using an external service like [Hashicorp Vault](https://www.vaultproject.io/){target="_blank"}, a widely popular key-value and secret storage and the other one using [Amazon S3](https://aws.amazon.com/s3/?nc1=h_ls){target="_blank"}, a popular object storage. === "working_with_own_provider_vault.py" ```python hl_lines="5 13 20 24" @@ -391,15 +391,15 @@ Here is the mapping between this utility's functions and methods and the underly | Provider | Function/Method | Client name | Function name | | ------------------- | ------------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| SSM Parameter Store | `get_parameter` | `ssm` | [get_parameter](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameter) | -| SSM Parameter Store | `get_parameters` | `ssm` | [get_parameters_by_path](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameters_by_path) | -| SSM Parameter Store | `SSMProvider.get` | `ssm` | [get_parameter](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameter) | -| SSM Parameter Store | `SSMProvider.get_multiple` | `ssm` | [get_parameters_by_path](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameters_by_path) | -| Secrets Manager | `get_secret` | `secretsmanager` | [get_secret_value](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html#SecretsManager.Client.get_secret_value) | -| Secrets Manager | `SecretsProvider.get` | `secretsmanager` | [get_secret_value](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html#SecretsManager.Client.get_secret_value) | -| DynamoDB | `DynamoDBProvider.get` | `dynamodb` | ([Table resource](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#table)) | [get_item](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Table.get_item) | -| DynamoDB | `DynamoDBProvider.get_multiple` | `dynamodb` | ([Table resource](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#table)) | [query](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Table.query) | -| App Config | `get_app_config` | `appconfigdata` | [start_configuration_session](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/appconfigdata.html#AppConfigData.Client.start_configuration_session) and [get_latest_configuration](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/appconfigdata.html#AppConfigData.Client.get_latest_configuration) | +| SSM Parameter Store | `get_parameter` | `ssm` | [get_parameter](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameter){target="_blank"} | +| SSM Parameter Store | `get_parameters` | `ssm` | [get_parameters_by_path](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameters_by_path){target="_blank"} | +| SSM Parameter Store | `SSMProvider.get` | `ssm` | [get_parameter](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameter){target="_blank"} | +| SSM Parameter Store | `SSMProvider.get_multiple` | `ssm` | [get_parameters_by_path](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameters_by_path){target="_blank"} | +| Secrets Manager | `get_secret` | `secretsmanager` | [get_secret_value](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html#SecretsManager.Client.get_secret_value){target="_blank"} | +| Secrets Manager | `SecretsProvider.get` | `secretsmanager` | [get_secret_value](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html#SecretsManager.Client.get_secret_value){target="_blank"} | +| DynamoDB | `DynamoDBProvider.get` | `dynamodb` | ([Table resource](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#table){target="_blank"}) | [get_item](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Table.get_item) | +| DynamoDB | `DynamoDBProvider.get_multiple` | `dynamodb` | ([Table resource](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#table){target="_blank"}) | [query](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Table.query) | +| App Config | `get_app_config` | `appconfigdata` | [start_configuration_session](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/appconfigdata.html#AppConfigData.Client.start_configuration_session){target="_blank"} and [get_latest_configuration](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/appconfigdata.html#AppConfigData.Client.get_latest_configuration){target="_blank"} | ### Bring your own boto client @@ -424,7 +424,7 @@ Bringing them together in a single code snippet would look like this: ### Customizing boto configuration -The **`config`** , **`boto3_session`**, and **`boto3_client`** parameters enable you to pass in a custom [botocore config object](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html) , [boto3 session](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html), or a [boto3 client](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/boto3.html) when constructing any of the built-in provider classes. +The **`config`** , **`boto3_session`**, and **`boto3_client`** parameters enable you to pass in a custom [botocore config object](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html){target="_blank"}, [boto3 session](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html){target="_blank"}, or a [boto3 client](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/boto3.html){target="_blank"} when constructing any of the built-in provider classes. ???+ tip You can use a custom session for retrieving parameters cross-account/region and for snapshot testing. @@ -450,7 +450,7 @@ The **`config`** , **`boto3_session`**, and **`boto3_client`** parameters enabl ### Mocking parameter values -For unit testing your applications, you can mock the calls to the parameters utility to avoid calling AWS APIs. This can be achieved in a number of ways - in this example, we use the [pytest monkeypatch fixture](https://docs.pytest.org/en/latest/how-to/monkeypatch.html) to patch the `parameters.get_parameter` method: +For unit testing your applications, you can mock the calls to the parameters utility to avoid calling AWS APIs. This can be achieved in a number of ways - in this example, we use the [pytest monkeypatch fixture](https://docs.pytest.org/en/latest/how-to/monkeypatch.html){target="_blank"} to patch the `parameters.get_parameter` method: === "test_single_mock.py" ```python hl_lines="4 8" @@ -470,8 +470,8 @@ If we need to use this pattern across multiple tests, we can avoid repetition by ``` Alternatively, if we need more fully featured mocking (for example checking the arguments passed to `get_parameter`), we -can use [unittest.mock](https://docs.python.org/3/library/unittest.mock.html) from the python stdlib instead of pytest's `monkeypatch` fixture. In this example, we use the -[patch](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch) decorator to replace the `aws_lambda_powertools.utilities.parameters.get_parameter` function with a [MagicMock](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.MagicMock) +can use [unittest.mock](https://docs.python.org/3/library/unittest.mock.html){target="_blank"} from the python stdlib instead of pytest's `monkeypatch` fixture. In this example, we use the +[patch](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch){target="_blank"} decorator to replace the `aws_lambda_powertools.utilities.parameters.get_parameter` function with a [MagicMock](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.MagicMock){target="_blank"} object named `get_parameter_mock`. === "test_with_monkeypatch.py" diff --git a/docs/utilities/parser.md b/docs/utilities/parser.md index 5ff419f8777..b0852a0129f 100644 --- a/docs/utilities/parser.md +++ b/docs/utilities/parser.md @@ -4,7 +4,7 @@ description: Utility --- -This utility provides data parsing and deep validation using [Pydantic](https://pydantic-docs.helpmanual.io/). +This utility provides data parsing and deep validation using [Pydantic](https://pydantic-docs.helpmanual.io/){target="_blank"}. ## Key features @@ -24,7 +24,7 @@ Add `aws-lambda-powertools[parser]` as a dependency in your preferred tool: _e.g This will increase the compressed package size by >10MB due to the Pydantic dependency. To reduce the impact on the package size at the expense of 30%-50% of its performance [Pydantic can also be - installed without binary files](https://pydantic-docs.helpmanual.io/install/#performance-vs-package-size-trade-off): + installed without binary files](https://pydantic-docs.helpmanual.io/install/#performance-vs-package-size-trade-off){target="_blank"}: Pip example: `SKIP_CYTHON=1 pip install --no-binary pydantic aws-lambda-powertools[parser]` @@ -245,13 +245,13 @@ for order_item in ret.detail.items: ???+ tip When extending a `string` field containing JSON, you need to wrap the field - with [Pydantic's Json Type](https://pydantic-docs.helpmanual.io/usage/types/#json-type): + with [Pydantic's Json Type](https://pydantic-docs.helpmanual.io/usage/types/#json-type){target="_blank"}: ```python hl_lines="14 18-19" --8<-- "examples/parser/src/extending_built_in_models_with_json_mypy.py" ``` - Alternatively, you could use a [Pydantic validator](https://pydantic-docs.helpmanual.io/usage/validators/) to transform the JSON string into a dict before the mapping: + Alternatively, you could use a [Pydantic validator](https://pydantic-docs.helpmanual.io/usage/validators/){target="_blank"} to transform the JSON string into a dict before the mapping: ```python hl_lines="18-20 24-25" --8<-- "examples/parser/src/extending_built_in_models_with_json_validator.py" @@ -495,14 +495,14 @@ parse(model=UserModel, event=payload) ### Advanced use cases ???+ tip "Tip: Looking to auto-generate models from JSON, YAML, JSON Schemas, OpenApi, etc?" - Use Koudai Aono's [data model code generation tool for Pydantic](https://github.com/koxudaxi/datamodel-code-generator) + Use Koudai Aono's [data model code generation tool for Pydantic](https://github.com/koxudaxi/datamodel-code-generator){target="_blank"} -There are number of advanced use cases well documented in Pydantic's doc such as creating [immutable models](https://pydantic-docs.helpmanual.io/usage/models/#faux-immutability), [declaring fields with dynamic values](https://pydantic-docs.helpmanual.io/usage/models/#field-with-dynamic-default-value). +There are number of advanced use cases well documented in Pydantic's doc such as creating [immutable models](https://pydantic-docs.helpmanual.io/usage/models/#faux-immutability){target="_blank"}, [declaring fields with dynamic values](https://pydantic-docs.helpmanual.io/usage/models/#field-with-dynamic-default-value){target="_blank"}. ???+ tip "Pydantic helper functions" - Pydantic also offers [functions](https://pydantic-docs.helpmanual.io/usage/models/#helper-functions) to parse models from files, dicts, string, etc. + Pydantic also offers [functions](https://pydantic-docs.helpmanual.io/usage/models/#helper-functions){target="_blank"} to parse models from files, dicts, string, etc. -Two possible unknown use cases are Models and exception' serialization. Models have methods to [export them](https://pydantic-docs.helpmanual.io/usage/exporting_models/) as `dict`, `JSON`, `JSON Schema`, and Validation exceptions can be exported as JSON. +Two possible unknown use cases are Models and exception' serialization. Models have methods to [export them](https://pydantic-docs.helpmanual.io/usage/exporting_models/){target="_blank"} as `dict`, `JSON`, `JSON Schema`, and Validation exceptions can be exported as JSON. ```python hl_lines="21 28-31" title="Converting data models in various formats" from aws_lambda_powertools.utilities import Logger diff --git a/docs/utilities/streaming.md b/docs/utilities/streaming.md index ed10d436550..fe4d59f9f94 100644 --- a/docs/utilities/streaming.md +++ b/docs/utilities/streaming.md @@ -85,11 +85,11 @@ That said, you can still open a specific file as a stream, reading only the nece We provide popular built-in transformations that you can apply against your streaming data. -| Name | Description | Class name | -| -------- | ------------------------------------------------------------------------------------------------ | ------------- | -| **Gzip** | Gunzips the stream of data using the [gzip library](https://docs.python.org/3/library/gzip.html) | GzipTransform | -| **Zip** | Exposes the stream as a [ZipFile object](https://docs.python.org/3/library/zipfile.html) | ZipTransform | -| **CSV** | Parses each CSV line as a CSV object, returning dictionary objects | CsvTransform | +| Name | Description | Class name | +| -------- | ----------------------------------------------------------------------------------------------------------------- | ------------- | +| **Gzip** | Gunzips the stream of data using the [gzip library](https://docs.python.org/3/library/gzip.html){target="_blank"} | GzipTransform | +| **Zip** | Exposes the stream as a [ZipFile object](https://docs.python.org/3/library/zipfile.html){target="_blank"} | ZipTransform | +| **CSV** | Parses each CSV line as a CSV object, returning dictionary objects | CsvTransform | ## Advanced @@ -131,11 +131,11 @@ You found out that each row has 8 bytes, the header line has 21 bytes, and every We will propagate additional options to the underlying implementation for each transform class. -| Name | Available options | -| ----------------- | ------------------------------------------------------------------------------------- | -| **GzipTransform** | [GzipFile constructor](https://docs.python.org/3/library/gzip.html#gzip.GzipFile) | -| **ZipTransform** | [ZipFile constructor](https://docs.python.org/3/library/zipfile.html#zipfile.ZipFile) | -| **CsvTransform** | [DictReader constructor](https://docs.python.org/3/library/csv.html#csv.DictReader) | +| Name | Available options | +| ----------------- | ------------------------------------------------------------------------------------------------------ | +| **GzipTransform** | [GzipFile constructor](https://docs.python.org/3/library/gzip.html#gzip.GzipFile){target="_blank"} | +| **ZipTransform** | [ZipFile constructor](https://docs.python.org/3/library/zipfile.html#zipfile.ZipFile){target="_blank"} | +| **CsvTransform** | [DictReader constructor](https://docs.python.org/3/library/csv.html#csv.DictReader){target="_blank"} | For instance, take `ZipTransform`. You can use the `compression` parameter if you want to unzip an S3 object compressed with `LZMA`. @@ -185,6 +185,6 @@ Create an input payload using `io.BytesIO` and assert the response of the transf ### AWS X-Ray segment size limit -We make multiple API calls to S3 as you read chunks from your S3 object. If your function is decorated with [Tracer](./../core/tracer.md), you can easily hit [AWS X-Ray 64K segment size](https://docs.aws.amazon.com/general/latest/gr/xray.html#limits_xray) when processing large files. +We make multiple API calls to S3 as you read chunks from your S3 object. If your function is decorated with [Tracer](./../core/tracer.md){target="_blank"}, you can easily hit [AWS X-Ray 64K segment size](https://docs.aws.amazon.com/general/latest/gr/xray.html#limits_xray){target="_blank"} when processing large files. !!! tip "Use tracer decorators in parts where you don't read your `S3Object` instead." diff --git a/docs/utilities/validation.md b/docs/utilities/validation.md index cef3b1e3134..c9fd08e31cc 100644 --- a/docs/utilities/validation.md +++ b/docs/utilities/validation.md @@ -91,7 +91,7 @@ You can also gracefully handle schema validation errors by catching `SchemaValid You might want to validate only a portion of your event - This is what the `envelope` parameter is for. -Envelopes are [JMESPath expressions](https://jmespath.org/tutorial.html) to extract a portion of JSON you want before applying JSON Schema validation. +Envelopes are [JMESPath expressions](https://jmespath.org/tutorial.html){target="_blank"} to extract a portion of JSON you want before applying JSON Schema validation. Here is a sample custom EventBridge event, where we only validate what's inside the `detail` key: @@ -113,7 +113,7 @@ Here is a sample custom EventBridge event, where we only validate what's inside --8<-- "examples/validation/src/getting_started_validator_unwrapping_payload.json" ``` -This is quite powerful because you can use JMESPath Query language to extract records from [arrays](https://jmespath.org/tutorial.html#list-and-slice-projections), combine [pipe](https://jmespath.org/tutorial.html#pipe-expressions) and [function expressions](https://jmespath.org/tutorial.html#functions). +This is quite powerful because you can use JMESPath Query language to extract records from [arrays](https://jmespath.org/tutorial.html#list-and-slice-projections){target="_blank"}, combine [pipe](https://jmespath.org/tutorial.html#pipe-expressions){target="_blank"} and [function expressions](https://jmespath.org/tutorial.html#functions){target="_blank"}. When combined, these features allow you to extract what you need before validating the actual payload. @@ -194,7 +194,7 @@ For each format defined in a dictionary key, you must use a regex, or a function You might have events or responses that contain non-encoded JSON, where you need to decode before validating them. -You can use our built-in [JMESPath functions](./jmespath_functions.md) within your expressions to do exactly that to [deserialize JSON Strings](./jmespath_functions.md#powertools_json-function), [decode base64](./jmespath_functions.md#powertools_base64-function), and [decompress gzip data](./jmespath_functions.md#powertools_base64_gzip-function). +You can use our built-in [JMESPath functions](./jmespath_functions.md){target="_blank"} within your expressions to do exactly that to [deserialize JSON Strings](./jmespath_functions.md#powertools_json-function){target="_blank"}, [decode base64](./jmespath_functions.md#powertools_base64-function){target="_blank"}, and [decompress gzip data](./jmespath_functions.md#powertools_base64_gzip-function){target="_blank"}. ???+ info We use these for [built-in envelopes](#built-in-envelopes) to easily to decode and unwrap events from sources like Kinesis, CloudWatch Logs, etc. diff --git a/poetry.lock b/poetry.lock index 5b423b983da..8a4cc9242c0 100644 --- a/poetry.lock +++ b/poetry.lock @@ -1,4 +1,4 @@ -# This file is automatically @generated by Poetry and should not be changed by hand. +# This file is automatically @generated by Poetry 1.4.2 and should not be changed by hand. [[package]] name = "anyio" @@ -3078,7 +3078,7 @@ docs = ["furo", "jaraco.packaging (>=9)", "jaraco.tidelift (>=1.4)", "rst.linker testing = ["big-O", "flake8 (<5)", "jaraco.functools", "jaraco.itertools", "more-itertools", "pytest (>=6)", "pytest-black (>=0.3.7)", "pytest-checkdocs (>=2.4)", "pytest-cov", "pytest-enabler (>=1.3)", "pytest-flake8", "pytest-mypy (>=0.9.1)"] [extras] -all = ["pydantic", "aws-xray-sdk", "fastjsonschema"] +all = ["aws-xray-sdk", "fastjsonschema", "pydantic"] aws-sdk = ["boto3"] parser = ["pydantic"] tracer = ["aws-xray-sdk"]