Skip to content

docs(roadmap): april updates #4181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/workflows/on_schedule_monthly_roadmap_reminder.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,3 @@ jobs:
uses: aws-powertools/actions/.github/workflows/monthly_roadmap_reminder.yml@fd4575466e5c2ac10703ac16f5aa9fb8890f532a
with:
token: ${{ github.token }}


146 changes: 77 additions & 69 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
<!-- markdownlint-disable MD043 -->

## Overview
<!-- markdownlint-disable MD043 MD041 -->
# Overview

Our public roadmap outlines the high level direction we are working towards. We update this document when our priorities change: security and stability are our top priority.

Expand All @@ -10,42 +9,60 @@ Our public roadmap outlines the high level direction we are working towards. We

Security and operational excellence take precedence above all else. This means bug fixing, stability, customer's support, and internal compliance may delay one or more key areas below.

### Amazon Bedrock Agent Event Handler
**Missing something or want us to prioritize an existing area?**

Based on [customers](https://github.com/aws-powertools/powertools-lambda-python#connect){target="_blank"} at re:Invent 2023, we will add a new Event Handler resolver to improve authoring and maintenance of Amazon Bedrock Agents.
You can help us prioritize by [upvoting existing feature requests](https://github.com/aws-powertools/powertools-lambda-python/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Afeature-request), leaving a comment on what use cases it could unblock for you, and by joining our discussions on Discord.

### Observability providers

We want to extend Tracer, Metrics, and Logger to support any [AWS Lambda certified observability partner](https://go.aws/3HtU6CZ){target="_blank"}, along with OpenTelemetry.

At launch, we will support Datadog since it's [most requested observability provider](https://github.com/aws-powertools/powertools-lambda-python/issues/1433). OpenTelemetry will be a fast follow-up as we need to decide on a stable solution to cold start penalty.

!!! tip "Help us identify which observability providers we should integrate next. Open [feature request](https://github.com/aws-powertools/powertools-lambda-python/issues/new?assignees=&labels=feature-request%2Ctriage&projects=&template=feature_request.yml&title=Feature+request%3A+TITLE){target="_blank"} or by voting `+1` in existing issues"

**Major updates**

* [x] [Event Source Data Classes support](https://github.com/aws-powertools/powertools-lambda-python/pull/3262)
* [x] [Pydantic model _(Parser)_ support](https://github.com/aws-powertools/powertools-lambda-python/pull/3286)
* [x] [MVP Event Handler](https://github.com/aws-powertools/powertools-lambda-python/pull/3285)
* [ ] [New feature documentation](https://github.com/aws-powertools/powertools-lambda-python/pull/3602)
* [ ] Video to walkthrough use cases for anyone new to LLM Agents
* [ ] Launch amplifier (_e.g., What's New, Blog post_)
* [x] [Document how customers can use any provider with Logger](https://docs.powertools.aws.dev/lambda/python/latest/core/logger/#observability-providers)
* [x] [Extend Metrics to add support for any Provider](https://github.com/aws-powertools/powertools-lambda-python/pull/2194)
* [ ] [Extend Tracer to add support for any Provider](https://github.com/aws-powertools/powertools-lambda-python/pull/2342#issuecomment-2061734362)
* [ ] Investigate alternative solution to OpenTelemetry cold start performance

### Setting Parameters and Secrets
### Lambda Layer in GovCloud

As of today, the [Parameters](./utilities/parameters.md){target="_blank"} feature is used to retrieve data, not to create or update existing parameters. Based on community feedback, we plan to enhance Parameters to allow set operations.
We want to investigate security and scaling requirements for these special regions, so they're in sync for every release.

!!! note "Help us prioritize it by reaching out to your AWS representatives or [via email](mailto:[email protected])."

**Major updates**

* [x] [RFC](https://github.com/aws-powertools/powertools-lambda-python/issues/3040)
* [ ] [MVP](https://github.com/aws-powertools/powertools-lambda-python/pull/2858)
* [x] Gather agencies and customers name to prioritize it
* [x] Investigate security requirements for special regions
* [x] Create additional infrastructure for special regions
* [x] AppSec review
* [x] Update CDK Layer construct to include regions
* [x] Distribution sign-off
* [ ] Distribute latest version
* [ ] Update Layer section with new AWS Accounts

### Observability providers
### V3

We want to extend Tracer, Metrics, and Logger to support any [AWS Lambda certified observability partner](https://go.aws/3HtU6CZ){target="_blank"}, along with OpenTelemetry.
We are in the process of planning the roadmap for v3. As always, [our approach](./versioning.md){target="_blank"} includes providing sufficient advance notice, a comprehensive upgrade guide, and minimizing breaking changes to facilitate a smooth transition (e.g., it took ~7 months from v2 to surpass v1 downloads).

At launch, we will support Datadog since it's [most requested observability provider](https://github.com/aws-powertools/powertools-lambda-python/issues/1433). OpenTelemetry will be a fast follow-up as we need to decide on a stable solution to cold start penalty.
For example, these are on our mind but not settled yet until we have a public tracker to discuss what these means in detail.

!!! tip "Help us identify which observability providers we should integrate next. Open [feature request](https://github.com/aws-powertools/powertools-lambda-python/issues/new?assignees=&labels=feature-request%2Ctriage&projects=&template=feature_request.yml&title=Feature+request%3A+TITLE){target="_blank"} or by voting `+1` in existing issues"
* **Parser**: Drop Pydantic v1
* **Parser**: Deserialize Amazon DynamoDB data types automatically (like Event Source Data Classes)
* **Parameters**: Increase default `max_age` for `get_secret`
* **Event Source Data Classes**: Return sane defaults for any property that has `Optional[<type>]` returns
* **Batch**: Stop at first error for Amazon DynamoDB Streams and Amazon Kinesis Data Streams (e.g., `stop_on_failure=True`)

**Major updates**

* [x] [Document how customers can use any provider with Logger](https://docs.powertools.aws.dev/lambda/python/latest/core/logger/#observability-providers)
* [x] [Extend Metrics to add support for any Provider](https://github.com/aws-powertools/powertools-lambda-python/pull/2194)
* [ ] [Extend Tracer to add support for any Provider](https://github.com/aws-powertools/powertools-lambda-python/issues/2030)
* [ ] Investigate alternative solution to OpenTelemetry cold start performance
* [ ] Create an issue to track breaking changes we consider making
* [ ] Create a v3 branch to allow early experimentation
* [ ] Create workflows to allow pre-releases
* [ ] Create a mechanism to keep ideas for breaking change somewhere regardless of v3

### Revamp Event Handler

Expand All @@ -63,6 +80,17 @@ Based on customers feedback, we want to provide [middleware authoring support](h
* [x] [MVP for OpenAPI and SwaggerUI](https://github.com/aws-powertools/powertools-lambda-python/pull/3109)
* [ ] [MVP for AppSync Batch invoke and partial failure support](https://github.com/aws-powertools/powertools-lambda-python/pull/1998)

### Authentication (SigV4)

[During customers interview](https://github.com/aws-powertools/powertools-lambda-python#connect){target="_blank"}, we hear that signing requests using [AWS SigV4](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html){target="_blank"} could be easier.

Since JWT is a close second, this new utility would cover higher level functions to sign and verify requests more easily.

**Major updates**

* [x] [Issue to outline challenges](https://github.com/aws-powertools/powertools-lambda-python/issues/2493), alternative solutions and desired experience
* [ ] [MVP for AWS SigV4](https://github.com/aws-powertools/powertools-lambda-python/pull/2435)

### Office hours

We heard from [customers](https://github.com/aws-powertools/powertools-lambda-python#connect){target="_blank"} that Powertools for AWS Lambda and its community can move faster than they are able to catch up. While documentation and release notes take these into account, they notice they don't always know advanced tricks, or what other customers tend to do in similar situations.
Expand All @@ -79,17 +107,6 @@ Timezones being tricky, we plan to experiment with an afternoon slot in Central
* [ ] Invite Discord community
* [ ] Update roadmap page with Discord event

### Authentication (SigV4)

[During customers interview](https://github.com/aws-powertools/powertools-lambda-python#connect){target="_blank"}, we hear that signing requests using [AWS SigV4](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html){target="_blank"} could be easier.

Since JWT is a close second, this new utility would cover higher level functions to sign and verify requests more easily.

**Major updates**

* [ ] RFC to outline challenges, alternative solutions and desired experience
* [ ] [MVP for AWS SigV4](https://github.com/aws-powertools/powertools-lambda-python/pull/2435)

### Enhanced operational metrics

[Through customers interview](https://github.com/aws-powertools/powertools-lambda-python#connect){target="_blank"}, [Discord](https://discord.gg/B8zZKbbyET){target="_blank" rel="nofollow"}, and [1:1 customer enablement](https://github.com/aws-powertools/powertools-lambda-python#connect){target="_blank"}, we noticed customers often create the same set of custom operational metrics.
Expand All @@ -105,41 +122,6 @@ We want to make this easier by extending certain utilities to accept a `metrics`
* [ ] RFC to outline metrics for Event Handler (_e.g., validation errors_ )
* [ ] RFC to outline metrics for Idempotency (_e.g., cache hit_)

### Lambda Layer in GovCloud and China region

We want to investigate security and scaling requirements for these special regions, so they're in sync for every release.

!!! note "Help us prioritize it by reaching out to your AWS representatives or [via email](mailto:[email protected])."

**Major updates**

* [x] Gather agencies and customers name to prioritize it
* [x] Investigate security requirements for special regions
* [x] Create additional infrastructure for special regions
* [ ] AppSec review
* [ ] Distribution sign-off
* [ ] Update CDK Layer construct to include regions

### V3

We are in the process of planning the roadmap for v3. As always, [our approach](./versioning.md){target="_blank"} includes providing sufficient advance notice, a comprehensive upgrade guide, and minimizing breaking changes to facilitate a smooth transition (e.g., it took ~7 months from v2 to surpass v1 downloads).

For example, these are on our mind but not settled yet until we have a public tracker to discuss what these means in detail.

* **Parser**: Drop Pydantic v1
* **Parser**: Deserialize Amazon DynamoDB data types automatically (like Event Source Data Classes)
* **Parameters**: Increase default `max_age` for `get_secret`
* **Event Source Data Classes**: Return sane defaults for any property that has `Optional[<type>]` returns
* **Upgrade tool**: Consider building a CST (Concrete Syntax Tree) tool to ease certain upgrade actions like `pyupgrade` and `django-upgrade`
* **Batch**: Stop at first error for Amazon DynamoDB Streams and Amazon Kinesis Data Streams (e.g., `stop_on_failure=True`)

**Major updates**

* [ ] Create an issue to track breaking changes we consider making
* [ ] Create a v3 branch to allow early experimentation
* [ ] Create workflows to allow pre-releases
* [ ] Create a mechanism to keep ideas for breaking change somewhere regardless of v3

## Roadmap status definition

<center>
Expand Down Expand Up @@ -209,6 +191,32 @@ A: For existing features, you can directly comment on issues. For anything else,

## Launched

### Setting Parameters and Secrets

> [Docs](./utilities/parameters.md#setting-parameters)

As of today, the [Parameters](./utilities/parameters.md){target="_blank"} feature is used to retrieve data, not to create or update existing parameters. Based on community feedback, we plan to enhance Parameters to allow set operations.

**Major updates**

* [x] [RFC](https://github.com/aws-powertools/powertools-lambda-python/issues/3040)
* [x] [MVP](https://github.com/aws-powertools/powertools-lambda-python/pull/2858)

### Amazon Bedrock Agent Event Handler

> [Docs](./core/event_handler/bedrock_agents.md)

Based on [customers](https://github.com/aws-powertools/powertools-lambda-python#connect){target="_blank"} at re:Invent 2023, we will add a new Event Handler resolver to improve authoring and maintenance of Amazon Bedrock Agents.

**Major updates**

* [x] [Event Source Data Classes support](https://github.com/aws-powertools/powertools-lambda-python/pull/3262)
* [x] [Pydantic model _(Parser)_ support](https://github.com/aws-powertools/powertools-lambda-python/pull/3286)
* [x] [MVP Event Handler](https://github.com/aws-powertools/powertools-lambda-python/pull/3285)
* [x] [New feature documentation](https://github.com/aws-powertools/powertools-lambda-python/pull/3602)
* [x] [Video to walkthrough](https://docs.powertools.aws.dev/lambda/python/latest/core/event_handler/bedrock_agents/#video-walkthrough) use cases for anyone new to LLM Agents
* [ ] Launch amplifier (_e.g., What's New, Blog post_)

### Sensitive Data Masking

> [Docs](./utilities/data_masking.md)
Expand Down