Bug: Event Handler Data Validation KeyError: 'multiValueHeaders' regression when running locally

[heitorlessa]

Expected Behaviour

Work as expected and described in the docs: https://docs.powertools.aws.dev/lambda/python/2.34.0/core/event_handler/api_gateway/#testing-your-code

Current Behaviour

When running locally, request fails when multiValueHeaders key isn't present despite not needing it.

Full stack trace.

> python blah.py
Traceback (most recent call last):
  File "/Users/lessa/DEV/pt-experiments/blah.py", line 24, in <module>
    ret = app.lambda_handler(minimal_event, LambdaContext())  # type: ignore
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/logging/logger.py", line 449, in decorate
    return lambda_handler(event, context, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/tracing/tracer.py", line 313, in decorate
    response = lambda_handler(event, context, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/todo_api/app.py", line 67, in lambda_handler
    return app.resolve(event, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/api_gateway.py", line 1800, in resolve
    response = self._resolve().build(self.current_event, self._cors)
               ^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/api_gateway.py", line 1907, in _resolve
    return self._call_route(route, route_keys)  # pass fn args
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/api_gateway.py", line 1985, in _call_route
    route(router_middlewares=self._router_middlewares, app=self, route_arguments=route_arguments),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/api_gateway.py", line 400, in __call__
    return self._middleware_stack(app)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/api_gateway.py", line 1291, in __call__
    return self.current_middleware(app, self.next_middleware)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/middlewares/base.py", line 121, in __call__
    return self.handler(app, next_middleware)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/middlewares/openapi_validation.py", line 86, in handler
    app.current_event.resolved_headers_field,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/utilities/data_classes/api_gateway_proxy_event.py", line 132, in resolved_headers_field
    if self.multi_value_headers:
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/utilities/data_classes/api_gateway_proxy_event.py", line 115, in multi_value_headers
    return self["multiValueHeaders"]
           ~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lessa/DEV/pt-experiments/.venv/lib/python3.11/site-packages/aws_lambda_powertools/utilities/data_classes/common.py", line 31, in __getitem__
    return self._data[key]
           ~~~~~~~~~~^^^^^
KeyError: 'multiValueHeaders'

Code snippet

import requests
from aws_lambda_powertools import Logger, Tracer
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.utilities.typing import LambdaContext

tracer = Tracer()
logger = Logger()
app = APIGatewayRestResolver(enable_validation=True)

@app.get("/todos")
@tracer.capture_method
def get_todos():

    todo = requests.get("https://jsonplaceholder.typicode.com/todos/")
    todo.raise_for_status()

    from aws_lambda_powertools import __version__

    return {"todos": todo.json()[0], "powertools_version": __version__}


@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
@tracer.capture_lambda_handler
def lambda_handler(event: dict, context: LambdaContext) -> dict:
    return app.resolve(event, context)

Possible Solution

Use .get over Key for a more defensive programming. Accidental regression was introduced in 2.33.0

https://github.com/aws-powertools/powertools-lambda-python/pull/3687/files#diff-4188087d42ec470125206481864423dbc79a7d50ece2a5338b4be75e9cbf63a9R49

Steps to Reproduce

1. Create a file named todo_api/app.py with the contents earlier
2. Create another file (blah.py) that will be run locally with the following contents
3. Run the script locally

Uncomment either line multiValueHeaders and the problem will go away.

from dataclasses import dataclass
from todo_api import app


@dataclass
class LambdaContext:
    function_name: str = "test"
    memory_limit_in_mb: int = 128
    invoked_function_arn: str = "arn:aws:lambda:eu-west-1:123456789012:function:test"
    aws_request_id: str = "da658bd3-2d6f-4e7b-8ec2-937234644fdc"


minimal_event = {
    "path": "/todos",
    "httpMethod": "GET",
    # "multiValueHeaders": {},
    # "multiValueHeaders": {"a-header": ["value"]},  # or if you need the headers
    "requestContext": {
        "requestId": "227b78aa-779d-47d4-a48e-ce62120393b8"
    },  # correlation ID
}


ret = app.lambda_handler(minimal_event, LambdaContext())  # type: ignore

print(ret)

Powertools for AWS Lambda (Python) version

latest

AWS Lambda function runtime

3.8

Packaging format used

Lambda Layers

Debugging logs

No response

[heitorlessa]

Important

This only happens locally or outside Lambda, and when Data Validation feature is used

Reported on Discord -- agreed with customer to work on a fix and make a patch release tomorrow, as it blocks customers from upgrading as unit tests are failing.

Our tests don't fail because multiValueHeaders are always present in the real but fake event.

Todo:

• Update dataclasses property to not access a key directly to prevent KeyError
• Create a functional test with minimal event as described in the docs to prevent regression like this

---

functional test

def test_validate_with_minimal_event():
    # GIVEN an APIGatewayRestResolver with validation enabled
    app = APIGatewayRestResolver(enable_validation=True)

    # WHEN a handler is defined with a default scalar parameter
    @app.get("/users/<user_id>")
    def handler(user_id: int = 123):
        print(user_id)

    minimal_event = {
        "path": "/users/123",
        "httpMethod": "GET",
        "requestContext": {"requestId": "227b78aa-779d-47d4-a48e-ce62120393b8"},  # correlation ID
    }

    # THEN the handler should be invoked and return 200
    result = app(minimal_event, {})
    assert result["statusCode"] == 200

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions]

This is now released under 2.34.1 version!