Bug: APIGatewayHttpResolver with validation enabled returns HTTP 422 when using single value query parameters

[DariusKunce]

Expected Behaviour

When using Amazon Api Gateway REST API to invoke an AWS Lambda using proxy integration, calling API with a single query parameter should return HTTP 200 with Lambda Powertools validation enabled.

This is also how it is documented here: https://docs.powertools.aws.dev/lambda/python/latest/core/event_handler/api_gateway/#validating-query-strings

Current Behaviour

Calling API with GET /my-path?productType=Category1 returns HTTP 422:

{
    "statusCode": 422,
    "detail": [{
        "loc": ["query", "productType"], 
        "type": "type_error.str"
    }]
}

Code snippet

Bug can be reproduced with this code inside a Lambda function:


app = api_gateway.APIGatewayRestResolver(
    enable_validation=True
)

@app.get("/my-path")
def my_handler(
    product_type: Annotated[str | None, Query(alias="productType")] = None,
) -> api_gateway.Response[api_model.GetAvailableProductsResponse]:
    ...

Possible Solution

Can be fixed by changing query param type to list:

@app.get("/my-path")
def my_handler(
    product_type: Annotated[list[str] | None, Query(alias="productType")] = None,
) -> api_gateway.Response[api_model.GetAvailableProductsResponse]:
    """Lists available products"""
    ...

Steps to Reproduce

In AWS:

1. Create a REST API in the Amazon Api Gateway Service.
2. Create a Lambda function with Lambda Powertools configured as in the code examples.
3. Configure the API Gateway with proxy integration to the Lambda function.
4. Invoke the API with a query parameter.

Locally:

1. Create unit tests for a Lambda function as described here: https://docs.powertools.aws.dev/lambda/python/latest/core/event_handler/api_gateway/#testing-your-code
2. Run the test using REST API proxy integration payload. This payload always contains 2 attributes for the query string parameters, regardless if it's single value or multi value in REST API proxy integration:

   {
       ...
       "queryStringParameters": { "productType": "Category1" },
       "multiValueQueryStringParameters": { "productType": ["Category1"] }
   }

The issue is this line of code: https://github.com/aws-powertools/powertools-lambda-python/blob/develop/aws_lambda_powertools/utilities/data_classes/api_gateway_proxy_event.py#L123

Lambda Powertools should additionally check dictionary values in multiValueQueryStringParameters if they contain only one value, and validate accordingly to be of type str instead of list[str].

Powertools for AWS Lambda (Python) version

2.33.0

AWS Lambda function runtime

3.12

Packaging format used

PyPi

Debugging logs

No response

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[sthulb]

Hi @DariusKunce this is on my list for verification tomorrow, thanks for submitting it!

[sthulb]

@DariusKunce Confirmed! We'll add this into the next sprint to fix

def test_weird_issue():
    # GIVEN a Http API V2 proxy type event
    app = APIGatewayRestResolver(enable_validation=True)

    class FunkyTown(BaseModel):
        parameter: str

    @app.get("/my/path")
    def my_path(
        parameter: Annotated[str | None, Query(alias="parameter1")] = None,
    ) -> Response[FunkyTown]:
        assert isinstance(app.current_event, APIGatewayProxyEvent)

        assert parameter == "value"
        return Response(200, content_types.APPLICATION_JSON, FunkyTown(parameter=parameter))

    result = app(load_event("apiGatewayProxyEvent.json"), {})

    print(result)
    assert result["statusCode"] == 200

[heitorlessa]

@DariusKunce we're aiming to make a release tomorrow with this fix ahead of schedule -- let us know if you're blocked by this and we can make it today.

[jpoissant]

The issue isn't related to the alias but the fact that the openapi_validation.py handles all query strings with _normalize_multi_query_string_with_param and takes the first value of the item.

With "queryStringParameters": { "productType": "Category1" } it's normalize to: "productType": "C"

With "multiValueQueryStringParameters": { "productType": ["Category1"] } it's normalize correctly to: "productType": "Category1"

I was able to find this as I was trying to validate the min length of a query string parameter (def get_hello(fname: Annotated[Optional[str], Query(min_length=4)] = None) -> HelloWorld: and fname was always too short... as it was always truncated to 1 character.

The line truncating the value is here

[heitorlessa]

Thanks a lot for the timely comment @jpoissant.

@rubenfonseca i think we had mixed messaging and this wasn't fixed -- if you can't look into it tomorrow morning before release, let me know pretty plz

[rubenfonseca]

Thank you so much @jpoissant I'm looking at this right now

[rubenfonseca]

The issue isn't related to the alias but the fact that the openapi_validation.py handles all query strings with _normalize_multi_query_string_with_param and takes the first value of the item.

With "queryStringParameters": { "productType": "Category1" } it's normalize to: "productType": "C"

With "multiValueQueryStringParameters": { "productType": ["Category1"] } it's normalize correctly to: "productType": "Category1"

@jpoissant I need to understand the problem a little bit better, since I'm unable to reproduce the problem. Can you please confirm which event handler are you using? Is it the APIGatewayRestResolver ?

Reason I'm asking is that the REST API Gateway will always send multiValueQueryStringParameters, so the existing code already works correctly.

So I need to understand what scenario do you have where you only have queryStringParameters. Can you please clarify?

[jpoissant]

@rubenfonseca I am using the APIGatewayHttpResolver resolver. I just tried with the APIGatewayRestResolver and it work. So the bug is only related to the API Gateway v2 proxy payload format

[jpoissant]

@rubenfonseca would it be better to open a new bug for the APIGatewayHttpResolver resolver?

[rubenfonseca]

No that helps, let me investigate today and I'll get back to you today!

[rubenfonseca]

Got it, was able to reproduce the problem! Fix incoming

[heitorlessa]

Coming up in this week's release! Just merged Ruben's PR

[heitorlessa]

We're making a release today after we push a non-related and last bugfix for this release 🚀

[github-actions]

This is now released under 2.34.0 version!