Using parser functionalities jointly with APIGatewayRestResolver

[gomboc1985]

``Hi folks,

I'm trying to use the parser functionalities jointly with APIGatewayRestResolver. The idea would be close to that of using pydantic jointly with FastAPI (see for example the code snippet here). I am looking for a boilerplate/blueprint that I might re-use.

The goal is to ensure that the body of the REST POST API looks like the one in the following example (see also class ParallelizationBody below):

{
"function_name": "slow-sum",
"input_array": [[1, 2], [3, 4], [5, 6]],
"calculation_settings": {
"invocation_type": "RequestResponse",
"sleeping_time": 5
}
}

What I tried to do is the following:

# appropriate imports here

logger = Logger(service="APP")
app = APIGatewayRestResolver()

class ParallelizationBody(BaseModel):
    function_name: str
    input_array: List[Any]
    calculation_settings: Optional[Dict]


class ParallelizationPayload(APIGatewayProxyEventV2Model):
    body: ParallelizationBody

# other stuff being done here

@logger.inject_lambda_context(
    correlation_id_path=correlation_paths.API_GATEWAY_REST, log_event=True
)
@event_parser(model=ParallelizationPayload)
def handler(event: ParallelizationPayload, context: LambdaContext):
    """
    Lambda entry point - currently supporting only:
    + one POST method for submitting calculations
    """
    return app.resolve(event, context)

I also tried using APIGatewayProxyEventModel and a couple of other tweaks but I always get a lot of validation errors. For example (and with reference to the code above):
Lambda execution failed with status 200 due to customer function error: 10 validation errors for ParallelizationPayload
version

field required (type=value_error.missing)
routeKey
field required (type=value_error.missing)
rawPath
field required (type=value_error.missing)
rawQueryString
field required (type=value_error.missing)
headers
none is not an allowed value (type=type_error.none.not_allowed)
requestContext -> routeKey
field required (type=value_error.missing)
requestContext -> time
field required (type=value_error.missing)
requestContext -> timeEpoch
field required (type=value_error.missing)
requestContext -> http
field required (type=value_error.missing)
body
value is not a valid dict (type=type_error.dict). Lambda request id: *******************
***********************: Method completed with status: 502

To reiterate the question on top: is there any blueprint or boilerplate that I might re-use?

[gomboc1985]

Updates:

1. APIGatewayProxyEventModel should be used for REST API (not the V2 version). I'd suggest to make the documentation a bit more clear.
2. Using APIGatewayProxyEventModel I get 3 validation errors:
   1. "headers": "none is not an allowed value"
   2. "multiValueHeader": "none is not an allowed value"
   3. "body": value is not a valid dict

As for the first 2 I believe there is a bug in class APIGatewayProxyEventModel (see lines 86, 87 here) in that both headers and multiValueHeaders are required, whereas these can be optional (I am allowed to test the API without providing headers):

Hence, the current version:

    headers: Dict[str, str]
    multiValueHeaders: Dict[str, List[str]]

should be changed to

    headers: Optional[Dict[str, str]]
    multiValueHeaders: Optional[Dict[str, List[str]]]

As for the third one, I am still a bit puzzled by the meaning of the error and I should dive a bit further.

[leandrodamascena]

Hi @gomboc1985!

I'll look into it tomorrow and get back to you.

Thank you

[leandrodamascena]

Hi @gomboc1985, thanks for bringing this discussion on board. I'm going to split this answer into 3 parts for easier explanation.

1 - Documentation of APIGatewayProxyEventModel
Yes, I agree with you that we have room to improve our documentation and make it explicit that this template only works with v1.

2 - multiValueHeader and Headers fields
The APIGatewayProxyEventModel Model is intended to work with RestAPI and expects to receive an event from API Gateway. Even if you don't add any header on the client side (curl/postman/others) the API Gateway will inject headers automatically and pass them to Lambda since its using a PROXY integration. So, in this case, and according to AWS Documentation the headers multiValueHeader and Headers always will be present in the request. If you use this model with an event that is not of the API Gateway v1 type, it may not work at all.

3 - The body value is not a valid dict
This error is expected if you try to convert a str field to a class. The field body returned by API Gateway is a JSON string and not a dict, so in this case, you need to convert that field to a dict before using it.

I created a code and you can see these things working together.

import json

from typing import Dict, Optional, List, Any

from aws_lambda_powertools import Logger, Tracer
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.utilities.typing import LambdaContext
from aws_lambda_powertools.utilities.parser.models import APIGatewayProxyEventModel
from aws_lambda_powertools.utilities.parser import BaseModel, parse, validator

tracer = Tracer()
logger = Logger()
app = APIGatewayRestResolver(debug=True)

class ParallelizationBody(BaseModel):
    function_name: str
    input_array: List[Any]
    calculation_settings: Optional[Dict]


class ParallelizationPayload(APIGatewayProxyEventModel):
    body: ParallelizationBody

    @validator("body", pre=True, allow_reuse=True)
    def body_str_to_dict(cls, value):
        return json.loads(value)



@app.post("/hello")
@tracer.capture_method
def post_todos():
    print(app.current_event.body)
    
    return {"hello": "word"}


@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
@tracer.capture_lambda_handler
def lambda_handler(event: ParallelizationPayload, context: LambdaContext) -> dict:
    # Use standalone parse function instead the event_parser because the decorator event_parser can't deserialize the custom class ParallelizationPayload
    # See https://awslabs.github.io/aws-lambda-powertools-python/2.5.0/utilities/parser/#parse-function
    parse_event: ParallelizationPayload = parse(event=event, model=ParallelizationPayload)
    logger.info(parse_event)

    return app.resolve(event, context)

API Call
curl --location --request POST 'https://xxxxxx.amazonaws.com/hello' --header 'Content-Type: application/json' --data-raw '{"function_name": "a","input_array": ["xxxx"]}'

Please let me know if you need any other help and we can make it work together.

Thank you

[gomboc1985]

Thanks a lot indeed! Looking into this right away!! Have a nice weekend and best regards!!

[gomboc1985]

@leandrodamascena, I've tested with curl it and it works fine. One needs to be aware though that testing it from the API Gateway management console does not work because Headers and multiValueHeader are missing in such a case (hence raising the validation error).

[leandrodamascena]

Hello @gomboc1985! Good to know it worked as expected.

@heitorlessa could you help us understand this Console question?

[heitorlessa]

Apologies, dropped the ball here. Please don't rely on API Gateway Console Test button feature as it generates an incorrect payload (API Gateway team is aware).

When API Gateway invokes a Lambda function it constructs the correct payload hence why this only happens in this case. The same occurs with validation on Source IP Address field which sends an incorrect value -- described here: #1562