Bug: APIGatewayHttpResolver fails with ValueError: [TypeError('cannot convert dictionary update sequence element #0 to a sequence'), TypeError('vars() argument must have __dict__ attribute')]

[toddcooke]

Expected Behaviour

I expect the APIGatewayHttpResolver to route my request to the proper annotated function, eg:

@app.get("/something")
def something():
    # does something...

Current Behaviour

For some requests, I get the error ValueError: [TypeError('cannot convert dictionary update sequence element #0 to a sequence'), TypeError('vars() argument must have __dict__ attribute')]

I'm using the serverless framework to package and deploy my app. I'm also using the serverless-python-requirements plugin to package my app.

See attached snippet for full traceback.

Code snippet

ANY /teams (λ: handler)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/openapi/encoders.py", line 272, in _dump_other
backend-1  |
    data = dict(obj)
           ^^^^^^^^^
TypeError: cannot convert dictionary update sequence element #0 to a sequence
backend-1  |
During handling of the above exception, another exception occurred:
backend-1  |
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/openapi/encoders.py", line 276, in _dump_other
    data = vars(obj)
           ^^^^^^^^^
TypeError: vars() argument must have __dict__ attribute
backend-1  |
The above exception was the direct cause of the following exception:
backend-1  |
Traceback (most recent call last):
  File "/app/node_modules/serverless-offline/src/lambda/handler-runner/python-runner/invoke.py", line 97, in <module>
backend-1  |
    result = handler(input['event'], context)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/tracing/tracer.py", line 317, in decorate
    response = lambda_handler(event, context, **kwargs)
               ^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/app/handlers.py", line 192, in handler
    return app.resolve(event, context)
           ^^^^^^^^^^^^^^^^^^^
^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/api_gateway.py", line 1918, in resolve
backend-1  |
    response = self._resolve().build(self.current_event, self._cors)
               ^^^^^^^^^^^^^^
^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/api_gateway.py", line 2025, in _resolve
backend-1  |
    return self._call_route(route, route_keys)  # pass fn args
backend-1  |
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/api_gateway.py", line 2103, in _call_route
backend-1  |
    route(router_middlewares=self._router_middlewares, app=self, route_arguments=route_arguments),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/api_gateway.py", line 407, in __call__
backend-1  |
    return self._middleware_stack(app)
backend-1  |
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/api_gateway.py", line 1314, in __call__
backend-1  |
    return self.current_middleware(app, self.next_middleware)
backend-1  |
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/middlewares/base.py", line 121, in __call__
    return self.handler(app, next_middleware)
backend-1  |
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/middlewares/openapi_validation.py", line 133, in handler
    return self._handle_response(route=route, response=response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/middlewares/openapi_validation.py", line 140, in _handle_response
    response.body = self._serialize_response(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/middlewares/openapi_validation.py", line 200, in _serialize_response
    return jsonable_encoder(response_content, custom_serializer=self._validation_serializer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/openapi/encoders.py", line 122, in jsonable_encoder
backend-1  |
    return _dump_sequence(
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/openapi/encoders.py", line 245, in _dump_sequence
    jsonable_encoder(
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/openapi/encoders.py", line 145, in jsonable_encoder
    return _dump_other(
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aws_lambda_powertools/event_handler/openapi/encoders.py", line 279, in _dump_other
    raise ValueError(errors) from e
ValueError: [TypeError('cannot convert dictionary update sequence element #0 to a sequence'), TypeError('vars() argument must have __dict__ attribute')]

### Possible Solution

_No response_

### Steps to Reproduce

My project is not easy to reproduce locally and I can't share it publicly, but I would be OK with sharing my screen or sending certain files privately to powertools-lambda-python contributors.

### Powertools for AWS Lambda (Python) version

latest

### AWS Lambda function runtime

3.11

### Packaging format used

PyPi

### Debugging logs

_No response_

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[leandrodamascena]

Hi @toddcooke, thanks for opening this issue and reporting what might be a bug. I'm looking at the traceback and I can imagine some situations where this could happen, but I may not be able to reproduce the exact problem.

Can you share some minimal reproducible code or stack, please? This would help a lot in identifying exactly the problem.

[toddcooke]

@leandrodamascena I've added you as a collaborator to a private repo which reproduces the issue. The steps to reproduce are in the readme

[leandrodamascena]

Hey @toddcooke! Thanks for sharing the repository with the minimal reproducible code. I enjoyed debugging the code using SQLAlchemy. This reminds me of the time I had to maintain a system that was a mix of Django, Django ORM, SQLAlchemy, and other technologies. It was an interesting experience!

The issue you're facing is that you're trying to return a list of app.models.Te... objects, which are SQLAlchemy models. Currently, we don't support serializing this kind of SQLAlchemy models out of the box. To resolve this, you have a couple of options:

1. Serialize the objects to JSON inside your model and return this as JSON to your function/route.

2. You can define a custom serializer function and pass it to the APIGatewayHttpResolverconstructor. This function should take the object(s) you want to serialize and return a JSON object. For example:

class DataclassCustomEncoder(JSONEncoder):
    def default(self, obj):
        return asdict(obj) if is_dataclass(obj) else super().default(obj)

def custom_serializer(obj) -> str:
    return json.dumps(obj, separators=(",", ":"), cls=DataclassCustomEncoder)

app = APIGatewayRestResolver(serializer=custom_serializer)

This code might fail because you may have other classes and this serialize won't be able to serialize. You need to change this code to match your classes.

Event Handler utility supports serializing objects of various types, including scalar types (e.g., strings, numbers), Pydantic models, dataclasses, Enum, PurePath objects, dictionaries, sequences (lists, tuples), and few others. However, it currently does not support serializing SQLAlchemy models out of the box.

We currently have no plans to implement native SQLAlchemy model serialization. The reason for this is that SQLAlchemy models can have complex relationships and lazy-loading behavior, which can make serialization non-trivial and potentially lead to performance issues or unexpected behavior.

Thanks and please let me know if you any other question or I can close this issue.

[toddcooke]

Gotcha! If I may make a suggestion, it would be helpful to say explicitly on the powertools docs site that sqlalchemy is not supported.

Thanks for figuring out this issue!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[leandrodamascena]

Hi @toddcooke! I'm reopening this issue because it looks like we have room to improve the customer experience.

If you and I have ever spent time trying to figure out behavior that isn't very clear on the Powertools side, it means something needs to be improved. I see two things we need to improve:

1 - We need to place this code in a try/except block and raise an error with a clear message saying that we do not support serialization of this type of object.
2 - Since sqlalchemy is widely used in the Python community, add a message saying that we do not serialize sqlalchemy models.

Does this make sense to you? Would you like to add anything else?

Thanks

[toddcooke]

@leandrodamascena Those both sound like excellent additions, I think that would cover it.

[heitorlessa]

Coming out next Thursday (13th) ;)

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions]

This is now released under 2.39.0 version!