Bug: Logger(json_serializer=json.dumps) cannot log subclassed exceptions

[fraser-langton]

Expected Behaviour

Traceback (most recent call last):
  File "/Users/.../temp.py", line 12, in <module>
    raise MyException
MyException

Current Behaviour

--- Logging error ---
Traceback (most recent call last):
  File "/path/to/project/temp.py", line 12, in <module>
    raise MyException
MyException

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/path/to/python/lib/python3.12/logging/__init__.py", line 1160, in emit
    msg = self.format(record)
          ^^^^^^^^^^^^^^^^^^^
  File "/path/to/python/lib/python3.12/logging/__init__.py", line 999, in format
    return fmt.format(record)
           ^^^^^^^^^^^^^^^^^^
  File "/path/to/project/.venv/lib/python3.12/site-packages/aws_lambda_powertools/logging/formatter.py", line 201, in format
    return self.serialize(log=formatted_log)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/project/.venv/lib/python3.12/site-packages/aws_lambda_powertools/logging/formatter.py", line 183, in serialize
    return self.json_serializer(log)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/python/lib/python3.12/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/python/lib/python3.12/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/python/lib/python3.12/json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
  File "/path/to/python/lib/python3.12/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type MyException is not JSON serializable
Call stack:
  File "/path/to/project/temp.py", line 14, in <module>
    logger.exception("Exception occurred")
  File "/path/to/project/.venv/lib/python3.12/site-packages/aws_lambda_powertools/logging/logger.py", line 508, in exception
    return self._logger.exception(
Message: 'Exception occurred'
Arguments: ()

Code snippet

from aws_lambda_powertools import Logger

logger = Logger()

class MyException(Exception): ...

try:
    raise MyException
except MyException:
    logger.exception("Exception occurred")

Possible Solution

No response

Steps to Reproduce

import json

from aws_lambda_powertools import Logger

logger = Logger(json_serializer=json.dumps)


class MyException(Exception): ...


try:
    raise MyException
except MyException:
    logger.exception("Exception occurred")

Powertools for AWS Lambda (Python) version

sentry-sdk==2.22.0

AWS Lambda function runtime

3.11

Packaging format used

Lambda Layers

Debugging logs

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[leandrodamascena]

Hey @fraser-langton, thanks for opening this issue! When using Powertools without a custom serializer, we can automatically serialize exceptions and return a formatted JSON. However, when you're directly using json.dumps to serialize custom exceptions like MyException, it fails because json.dumps cannot encode objects it doesn't inherently understand. The MyException object is not JSON serializable by default. You can check a list of objects that json module encode/decode by default here.

I'm curious about your specific requirements for the JSON serializer. Could you provide more context about what functionality you're looking to implement that isn't currently working with Powertools?

Just in case If you want to do this, you must create a CustomJSONEncoder that understand this object and serialize it. Something like this:

import json
import traceback

from aws_lambda_powertools import Logger

class CustomJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        # Handle exceptions
        if isinstance(obj, BaseException):
            return {
                'type': type(obj).__name__,
                'message': str(obj),
                'traceback': traceback.format_exc()
            }
        
        # Handle objects with __dict__
        if hasattr(obj, '__dict__'):
            return obj.__dict__
        
        # For any other non-serializable objects, convert to string
        return str(obj)

# Custom JSON serializer using the custom encoder
def custom_json_serializer(obj):
    return json.dumps(obj, cls=CustomJSONEncoder)

logger = Logger(json_serializer=custom_json_serializer)

class MyException(Exception): ...

try:
    raise MyException
except MyException:
    logger.exception("Exception occurred")

Thanks.

[fraser-langton]

Ok thanks for the added context, my assumption was the Exception was converted to dict form first regardless of the serialiser. We incorrectly assumed json dumps was the default as we have a different serialiser for different environments so we just reverted to using the default.

Thanks you!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.