Bug: DynamoDBStreamChangedRecordModel doesn't deserialize ApproximateCreationDateTime with time components when using Pydantic V2

[waveFrontSet]

Expected Behaviour

DynamoDB Stream events with ApproximateCreationDateTime with time components should be deserialized.

Current Behaviour

In v2, pydantic raises the validation error date_from_datetime_inexact. In v1, pydantic drops time components silently when parsing timestamps into date objects.

Code snippet

from aws_lambda_powertools.utilities.parser.models import DynamoDBStreamModel

# This is an example from the official AWS documention:
# https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.Tutorial.html#Streams.Lambda.Tutorial.LambdaFunction
event = {
    "Records": [
        {
            "eventID": "7de3041dd709b024af6f29e4fa13d34c",
            "eventName": "INSERT",
            "eventVersion": "1.1",
            "eventSource": "aws:dynamodb",
            "awsRegion": "region",
            "dynamodb": {
                "ApproximateCreationDateTime": 1479499740,
                "Keys": {
                    "Timestamp": {"S": "2016-11-18:12:09:36"},
                    "Username": {"S": "John Doe"},
                },
                "NewImage": {
                    "Timestamp": {"S": "2016-11-18:12:09:36"},
                    "Message": {"S": "This is a bark from the Woofer social network"},
                    "Username": {"S": "John Doe"},
                },
                "SequenceNumber": "13021600000000001596893679",
                "SizeBytes": 112,
                "StreamViewType": "NEW_IMAGE",
            },
            "eventSourceARN": "arn:aws:dynamodb:region:account ID:table/BarkTable/stream/2016-11-16T20:42:48.104",
        }
    ]
}
DynamoDBStreamModel.model_validate(event)

Possible Solution

The simplest solution probably is to promote ApproximateCreationDateTime to an Optional[datetime] instead of Optional[date].
The test events in dynamoStreamEvent.json don't contain the attribute which is probably why this wasn't caught during the pydantic v2 upgrade.

Steps to Reproduce

1. Put the code snippet into a Python script.
2. Install latest aws-lambda-powertools + pydantic v2 versions.
3. Execute the Python script.

Powertools for AWS Lambda (Python) version

latest

AWS Lambda function runtime

3.11

Packaging format used

PyPi

Debugging logs

./python-3.11.4/lib/python3.11/site-packages/aws_lambda_powertools/package_logger.py:20: UserWarning: POWERTOOLS_DEBUG environment variable is enabled. Setting logging level to DEBUG.
  if powertools_debug_is_set():
Traceback (most recent call last):
  File "dynamodb_fail.py", line 30, in <module>
    DynamoDBStreamModel.model_validate(event)
  File "./python-3.11.4/lib/python3.11/site-packages/pydantic/main.py", line 504, in model_validate
    return cls.__pydantic_validator__.validate_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for DynamoDBStreamModel
Records.0.dynamodb.ApproximateCreationDateTime
  Datetimes provided to dates should have zero time - e.g. be exact dates [type=date_from_datetime_inexact, input_value=1479499740, input_type=int]
    For further information visit https://errors.pydantic.dev/2.3/v/date_from_datetime_inexact

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[leandrodamascena]

Hey @waveFrontSet! Thanks for reporting this possible bug. I'm looking at this now.

[leandrodamascena]

Hi @waveFrontSet! Yes, I can confirm that it is a bug and we need to fix it. Promoting the field to datetime seems like the right way to go, since it's an epoch timestamp with date+time. Even if we change the behavior by changing the field to datetime, we must do this, otherwise, clients may rely on inconsistent data.

I completely agree that we need to fix our tests to test this field, which is not happening today. So do you want to send a PR for this? Do you want me to fix it myself? We plan to launch a release on Friday, but if this is too urgent for you, we may be able to move this up to tomorrow.

Thanks.

[waveFrontSet]

Hi @leandrodamascena,
thanks so much for your blazingly fast response!
I'll gladly give it a shot and send a PR. Regarding the release: Friday definitely is early enough. 👍

[leandrodamascena]

Great @waveFrontSet! When you send the PR I can review and merge it.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.