feat(data-classes): fix SES date field typing and add missing fields

[michaelbrewer]

What were you trying to accomplish?

Use SESEvent with S3

• Related docs: https://docs.aws.amazon.com/ses/latest/dg/receiving-email-notifications-examples.html#receiving-email-notifications-examples-alert
• Example event: https://github.com/aws/aws-lambda-go/blob/main/events/testdata/ses-lambda-event.json
• types/aws-lambda - [aws-lambda] improve SES event DefinitelyTyped/DefinitelyTyped#50031
• https://docs.aws.amazon.com/ses/latest/APIReference/API_LambdaAction.html - includes references to topicArn
• Extend events.SimpleEmailReceiptAction fields aws/aws-lambda-go#211 - go pull request for additional fields.
• https://github.com/aws/aws-lambda-dotnet/blob/master/Libraries/src/Amazon.Lambda.SimpleEmailEvents/Actions/S3ReceiptAction.cs - DotNet lambda events also includes these fields.

Expected Behavior

• date field is set to a List[str] and not a str
• All fields should be included in the data classes

Current Behavior

Missing fields needs to be accessed via keys directly

Possible Solution

Add missing fields to SESEvent data class

Steps to Reproduce (for bugs)

Parsing regular SES message:

{
  "Records": [
    {
      "eventVersion": "1.0",
      "ses": {
        "mail": {
          "commonHeaders": {
            "from": [
              "Amazon Web Services <[email protected]>"
            ],
            "to": [
              "[email protected]"
            ],
            "returnPath": "[email protected]",
            "messageId": "<[email protected]>",
            "date": "Mon, 5 Dec 2016 18:40:08 -0800",
            "subject": "Test Subject"
          },
          "source": "[email protected]",
          "timestamp": "1970-01-01T00:00:00.123Z",
          "destination": [
            "[email protected]"
          ],
          "headers": [
            {
              "name": "Return-Path",
              "value": "<[email protected]>"
            },
            {
              "name": "Received",
              "value": "from mx.amazon.com (mx.amazon.com [127.0.0.1]) by inbound-smtp.us-east-1.amazonaws.com with SMTP id 6n4thuhcbhpfiuf25gshf70rss364fuejrvmqko1 for [email protected]; Tue, 06 Dec 2016 02:40:10 +0000 (UTC)"
            },
            {
              "name": "DKIM-Signature",
              "value": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=iatn.net; s=amazon; h=mime-version:from:date:message-id:subject:to; bh=chlJxa/vZ11+0O9lf4tKDM/CcPjup2nhhdITm+hSf3c=; b=SsoNPK0wX7umtWnw8pln3YSib+E09XO99d704QdSc1TR1HxM0OTti/UaFxVD4e5b0+okBqo3rgVeWgNZ0sWZEUhBaZwSL3kTd/nHkcPexeV0XZqEgms1vmbg75F6vlz9igWflO3GbXyTRBNMM0gUXKU/686hpVW6aryEIfM/rLY="
            },
            {
              "name": "MIME-Version",
              "value": "1.0"
            },
            {
              "name": "From",
              "value": "Amazon Web Services <[email protected]>"
            },
            {
              "name": "Date",
              "value": "Mon, 5 Dec 2016 18:40:08 -0800"
            },
            {
              "name": "Message-ID",
              "value": "<[email protected]>"
            },
            {
              "name": "Subject",
              "value": "Test Subject"
            },
            {
              "name": "To",
              "value": "[email protected]"
            },
            {
              "name": "Content-Type",
              "value": "multipart/alternative; boundary=94eb2c0742269658b10542f452a9"
            }
          ],
          "headersTruncated": false,
          "messageId": "6n4thuhcbhpfiuf25gshf70rss364fuejrvmqko1"
        },
        "receipt": {
          "recipients": [
            "[email protected]"
          ],
          "timestamp": "1970-01-01T00:00:00.123Z",
          "spamVerdict": {
            "status": "PASS"
          },
          "dkimVerdict": {
            "status": "PASS"
          },
          "dmarcVerdict": {
            "status": "PASS"
          },
          "dmarcPolicy": "reject",
          "processingTimeMillis": 574,
          "action": {
            "type": "Lambda",
            "invocationType": "Event",
            "functionArn": "arn:aws:lambda:us-east-1:000000000000:function:my-ses-lambda-function"
          },
          "spfVerdict": {
            "status": "PASS"
          },
          "virusVerdict": {
            "status": "PASS"
          }
        }
      },
      "eventSource": "aws:ses"
    }
  ]
}

Parse a test SES S3 event

{
  "Records": [
    {
      "eventVersion": "1.0",
      "ses": {
        "receipt": {
          "timestamp": "2015-09-11T20:32:33.936Z",
          "processingTimeMillis": 406,
          "recipients": [
            "[email protected]"
          ],
          "spamVerdict": {
            "status": "PASS"
          },
          "virusVerdict": {
            "status": "PASS"
          },
          "spfVerdict": {
            "status": "PASS"
          },
          "dkimVerdict": {
            "status": "PASS"
          },
          "dmarcVerdict": {
            "status": "PASS"
          },
          "dmarcPolicy": "reject",
          "action": {
            "type": "S3",
            "topicArn": "arn:aws:sns:us-east-1:012345678912:example-topic",
            "bucketName": "my-S3-bucket",
            "objectKey": "email"
          }
        },
        "mail": {
          "timestamp": "2015-09-11T20:32:33.936Z",
          "source": "0000014fbe1c09cf-7cb9f704-7531-4e53-89a1-5fa9744f5eb6-000000@amazonses.com",
          "messageId": "d6iitobk75ur44p8kdnnp7g2n800",
          "destination": [
            "[email protected]"
          ],
          "headersTruncated": false,
          "headers": [
            {
              "name": "Return-Path",
              "value": "<0000014fbe1c09cf-7cb9f704-7531-4e53-89a1-5fa9744f5eb6-000000@amazonses.com>"
            },
            {
              "name": "Received",
              "value": "from a9-183.smtp-out.amazonses.com (a9-183.smtp-out.amazonses.com [54.240.9.183]) by inbound-smtp.us-east-1.amazonaws.com with SMTP id d6iitobk75ur44p8kdnnp7g2n800 for [email protected]; Fri, 11 Sep 2015 20:32:33 +0000 (UTC)"
            },
            {
              "name": "DKIM-Signature",
              "value": "v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug; d=amazonses.com; t=1442003552; h=From:To:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Date:Message-ID:Feedback-ID; bh=DWr3IOmYWoXCA9ARqGC/UaODfghffiwFNRIb2Mckyt4=; b=p4ukUDSFqhqiub+zPR0DW1kp7oJZakrzupr6LBe6sUuvqpBkig56UzUwc29rFbJF hlX3Ov7DeYVNoN38stqwsF8ivcajXpQsXRC1cW9z8x875J041rClAjV7EGbLmudVpPX 4hHst1XPyX5wmgdHIhmUuh8oZKpVqGi6bHGzzf7g="
            },
            {
              "name": "From",
              "value": "[email protected]"
            },
            {
              "name": "To",
              "value": "[email protected]"
            },
            {
              "name": "Subject",
              "value": "Example subject"
            },
            {
              "name": "MIME-Version",
              "value": "1.0"
            },
            {
              "name": "Content-Type",
              "value": "text/plain; charset=UTF-8"
            },
            {
              "name": "Content-Transfer-Encoding",
              "value": "7bit"
            },
            {
              "name": "Date",
              "value": "Fri, 11 Sep 2015 20:32:32 +0000"
            },
            {
              "name": "Message-ID",
              "value": "<[email protected]>"
            },
            {
              "name": "X-SES-Outgoing",
              "value": "2015.09.11-54.240.9.183"
            },
            {
              "name": "Feedback-ID",
              "value": "1.us-east-1.Krv2FKpFdWV+KUYw3Qd6wcpPJ4Sv/pOPpEPSHn2u2o4=:AmazonSES"
            }
          ],
          "commonHeaders": {
            "returnPath": "0000014fbe1c09cf-7cb9f704-7531-4e53-89a1-5fa9744f5eb6-000000@amazonses.com",
            "from": [
              "[email protected]"
            ],
            "date": "Fri, 11 Sep 2015 20:32:32 +0000",
            "to": [
              "[email protected]"
            ],
            "messageId": "<[email protected]>",
            "subject": "Example subject"
          }
        }
      },
      "eventSource": "aws:ses"
    }
  ]
}

Environment

• Powertools version used: 1.25
• Packaging format (Layers, PyPi):
• AWS Lambda function runtime:
• Debugging logs

[heitorlessa]

@michaelbrewer @ran-isenberg I'm missing more information here - could you help me understand whether this is related to SES->Lambda, or a trigger to SES that then invokes Lambda with a slightly different invoke?

I appreciate the links but I'm trying to understand how you ran into this in the first place, so we can prioritize it accordingly.

We had a similar case with SNS->Lambda vs SNS->SQS->Lambda, so trying to figure out if this is a similar type of issue that we're proactively addressing vs an actual SES->Lambda that we didn't foresee previously.

Thanks a lot!

[michaelbrewer]

@heitorlessa i guess my confusion is as to why this was also added to the official GoLang AWS lambda events library.

Are these actual payloads coming to AWS lambda

[michaelbrewer]

@heitorlessa - i can split up the PR into the hotfix (typing), from the missing fields.

[michaelbrewer]

I created a new PR with simplifies the fix to only what we need for our Lambda use cases.

[michaelbrewer]

@sthulb can you clarify what this is being closed? It is a runtime error.

[michaelbrewer]

Ok I see that the related PR was merged. Thanks.

[sthulb]

Yeah, figured it was polite to close the issue when it was resolved 😀

[michaelbrewer]

Haha. I just assumed the issue was rejected.

[sthulb]

Someone I give a little too.

[github-actions]

This is now released under 1.25.10 version!