Feature request: parser helper to work with DynamoDB data structure

[dreamorosi]

Use case

When working with Amazon DynamoDB Stream events, the service sends the DynamoDB items in the database-specific format rather than regular JSON objects.

For example, a string might be represented as { "S": "foo" } rather than just foo. This makes parsing the actual changes within an event hard to parse and validate since it requires customers writing a Zod schema with this in mind.

We should work on adding a helper that eases the authoring of schemas by abstracting away the transformation (unmarshalling).

Solution/User Experience

The helper will need to depend on the @aws-sdk/util-dynamodb package, which should be an optional peer dependency for the Parser utility.

For this reason, we'll need to be intentional in how we expose the helper to avoid forcing the dependency on every Parser customer.

import { unmarshall } from '@aws-sdk/util-dynamodb';

const DynamoDBMarshalled = <T extends ZodTypeAny>(schema: T) =>
  z
    .record(z.string(), z.unknown())
    .transform((str, ctx) => {
      try {
        return unmarshall(str);
      } catch (err) {
        ctx.addIssue({
          code: 'custom',
          message: 'Could not unmarshall DynamoDB stream record',
          fatal: true,
        });

        return z.NEVER;
      }
    })
    .pipe(schema);

Which would then be used as:

import { DDBUnmarshalled } from '@aws-lambda-powertools/parser/helpers/dynamodb'; // <- new sub-path export
import { DynamoDBStreamSchema } from '@aws-lambda-powertools/parser/schemas/dynamodb';
import { z } from 'zod';

const MyItemSchema = z.object({
  foo: z.string(),
  bar: z.number().optional()
})

const EventSchema = DynamoDBStreamSchema.extend({
  dynamodb: z.object({
    NewImage: DDBUnmarshalled(MyItemSchema).optional(),
  }),
})

Note that we are also introducing a new sub-path export dedicated to this feature. By exporting the helper in its own path rather than in the main @aws-lambda-powertools/parser/helpers path we can make sure that only customers who actually want to use the helper need to install the extra peer dependency.

Alternative solutions

No response

Acknowledgment

• This feature request meets Powertools for AWS Lambda (TypeScript) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[arnabrahman]

I see a help-wanted label. Can I work on this or is this already WIP? @am29d

[dreamorosi]

Hi @arnabrahman, you can work on this.

I think @am29d's working on another issue.

Thanks for asking and nice to see you here!

[github-actions]

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions]

This is now released under v2.13.1 version!