RFC: Testing Factories for AWS Data Objects

[simonireilly]

ref https://github.com/awslabs/aws-lambda-powertools-typescript/issues/26for strucutre of my RFC

Description of the proposal

There should be a consistent way to produce test fixtures for AWS lambda environments

---

Name of the core utility (and consequently the folders that contain the libraries):

• packages/factories

Justification

When it comes to making effective use of AWS in unit and integration testing, developers will create data objects which have the interfaces of the 'aws-lambda' types package.

This is true, even within this project;

• Here, for testing the SQS event format is compatible with a function interface: https://github.com/awslabs/aws-lambda-powertools-typescript/pull/364/files#r780652925
• Here, where a collection of untyped JSON is used for testing fixtures: https://github.com/awslabs/aws-lambda-powertools-typescript/tree/main/tests/resources/events/aws

These untyped objects become a cause for churn, and often because a single source of truth is not present, they can misdiagnose behaviors through simple omission of fields.

Goals

• Provide users of the nodejs lambda platform effective utilities for testing lambda handler
• Provide consistent type-safe factories for AWS data structures and events
• Support version of event formats to test utilities against different versions

Proposed API

There is an example here that utilizes faker and fishery to create type compliant factories that can be deep merged to allow for standard testing objects:

• https://github.com/simonireilly/aws-cdk-utils/tree/main/packages/factories

Installation:

yarn add --dev @aws-lambda-powertools/factories
npm i -d @aws-lambda-powertools/factories

Usage

Consider the following test that is written in #364

test('when called, it set the correct segment', () => {
  // Prepare
  const provider: ProviderService = new ProviderService();
  const record: SQSRecord = {
    messageId: 'fd95260b-1600-4028-b252-590cfcc9fe6d',
    receiptHandle: 'test',
    body: 'Information about current NY Times fiction bestseller for week of 12/11/2016.',
    attributes: {
      ApproximateReceiveCount: '1',
      AWSTraceHeader: 'Root=1-61cc1005-53ff3b575736e3c74eae6bfb;Parent=1f57c53badf96998;Sampled=1',
      SentTimestamp: '1640763398126',
      SenderId: 'AROAT26JIZQWSOCCOUNE5:sqsProducer',
      ApproximateFirstReceiveTimestamp: '1640763398127',
    },
    messageAttributes: {},
    md5OfBody: 'bbdc5fdb8be7251f5c910905db994bab',
    eventSource: 'aws:sqs',
    eventSourceARN:
      'arn:aws:sqs:eu-west-1:123456789012:queue',
    awsRegion: 'eu-west-1',
  };

  const context = ContextExamples.helloworldContext;

  // Act
  provider.continueSQSRecordTrace(record, context);

  // Assess
  expect(setSegment).toHaveBeenCalledTimes(1);
});

This test becomes:

import factories from '@aws-lambda-powertools/factories';


test('when called, it set the correct segment', () => {
  // Prepare
  const provider: ProviderService = new ProviderService();
  const record = factories.sqs.record.build();
  const context = factories.sqs.context.build();

  // Act
  provider.continueSQSRecordTrace(record, context);

  // Assess
  expect(setSegment).toHaveBeenCalledTimes(1);
});

[michaelbrewer]

@heitorlessa it is worth thinking about this across both Python, Java and Typescript (aws-powertools/powertools-lambda-python#1169)

[saragerion]

Thanks for opening this issue @simonireilly. As a former AWS customer, I would like to say that this is something I wanted myself as well, back then.
The team and I will get back to you on this one.

[willfarrell]

I've been using @serverless/event-mocks for this, there are a few others, all are out of date and not maintained.

[michaelbrewer]

For Python is have a lot of mocks and sample events. So whichever solution we have in mind it would be great to has the potential for reuse across python, typescript, Java and .Net.

[driimus]

If any ESM users are looking for an utility similar to what's being proposed here:

I've been shipping my own set of test data factories for Lambda events, built using the same combo (fishery + @faker-js/faker): https://www.npmjs.com/package/@driimus/aws-event-factory,

[dreamorosi]

Hello,

I discussed this with the team and we came to the conclusion that at lest for the time being, we are not positioned to help with this type of use case in the way we would feel confident doing so.

Our position stems from the fact that even if we took the time to hand-roll data objects for each integration and variation of event that can trigger a Lambda function, we would not be able to detect changes to these interfaces on actual AWS resources. This is because as of today there's no centralized repository of such events, and we simply don't have capacity to deploy and maintain an environment with all these combinations deployed.

Because of this, we won't be moving forward with this for the time being. We do however provide Zod schemas for some popular payloads for Lambda integrations, which you can use in conjunction with 3rd packages like this one to generate an approximation of event payload to test your function.

We're also open to revisit the decision in the feature if circumstances change, but until then I'll close the issue as not planned.

[github-actions]

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.