Bug: API Gateway Console Test button fails validation with @parser envelope ApiGatewayEnvelope because headers & multiValueHeaders are null

[Cihaan]

Expected Behavior

We should be able to generate a test form the API Gateway console that allows the ApiGatewayProxyEventModel to be validated and parsed by Zod.

Current Behavior

When generating a test from the API Gateway console, the value for the headers and multiValueHeaders in requestContext -> null. Since this value is defined to be an optionnal but not nullable here And so test events from the API Gateway console fail with a 502 error because of the resulting Zod ValidationError.

Code snippet

  RestApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: !Ref Environment
      Name: !Sub '${ProjectName}-${Environment}-api'
      OpenApiVersion: '3.0.1' 
      TracingEnabled: true
      EndpointConfiguration: REGIONAL
      Auth: 
        DefaultAuthorizer: NONE
        AddDefaultAuthorizerToCorsPreflight: false
        ApiKeyRequired: false
      Cors:
        AllowMethods: "'DELETE,GET,HEAD,OPTIONS,POST,PUT'"
        AllowHeaders: "'*'"
        AllowOrigin: "'*'"
      MethodSettings:
        - HttpMethod: '*'
          ResourcePath: '/*'
          LoggingLevel: INFO
          DataTraceEnabled: true
          MetricsEnabled: true
          ThrottlingBurstLimit: 500
          ThrottlingRateLimit: 1000
      AccessLogSetting:
        DestinationArn: !GetAtt ApiAccessLogGroup.Arn
        Format: >-
          {
          "status": "$context.status",
          "traceId": "$context.xrayTraceId",
          "requestId": "$context.requestId",
          "requestTime": "$context.requestTime",
          "httpMethod": "$context.httpMethod",
          "responseLength": "$context.responseLength",
          "resourcePath": "$context.resourcePath",
          "resourceId": "$context.resourceId",
          "sourceIp": "$context.identity.sourceIp",
          "userAgent": "$context.identity.userAgent",
          "accountId": "$context.identity.accountId",
          "caller": "$context.identity.caller",
          "user": "$context.identity.user",
          "userArn": "$context.identity.userArn"
          }
      Tags:
        APPLI: !Ref ApplicationName
        ProjectName: !Ref ProjectName
        ENV: !Ref Environment
        Owner: !Ref Owner

// Lambda parses input and returns back the metadata field with a 200 status code
  UploadFileFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub '${ProjectName}-${Environment}-upload-file'
      CodeUri: ../../../dist/packages/backend/upload-file/
      CodeSigningConfigArn: !If [IsSandbox, !Ref AWS::NoValue, !Ref SignedCodeSigningConfigArn]
      Events:
        Api:
          Type: Api
          Properties:
            Path: /upload
            Method: POST
            RestApiId: !Ref RestApi
      Environment:
        Variables:
          BUCKET_NAME: !Ref S3Bucket
      Policies:
        - S3ReadPolicy:
            BucketName: !Ref S3Bucket
        - S3WritePolicy:
            BucketName: !Ref S3Bucket
    Metadata:
      BuildMethod: esbuild
      BuildProperties:
        EntryPoints:
          - index.js

// index.ts
import type { LambdaInterface } from '@aws-lambda-powertools/commons/types';
import { parser } from '@aws-lambda-powertools/parser';
import { ApiGatewayEnvelope } from '@aws-lambda-powertools/parser/envelopes';
import { Tracer } from '@aws-lambda-powertools/tracer';
import Logger from '@fec-module/logger';
import { z } from 'zod';

import { uploadFileSchema } from 'packages/backend/src/upload-file/schema';

const tracer = new Tracer();

type UploadInput = z.infer<typeof uploadFileSchema>;

type UploadResponse = {
  cookies?: string[];
  isBase64Encoded?: true | false;
  statusCode: number;
  headers?: { [header: string]: string | number | boolean };
  body?: string;
};

class Lambda implements LambdaInterface {
  @Logger.injectLambdaContext({ logEvent: true })
  @tracer.captureLambdaHandler()
  @parser({ schema: uploadFileSchema, envelope: ApiGatewayEnvelope })
  public async handler(event: UploadInput): Promise<UploadResponse> {
    Logger.info('Received context', event);

    return {
      statusCode: 200,
      body: JSON.stringify({
        message: 'Request Received successfully with body',
        metadata: event.metadata,
      }),
    };
  }
}

// schema.ts
const uploadFileSchema = z.object({
  metadata: z.object({
    source_system: z.string(),
    category: z.string(),
    filename: z.string(),
    display_name: z.string().optional(),
    external_id: z.string().optional(),
    description: z.string().optional(),
    ttl: z.number().optional(),
    application_metadata: z.record(z.unknown()).optional(),
  }),
});

Steps to Reproduce

• deploy the Api Gateway and the Lambda using SAM
• go to the Test tab of the /upload route in the Api Gateway AWS Console
• place following headers :

{
  "metadata": {
    "source_system": "source_system",
    "category": "category",
    "filename": "filename",
    "display_name": "display_name",
    "external_id": "external_id",
    "description": "descrition",
    "ttl": 1,
    "application_metadata": {
        "other": "other"
    }
  }
}

• run test

Possible Solution

set dummy headers in the UI for testing.
treat it as edge case and mention that the AWS Console ApiGateway UI in moody in the documentation.

Powertools for AWS Lambda (TypeScript) version

latest

AWS Lambda function runtime

20.x

Packaging format used

npm

Execution logs

No response

[dreamorosi]

Thank you for taking the time to open this issue.

As I mentioned on Discord from a preliminary investigation the issue seems to be due to another quirk of API Gateway's Test UI, just like #2526.

I'll need to investigate more to see if these two fields can be null only when using this feature or also in other cases. If it happens only with the Test UI then I'm inclined to agree with what you said and treat this as an edge case, meaning that we won't make the fields .nullable() nor .optional() but always expect an object (even if empty).

If we go that direction, then I'll add a callout to the docs to record this for future customers.

[Cihaan]

sounds good!

[dreamorosi]

Hi, just wanted to update that I am still waiting for a conclusive answer on the topic. I have however identified the right team to ask the question so hopefully we'll get more clarity soon

Thank you for your patience!

[dreamorosi]

We haven't heard back from the team yet - for now we have marked the fields as .nullish() in the schemas as per #2624, which also addresses this issue.

[github-actions]

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.