Bug: ESM Require not supported use import with suggested banner.

[Xenoha]

Expected Behavior

I would expect powertools to be able to compile to ESM with included banner.

Current Behavior

{
"errorType": "ReferenceError",
"errorMessage": "require is not defined in ES module scope, you can use import instead",
"stack": [
"ReferenceError: require is not defined in ES module scope, you can use import instead",
" at ProviderService.captureHTTPsGlobal (file:///opt/nodejs/node_modules/@aws-lambda-powertools/tracer/lib/esm/provider/ProviderService.js:25:9)",
" at new Tracer (file:///opt/nodejs/node_modules/@aws-lambda-powertools/tracer/lib/esm/Tracer.js:122:27)",
" at (/infra/src/pipeline/config/lambda-fns/common/powertools.ts:24:16)",
" at ModuleJob.run (node:internal/modules/esm/module_job:222:25)",
" at async ModuleLoader.import (node:internal/modules/esm/loader:323:24)",
" at async _tryAwaitImport (file:///var/runtime/index.mjs:1008:16)",
" at async _tryRequire (file:///var/runtime/index.mjs:1057:86)",
" at async _loadUserApp (file:///var/runtime/index.mjs:1081:16)",
" at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1119:21)",
" at async start (file:///var/runtime/index.mjs:1282:23)"
]
}

Code snippet

		const powerToolsLayer = LayerVersion.fromLayerVersionArn(scope, `${id}-powertools-layer`, `arn:aws:lambda:${Stack.of(scope).region}:094274105915:layer:AWSLambdaPowertoolsTypeScriptV2:2`)

		super(scope, id, {
			...rest,
			handler: 'handler',
			runtime: Runtime.NODEJS_20_X,
			tracing: Tracing.ACTIVE,
			architecture: Architecture.ARM_64,
			timeout: Duration.seconds(30),
			layers: rest.layers ? [...rest.layers, powerToolsLayer] : [powerToolsLayer],
			environment: {
				NODE_OPTIONS: '--enable-source-maps',
				...props.environment,
			},
			logGroup: new LogGroup(scope, `${id}-log-group`, {
				logGroupName: `/aws/lambda/${id}`,
				retention: getRetention(stage),
				removalPolicy: stage === 'Prod' ? RemovalPolicy.RETAIN : RemovalPolicy.DESTROY,
			}),
			bundling: {
				minify: true,
				sourceMap: true,
				keepNames: true,
				sourcesContent: true,
				format: OutputFormat.ESM,
				mainFields: ['module', 'main'],
				esbuildArgs: { '--tree-shaking': 'true' },
				banner: "import { createRequire } from 'module';const require = createRequire(import.meta.url);",
				externalModules: ['@aws-sdk/*', '@aws-lambda-powertools/*'],
			},
		})

Steps to Reproduce

Lambda..

import { captureLambdaHandler } from '@aws-lambda-powertools/tracer/middleware'
import { injectLambdaContext } from '@aws-lambda-powertools/logger/middleware'
import { logMetrics } from '@aws-lambda-powertools/metrics/middleware'
import { logger, metrics, tracer } from './common/powertools'

import { PipelineEvent } from '../../../../../types'

const { SLACK_CHANNEL = '' } = process.env

const lambdaHandler = async (event: PipelineEvent) => {
	await sendToSlack(event)
}

const handler = middy(lambdaHandler)
	.use(captureLambdaHandler(tracer))
	.use(logMetrics(metrics, { captureColdStartMetric: true }))
	.use(injectLambdaContext(logger, { clearState: true }))

export { handler }```


### Possible Solution

any work being done on ESM bundling, or should we just switch to CJS?

### Powertools for AWS Lambda (TypeScript) version

latest

### AWS Lambda function runtime

20.x

### Packaging format used

Lambda Layers

### Execution logs

```Shell
{
    "errorType": "ReferenceError",
    "errorMessage": "require is not defined in ES module scope, you can use import instead",
    "stack": [
        "ReferenceError: require is not defined in ES module scope, you can use import instead",
        "    at ProviderService.captureHTTPsGlobal (file:///opt/nodejs/node_modules/@aws-lambda-powertools/tracer/lib/esm/provider/ProviderService.js:25:9)",
        "    at new Tracer (file:///opt/nodejs/node_modules/@aws-lambda-powertools/tracer/lib/esm/Tracer.js:122:27)",
        "    at <anonymous> (/infra/src/pipeline/config/lambda-fns/common/powertools.ts:24:16)",
        "    at ModuleJob.run (node:internal/modules/esm/module_job:222:25)",
        "    at async ModuleLoader.import (node:internal/modules/esm/loader:323:24)",
        "    at async _tryAwaitImport (file:///var/runtime/index.mjs:1008:16)",
        "    at async _tryRequire (file:///var/runtime/index.mjs:1057:86)",
        "    at async _loadUserApp (file:///var/runtime/index.mjs:1081:16)",
        "    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1119:21)",
        "    at async start (file:///var/runtime/index.mjs:1282:23)"
    ]
}

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #typescript channel on our Powertools for AWS Lambda Discord: Invite link

[dreamorosi]

Hi, could you please try again with a newer version of the Lambda layer?

In later versions we added the banner right in the layer so you shouldn't have to add it in your NodejsFunction bundling options.

[dreamorosi]

Just went to check and we added the banner in the layer in v2.0.3 which corresponds to Layer version 3.

If you get the latest (6) you should not need the banner when Tracer is being imported from the layer (which seems to be the case according to the path in your logs).

[Xenoha]

Hello @dreamorosi

Thanks for the quick reply.

Lambda Code:

const lambdaHandler: APIGatewayProxyHandlerV2WithIAMAuthorizer = async (event, context) => {
	logger.info('Context', { context })
	const { data } = await Application.query.applications({}).go()
	return {
		statusCode: 200,
		body: JSON.stringify(data),
	}
}

const handler = middy(lambdaHandler)
	.use(captureLambdaHandler(tracer))
	.use(logMetrics(metrics, { captureColdStartMetric: true }))
	.use(injectLambdaContext(logger, { clearState: true, logEvent: true }))

export { handler }

Updated CDK Construct:

const powerToolsLayer = LayerVersion.fromLayerVersionArn(scope, `${id}-powertools-layer`, `arn:aws:lambda:${Stack.of(scope).region}:094274105915:layer:AWSLambdaPowertoolsTypeScriptV2:6`)

super(scope, id, {
	...rest,
	handler: 'handler',
	runtime: Runtime.NODEJS_20_X,
	tracing: Tracing.ACTIVE,
	architecture: Architecture.ARM_64,
	timeout: Duration.seconds(30),
	layers: rest.layers ? [...rest.layers, powerToolsLayer] : [powerToolsLayer],
	environment: {
		NODE_OPTIONS: '--enable-source-maps',
		...rest.environment,
	},
	logGroup: new LogGroup(scope, `${id}-log-group`, {
		logGroupName: `/aws/lambda/${id}`,
		retention: getRetention(stage),
		removalPolicy: stage === 'Prod' ? RemovalPolicy.RETAIN : RemovalPolicy.DESTROY,
	}),
	bundling: {
		minify: true,
		sourceMap: true,
		keepNames: true,
		sourcesContent: true,
		format: OutputFormat.ESM,
		esbuildArgs: { '--tree-shaking': 'true' },
		externalModules: ['@aws-sdk/*', '@aws-lambda-powertools/*'],
	},
})

Here are the updated logs, which now have moved to include electroDB and a slightly different error on the x-ray-tracing. NOTE: This is a different function that is WAY easier to test different imports as it's in the pipeline and not in the support stack.

2024-05-15T14:18:28.303Z
INIT_START Runtime Version: nodejs:20.v22	Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:da57c20c4b965d5b75540f6865a35fc8030358e33ec44ecfed33e90901a27a72

INIT_START Runtime Version: nodejs:20.v22 Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:da57c20c4b965d5b75540f6865a35fc8030358e33ec44ecfed33e90901a27a72
2024-05-15T14:18:29.027Z
2024-05-15T14:18:29.027Z	undefined	ERROR	Uncaught Exception 	
{
    "errorType": "Error",
    "errorMessage": "Dynamic require of \"url\" is not supported",
    "stack": [
        "Error: Dynamic require of \"url\" is not supported",
        "    at file:///var/task/index.mjs:1:431",
        "    at <anonymous> (/var/asset-input/node_modules/.pnpm/[email protected]/node_modules/jsonschema/lib/validator.js:3:14)",
        "    at file:///var/task/index.mjs:1:508",
        "    at <anonymous> (/var/asset-input/node_modules/.pnpm/[email protected]/node_modules/jsonschema/lib/index.js:3:44)",
        "    at file:///var/task/index.mjs:1:508",
        "    at <anonymous> (/var/asset-input/node_modules/.pnpm/[email protected]_@[email protected]/node_modules/electrodb/src/validations.js:4:19)",
        "    at file:///var/task/index.mjs:1:508",
        "    at <anonymous> (/var/asset-input/node_modules/.pnpm/[email protected]_@[email protected]/node_modules/electrodb/src/util.js:3:11)",
        "    at file:///var/task/index.mjs:1:508",
        "    at <anonymous> (/var/asset-input/node_modules/.pnpm/[email protected]_@[email protected]/node_modules/electrodb/src/schema.js:20:11)"
    ]
}


2024-05-15T14:18:29.027Z undefined ERROR Uncaught Exception {"errorType":"Error","errorMessage":"Dynamic require of \"url\" is not supported","stack":["Error: Dynamic require of \"url\" is not supported"," at file:///var/task/index.mjs:1:431"," at <anonymous> (/var/asset-input/node_modules/.pnpm/[email protected]/node_modules/jsonschema/lib/validator.js:3:14)"," at file:///var/task/index.mjs:1:508"," at <anonymous> (/var/asset-input/node_modules/.pnpm/[email protected]/node_modules/jsonschema/lib/index.js:3:44)"," at file:///var/task/index.mjs:1:508"," at <anonymous> (/var/asset-input/node_modules/.pnpm/[email protected]_@[email protected]/node_modules/electrodb/src/validations.js:4:19)"," at file:///var/task/index.mjs:1:508"," at <anonymous> (/var/asset-input/node_modules/.pnpm/[email protected]_@[email protected]/node_modules/electrodb/src/util.js:3:11)"," at file:///var/task/index.mjs:1:508"," at <anonymous> (/var/asset-input/node_modules/.pnpm/[email protected]_@[email protected]/node_modules/electrodb/src/schema.js:20:11)"]}
2024-05-15T14:18:29.037Z
2024-05-15T14:18:29.037Z	undefined	ERROR	Error: Missing AWS Lambda trace data for X-Ray. Ensure Active Tracing is enabled and no subsegments are created outside the function handler.
    at Object.contextMissingLogError [as contextMissing] (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/context_utils.js:22:27)
    at Segment.resolveLambdaTraceData (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/env/aws_lambda.js:86:49)
    at Object.getSegment (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/context_utils.js:92:25)
    at Object.resolveSegment (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/context_utils.js:71:25)
    at captureOutgoingHTTPs (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/patchers/http_p.js:84:37)
    at Object.captureHTTPsRequest [as request] (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/patchers/http_p.js:173:16)
    at RAPIDClient._post (file:///var/runtime/index.mjs:510:33)
    at RAPIDClient.postInitError (file:///var/runtime/index.mjs:460:14)
    at Object.uncaughtException (file:///var/runtime/index.mjs:1265:14)
    at process.<anonymous> (file:///var/runtime/index.mjs:1273:20)

2024-05-15T14:18:29.037Z undefined ERROR Error: Missing AWS Lambda trace data for X-Ray. Ensure Active Tracing is enabled and no subsegments are created outside the function handler. at Object.contextMissingLogError [as contextMissing] (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/context_utils.js:22:27) at Segment.resolveLambdaTraceData (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/env/aws_lambda.js:86:49) at Object.getSegment (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/context_utils.js:92:25) at Object.resolveSegment (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/context_utils.js:71:25) at captureOutgoingHTTPs (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/patchers/http_p.js:84:37) at Object.captureHTTPsRequest [as request] (/opt/nodejs/node_modules/aws-xray-sdk-core/dist/lib/patchers/http_p.js:173:16) at RAPIDClient._post (file:///var/runtime/index.mjs:510:33) at RAPIDClient.postInitError (file:///var/runtime/index.mjs:460:14) at Object.uncaughtException (file:///var/runtime/index.mjs:1265:14) at process.<anonymous> (file:///var/runtime/index.mjs:1273:20)
2024-05-15T14:18:29.060Z
INIT_REPORT Init Duration: 757.47 ms	Phase: init	Status: error	Error Type: Runtime.ExitError

[dreamorosi]

Hi, the stack trace you just shared suggests that the error is originating from ElectroDB, which is attempting to import the url module using the require keyword, and not from Powertools. We don't use the url module at all, you can verify this by searching for it in the repo.

The second error, related to X-Ray, seems instead to be related to the fact that your code, or code that you are importing might be trying to interact with trace data outside of the handler scope. This is a known limitation of X-Ray integration with AWS Lambda (see X-Ray docs - callout titled "Important") and also not up Powertools.

[Xenoha]

Glad we confirmed the original issue as fixed in the latest Powertools release.

[github-actions]

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[dreamorosi]

In terms of ways of how to move forward, I think you have these options which I am sharing purely for information purposes:

• Try setting a banner in your function as well as using a Powertools Lambda layer that has the banner. I haven't tested this but there's a chance the banner we include in the layer will apply only to the code required under /opt/* (aka the dependencies in the Layer) and the banner you set in your bundle will apply to your code + the dependencies you are bundling. Ideally these two banners won't conflict.
• Avoid using the Powertools Layer and instead bundle Powertools together with your code & other dependencies + add back the banner into your function bundling config. This way the require polyfill might apply to both dependencies.
• Investigate if ElectroDB or its dependencies ship an ESM-compatible version, and if so, use it together with the Powertools layer.

These option might fix the first error, but not the second. For the other one you'll have to try seeing where the code is interacting with X-Ray and try removing it. If this is not an option, you could explore the option of silencing that error/log using the environment variables provided by the X-Ray SDK that you can see here (set AWS_XRAY_CONTEXT_MISSING to either LOG_ERROR or IGNORE_ERROR if right now is set to RUNTIME_ERROR).

Hope this helps, @Xenoha!

[Xenoha]

Thanks @dreamorosi

TLDR: I was working on bundler size of my lambda's and thought going to ESM would assist when I came across the first error. I can package with CJS and now have an issue, so I am going back to that as I found root cause of my bloated bundles.

Thanks for your quick response and awesome package.