Skip to content

feat(logger): flush buffer on uncaught error decorator #3676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 28, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 66 additions & 31 deletions packages/logger/src/Logger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,12 @@ import type {
import type { Context, Handler } from 'aws-lambda';
import merge from 'lodash.merge';
import { EnvironmentVariablesService } from './config/EnvironmentVariablesService.js';
import { LogJsonIndent, LogLevelThreshold, ReservedKeys } from './constants.js';
import {
LogJsonIndent,
LogLevelThreshold,
ReservedKeys,
UncaughtErrorLogMessage,
} from './constants.js';
import type { LogFormatter } from './formatter/LogFormatter.js';
import type { LogItem } from './formatter/LogItem.js';
import { PowertoolsLogFormatter } from './formatter/PowertoolsLogFormatter.js';
Expand Down Expand Up @@ -39,6 +44,7 @@ import type {
* * Capture key fields from AWS Lambda context, cold start, and structure log output as JSON
* * Append additional keys to one or all log items
* * Switch log level to `DEBUG` based on a sample rate value for a percentage of invocations
* * Ability to buffer logs in memory and flush them when there's an error
*
* After initializing the Logger class, you can use the methods to log messages at different levels.
*
Expand All @@ -54,7 +60,7 @@ import type {
* };
* ```
*
* To enrich the log items with information from the Lambda context, you can use the {@link Logger.addContext | addContext()} method.
* To enrich the log items with information from the Lambda context, you can use the {@link Logger.addContext | `addContext()`} method.
*
* @example
* ```typescript
Expand All @@ -69,7 +75,7 @@ import type {
* };
* ```
*
* You can also add additional attributes to all log items using the {@link Logger.appendKeys | appendKeys()} method.
* You can also add additional attributes to all log items using the {@link Logger.appendKeys | `appendKeys()`} method.
*
* @example
* ```typescript
Expand All @@ -82,10 +88,10 @@ import type {
* };
*```
*
* If you write your functions as classes and use TypeScript, you can use the {@link Logger.injectLambdaContext} class method decorator
* If you write your functions as classes and use TypeScript, you can use the {@link Logger.injectLambdaContext | `injectLambdaContext()`} class method decorator
* to automatically add context to your logs and clear the state after the invocation.
*
* If instead you use Middy.js middlewares, you use the {@link "middleware/middy".injectLambdaContext | injectLambdaContext()} middleware.
* If instead you use Middy.js middlewares, you use the {@link "middleware/middy".injectLambdaContext | `injectLambdaContext()`} middleware.
*
* @see https://docs.powertools.aws.dev/lambda/typescript/latest/core/logger/
*/
Expand All @@ -97,7 +103,7 @@ class Logger extends Utility implements LoggerInterface {
* full control over the output of the logs. In testing environments, we use the
* default console instance.
*
* This property is initialized in the constructor in setOptions().
* This property is initialized in the constructor in `setOptions()`.
*/
private console!: Console;
/**
Expand Down Expand Up @@ -191,7 +197,7 @@ class Logger extends Utility implements LoggerInterface {
#maxBufferBytesSize = 20480;

/**
* Contains buffered logs, grouped by _X_AMZN_TRACE_ID, each group with a max size of `maxBufferBytesSize`
* Contains buffered logs, grouped by `_X_AMZN_TRACE_ID`, each group with a max size of `maxBufferBytesSize`
*/
#buffer?: CircularMap<string>;

Expand Down Expand Up @@ -379,8 +385,8 @@ class Logger extends Utility implements LoggerInterface {
* Class method decorator that adds the current Lambda function context as extra
* information in all log items.
*
* This decorator is useful when you want to add the Lambda context to all log items
* and it works only when decorating a class method that is a Lambda function handler.
* This decorator is useful when you want to enrich your logs with information
* from the function context, such as the function name, version, and request ID, and more.
*
* @example
* ```typescript
Expand All @@ -401,7 +407,18 @@ class Logger extends Utility implements LoggerInterface {
* export const handler = handlerClass.handler.bind(handlerClass);
* ```
*
* The decorator can also be used to log the Lambda invocation event; this can be configured both via
* the `logEvent` parameter and the `POWERTOOLS_LOGGER_LOG_EVENT` environment variable. When both
* are set, the `logEvent` parameter takes precedence.
*
* Additionally, the decorator can be used to reset the temporary keys added with the `appendKeys()` method
* after the invocation, or to flush the buffer when an uncaught error is thrown in the handler.
*
* @see https://www.typescriptlang.org/docs/handbook/decorators.html#method-decorators
*
* @param options.logEvent - When `true` the logger will log the event.
* @param options.resetKeys - When `true` the logger will clear temporary keys added with {@link Logger.appendKeys() `appendKeys()`} method.
* @param options.flushBufferOnUncaughtError - When `true` the logger will flush the buffer when an uncaught error is thrown in the handler.
*/
public injectLambdaContext(
options?: InjectLambdaContextOptions
Expand All @@ -420,16 +437,24 @@ class Logger extends Utility implements LoggerInterface {
callback
) {
loggerRef.refreshSampleRateCalculation();
Logger.injectLambdaContextBefore(loggerRef, event, context, options);
loggerRef.addContext(context);
loggerRef.logEventIfEnabled(event, options?.logEvent);

let result: unknown;
try {
result = await originalMethod.apply(this, [event, context, callback]);
return await originalMethod.apply(this, [event, context, callback]);
} catch (error) {
if (options?.flushBufferOnUncaughtError) {
loggerRef.flushBuffer();
loggerRef.error({
message: UncaughtErrorLogMessage,
error,
});
}
throw error;
/* v8 ignore next */
} finally {
if (options?.clearState || options?.resetKeys) loggerRef.resetKeys();
}

return result;
};
};
}
Expand All @@ -450,7 +475,7 @@ class Logger extends Utility implements LoggerInterface {
/**
* @deprecated - This method is deprecated and will be removed in the next major version.
*/
public static injectLambdaContextBefore(
/* v8 ignore start */ public static injectLambdaContextBefore(
logger: Logger,
event: unknown,
context: Context,
Expand All @@ -463,7 +488,7 @@ class Logger extends Utility implements LoggerInterface {
shouldLogEvent = options.logEvent;
}
logger.logEventIfEnabled(event, shouldLogEvent);
}
} /* v8 ignore stop */

/**
* Log the AWS Lambda event payload for the current invocation if the environment variable `POWERTOOLS_LOGGER_LOG_EVENT` is set to `true`.
Expand Down Expand Up @@ -1239,6 +1264,11 @@ class Logger extends Utility implements LoggerInterface {
persistentKeys && this.appendPersistentKeys(persistentKeys);
}

/**
* Configure the buffer settings for the Logger instance.
*
* @param options - Options to configure the Logger instance
*/
#setLogBuffering(
options: NonNullable<ConstructorOptions['logBufferOptions']>
) {
Expand Down Expand Up @@ -1269,30 +1299,34 @@ class Logger extends Utility implements LoggerInterface {
}

/**
* Add a log to the buffer
* @param xrayTraceId - _X_AMZN_TRACE_ID of the request
* Add a log to the buffer.
*
* @param xrayTraceId - `_X_AMZN_TRACE_ID` of the request
* @param log - Log to be buffered
* @param logLevel - level of log to be buffered
* @param logLevel - The level of log to be buffered
*/
protected bufferLogItem(
xrayTraceId: string,
log: LogItem,
logLevel: number
): void {
log.prepareForPrint();

const stringified = JSON.stringify(
log.getAttributes(),
this.getJsonReplacer(),
this.logIndentation
this.#buffer?.setItem(
xrayTraceId,
JSON.stringify(
log.getAttributes(),
this.getJsonReplacer(),
this.logIndentation
),
logLevel
);

this.#buffer?.setItem(xrayTraceId, stringified, logLevel);
}

/**
* Flushes all items of the respective _X_AMZN_TRACE_ID within
* the buffer.
* Flush all logs in the request buffer.
*
* This is called automatically when you use the {@link injectLambdaContext | `@logger.injectLambdaContext()`} decorator and
* your function throws an error.
*/
public flushBuffer(): void {
const traceId = this.envVarsService.getXrayTraceId();
Expand Down Expand Up @@ -1328,9 +1362,10 @@ class Logger extends Utility implements LoggerInterface {
this.#buffer?.delete(traceId);
}
/**
* Tests if the log meets the criteria to be buffered
* @param traceId - _X_AMZN_TRACE_ID of the request
* @param logLevel - The level of the log being considered
* Test if the log meets the criteria to be buffered.
*
* @param traceId - `_X_AMZN_TRACE_ID` of the request
* @param logLevel - The level of the log being considered
*/
protected shouldBufferLog(
traceId: string | undefined,
Expand Down
14 changes: 13 additions & 1 deletion packages/logger/src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,16 @@ const ReservedKeys = [
'timestamp',
];

export { LogJsonIndent, LogLevel, LogLevelThreshold, ReservedKeys };
/**
* Message logged when an uncaught error occurs in a Lambda function.
*/
const UncaughtErrorLogMessage =
'Uncaught error detected, flushing log buffer before exit';

export {
LogJsonIndent,
LogLevel,
LogLevelThreshold,
ReservedKeys,
UncaughtErrorLogMessage,
};
8 changes: 7 additions & 1 deletion packages/logger/src/types/Logger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,12 @@ type InjectLambdaContextOptions = {
* If `true`, the logger will reset the keys added via {@link LoggerInterface.appendKeys()}
*/
resetKeys?: boolean;
/**
* Whether to flush the log buffer when an uncaught error is logged.
*
* @default `false`
*/
flushBufferOnUncaughtError?: boolean;
};

/**
Expand Down Expand Up @@ -197,7 +203,7 @@ type LogBufferOption = {
/**
* The threshold to buffer logs. Logs with a level below
* this threshold will be buffered
* @default `'DEBUG'`
* @default `DEBUG`
*/
bufferAtVerbosity?: Omit<
LogLevel,
Expand Down
Loading