Skip to content

feat(idempotency): add idempotency decorator #1723

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Sep 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/snippets/idempotency/idempotentDecoratorBase.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import type { Context } from 'aws-lambda';
import { LambdaInterface } from '@aws-lambda-powertools/commons';
import {
IdempotencyConfig,
idempotent,
} from '@aws-lambda-powertools/idempotency';
import { DynamoDBPersistenceLayer } from '@aws-lambda-powertools/idempotency/dynamodb';
import { Request, Response } from './types';

const dynamoDBPersistenceLayer = new DynamoDBPersistenceLayer({
tableName: 'idempotencyTableName',
});

const config = new IdempotencyConfig({});

class MyLambda implements LambdaInterface {
@idempotent({ persistenceStore: dynamoDBPersistenceLayer, config: config })
public async handler(_event: Request, _context: Context): Promise<Response> {
// ... process your event
return {
message: 'success',
statusCode: 200,
};
}
}

const defaultLambda = new MyLambda();
export const handler = defaultLambda.handler.bind(defaultLambda);
16 changes: 16 additions & 0 deletions docs/utilities/idempotency.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,22 @@ When using `makeIdempotent` on arbitrary functions, you can tell us which argume

The function this example has two arguments, note that while wrapping it with the `makeIdempotent` high-order function, we specify the `dataIndexArgument` as `1` to tell the decorator that the second argument is the one that contains the data we should use to make the function idempotent. Remember that arguments are zero-indexed, so the first argument is `0`, the second is `1`, and so on.

### Idempotent Decorator

You can also use the `@idempotent` decorator to make your Lambda handler idempotent, similar to the `makeIdempotent` function wrapper.

=== "index.ts"

```typescript hl_lines="17"
--8<-- "docs/snippets/idempotency/idempotentDecoratorBase.ts"
```

=== "types.ts"

```typescript

You can use the decorator on your Lambda handler or on any function that returns a response to make it idempotent. This is useful when you want to make a specific logic idempotent, for example when your Lambda handler performs multiple side effects and you only want to make a specific one idempotent.
The configuration options for the `@idempotent` decorator are the same as the ones for the `makeIdempotent` function wrapper.

### MakeHandlerIdempotent Middy middleware

Expand Down
66 changes: 65 additions & 1 deletion packages/idempotency/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ You can use the package in both TypeScript and JavaScript code bases.
- [Key features](#key-features)
- [Usage](#usage)
- [Function wrapper](#function-wrapper)
- [Decorator](#decorator)
- [Middy middleware](#middy-middleware)
- [DynamoDB persistence layer](#dynamodb-persistence-layer)
- [Contribute](#contribute)
Expand All @@ -24,7 +25,7 @@ You can use the package in both TypeScript and JavaScript code bases.
## Intro

This package provides a utility to implement idempotency in your Lambda functions.
You can either use it to wrap a function, or as Middy middleware to make your AWS Lambda handler idempotent.
You can either use it to wrap a function, decorate a function, or as Middy middleware to make your AWS Lambda handler idempotent.

The current implementation provides a persistence layer for Amazon DynamoDB, which offers a variety of configuration options. You can also bring your own persistence layer by extending the `BasePersistenceLayer` class.

Expand Down Expand Up @@ -163,6 +164,69 @@ export const handler = makeIdempotent(myHandler, {

Check the [docs](https://docs.powertools.aws.dev/lambda/typescript/latest/utilities/idempotency/) for more examples.

### Decorator

You can make any function idempotent, and safe to retry, by decorating it using the `@idempotent` decorator.

```ts
import { idempotent } from '@aws-lambda-powertools/idempotency';
import { LambdaInterface } from '@aws-lambda-powertools/commons';
import { DynamoDBPersistenceLayer } from '@aws-lambda-powertools/idempotency/dynamodb';
import type { Context, APIGatewayProxyEvent } from 'aws-lambda';

const persistenceStore = new DynamoDBPersistenceLayer({
tableName: 'idempotencyTableName',
});

class MyHandler extends LambdaInterface {
@idempotent({ persistenceStore: dynamoDBPersistenceLayer })
public async handler(
event: APIGatewayProxyEvent,
context: Context
): Promise<void> {
// your code goes here here
}
}

const handlerClass = new MyHandler();
export const handler = handlerClass.handler.bind(handlerClass);
```

Using the same decorator, you can also make any other arbitrary function idempotent.

```ts
import { idempotent } from '@aws-lambda-powertools/idempotency';
import { LambdaInterface } from '@aws-lambda-powertools/commons';
import { DynamoDBPersistenceLayer } from '@aws-lambda-powertools/idempotency/dynamodb';
import type { Context } from 'aws-lambda';

const persistenceStore = new DynamoDBPersistenceLayer({
tableName: 'idempotencyTableName',
});

class MyHandler extends LambdaInterface {

public async handler(
event: unknown,
context: Context
): Promise<void> {
for(const record of event.Records) {
await this.processIdempotently(record);
}
}

@idempotent({ persistenceStore: dynamoDBPersistenceLayer })
private async process(record: unknown): Promise<void> {
// process each code idempotently
}
}

const handlerClass = new MyHandler();
export const handler = handlerClass.handler.bind(handlerClass);
```

The decorator configuration options are identical with the ones of the `makeIdempotent` function. Check the [docs](https://docs.powertools.aws.dev/lambda/typescript/latest/utilities/idempotency/) for more examples.

### Middy middleware

If instead you use Middy, you can use the `makeHandlerIdempotent` middleware. When using the middleware your Lambda handler becomes idempotent.
Expand Down
69 changes: 69 additions & 0 deletions packages/idempotency/src/idempotencyDecorator.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
import { AnyFunction, ItempotentFunctionOptions } from './types';
import { makeIdempotent } from './makeIdempotent';

/**
* Use this decorator to make your lambda handler itempotent.
* You need to provide a peristance layer to store the idempotency information.
* At the moment we only support `DynamodbPersistenceLayer`.
*
* @example
* ```ts
* import {
* DynamoDBPersistenceLayer,
* idempotentLambdaHandler
* } from '@aws-lambda-powertools/idempotency'
*
* class MyLambdaFunction {
* @idempotent({ persistenceStore: new DynamoDBPersistenceLayer() })
* async handler(event: any, context: any) {
* return "Hello World";
* }
* }
* export myLambdaHandler new MyLambdaFunction();
* export const handler = myLambdaHandler.handler.bind(myLambdaHandler);
* ```
*
* Similar to decoratoring a handler you can use the decorator on any other function.
* @example
* ```ts
* import {
* DynamoDBPersistenceLayer,
* idempotentFunction
* } from '@aws-lambda-powertools/idempotency'
*
* class MyClass {
*
* public async handler(_event: any, _context: any) {
* for(const record of _event.records){
* await this.process(record);
* }
* }
*
* @idemptent({ persistenceStore: new DynamoDBPersistenceLayer() })
* public async process(record: Record<stiring, unknown) {
* // do some processing
* }
* ```
* @see {@link DynamoDBPersistenceLayer}
* @see https://www.typescriptlang.org/docs/handbook/decorators.html
*/
const idempotent = function (
options: ItempotentFunctionOptions<Parameters<AnyFunction>>
): (
target: unknown,
propertyKey: string,
descriptor: PropertyDescriptor
) => PropertyDescriptor {
return function (
_target: unknown,
_propertyKey: string,
descriptor: PropertyDescriptor
) {
const childFunction = descriptor.value;
descriptor.value = makeIdempotent(childFunction, options);

return descriptor;
};
};

export { idempotent };
1 change: 1 addition & 0 deletions packages/idempotency/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
export * from './errors';
export * from './IdempotencyConfig';
export * from './makeIdempotent';
export * from './idempotencyDecorator';
export { IdempotencyRecordStatus } from './constants';
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
import type { Context } from 'aws-lambda';
import { LambdaInterface } from '@aws-lambda-powertools/commons';
import { idempotent } from '../../src';
import { Logger } from '../../../logger';
import { DynamoDBPersistenceLayer } from '../../src/persistence/DynamoDBPersistenceLayer';
import { IdempotencyConfig } from '../../src/';

const IDEMPOTENCY_TABLE_NAME =
process.env.IDEMPOTENCY_TABLE_NAME || 'table_name';
const dynamoDBPersistenceLayer = new DynamoDBPersistenceLayer({
tableName: IDEMPOTENCY_TABLE_NAME,
});

const dynamoDBPersistenceLayerCustomized = new DynamoDBPersistenceLayer({
tableName: IDEMPOTENCY_TABLE_NAME,
dataAttr: 'dataAttr',
keyAttr: 'customId',
expiryAttr: 'expiryAttr',
statusAttr: 'statusAttr',
inProgressExpiryAttr: 'inProgressExpiryAttr',
staticPkValue: 'staticPkValue',
validationKeyAttr: 'validationKeyAttr',
});

const config = new IdempotencyConfig({});

class DefaultLambda implements LambdaInterface {
@idempotent({ persistenceStore: dynamoDBPersistenceLayer })
public async handler(
_event: Record<string, unknown>,
_context: Context
): Promise<string> {
logger.info(`Got test event: ${JSON.stringify(_event)}`);
// sleep to enforce error with parallel execution
await new Promise((resolve) => setTimeout(resolve, 1000));

return 'Hello World';
}

@idempotent({
persistenceStore: dynamoDBPersistenceLayerCustomized,
config: config,
})
public async handlerCustomized(
event: { foo: string },
context: Context
): Promise<string> {
config.registerLambdaContext(context);
logger.info('Processed event', { details: event.foo });

return event.foo;
}

@idempotent({
persistenceStore: dynamoDBPersistenceLayer,
config: new IdempotencyConfig({
useLocalCache: false,
expiresAfterSeconds: 1,
eventKeyJmesPath: 'foo',
}),
})
public async handlerExpired(
event: { foo: string; invocation: number },
context: Context
): Promise<{ foo: string; invocation: number }> {
logger.addContext(context);

logger.info('Processed event', { details: event.foo });

return {
foo: event.foo,
invocation: event.invocation,
};
}

@idempotent({ persistenceStore: dynamoDBPersistenceLayer })
public async handlerParallel(
event: { foo: string },
context: Context
): Promise<string> {
logger.addContext(context);

await new Promise((resolve) => setTimeout(resolve, 1500));

logger.info('Processed event', { details: event.foo });

return event.foo;
}

@idempotent({
persistenceStore: dynamoDBPersistenceLayer,
config: new IdempotencyConfig({
eventKeyJmesPath: 'foo',
}),
})
public async handlerTimeout(
event: { foo: string; invocation: number },
context: Context
): Promise<{ foo: string; invocation: number }> {
logger.addContext(context);

if (event.invocation === 0) {
await new Promise((resolve) => setTimeout(resolve, 4000));
}

logger.info('Processed event', {
details: event.foo,
});

return {
foo: event.foo,
invocation: event.invocation,
};
}
}

const defaultLambda = new DefaultLambda();
const handler = defaultLambda.handler.bind(defaultLambda);
const handlerParallel = defaultLambda.handlerParallel.bind(defaultLambda);

const handlerCustomized = defaultLambda.handlerCustomized.bind(defaultLambda);

const handlerTimeout = defaultLambda.handlerTimeout.bind(defaultLambda);

const handlerExpired = defaultLambda.handlerExpired.bind(defaultLambda);

const logger = new Logger();

class LambdaWithKeywordArgument implements LambdaInterface {
public async handler(
event: { id: string },
_context: Context
): Promise<string> {
config.registerLambdaContext(_context);
await this.process(event.id, 'bar');

return 'Hello World Keyword Argument';
}

@idempotent({
persistenceStore: dynamoDBPersistenceLayer,
config: config,
dataIndexArgument: 1,
})
public async process(id: string, foo: string): Promise<string> {
logger.info('Got test event', { id, foo });

return 'idempotent result: ' + foo;
}
}

const handlerDataIndexArgument = new LambdaWithKeywordArgument();
const handlerWithKeywordArgument = handlerDataIndexArgument.handler.bind(
handlerDataIndexArgument
);

export {
handler,
handlerCustomized,
handlerExpired,
handlerWithKeywordArgument,
handlerTimeout,
handlerParallel,
};
Loading