Skip to content

Commit 221855e

Browse files
jeromevdljeromevdl
committed
PR #717: update documentation
- add installation guide - sam template indentation - local cache disabled by default - + minors changes
1 parent 1550cc4 commit 221855e

File tree

1 file changed

+92
-40
lines changed

1 file changed

+92
-40
lines changed

‎docs/utilities/idempotency.md

+92-40
Original file line numberDiff line numberDiff line change
@@ -18,13 +18,59 @@ times with the same parameters**. This makes idempotent operations safe to retry
1818

1919
## Key features
2020

21-
* Prevent Lambda handler from executing more than once on the same event payload during a time window
21+
* Prevent Lambda handler function from executing more than once on the same event payload during a time window
2222
* Ensure Lambda handler returns the same result when called with the same payload
2323
* Select a subset of the event as the idempotency key using JMESPath expressions
2424
* Set a time window in which records with the same payload should be considered duplicates
2525

2626
## Getting started
2727

28+
### Installation
29+
=== "Maven"
30+
```xml hl_lines="3-7 24-27"
31+
<dependencies>
32+
...
33+
<dependency>
34+
<groupId>software.amazon.lambda</groupId>
35+
<artifactId>powertools-idempotency</artifactId>
36+
<version>{{ powertools.version }}</version>
37+
</dependency>
38+
...
39+
</dependencies>
40+
41+
<!-- configure the aspectj-maven-plugin to compile-time weave (CTW) the aws-lambda-powertools-java aspects into your project -->
42+
<build>
43+
<plugins>
44+
...
45+
<plugin>
46+
<groupId>org.codehaus.mojo</groupId>
47+
<artifactId>aspectj-maven-plugin</artifactId>
48+
<version>1.14.0</version>
49+
<configuration>
50+
<source>1.8</source>
51+
<target>1.8</target>
52+
<complianceLevel>1.8</complianceLevel>
53+
<aspectLibraries>
54+
<aspectLibrary>
55+
<groupId>software.amazon.lambda</groupId>
56+
<artifactId>powertools-idempotency</artifactId>
57+
</aspectLibrary>
58+
...
59+
</aspectLibraries>
60+
</configuration>
61+
<executions>
62+
<execution>
63+
<goals>
64+
<goal>compile</goal>
65+
</goals>
66+
</execution>
67+
</executions>
68+
</plugin>
69+
...
70+
</plugins>
71+
</build>
72+
```
73+
2874
### Required resources
2975

3076
Before getting started, you need to create a persistent storage layer where the idempotency utility can store its state - your Lambda functions will need read and write access to it.
@@ -43,33 +89,33 @@ If you're not [changing the default configuration for the DynamoDB persistence l
4389
!!! Tip "Tip: You can share a single state table for all functions"
4490
You can reuse the same DynamoDB table to store idempotency state. We add your function name in addition to the idempotency key as a hash key.
4591

46-
```yaml hl_lines="5-13 21-23" title="AWS Serverless Application Model (SAM) example"
92+
```yaml hl_lines="5-13 21-23 26" title="AWS Serverless Application Model (SAM) example"
4793
Resources:
4894
IdempotencyTable:
49-
Type: AWS::DynamoDB::Table
50-
Properties:
51-
AttributeDefinitions:
52-
- AttributeName: id
53-
AttributeType: S
54-
KeySchema:
55-
- AttributeName: id
56-
KeyType: HASH
57-
TimeToLiveSpecification:
58-
AttributeName: expiration
59-
Enabled: true
60-
BillingMode: PAY_PER_REQUEST
61-
62-
HelloWorldFunction:
95+
Type: AWS::DynamoDB::Table
96+
Properties:
97+
AttributeDefinitions:
98+
- AttributeName: id
99+
AttributeType: S
100+
KeySchema:
101+
- AttributeName: id
102+
KeyType: HASH
103+
TimeToLiveSpecification:
104+
AttributeName: expiration
105+
Enabled: true
106+
BillingMode: PAY_PER_REQUEST
107+
108+
IdempotencyFunction:
63109
Type: AWS::Serverless::Function
64110
Properties:
65-
Runtime: python3.8
66-
...
67-
Policies:
68-
- DynamoDBCrudPolicy:
69-
TableName: !Ref IdempotencyTable
111+
CodeUri: Function
112+
Handler: helloworld.App::handleRequest
113+
Policies:
114+
- DynamoDBCrudPolicy:
115+
TableName: !Ref IdempotencyTable
70116
Environment:
71117
Variables:
72-
TABLE_NAME: !Ref IdempotencyTable
118+
IDEMPOTENCY_TABLE: !Ref IdempotencyTable
73119
```
74120
75121
!!! warning "Warning: Large responses with DynamoDB persistence layer"
@@ -227,7 +273,7 @@ Imagine the function executes successfully, but the client never receives the re
227273
!!! warning "Warning: Idempotency for JSON payloads"
228274
The payload extracted by the `EventKeyJMESPath` is treated as a string by default, so will be sensitive to differences in whitespace even when the JSON payload itself is identical.
229275

230-
To alter this behaviour, you can use the [JMESPath built-in function](jmespath_functions.md#powertools_json-function) `powertools_json()` to treat the payload as a JSON object rather than a string.
276+
To alter this behaviour, you can use the [JMESPath built-in function](utilities.md#powertools_json-function) `powertools_json()` to treat the payload as a JSON object rather than a string.
231277

232278
=== "PaymentFunction.java"
233279

@@ -252,7 +298,6 @@ Imagine the function executes successfully, but the client never receives the re
252298
APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent();
253299
254300
try {
255-
// TODO you can use our Jackson ObjectMapper if you want
256301
Subscription subscription = JsonConfig.get().getObjectMapper().readValue(event.getBody(), Subscription.class);
257302
258303
SubscriptionPayment payment = createSubscriptionPayment(
@@ -393,7 +438,7 @@ IdempotencyConfig.builder()
393438
.withPayloadValidationJMESPath("paymentId")
394439
.withThrowOnNoIdempotencyKey(true)
395440
.withExpiration(Duration.of(5, ChronoUnit.MINUTES))
396-
.withUseLocalCache(false)
441+
.withUseLocalCache(true)
397442
.withLocalCacheMaxItems(432)
398443
.withHashFunction("SHA-256")
399444
.build()
@@ -403,11 +448,11 @@ These are the available options for further configuration:
403448

404449
| Parameter | Default | Description |
405450
|---------------------------------------------------|---------|----------------------------------------------------------------------------------------------------------------------------------|
406-
| **EventKeyJMESPath** | `""` | JMESPath expression to extract the idempotency key from the event record. See available [built-in functions](utilities) |
451+
| **EventKeyJMESPath** | `""` | JMESPath expression to extract the idempotency key from the event record. See available [built-in functions](serialization) |
407452
| **PayloadValidationJMESPath** | `""` | JMESPath expression to validate whether certain parameters have changed in the event |
408453
| **ThrowOnNoIdempotencyKey** | `False` | Throw exception if no idempotency key was found in the request |
409454
| **ExpirationInSeconds** | 3600 | The number of seconds to wait before a record is expired |
410-
| **UseLocalCache** | `true` | Whether to locally cache idempotency results (LRU cache) |
455+
| **UseLocalCache** | `false` | Whether to locally cache idempotency results (LRU cache) |
411456
| **LocalCacheMaxItems** | 256 | Max number of items to store in local cache |
412457
| **HashFunction** | `MD5` | Algorithm to use for calculating hashes, as supported by `java.security.MessageDigest` (eg. SHA-1, SHA-256, ...) |
413458

@@ -424,18 +469,18 @@ This is a locking mechanism for correctness. Since we don't know the result from
424469

425470
### Using in-memory cache
426471

427-
**By default, in-memory local caching is enabled**, to improve performance of your Lambda function.
428-
We cache a maximum of 256 records in each Lambda execution environment - You can change it with the **`LocalCacheMaxItems`** parameter.
472+
**By default, in-memory local caching is disabled**, to avoid using memory in an unpredictable way.
429473

430474
!!! warning Memory configuration of your function
431475
Be sure to configure the Lambda memory according to the number of records and the potential size of each record.
432476

433-
You can disable it as seen before with:
434-
```java title="Disable local cache"
477+
You can enable it as seen before with:
478+
```java title="Enable local cache"
435479
IdempotencyConfig.builder()
436-
.withUseLocalCache(false)
480+
.withUseLocalCache(true)
437481
.build()
438482
```
483+
When enabled, we cache a maximum of 256 records in each Lambda execution environment - You can change it with the **`LocalCacheMaxItems`** parameter.
439484

440485
!!! note "Note: This in-memory cache is local to each Lambda execution environment"
441486
This means it will be effective in cases where your function's concurrency is low in comparison to the number of "retry" invocations with the same payload, because cache might be empty.
@@ -597,13 +642,19 @@ This means that we will throw **`IdempotencyKeyException`** if the evaluation of
597642

598643
When creating the `DynamoDBPersistenceStore`, you can set a custom [`DynamoDbClient`](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/dynamodb/DynamoDbClient.html) if you need to customize the configuration:
599644

600-
=== "Custom DynamoDbClient"
645+
=== "Custom DynamoDbClient with X-Ray interceptor"
601646

602-
```java hl_lines="2 7"
647+
```java hl_lines="2-8 13"
603648
public App() {
604-
DynamoDbClient customClient = DynamoDbClient.builder().httpClient(AwsCrtAsyncHttpClient.create());
605-
606-
Idempotency.config().withPersistenceStore(
649+
DynamoDbClient customClient = DynamoDbClient.builder()
650+
.region(Region.US_WEST_2)
651+
.overrideConfiguration(ClientOverrideConfiguration.builder()
652+
.addExecutionInterceptor(new TracingInterceptor())
653+
.build()
654+
)
655+
.build();
656+
657+
Idempotency.config().withPersistenceStore(
607658
DynamoDBPersistenceStore.builder()
608659
.withTableName(System.getenv("TABLE_NAME"))
609660
.withDynamoDbClient(customClient)
@@ -674,16 +725,16 @@ public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent in
674725
```
675726

676727
!!! tip "Tip: JMESPath Powertools functions are also available"
677-
Built-in functions like `powertools_json`, `powertools_base64`, `powertools_base64_gzip` are also available to use in this utility. See [JMESPath Powertools functions](utilities.md)
728+
Built-in functions like `powertools_json`, `powertools_base64`, `powertools_base64_gzip` are also available to use in this utility. See [JMESPath Powertools functions](serialization.md)
678729

679730

680731
## Testing your code
681732

682733
The idempotency utility provides several routes to test your code.
683734

684735
### Disabling the idempotency utility
685-
When testing your code, you may wish to disable the idempotency logic altogether and focus on testing your business logic. To do this, you can set the environment variable `POWERTOOLS_IDEMPOTENCY_DISABLED`
686-
with a truthy value. If you prefer setting this for specific tests, and are using JUnit 5, you can use [junit-pioneer](https://junit-pioneer.org/docs/environment-variables/) library:
736+
When testing your code, you may wish to disable the idempotency logic altogether and focus on testing your business logic. To do this, you can set the environment variable `POWERTOOLS_IDEMPOTENCY_DISABLED` to true.
737+
If you prefer setting this for specific tests, and are using JUnit 5, you can use [junit-pioneer](https://junit-pioneer.org/docs/environment-variables/) library:
687738

688739
=== "MyFunctionTest.java"
689740

@@ -739,6 +790,7 @@ To unit test your function with DynamoDB Local, you can refer to this guide to [
739790
</dependencies>
740791
<repositories>
741792
<!-- custom repository to get the dependency -->
793+
<!-- see https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html#apache-maven -->
742794
<repository>
743795
<id>dynamodb-local-oregon</id>
744796
<name>DynamoDB Local Release Repository</name>

0 commit comments

Comments
 (0)