Skip to content

Commit f8220fe

Browse files
author
Pankaj Agrawal
committed
docs(batch-processing): Support for moving non retryable msg to DLQ
1 parent e0bbaa6 commit f8220fe

File tree

2 files changed

+221
-1
lines changed

2 files changed

+221
-1
lines changed

‎powertools-sqs/src/main/java/software/amazon/lambda/powertools/sqs/SqsBatch.java

+20-1
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
* calling {@link SqsMessageHandler#process(SQSMessage)} method for each {@link SQSMessage} in the received {@link SQSEvent}
1818
* </p>
1919
*
20-
* </p>
20+
* <p>
2121
* If any exception is thrown from {@link SqsMessageHandler#process(SQSMessage)} during processing of a messages, Utility
2222
* will take care of deleting all the successful messages from SQS. When one or more single message fails processing due
2323
* to exception thrown from {@link SqsMessageHandler#process(SQSMessage)}, Lambda execution will fail
@@ -32,6 +32,24 @@
3232
* {@link SqsBatch#suppressException()} to true. By default its value is false
3333
* </p>
3434
*
35+
* <p>
36+
* If you want certain exceptions to be treated as permanent failures, i.e. exceptions where the result of retrying will
37+
* always be a failure and want these can be immediately moved to the dead letter queue associated to the source SQS queue,
38+
*
39+
* you can use {@link SqsBatch#nonRetryableExceptions()} to configure such exceptions.
40+
* Make sure function execution role has sqs:GetQueueAttributes permission on source SQS queue and sqs:SendMessage,
41+
* sqs:SendMessageBatch permission for configured DLQ.
42+
*
43+
* If you want such messages to be deleted instead, set {@link SqsBatch#deleteNonRetryableMessageFromQueue()} to true.
44+
* By default its value is false.
45+
*
46+
* If there is no DLQ configured on source SQS queue and {@link SqsBatch#nonRetryableExceptions()} attribute is set, if
47+
* nonRetryableExceptions occurs from {@link SqsMessageHandler}, such exceptions will still be treated as temporary
48+
* exceptions and the message will be moved back to source SQS queue for reprocessing. The same behaviour will occur if
49+
* for some reason the utility is unable to move the message to the DLQ. An example of this could be because the function
50+
* is missing the correct permissions.
51+
* </p>
52+
*
3553
* <pre>
3654
* public class SqsMessageHandler implements RequestHandler<SQSEvent, String> {
3755
*
@@ -51,6 +69,7 @@
5169
*
5270
* ...
5371
* </pre>
72+
* @see <a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html">Amazon SQS dead-letter queues</a>
5473
*/
5574
@Retention(RetentionPolicy.RUNTIME)
5675
@Target(ElementType.METHOD)

‎powertools-sqs/src/main/java/software/amazon/lambda/powertools/sqs/SqsUtils.java

+201
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,54 @@ public static <R> List<R> batchProcessor(final SQSEvent event,
131131
return batchProcessor(event, false, handler);
132132
}
133133

134+
/**
135+
* This utility method is used to processes each {@link SQSMessage} inside received {@link SQSEvent}
136+
*
137+
* <p>
138+
* Utility will take care of calling {@link SqsMessageHandler#process(SQSMessage)} method for each {@link SQSMessage}
139+
* in the received {@link SQSEvent}
140+
* </p>
141+
*
142+
* <p>
143+
* If any exception is thrown from {@link SqsMessageHandler#process(SQSMessage)} during processing of a messages,
144+
* Utility will take care of deleting all the successful messages from SQS. When one or more single message fails
145+
* processing due to exception thrown from {@link SqsMessageHandler#process(SQSMessage)}
146+
* {@link SQSBatchProcessingException} is thrown with all the details of successful and failed messages.
147+
*
148+
* </p>
149+
*
150+
* <p>
151+
* If all the messages are successfully processes, No SQS messages are deleted explicitly but is rather delegated to
152+
* Lambda execution context for deletion.
153+
* </p>
154+
*
155+
* <p>
156+
* If you dont want to utility to throw {@link SQSBatchProcessingException} in case of failures but rather suppress
157+
* it, Refer {@link SqsUtils#batchProcessor(SQSEvent, boolean, Class)}
158+
* </p>
159+
*
160+
* <p>
161+
* If you want certain exceptions to be treated as permanent failures, i.e. exceptions where the result of retrying will
162+
* always be a failure and want these can be immediately moved to the dead letter queue associated to the source SQS queue,
163+
* you can use nonRetryableExceptions parameter to configure such exceptions.
164+
*
165+
* Make sure function execution role has sqs:GetQueueAttributes permission on source SQS queue and sqs:SendMessage,
166+
* sqs:SendMessageBatch permission for configured DLQ.
167+
*
168+
* If there is no DLQ configured on source SQS queue and {@link SqsBatch#nonRetryableExceptions()} attribute is set, if
169+
* nonRetryableExceptions occurs from {@link SqsMessageHandler}, such exceptions will still be treated as temporary
170+
* exceptions and the message will be moved back to source SQS queue for reprocessing. The same behaviour will occur if
171+
* for some reason the utility is unable to move the message to the DLQ. An example of this could be because the function
172+
* is missing the correct permissions.
173+
* </p>
174+
* @see <a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html">Amazon SQS dead-letter queues</a>
175+
* @param event {@link SQSEvent} received by lambda function.
176+
* @param handler Class implementing {@link SqsMessageHandler} which will be called for each message in event.
177+
* @param nonRetryableExceptions exception classes that are to be treated as permanent exceptions and to be moved
178+
* to DLQ.
179+
* @return List of values returned by {@link SqsMessageHandler#process(SQSMessage)} while processing each message.
180+
* @throws SQSBatchProcessingException if some messages fail during processing.
181+
*/
134182
@SafeVarargs
135183
public static <R> List<R> batchProcessor(final SQSEvent event,
136184
final Class<? extends SqsMessageHandler<R>> handler,
@@ -173,6 +221,57 @@ public static <R> List<R> batchProcessor(final SQSEvent event,
173221
return batchProcessor(event, suppressException, handlerInstance);
174222
}
175223

224+
/**
225+
* This utility method is used to processes each {@link SQSMessage} inside received {@link SQSEvent}
226+
*
227+
* <p>
228+
* Utility will take care of calling {@link SqsMessageHandler#process(SQSMessage)} method for each {@link SQSMessage}
229+
* in the received {@link SQSEvent}
230+
* </p>
231+
*
232+
* <p>
233+
* If any exception is thrown from {@link SqsMessageHandler#process(SQSMessage)} during processing of a messages,
234+
* Utility will take care of deleting all the successful messages from SQS. When one or more single message fails
235+
* processing due to exception thrown from {@link SqsMessageHandler#process(SQSMessage)}
236+
* {@link SQSBatchProcessingException} is thrown with all the details of successful and failed messages.
237+
*
238+
* </p>
239+
*
240+
* <p>
241+
* If all the messages are successfully processes, No SQS messages are deleted explicitly but is rather delegated to
242+
* Lambda execution context for deletion.
243+
* </p>
244+
*
245+
* <p>
246+
* If you dont want to utility to throw {@link SQSBatchProcessingException} in case of failures but rather suppress
247+
* it, Refer {@link SqsUtils#batchProcessor(SQSEvent, boolean, Class)}
248+
* </p>
249+
*
250+
* <p>
251+
* If you want certain exceptions to be treated as permanent failures, i.e. exceptions where the result of retrying will
252+
* always be a failure and want these can be immediately moved to the dead letter queue associated to the source SQS queue,
253+
* you can use nonRetryableExceptions parameter to configure such exceptions.
254+
*
255+
* Make sure function execution role has sqs:GetQueueAttributes permission on source SQS queue and sqs:SendMessage,
256+
* sqs:SendMessageBatch permission for configured DLQ.
257+
*
258+
* If there is no DLQ configured on source SQS queue and {@link SqsBatch#nonRetryableExceptions()} attribute is set, if
259+
* nonRetryableExceptions occurs from {@link SqsMessageHandler}, such exceptions will still be treated as temporary
260+
* exceptions and the message will be moved back to source SQS queue for reprocessing. The same behaviour will occur if
261+
* for some reason the utility is unable to move the message to the DLQ. An example of this could be because the function
262+
* is missing the correct permissions.
263+
* </p>
264+
* @see <a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html">Amazon SQS dead-letter queues</a>
265+
*
266+
* @param event {@link SQSEvent} received by lambda function.
267+
* @param suppressException if this is set to true, No {@link SQSBatchProcessingException} is thrown even on failed
268+
* messages.
269+
* @param handler Class implementing {@link SqsMessageHandler} which will be called for each message in event.
270+
* @param nonRetryableExceptions exception classes that are to be treated as permanent exceptions and to be moved
271+
* to DLQ.
272+
* @return List of values returned by {@link SqsMessageHandler#process(SQSMessage)} while processing each message.
273+
* @throws SQSBatchProcessingException if some messages fail during processing.
274+
*/
176275
@SafeVarargs
177276
public static <R> List<R> batchProcessor(final SQSEvent event,
178277
final boolean suppressException,
@@ -183,6 +282,59 @@ public static <R> List<R> batchProcessor(final SQSEvent event,
183282
return batchProcessor(event, suppressException, handlerInstance, false, nonRetryableExceptions);
184283
}
185284

285+
/**
286+
* This utility method is used to processes each {@link SQSMessage} inside received {@link SQSEvent}
287+
*
288+
* <p>
289+
* Utility will take care of calling {@link SqsMessageHandler#process(SQSMessage)} method for each {@link SQSMessage}
290+
* in the received {@link SQSEvent}
291+
* </p>
292+
*
293+
* <p>
294+
* If any exception is thrown from {@link SqsMessageHandler#process(SQSMessage)} during processing of a messages,
295+
* Utility will take care of deleting all the successful messages from SQS. When one or more single message fails
296+
* processing due to exception thrown from {@link SqsMessageHandler#process(SQSMessage)}
297+
* {@link SQSBatchProcessingException} is thrown with all the details of successful and failed messages.
298+
*
299+
* </p>
300+
*
301+
* <p>
302+
* If all the messages are successfully processes, No SQS messages are deleted explicitly but is rather delegated to
303+
* Lambda execution context for deletion.
304+
* </p>
305+
*
306+
* <p>
307+
* If you dont want to utility to throw {@link SQSBatchProcessingException} in case of failures but rather suppress
308+
* it, Refer {@link SqsUtils#batchProcessor(SQSEvent, boolean, Class)}
309+
* </p>
310+
*
311+
* <p>
312+
* If you want certain exceptions to be treated as permanent failures, i.e. exceptions where the result of retrying will
313+
* always be a failure and want these can be immediately moved to the dead letter queue associated to the source SQS queue,
314+
* you can use nonRetryableExceptions parameter to configure such exceptions.
315+
*
316+
* Make sure function execution role has sqs:GetQueueAttributes permission on source SQS queue and sqs:SendMessage,
317+
* sqs:SendMessageBatch permission for configured DLQ.
318+
*
319+
* If you want such messages to be deleted instead, set deleteNonRetryableMessageFromQueue to true.
320+
*
321+
* If there is no DLQ configured on source SQS queue and {@link SqsBatch#nonRetryableExceptions()} attribute is set, if
322+
* nonRetryableExceptions occurs from {@link SqsMessageHandler}, such exceptions will still be treated as temporary
323+
* exceptions and the message will be moved back to source SQS queue for reprocessing. The same behaviour will occur if
324+
* for some reason the utility is unable to move the message to the DLQ. An example of this could be because the function
325+
* is missing the correct permissions.
326+
* </p>
327+
* @see <a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html">Amazon SQS dead-letter queues</a>
328+
* @param event {@link SQSEvent} received by lambda function.
329+
* @param suppressException if this is set to true, No {@link SQSBatchProcessingException} is thrown even on failed
330+
* messages.
331+
* @param handler Class implementing {@link SqsMessageHandler} which will be called for each message in event.
332+
* @param deleteNonRetryableMessageFromQueue If messages with nonRetryableExceptions are to be deleted from SQS queue.
333+
* @param nonRetryableExceptions exception classes that are to be treated as permanent exceptions and to be moved
334+
* to DLQ.
335+
* @return List of values returned by {@link SqsMessageHandler#process(SQSMessage)} while processing each message.
336+
* @throws SQSBatchProcessingException if some messages fail during processing.
337+
*/
186338
@SafeVarargs
187339
public static <R> List<R> batchProcessor(final SQSEvent event,
188340
final boolean suppressException,
@@ -227,6 +379,55 @@ public static <R> List<R> batchProcessor(final SQSEvent event,
227379
return batchProcessor(event, false, handler);
228380
}
229381

382+
383+
/**
384+
* This utility method is used to processes each {@link SQSMessage} inside received {@link SQSEvent}
385+
*
386+
* <p>
387+
* Utility will take care of calling {@link SqsMessageHandler#process(SQSMessage)} method for each {@link SQSMessage}
388+
* in the received {@link SQSEvent}
389+
* </p>
390+
*
391+
* <p>
392+
* If any exception is thrown from {@link SqsMessageHandler#process(SQSMessage)} during processing of a messages,
393+
* Utility will take care of deleting all the successful messages from SQS. When one or more single message fails
394+
* processing due to exception thrown from {@link SqsMessageHandler#process(SQSMessage)}
395+
* {@link SQSBatchProcessingException} is thrown with all the details of successful and failed messages.
396+
*
397+
* </p>
398+
*
399+
* <p>
400+
* If all the messages are successfully processes, No SQS messages are deleted explicitly but is rather delegated to
401+
* Lambda execution context for deletion.
402+
* </p>
403+
*
404+
* <p>
405+
* If you dont want to utility to throw {@link SQSBatchProcessingException} in case of failures but rather suppress
406+
* it, Refer {@link SqsUtils#batchProcessor(SQSEvent, boolean, Class)}
407+
* </p>
408+
*
409+
* <p>
410+
* If you want certain exceptions to be treated as permanent failures, i.e. exceptions where the result of retrying will
411+
* always be a failure and want these can be immediately moved to the dead letter queue associated to the source SQS queue,
412+
* you can use nonRetryableExceptions parameter to configure such exceptions.
413+
*
414+
* Make sure function execution role has sqs:GetQueueAttributes permission on source SQS queue and sqs:SendMessage,
415+
* sqs:SendMessageBatch permission for configured DLQ.
416+
*
417+
* If there is no DLQ configured on source SQS queue and {@link SqsBatch#nonRetryableExceptions()} attribute is set, if
418+
* nonRetryableExceptions occurs from {@link SqsMessageHandler}, such exceptions will still be treated as temporary
419+
* exceptions and the message will be moved back to source SQS queue for reprocessing.The same behaviour will occur if
420+
* for some reason the utility is unable to moved the message to the DLQ. An example of this could be because the function
421+
* is missing the correct permissions.
422+
* </p>
423+
* @see <a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html">Amazon SQS dead-letter queues</a>
424+
* @param event {@link SQSEvent} received by lambda function.
425+
* @param handler Instance of class implementing {@link SqsMessageHandler} which will be called for each message in event.
426+
* @param nonRetryableExceptions exception classes that are to be treated as permanent exceptions and to be moved
427+
* to DLQ.
428+
* @return List of values returned by {@link SqsMessageHandler#process(SQSMessage)} while processing each message.
429+
* @throws SQSBatchProcessingException if some messages fail during processing.
430+
*/
230431
@SafeVarargs
231432
public static <R> List<R> batchProcessor(final SQSEvent event,
232433
final SqsMessageHandler<R> handler,

0 commit comments

Comments
 (0)