You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/utilities/batch.md
+34-6
Original file line number
Diff line number
Diff line change
@@ -143,12 +143,9 @@ Processing batches from SQS works in three stages:
143
143
144
144
When working with [SQS FIFO queues](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html){target="_blank"}, you should know that a batch may include messages from different group IDs.
145
145
146
-
By default, we will stop processing at the first failure and mark unprocessed messages as failed to preserve ordering.
146
+
By default, we will stop processing at the first failure and mark unprocessed messages as failed to preserve ordering. However, this behavior may not be optimal for customers who wish to proceed with processing messages from a different group ID.
147
147
148
-
By default, message processing halts after the initial failure, returning all failed and unprocessed messages in `batchItemFailures` to preserve the ordering of messages in your queue. However, customers can opt to continue processing messages and retrieve failed messages within a message group ID by setting `return_on_first_error` to False.
149
-
150
-
???+ notice "Having problems with DLQ?"
151
-
`AsyncBatchProcessor` uses `asyncio.gather`. This might cause [side effects and reach trace limits at high concurrency](../core/tracer.md#concurrent-asynchronous-functions){target="_blank"}.
148
+
Enable the `skip_group_on_error` option for seamless processing of messages from various group IDs. This setup ensures that messages from a failed group ID are sent back to SQS, enabling uninterrupted processing of messages from the subsequent group ID.
152
149
153
150
=== "Recommended"
154
151
@@ -170,6 +167,12 @@ By default, message processing halts after the initial failure, returning all fa
Processing batches from Kinesis works in three stages:
@@ -317,7 +320,7 @@ sequenceDiagram
317
320
318
321
> Read more about [Batch Failure Reporting feature in AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html#services-sqs-batchfailurereporting){target="_blank"}.
319
322
320
-
Sequence diagram to explain how [`SqsFifoPartialProcessor` works](#fifo-queues) with SQS FIFO queues.
323
+
Sequence diagram to explain how [`SqsFifoPartialProcessor` works](#fifo-queues) with SQS FIFO queues without `skip_group_on_error` flag.
321
324
322
325
<center>
323
326
```mermaid
@@ -341,6 +344,31 @@ sequenceDiagram
341
344
<i>SQS FIFO mechanism with Batch Item Failures</i>
342
345
</center>
343
346
347
+
Sequence diagram to explain how [`SqsFifoPartialProcessor` works](#fifo-queues) with SQS FIFO queues with `skip_group_on_error` flag.
<i>SQS FIFO mechanism with Batch Item Failures</i>
370
+
</center>
371
+
344
372
#### Kinesis and DynamoDB Streams
345
373
346
374
> Read more about [Batch Failure Reporting feature](https://docs.aws.amazon.com/lambda/latest/dg/with-kinesis.html#services-kinesis-batchfailurereporting){target="_blank"}.
0 commit comments