Skip to content

Send messages to the DLQ before deleting from SQS #729

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
VictorJuliani opened this issue Jan 28, 2022 · 2 comments
Closed

Send messages to the DLQ before deleting from SQS #729

VictorJuliani opened this issue Jan 28, 2022 · 2 comments
Assignees
Labels
bug Something isn't working documentation Improvements or additions to documentation

Comments

@VictorJuliani
Copy link

Is your feature request related to a problem? Please describe.

I came across a problem where messages were failing and throwing exceptions, but they weren't sent to the DLQ. They were consumed as successful.

After investigation, I found out that the DLQ#SendMessage call from PowerTools was failing due to a missing KMS permission in my lambda.

That caused some messages to be lost from the queues.

Describe the solution you'd like

  1. Update the documentation to point out that Encrypt/Decrypt and GenerateDataKey* permissions are required if the SQS is encrypted (at least it's not documented here)
  2. Perform the DLQ#SendMessage call before the SQS#DeleteMessage (code ref) to make sure the message won't be lost in case of any DLQ misconfiguration.

Describe alternatives you've considered

Additional context

@pankajagrawal16
Copy link
Contributor

pankajagrawal16 commented Jan 29, 2022

Thanks for opening the issue @VictorJuliani . fair points. Regarding the second point, send message call is actually done before delete message call today.

Its a bug that any failure to send message call to DLQ is assumed to be always successful, so we need to respect failure response and mark batch as failure.

Thanks for raising this, I will create a bug fix and also improve the docs.

@pankajagrawal16 pankajagrawal16 added documentation Improvements or additions to documentation feature-request New feature or request bug Something isn't working labels Jan 29, 2022
@pankajagrawal16 pankajagrawal16 self-assigned this Jan 31, 2022
@pankajagrawal16 pankajagrawal16 added status/staged-next-release and removed feature-request New feature or request labels Jan 31, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Feb 1, 2022

This is now released under 1.10.3 version!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

2 participants