Adding two features, retain S3 objects after messages deletion and custom S3 object key generation. #6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…when set to 'true' prevents the companion S3 object of a message being removed from S3 on the sqs 'deleteMessage()' action. The S3 bucket will then retain a log of all messages (if the 'alwaysThroughS3' is also set to true) after they have been processed and deleted. Also added the S3KeyGenerator interface and a default implementation to the ExtendedClientConfiguration class. The S3KeyGenerator interface allows custom implementations to add structure to the S3 bucket objects create (such as partionting on date created).
|
Any chance of getting this merged in? I was about to implement similar functionality myself |
|
bump :) |
|
bump again 👍 👍 |
|
We want to use this for SNS, too, so we need SQS subscribers to leave the message alone, until all subscribers have had the chance to read the message. |
|
also looking for this functionality |
|
Looking for such functionality also... |
acruise
reviewed
May 4, 2022
| * the queue. | ||
| * | ||
| * @param retainS3Messages | ||
| * Whether or not messages are deleted in S3 when they are delete |
There was a problem hiding this comment.
The method name is retain but the description is delete :)
There was a problem hiding this comment.
Well, that is because this method controls if the S3 objects are either 'retained' or 'deleted', so yes the description uses the word 'deleted'. Not sure what the issue is tbh? 🤔
|
+1 |
|
bump! Is there anything which blocks getting this very needed feature merged? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adding ExtendedClientConfiguration parameter 'retainS3Messages' that when set to 'true' prevents the companion S3 object of a message being removed from S3 on the sqs 'deleteMessage()' action. The S3 bucket will then retain a log of all messages (if the 'alwaysThroughS3' is also set to true) after they have been processed and deleted.
Also added the S3KeyGenerator interface and a default implementation to the ExtendedClientConfiguration class. The S3KeyGenerator interface allows custom implementations to add structure to the S3 bucket objects create (such as partitioning on date created).