Skip to content

Feature request: Reduce Batch boilerplate with a facade #2083

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 of 2 tasks
heitorlessa opened this issue Apr 4, 2023 · 3 comments
Closed
1 of 2 tasks

Feature request: Reduce Batch boilerplate with a facade #2083

heitorlessa opened this issue Apr 4, 2023 · 3 comments
Assignees
Labels
batch Batch processing utility feature-request feature request

Comments

@heitorlessa
Copy link
Contributor

Use case

Customers have to decorate their Lambda handler with batch_processor decorator to repeatedly call a record handler for each record it walks through.

Historically, this was kept for backwards compatibility reasons as we moved from the original Batch that used AWS SDK to delete successful messages to the native Lambda integration launched two years later.

The new integration required a strict response to make it work, so we introduced a new property .response in the processor, so the Lambda handler could return it in the right format.

Now that we introduced async along with some refactoring and the new typing extensions, we have access to better tools and we can make it easier to customers onboard to the Batch feature by removing this unnecessary boilerplate in a backwards compatible way.

image

Solution/User Experience

We can introduce a new function, call it process_partial_response for the sake of argument, that is responsible to:

  • Extract Records[*] from the event, whether that is a dict or a Parser's SQS Model
  • Call the processor .process sync using the given record handler
  • Extract the final response from the processor and simply return it to the caller

Benefits. Besides reducing boilerplate and accelerate on-boarding, it'll reduce the call stack in Python improving performance a tiny wee bit.

image

Alternative solutions

No response

Acknowledgment

@heitorlessa heitorlessa added triage Pending triage from maintainers feature-request feature request labels Apr 4, 2023
@heitorlessa heitorlessa added the batch Batch processing utility label Apr 5, 2023
@leandrodamascena
Copy link
Contributor

Hi @heitorlessa! I understand the developer experience proposition behind this change and it makes sense to create this new function process_partial_response. When you send the PR I can review.

@leandrodamascena leandrodamascena removed the triage Pending triage from maintainers label Apr 6, 2023
@github-actions github-actions bot added the pending-release Fix or implementation already in dev waiting to be released label Apr 7, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Apr 7, 2023

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 7, 2023

This is now released under 2.12.0 version!

@github-actions github-actions bot removed the pending-release Fix or implementation already in dev waiting to be released label Apr 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
batch Batch processing utility feature-request feature request
Projects
None yet
Development

No branches or pull requests

3 participants