Skip to content

feat(batch): new BatchProcessor for SQS, DynamoDB, Kinesis #886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
b1ca7f1
feat(batch): new BatchProcessor for SQS, DynamoDB, Kinesis
heitorlessa Dec 10, 2021
113a44f
refactor: response over report
heitorlessa Dec 11, 2021
5ab2ec7
fix: mutability bug
heitorlessa Dec 11, 2021
4652237
feat(batch): add Kinesis Data streams support
heitorlessa Dec 12, 2021
4cfcd34
fix: item identifier key should be constant
heitorlessa Dec 12, 2021
139f52b
feat(batch): add DynamoDB Streams support
heitorlessa Dec 12, 2021
1822456
feat(batch): use event source data classes by default
heitorlessa Dec 12, 2021
c757316
chore: permanent exceptions TBD in separate PR
heitorlessa Dec 12, 2021
3217097
feat: mypy support
heitorlessa Dec 12, 2021
09257ba
feat: draft implementation
heitorlessa Dec 12, 2021
251541c
fix: mypy typing
heitorlessa Dec 13, 2021
4c95d39
chore: improve mypy support on success/failure
heitorlessa Dec 13, 2021
7756d2c
fix: failure handler record types
heitorlessa Dec 13, 2021
1b242eb
fix: copy data not pointer if one subclasses it
heitorlessa Dec 17, 2021
4a7ceea
chore: linting
heitorlessa Dec 17, 2021
53b8e75
chore: address Tom's feedback on type name
heitorlessa Dec 17, 2021
b0f170e
chore: test model support
heitorlessa Dec 17, 2021
01eb5a7
Merge branch 'develop' of https://github.com/awslabs/aws-lambda-power…
heitorlessa Dec 17, 2021
77a7ab5
chore: remove leftovers
heitorlessa Dec 17, 2021
11ab825
docs: new BatchProcessor for SQS, Kinesis, DynamoDB
heitorlessa Dec 19, 2021
0d5d24e
fix: ensure BatchProcessorError is raised when entire batch fails
heitorlessa Dec 19, 2021
e1dc4cf
fix: exception leftover
heitorlessa Dec 19, 2021
cf3b01a
chore: cleanup exceptions
heitorlessa Dec 19, 2021
3fc3e40
docs: update mechanics section
heitorlessa Dec 19, 2021
9ceb74b
docs: update IAM permission
heitorlessa Dec 19, 2021
be8ab03
docs: keep old section name for stats
heitorlessa Dec 19, 2021
27f2937
docs: update accessing processed messages section
heitorlessa Dec 19, 2021
1496b6f
docs: update sentry section
heitorlessa Dec 19, 2021
9057791
docs: add extension, update create own processor
heitorlessa Dec 19, 2021
580eeae
docs: add pydantic section
heitorlessa Dec 19, 2021
f380f57
docs: add migration guide
heitorlessa Dec 19, 2021
58d78ca
docs: add caveat section
heitorlessa Dec 19, 2021
95715d0
docs: add testing section
heitorlessa Dec 19, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,6 @@ repos:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-toml
- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.5.1
hooks:
- id: python-use-type-annotations
- repo: local
hooks:
- id: black
Expand Down
24 changes: 21 additions & 3 deletions aws_lambda_powertools/utilities/batch/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,25 @@
Batch processing utility
"""

from .base import BasePartialProcessor, batch_processor
from .sqs import PartialSQSProcessor, sqs_batch_processor
from aws_lambda_powertools.utilities.batch.base import (
BasePartialProcessor,
BatchProcessor,
EventType,
ExceptionInfo,
FailureResponse,
SuccessResponse,
batch_processor,
)
from aws_lambda_powertools.utilities.batch.sqs import PartialSQSProcessor, sqs_batch_processor

__all__ = ("BasePartialProcessor", "PartialSQSProcessor", "batch_processor", "sqs_batch_processor")
__all__ = (
"BatchProcessor",
"BasePartialProcessor",
"ExceptionInfo",
"EventType",
"FailureResponse",
"PartialSQSProcessor",
"SuccessResponse",
"batch_processor",
"sqs_batch_processor",
)
Loading