fix(batch): handle early validation errors for pydantic models (poison pill) #2091

[nepshshsh]

Issue number: #2091

Summary

Lambda shouldn't failure if there is at least one successful record and return {batchItemFailures: [...] } when we use pydantic model check. For resolution, checking model function has been moved under try section in batch processing for BatchProcessor and AsyncBatchProcessor classes

Changes

Move pydantic model check line under try section in batch processing.

User experience

Now only records which aren't pass pydantic modeling check will mark as failed records

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• Meet tenets criteria
• I have performed a self-review of this change
• Changes have been tested
• Changes are documented
• PR title follows conventional commit semantics

Is this a breaking change?

RFC issue number:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[boring-cyborg]

Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need.
In the meantime, check out the #python channel on our AWS Lambda Powertools Discord: Invite link

[heitorlessa]

Looking

[heitorlessa]

Sending a few fixes and we can merge and add it to the release

[nepshshsh]

Thank you @heitorlessa, now it looks much better

[codecov-commenter]

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (ec1b346) 97.45% compared to head (b4cca25) 97.45%.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #2099   +/-   ##
========================================
  Coverage    97.45%   97.45%           
========================================
  Files          146      146           
  Lines         6748     6767   +19     
  Branches       477      483    +6     
========================================
+ Hits          6576     6595   +19     
  Misses         136      136           
  Partials        36       36           

Impacted Files	Coverage Δ
aws_lambda_powertools/utilities/batch/base.py	95.58% <100.00%> (+0.45%)	⬆️
...wertools/utilities/idempotency/persistence/base.py	99.39% <100.00%> (ø)
...ools/utilities/idempotency/persistence/dynamodb.py	100.00% <100.00%> (ø)
...bda_powertools/utilities/parser/models/dynamodb.py	100.00% <100.00%> (ø)
...mbda_powertools/utilities/parser/models/kinesis.py	100.00% <100.00%> (ø)
...s_lambda_powertools/utilities/parser/models/sqs.py	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[heitorlessa]

Just need to finish the async and we can merge it ;) THANK YOU for spotting this subtle but important bug and taking the initiative all along.

[heitorlessa]

Done - just waiting for CI to confirm everything works as expected and I'll merge it.

Took longer than expected as we had a bunch of tech debts to pay in test authoring - all cleaned up now.

PS: Working on a better DX (backwards compatible) for Batch processing to reduce boilerplate in parallel: #2090.

[heitorlessa]

Thank you so much @LuckIlNe for kicking this off. Bunch of tech debts we've wanted to address so we took the liberty to do it too. Mypy/Static typing linters will no also no longer complain about the use of body: Json[Model] so it'll also cut some boilerplate for you: https://github.com/awslabs/aws-lambda-powertools-python/pull/2099/files#diff-24525fde52e7050cebd8041038fb1ca36688c0f14e16c2f3960fa178c0e1d63fR20

Merging!

[boring-cyborg]

Awesome work, congrats on your first merged pull request and thank you for helping improve everyone's experience!