-
Notifications
You must be signed in to change notification settings - Fork 107
build: recognize collection_model_binding_data for batch inputs #1655
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
tests/extension_tests/deferred_bindings_tests/test_deferred_bindings.py
Outdated
Show resolved
Hide resolved
return binding.decode(datum, | ||
trigger_metadata=metadata, | ||
pytype=pytype) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NIT: Improvement suggestion:
validSources = {
"AzureEventHubsEventData",
"AzureServiceBusReceivedMessage"
}
if datum.type == "collection_model_binding_data" or datum.value.source in valid_sources:
return binding.decode(datum, trigger_metadata=metadata, pytype=pytype)
Also, can type can be enum in your datum model? That will avoid the string comparisons and improve the codebae as less strings constant will flow.
* build: recognize collection_model_binding_data for batch inputs (#1655) * add cmbd * Add * Add * Rm newline * Add tests * Fix cmbd * Fix test * Lint * Rm * Rm * Add back newline * rm ws * Rm list * Rm cmbd from cache * Avoid caching * Keep cmbd check * Add comment * Lint --------- Co-authored-by: Evan Roman <[email protected]> Co-authored-by: hallvictoria <[email protected]> * build: update Python Worker Version to 4.36.1 (#1660) Co-authored-by: AzureFunctionsPython <[email protected]> * initial changes * Update Python SDK Version to 1.23.0 (#1663) Co-authored-by: AzureFunctionsPython <[email protected]> * merges from ADO * merge fixes * merge fixes * merge fixes * merge fixes * don't run 313 unit tests yet * changes for builds --------- Co-authored-by: Evan <[email protected]> Co-authored-by: Evan Roman <[email protected]> Co-authored-by: AzureFunctionsPython <[email protected]> Co-authored-by: AzureFunctionsPython <[email protected]>
* Proxy Worker: Initial Commit * Updated worker config to include 3.13 * Updated test_setup * Updated worker.py * Updated dispatcher * Updated syspath in worker.py * Updated path in worker.py * Updated worker.py * Removed reload in dispatcher * Updating v1 library worker name * Added dispatcher logs * Added dispatcher try/catch logs * Updated sys path * Dispatcher and dependency manager updates * Updated dispatcher and pyproject * Testing updates and refactoring * Bug fixes and refactoring * Added more unit tests * Added tests and fixed test setup * Updated test_setup * Updated test setup to add grpc dir copy * build: proxy worker build & test setup (#1664) * build: recognize collection_model_binding_data for batch inputs (#1655) * add cmbd * Add * Add * Rm newline * Add tests * Fix cmbd * Fix test * Lint * Rm * Rm * Add back newline * rm ws * Rm list * Rm cmbd from cache * Avoid caching * Keep cmbd check * Add comment * Lint --------- Co-authored-by: Evan Roman <[email protected]> Co-authored-by: hallvictoria <[email protected]> * build: update Python Worker Version to 4.36.1 (#1660) Co-authored-by: AzureFunctionsPython <[email protected]> * initial changes * Update Python SDK Version to 1.23.0 (#1663) Co-authored-by: AzureFunctionsPython <[email protected]> * merges from ADO * merge fixes * merge fixes * merge fixes * merge fixes * don't run 313 unit tests yet * changes for builds --------- Co-authored-by: Evan <[email protected]> Co-authored-by: Evan Roman <[email protected]> Co-authored-by: AzureFunctionsPython <[email protected]> Co-authored-by: AzureFunctionsPython <[email protected]> * Merging changes * linting fixes * Addressed comments * Updated unit test and added missing protos files * fix e2e test reference * lint, mypy, add 3.13 to unittests * correct version check * syntax * syntax * fix unit tests, mypy * oops * format * lint * fix unittest dir for proxy * set env variable * update pyproject to use real deps * bump to a2 * bump v2 to a3 * Import v2 by default for LC * Refactoring and minor fixes --------- Co-authored-by: hallvictoria <[email protected]> Co-authored-by: Evan <[email protected]> Co-authored-by: Evan Roman <[email protected]> Co-authored-by: AzureFunctionsPython <[email protected]> Co-authored-by: AzureFunctionsPython <[email protected]> Co-authored-by: hallvictoria <[email protected]> Co-authored-by: Victoria Hall <[email protected]>
This pull request introduces several changes to enhance the handling of
collection_model_binding_data
. The most important changes include adding support forcollection_model_binding_data
in thedatumdef.py
andmeta.py
filesEnhancements to data handling:
azure_functions_worker/bindings/datumdef.py
: Added support forcollection_model_binding_data
in thefrom_typed_data
method.azure_functions_worker/bindings/meta.py
: Modified thedeferred_bindings_decode
method to handlecollection_model_binding_data
by accumulating the content of eachmodel_binding_data
. [1] [2]Description
Addresses https://github.com/Azure/azure-functions-pyfx-planning/issues/393
Today when a user uses a deferred binding, the host sends a model_binding_data object that is converted into its corresponding SDK type. This object contains important information about the SDK type, but most notably the content.
For triggers that can handle batch inputs such as EventHub, this means the cardinality="many" - in this case, the host sends us a collection_model_binding_data object, which is just 1 to N model_binding_data objects. Here is a sample for each:
model_binding_data:
collection_model_binding_data:
Notice that model_binding_data automatically has the fields accessible to do the conversion whereas collection_model_binding_data has each model_binding_data nested
From proto:
PR information
Quality of Code and Contribution Guidelines