Skip to content

feat: supporting EventHub trigger SDK bindings #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Apr 2, 2025
Merged
12 changes: 6 additions & 6 deletions azurefunctions-extensions-bindings-blob/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ Blob client types can be generated from:
* Blob Triggers
* Blob Input

[Source code](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob)
[Source code](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-blob)
[Package (PyPi)](https://pypi.org/project/azurefunctions-extensions-bindings-blob/)
| API reference documentation
| Product documentation
| [Samples](hhttps://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples)
| [Samples](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-blob/samples)


## Getting started
Expand Down Expand Up @@ -85,19 +85,19 @@ This list can be used for reference to catch thrown exceptions. To get the speci

### More sample code

Get started with our [Blob samples](hhttps://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples).
Get started with our [Blob samples](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-blob/samples).

Several samples are available in this GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Storage Blobs:

* [blob_samples_blobclient](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples/blob_samples_blobclient) - Examples for using the BlobClient type:
* [blob_samples_blobclient](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-blob/samples/blob_samples_blobclient) - Examples for using the BlobClient type:
* From BlobTrigger
* From BlobInput

* [blob_samples_containerclient](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples/blob_samples_containerclient) - Examples for using the ContainerClient type:
* [blob_samples_containerclient](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-blob/samples/blob_samples_containerclient) - Examples for using the ContainerClient type:
* From BlobTrigger
* From BlobInput

* [blob_samples_storagestreamdownloader](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples/blob_samples_storagestreamdownloader) - Examples for using the StorageStreamDownloader type:
* [blob_samples_storagestreamdownloader](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-blob/samples/blob_samples_storagestreamdownloader) - Examples for using the StorageStreamDownloader type:
* From BlobTrigger
* From BlobInput

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@


class BlobClient(SdkType):
def __init__(self, *, data: Union[bytes, Datum]) -> None:
def __init__(self, *, data: Datum) -> None:
# model_binding_data properties
self._data = data
self._using_managed_identity = False
Expand Down
8 changes: 4 additions & 4 deletions azurefunctions-extensions-bindings-blob/samples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,15 @@ These are code samples that show common scenario operations with the Azure Funct
These samples relate to the Azure Storage Blob client library being used as part of a Python Function App. For
examples on how to use the Azure Storage Blob client library, please see [Azure Storage Blob samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/storage/azure-storage-blob/samples)

* [blob_samples_blobclient](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples/blob_samples_blobclient) - Examples for using the BlobClient type:
* [blob_samples_blobclient](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-blob/samples/blob_samples_blobclient) - Examples for using the BlobClient type:
* From BlobTrigger
* From BlobInput

* [blob_samples_containerclient](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples/blob_samples_containerclient) - Examples for using the ContainerClient type:
* [blob_samples_containerclient](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-blob/samples/blob_samples_containerclient) - Examples for using the ContainerClient type:
* From BlobTrigger
* From BlobInput

* [blob_samples_storagestreamdownloader](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples/blob_samples_storagestreamdownloader) - Examples for using the StorageStreamDownloader type:
* [blob_samples_storagestreamdownloader](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-blob/samples/blob_samples_storagestreamdownloader) - Examples for using the StorageStreamDownloader type:
* From BlobTrigger
* From BlobInput

Expand Down Expand Up @@ -63,6 +63,6 @@ based on the type of function you wish to execute.

## Next steps

Visit the [SDK-type bindings in Python reference documentation]() to learn more about how to use SDK-type bindings in a Python Function App and the
Visit the [SDK-type bindings in Python reference documentation](https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python?tabs=get-started%2Casgi%2Capplication-level&pivots=python-mode-decorators#sdk-type-bindings-preview) to learn more about how to use SDK-type bindings in a Python Function App and the
[API reference documentation](https://aka.ms/azsdk-python-storage-blob-ref) to learn more about
what you can do with the Azure Storage Blob client library.
Empty file.
21 changes: 21 additions & 0 deletions azurefunctions-extensions-bindings-eventhub/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Copyright (c) Microsoft Corporation.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
3 changes: 3 additions & 0 deletions azurefunctions-extensions-bindings-eventhub/MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
recursive-include azure *.py *.pyi
recursive-include tests *.py
include LICENSE README.md
90 changes: 90 additions & 0 deletions azurefunctions-extensions-bindings-eventhub/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Azure Functions Extensions Bindings EventHub library for Python
This library allows an EventHub Trigger binding in Python Function Apps to recognize and bind to the types from the
Azure EventHub sdk (EventData).

EventHub types can be generated from:

[Source code](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-eventhub)
[Package (PyPi)](https://pypi.org/project/azurefunctions-extensions-bindings-eventhub/)
| API reference documentation
| Product documentation
| [Samples](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-eventhub/samples)


## Getting started

### Prerequisites
* Python 3.9 or later is required to use this package. For more details, please read our page on [Python Functions version support policy](https://learn.microsoft.com/en-us/azure/azure-functions/functions-versions?tabs=isolated-process%2Cv4&pivots=programming-language-python#languages).

* You must have an [Azure subscription](https://azure.microsoft.com/free/) and an
[Azure storage account](https://docs.microsoft.com/azure/storage/common/storage-account-overview) to use this package.

### Install the package
Install the Azure Functions Extensions Bindings EventHub library for Python with pip:

```bash
pip install azurefunctions-extensions-bindings-eventhub
```

### Create a storage account
If you wish to create a new storage account, you can use the
[Azure Portal](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-portal),
[Azure PowerShell](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-powershell),
or [Azure CLI](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-cli):

```bash
# Create a new resource group to hold the storage account -
# if using an existing resource group, skip this step
az group create --name my-resource-group --location westus2

# Create the storage account
az storage account create -n my-storage-account-name -g my-resource-group
```

### Bind to the SDK-type
The Azure Functions Extensions Bindings EventHub library for Python allows you to create a function app with an EventHub Trigger
and define the type as an EventData. Instead of receiving an EventHubEvent, when the function is executed, the type returned will be the defined SDK-type and have all of the properties and methods available as seen in the Azure EventHub library for Python.


```python
import logging
import azure.functions as func
import azurefunctions.extensions.bindings.eventhub as eh

@app.event_hub_message_trigger(
arg_name="eh_data", event_hub_name="EVENTHUB_NAME", connection="AzureWebJobsStorage"
)
def eventhub_trigger(eh_data: eh.EventHubData):
logging.info(
"Python EventHub trigger processed an event %s",
eh_data.body_as_str()
)
```

## Troubleshooting
### General
The SDK-types raise exceptions defined in [Azure Core](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md).

This list can be used for reference to catch thrown exceptions. To get the specific error code of the exception, use the `error_code` attribute, i.e, `exception.error_code`.

## Next steps

### More sample code

Get started with our [EventHub samples](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-eventhub/samples).

Several samples are available in this GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with EventHubs:

* [eventhub_samples_eventhubdata](https://github.com/Azure/azure-functions-python-extensions/tree/dev/azurefunctions-extensions-bindings-eventhub/samples/eventhub_samples_eventhubdata) - Examples for using the EventHubData type:
* From EventHubTrigger

### Additional documentation
For more information on the Azure EventHub SDK, see the [Azure EventHub documentation](https://learn.microsoft.com/en-us/azure/event-hubs/) on learn.microsoft.com
and the [Azure EventHub README](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/eventhub/azure-eventhub/README.md).

## Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__path__ = __import__("pkgutil").extend_path(__path__, __name__)
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__path__ = __import__("pkgutil").extend_path(__path__, __name__)
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__path__ = __import__("pkgutil").extend_path(__path__, __name__)
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.

from .eventHubData import EventHubData
from .eventHubDataConverter import EventHubDataConverter

__all__ = [
"EventHubData",
"EventHubDataConverter",
]

__version__ = "1.0.0b1"
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.

from typing import Optional, Union
import uamqp

from azure.eventhub import EventData
from azurefunctions.extensions.base import Datum, SdkType


class EventHubData(SdkType):
def __init__(self, *, data: Datum) -> None:
# model_binding_data properties
self._data = data
self._version = None
self._source = None
self._content_type = None
self._content = None
self.decoded_message = None
if self._data:
self._version = data.version
self._source = data.source
self._content_type = data.content_type
self._content = data.content
self.decoded_message = self.__get_eventhub_content(self._content)

def __get_eventhub_content(self, content):
"""
When receiving the EventBindingData, the content field is in the form of bytes.
This content must be decoded in order to construct an EventData object from the azure.eventhub SDK.
The .NET worker uses the Azure.Core.Amqp library to do this:
https://github.com/Azure/azure-functions-dotnet-worker/blob/main/extensions/Worker.Extensions.EventHubs/src/EventDataConverter.cs#L45
"""
if content:
try:
return uamqp.Message().decode_from_bytes(content)
except Exception as e:
raise ValueError(f"Failed to decode EventHub content: {e}") from e

return None

def get_sdk_type(self) -> Optional[EventData]:
"""
When receiving an EventHub message, the content portion after being decoded
is used in the constructor to create an EventData object. This will contains
fields such as message, enqueue_time, and more.
"""
# https://github.com/Azure/azure-sdk-for-python/issues/39711
if self.decoded_message:
return EventData._from_message(self.decoded_message)

return None
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.

from typing import Any, Optional

from azurefunctions.extensions.base import Datum, InConverter, OutConverter

from .eventHubData import EventHubData


class EventHubDataConverter(
InConverter,
OutConverter,
binding="eventHub",
trigger="eventHubTrigger",
):
@classmethod
def check_input_type_annotation(cls, pytype: type) -> bool:
return issubclass(
pytype, (EventHubData)
)

@classmethod
def decode(cls, data: Datum, *, trigger_metadata, pytype) -> Optional[Any]:
if data is None or data.type is None:
return None

if data.type != "model_binding_data":
raise ValueError(
"Unexpected type of data received for the 'eventhub' binding: "
+ repr(data.type)
)

content = data.value

# Determines which sdk type to return based on pytype
if pytype == EventHubData:
return EventHubData(data=content).get_sdk_type()

return None
49 changes: 49 additions & 0 deletions azurefunctions-extensions-bindings-eventhub/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
[build-system]
requires = ["setuptools >= 61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "azurefunctions-extensions-bindings-eventhub"
dynamic = ["version"]
requires-python = ">=3.9"
authors = [{ name = "Azure Functions team at Microsoft Corp.", email = "[email protected]"}]
description = "EventHub Python worker extension for Azure Functions."
readme = "README.md"
license = {text = "MIT License"}
classifiers= [
'License :: OSI Approved :: MIT License',
'Intended Audience :: Developers',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12',
'Operating System :: Microsoft :: Windows',
'Operating System :: POSIX',
'Operating System :: MacOS :: MacOS X',
'Environment :: Web Environment',
'Development Status :: 5 - Production/Stable',
]
dependencies = [
'azurefunctions-extensions-base',
'azure-eventhub~=5.13.0'
]

[project.optional-dependencies]
dev = [
'pytest',
'pytest-cov',
'coverage',
'pytest-instafail',
'pre-commit'
]

[tool.setuptools.dynamic]
version = {attr = "azurefunctions.extensions.bindings.eventhub.__version__"}

[tool.setuptools.packages.find]
exclude = [
'azurefunctions.extensions.bindings','azurefunctions.extensions',
'azurefunctions', 'tests', 'samples'
]

Loading