Skip to content

Commit 4b4f59d

Browse files
author
Evan Roman
committed
Chng
1 parent 959563a commit 4b4f59d

File tree

22 files changed

+658
-0
lines changed

22 files changed

+658
-0
lines changed

‎azurefunctions-extensions-bindings-eventhub/CHANGELOG.md

Whitespace-only changes.

‎azurefunctions-extensions-bindings-eventhub/LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
Copyright (c) Microsoft Corporation.
2+
3+
MIT License
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎azurefunctions-extensions-bindings-eventhub/MANIFEST.in

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
recursive-include azure *.py *.pyi
2+
recursive-include tests *.py
3+
include LICENSE README.md

‎azurefunctions-extensions-bindings-eventhub/README.md

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
# Azure Functions Extensions Bindings EventHub library for Python
2+
This library allows Blob Trigger and Blob Input bindings in Python Function Apps to recognize and bind to client types from the
3+
Azure Storage Blob sdk.
4+
5+
Blob client types can be generated from:
6+
7+
* Blob Triggers
8+
* Blob Input
9+
10+
[Source code](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob)
11+
[Package (PyPi)](https://pypi.org/project/azurefunctions-extensions-bindings-blob/)
12+
| API reference documentation
13+
| Product documentation
14+
| [Samples](hhttps://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples)
15+
16+
17+
## Getting started
18+
19+
### Prerequisites
20+
* Python 3.9 or later is required to use this package. For more details, please read our page on [Python Functions version support policy](https://learn.microsoft.com/en-us/azure/azure-functions/functions-versions?tabs=isolated-process%2Cv4&pivots=programming-language-python#languages).
21+
22+
* You must have an [Azure subscription](https://azure.microsoft.com/free/) and an
23+
[Azure storage account](https://docs.microsoft.com/azure/storage/common/storage-account-overview) to use this package.
24+
25+
### Install the package
26+
Install the Azure Functions Extensions Bindings Blob library for Python with pip:
27+
28+
```bash
29+
pip install azurefunctions-extensions-bindings-blob
30+
```
31+
32+
### Create a storage account
33+
If you wish to create a new storage account, you can use the
34+
[Azure Portal](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-portal),
35+
[Azure PowerShell](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-powershell),
36+
or [Azure CLI](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-cli):
37+
38+
```bash
39+
# Create a new resource group to hold the storage account -
40+
# if using an existing resource group, skip this step
41+
az group create --name my-resource-group --location westus2
42+
43+
# Create the storage account
44+
az storage account create -n my-storage-account-name -g my-resource-group
45+
```
46+
47+
### Bind to the SDK-type
48+
The Azure Functions Extensions Bindings Blob library for Python allows you to create a function app with a Blob Trigger or
49+
Blob Input and define the type as a BlobClient, ContainerClient, or StorageStreamDownloader. Instead of receiving
50+
an InputStream, when the function is executed, the type returned will be the defined SDK-type and have all of the
51+
properties and methods available as seen in the Azure Storage Blob library for Python.
52+
53+
54+
```python
55+
import logging
56+
import azure.functions as func
57+
import azurefunctions.extensions.bindings.blob as blob
58+
59+
@app.blob_trigger(arg_name="client",
60+
path="PATH/TO/BLOB",
61+
connection="AzureWebJobsStorage")
62+
def blob_trigger(client: blob.BlobClient):
63+
logging.info(f"Python blob trigger function processed blob \n"
64+
f"Properties: {client.get_blob_properties()}\n"
65+
f"Blob content head: {client.download_blob(encoding="utf-8").read(size=1)}")
66+
67+
68+
@app.route(route="file")
69+
@app.blob_input(arg_name="client",
70+
path="PATH/TO/BLOB",
71+
connection="AzureWebJobsStorage")
72+
def blob_input(req: func.HttpRequest, client: blob.BlobClient):
73+
logging.info(f"Python blob input function processed blob \n"
74+
f"Properties: {client.get_blob_properties()}\n"
75+
f"Blob content head: {client.download_blob(encoding="utf-8").read(size=1)}")
76+
```
77+
78+
## Troubleshooting
79+
### General
80+
The SDK-types raise exceptions defined in [Azure Core](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md).
81+
82+
This list can be used for reference to catch thrown exceptions. To get the specific error code of the exception, use the `error_code` attribute, i.e, `exception.error_code`.
83+
84+
## Next steps
85+
86+
### More sample code
87+
88+
Get started with our [Blob samples](hhttps://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples).
89+
90+
Several samples are available in this GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Storage Blobs:
91+
92+
* [blob_samples_blobclient](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples/blob_samples_blobclient) - Examples for using the BlobClient type:
93+
* From BlobTrigger
94+
* From BlobInput
95+
96+
* [blob_samples_containerclient](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples/blob_samples_containerclient) - Examples for using the ContainerClient type:
97+
* From BlobTrigger
98+
* From BlobInput
99+
100+
* [blob_samples_storagestreamdownloader](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-blob/samples/blob_samples_storagestreamdownloader) - Examples for using the StorageStreamDownloader type:
101+
* From BlobTrigger
102+
* From BlobInput
103+
104+
### Additional documentation
105+
For more information on the Azure Storage Blob SDK, see the [Azure Blob storage documentation](https://docs.microsoft.com/azure/storage/blobs/) on docs.microsoft.com
106+
and the [Azure Storage Blobs README](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/storage/azure-storage-blob).
107+
108+
## Contributing
109+
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
110+
111+
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
112+
113+
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [[email protected]](mailto:[email protected]) with any additional questions or comments.

‎azurefunctions-extensions-bindings-eventhub/azurefunctions/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__path__ = __import__("pkgutil").extend_path(__path__, __name__)

‎azurefunctions-extensions-bindings-eventhub/azurefunctions/extensions/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__path__ = __import__("pkgutil").extend_path(__path__, __name__)

‎azurefunctions-extensions-bindings-eventhub/azurefunctions/extensions/bindings/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__path__ = __import__("pkgutil").extend_path(__path__, __name__)

‎azurefunctions-extensions-bindings-eventhub/azurefunctions/extensions/bindings/eventhub/__init__.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Copyright (c) Microsoft Corporation. All rights reserved.
2+
# Licensed under the MIT License.
3+
4+
from .eventHubData import EventHubData
5+
from .eventHubDataConverter import EventHubDataConverter
6+
7+
__all__ = [
8+
"EventHubData",
9+
"EventHubDataConverter",
10+
]
11+
12+
__version__ = "1.0.0b2"

‎azurefunctions-extensions-bindings-eventhub/azurefunctions/extensions/bindings/eventhub/eventHubData.py

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Copyright (c) Microsoft Corporation. All rights reserved.
2+
# Licensed under the MIT License.
3+
4+
import re
5+
from typing import Union
6+
import uamqp
7+
8+
from azure.eventhub import EventData
9+
from azurefunctions.extensions.base import Datum, SdkType
10+
11+
12+
class EventHubData(SdkType):
13+
def __init__(self, *, data: Union[bytes, Datum]) -> None:
14+
# model_binding_data properties
15+
self._data = data
16+
self._version = None
17+
self._source = None
18+
self._content_type = None
19+
self._content = None
20+
self.decoded_message = None
21+
if self._data:
22+
self._version = data.version
23+
self._source = data.source
24+
self._content_type = data.content_type
25+
self._content = data.content
26+
self.decoded_message = self.__get_eventhub_content(self._content)
27+
28+
def __get_eventhub_content(self, content):
29+
if content:
30+
return uamqp.Message().decode_from_bytes(content)
31+
else:
32+
return None
33+
34+
def get_sdk_type(self):
35+
# https://github.com/Azure/azure-sdk-for-python/issues/39711
36+
if self.decoded_message:
37+
return EventData._from_message(self.decoded_message)
38+
else:
39+
return None

‎azurefunctions-extensions-bindings-eventhub/azurefunctions/extensions/bindings/eventhub/eventHubDataConverter.py

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Copyright (c) Microsoft Corporation. All rights reserved.
2+
# Licensed under the MIT License.
3+
4+
from typing import Any
5+
6+
from azurefunctions.extensions.base import Datum, InConverter, OutConverter
7+
8+
from .eventHubData import EventHubData
9+
10+
11+
class EventHubDataConverter(
12+
InConverter,
13+
OutConverter,
14+
binding="eventHub",
15+
trigger="eventHubTrigger",
16+
):
17+
@classmethod
18+
def check_input_type_annotation(cls, pytype: type) -> bool:
19+
return issubclass(
20+
pytype, (EventHubData)
21+
)
22+
23+
@classmethod
24+
def decode(cls, data: Datum, *, trigger_metadata, pytype) -> Any:
25+
if data is None or data.type is None:
26+
return None
27+
28+
data_type = data.type
29+
30+
if data_type == "model_binding_data":
31+
data = data.value
32+
else:
33+
raise ValueError(
34+
f'unexpected type of data received for the "eventhub" binding '
35+
f": {data_type!r}"
36+
)
37+
38+
# Determines which sdk type to return based on pytype
39+
if pytype == EventHubData:
40+
return EventHubData(data=data).get_sdk_type()
41+
else:
42+
return None

‎azurefunctions-extensions-bindings-eventhub/pyproject.toml

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
[build-system]
2+
requires = ["setuptools >= 61.0"]
3+
build-backend = "setuptools.build_meta"
4+
5+
[project]
6+
name = "azurefunctions-extensions-bindings-eventhub"
7+
dynamic = ["version"]
8+
requires-python = ">=3.9"
9+
authors = [{ name = "Azure Functions team at Microsoft Corp.", email = "[email protected]"}]
10+
description = "EventHub Python worker extension for Azure Functions."
11+
readme = "README.md"
12+
license = {text = "MIT License"}
13+
classifiers= [
14+
'License :: OSI Approved :: MIT License',
15+
'Intended Audience :: Developers',
16+
'Programming Language :: Python :: 3',
17+
'Programming Language :: Python :: 3.9',
18+
'Programming Language :: Python :: 3.10',
19+
'Programming Language :: Python :: 3.11',
20+
'Operating System :: Microsoft :: Windows',
21+
'Operating System :: POSIX',
22+
'Operating System :: MacOS :: MacOS X',
23+
'Environment :: Web Environment',
24+
'Development Status :: 5 - Production/Stable',
25+
]
26+
dependencies = [
27+
'azurefunctions-extensions-base',
28+
'azure-eventhub~=5.13.0',
29+
'azure-identity~=1.19.0'
30+
]
31+
32+
[project.optional-dependencies]
33+
dev = [
34+
'pytest',
35+
'pytest-cov',
36+
'coverage',
37+
'pytest-instafail',
38+
'pre-commit'
39+
]
40+
41+
[tool.setuptools.dynamic]
42+
version = {attr = "azurefunctions.extensions.bindings.eventhub.__version__"}
43+
44+
[tool.setuptools.packages.find]
45+
exclude = [
46+
'azurefunctions.extensions.bindings','azurefunctions.extensions',
47+
'azurefunctions', 'tests', 'samples'
48+
]
49+

‎azurefunctions-extensions-bindings-eventhub/samples/README.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
page_type: sample
3+
languages:
4+
- python
5+
products:
6+
- azure
7+
- azure-functions
8+
- azure-functions-extensions
9+
- azurefunctions-extensions-bindings-eventhub
10+
urlFragment: extension-eventhub-samples
11+
---
12+
13+
# Azure Functions Extension EventHub library for Python samples
14+
15+
These are code samples that show common scenario operations with the Azure Functions Extension EventHub library.
16+
17+
These samples relate to the Azure EventHub library being used as part of a Python Function App. For
18+
examples on how to use the Azure EventHub library, please see [Azure EventHub samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/eventhub/azure-eventhub/samples)
19+
20+
* [blob_samples_eventhubdata](https://github.com/Azure/azure-functions-python-extensions/tree/main/azurefunctions-extensions-bindings-eventhub/samples/eventhub_samples_eventhubdata) - Examples for using the EventData type:
21+
* From EventHubTrigger
22+
23+
## Prerequisites
24+
* Python 3.9 or later is required to use this package. For more details, please read our page on [Python Functions version support policy](https://learn.microsoft.com/en-us/azure/azure-functions/functions-versions?tabs=isolated-process%2Cv4&pivots=programming-language-python#languages).
25+
* You must have an [Azure subscription](https://azure.microsoft.com/free/) and an
26+
[Azure storage account](https://docs.microsoft.com/azure/storage/common/storage-account-overview) to use this package.
27+
28+
## Setup
29+
30+
1. Install [Core Tools](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local?tabs=windows%2Cisolated-process%2Cnode-v4%2Cpython-v2%2Chttp-trigger%2Ccontainer-apps&pivots=programming-language-python)
31+
2. Install the Azure Functions Extension EventHub library for Python with [pip](https://pypi.org/project/pip/):
32+
33+
```bash
34+
pip install azurefunctions-extensions-bindings-eventhub
35+
```
36+
37+
3. Clone or download this sample repository
38+
4. Open the sample folder in Visual Studio Code or your IDE of choice.
39+
40+
## Running the samples
41+
42+
1. Open a terminal window and `cd` to the directory that the sample you wish to run is saved in.
43+
2. Set the environment variables specified in the sample file you wish to run.
44+
3. Install the required dependencies
45+
```bash
46+
pip install -r requirements.txt
47+
```
48+
4. Start the Functions runtime
49+
```bash
50+
func start
51+
```
52+
5. Execute the function by uploading an event to the EventHub that is being targeted.
53+
54+
## Next steps
55+
56+
Visit the [SDK-type bindings in Python reference documentation]() to learn more about how to use SDK-type bindings in a Python Function App and the
57+
[API reference documentation](https://aka.ms/azsdk-python-storage-blob-ref) to learn more about
58+
what you can do with the Azure EventHub library.

‎azurefunctions-extensions-bindings-eventhub/samples/eventhub_samples_eventhubdata/function_app.py

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# coding: utf-8
2+
3+
# -------------------------------------------------------------------------
4+
# Copyright (c) Microsoft Corporation. All rights reserved.
5+
# Licensed under the MIT License. See License.txt in the project root for
6+
# license information.
7+
# --------------------------------------------------------------------------
8+
9+
import logging
10+
11+
import azure.functions as func
12+
import azurefunctions.extensions.bindings.eventhub as eh
13+
14+
app = func.FunctionApp(http_auth_level=func.AuthLevel.ANONYMOUS)
15+
16+
"""
17+
FOLDER: eventhub_samples_eventhubdata
18+
DESCRIPTION:
19+
These samples demonstrate how to obtain EventHubData from an EventHub Trigger.
20+
USAGE:
21+
There are different ways to connect to an EventHub via the connection property and
22+
envionrment variables specifiied in local.settings.json
23+
24+
The connection property can be:
25+
- The name of an application setting containing a connection string
26+
- The name of a shared prefix for multiple application settings, together defining an identity-based connection
27+
28+
For more information, see:
29+
https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-hubs-trigger?tabs=python-v2%2Cisolated-process%2Cnodejs-v4%2Cfunctionsv2%2Cextensionv5&pivots=programming-language-python
30+
"""
31+
32+
33+
@app.event_hub_message_trigger(
34+
arg_name="eh_data", event_hub_name="EVENTHUB_NAME", connection="AzureWebJobsStorage"
35+
)
36+
def eventhub_trigger(eh_data: eh.EventHubData):
37+
logging.info(
38+
"Python EventHub trigger processed an event %s",
39+
eh_data.body_as_str()
40+
)
41+

0 commit comments

Comments
 (0)