Skip to content

Commit c3899d8

Browse files
gavin-aguiarhallvictoriaEvanR-DevEvan RomanAzureFunctionsPython
authored
feat: Adding support for python 3.13 (#1666)
* Proxy Worker: Initial Commit * Updated worker config to include 3.13 * Updated test_setup * Updated worker.py * Updated dispatcher * Updated syspath in worker.py * Updated path in worker.py * Updated worker.py * Removed reload in dispatcher * Updating v1 library worker name * Added dispatcher logs * Added dispatcher try/catch logs * Updated sys path * Dispatcher and dependency manager updates * Updated dispatcher and pyproject * Testing updates and refactoring * Bug fixes and refactoring * Added more unit tests * Added tests and fixed test setup * Updated test_setup * Updated test setup to add grpc dir copy * build: proxy worker build & test setup (#1664) * build: recognize collection_model_binding_data for batch inputs (#1655) * add cmbd * Add * Add * Rm newline * Add tests * Fix cmbd * Fix test * Lint * Rm * Rm * Add back newline * rm ws * Rm list * Rm cmbd from cache * Avoid caching * Keep cmbd check * Add comment * Lint --------- Co-authored-by: Evan Roman <[email protected]> Co-authored-by: hallvictoria <[email protected]> * build: update Python Worker Version to 4.36.1 (#1660) Co-authored-by: AzureFunctionsPython <[email protected]> * initial changes * Update Python SDK Version to 1.23.0 (#1663) Co-authored-by: AzureFunctionsPython <[email protected]> * merges from ADO * merge fixes * merge fixes * merge fixes * merge fixes * don't run 313 unit tests yet * changes for builds --------- Co-authored-by: Evan <[email protected]> Co-authored-by: Evan Roman <[email protected]> Co-authored-by: AzureFunctionsPython <[email protected]> Co-authored-by: AzureFunctionsPython <[email protected]> * Merging changes * linting fixes * Addressed comments * Updated unit test and added missing protos files * fix e2e test reference * lint, mypy, add 3.13 to unittests * correct version check * syntax * syntax * fix unit tests, mypy * oops * format * lint * fix unittest dir for proxy * set env variable * update pyproject to use real deps * bump to a2 * bump v2 to a3 * Import v2 by default for LC * Refactoring and minor fixes --------- Co-authored-by: hallvictoria <[email protected]> Co-authored-by: Evan <[email protected]> Co-authored-by: Evan Roman <[email protected]> Co-authored-by: AzureFunctionsPython <[email protected]> Co-authored-by: AzureFunctionsPython <[email protected]> Co-authored-by: hallvictoria <[email protected]> Co-authored-by: Victoria Hall <[email protected]>
1 parent 57a24f0 commit c3899d8

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+3150
-159
lines changed

‎.flake8

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
ignore = W503,E402,E731
77

88
exclude = .git, __pycache__, build, dist, .eggs, .github, .local, docs/,
9-
Samples, azure_functions_worker/protos/,
9+
Samples, azure_functions_worker/protos/, proxy_worker/protos/,
1010
azure_functions_worker/_thirdparty/typing_inspect.py,
1111
tests/unittests/test_typing_inspect.py,
1212
tests/unittests/broken_functions/syntax_error/main.py,

‎azure_functions_worker/protos/.gitignore

-1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,2 @@
1-
/_src
21
*_pb2.py
32
*_pb2_grpc.py

‎eng/templates/jobs/ci-emulator-tests.yml

+2
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,8 @@ jobs:
2121
PYTHON_VERSION: '3.11'
2222
Python312:
2323
PYTHON_VERSION: '3.12'
24+
Python313:
25+
PYTHON_VERSION: '3.13'
2426
steps:
2527
- task: UsePythonVersion@0
2628
inputs:

‎eng/templates/jobs/ci-unit-tests.yml

+19-2
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,8 @@ jobs:
1616
PYTHON_VERSION: '3.11'
1717
Python312:
1818
PYTHON_VERSION: '3.12'
19-
19+
Python313:
20+
PYTHON_VERSION: '3.13'
2021
steps:
2122
- task: UsePythonVersion@0
2223
inputs:
@@ -34,8 +35,24 @@ jobs:
3435
displayName: 'Install dependencies'
3536
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
3637
- bash: |
37-
python -m pytest -q -n auto --dist loadfile --reruns 4 --instafail --cov=./azure_functions_worker --cov-report xml --cov-branch tests/unittests
38+
PY_VER="$(PYTHON_VERSION)"
39+
echo "Python version: $PY_VER"
40+
41+
# Extract minor version
42+
PY_MINOR="${PY_VER#*.}"
43+
44+
if [ "$PY_MINOR" -ge 13 ]; then
45+
echo "Running proxy_worker tests (Python >= 3.13)..."
46+
python -m pytest -q -n auto --dist loadfile --reruns 4 --instafail \
47+
--cov=./proxy_worker --cov-report xml --cov-branch tests/unittest_proxy
48+
else
49+
echo "Running unittests (Python < 3.13)..."
50+
python -m pytest -q -n auto --dist loadfile --reruns 4 --instafail \
51+
--cov=./azure_functions_worker --cov-report xml --cov-branch tests/unittests
52+
fi
3853
displayName: "Running $(PYTHON_VERSION) Unit Tests"
3954
# Skip running tests for SDK and Extensions release branches. Public pipeline doesn't have permissions to download artifact.
4055
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
56+
env:
57+
PYTHON_VERSION: $(PYTHON_VERSION)
4158

‎eng/templates/official/jobs/build-artifacts.yml

+15
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,9 @@ jobs:
2424
Python312V4:
2525
pythonVersion: '3.12'
2626
workerPath: 'python/prodV4/worker.py'
27+
Python313V4:
28+
pythonVersion: '3.13'
29+
workerPath: 'python/proxyV4/worker.py'
2730
templateContext:
2831
outputParentDirectory: $(Build.ArtifactStagingDirectory)
2932
outputs:
@@ -62,6 +65,9 @@ jobs:
6265
Python312V4:
6366
pythonVersion: '3.12'
6467
workerPath: 'python/prodV4/worker.py'
68+
Python313V4:
69+
pythonVersion: '3.13'
70+
workerPath: 'python/proxyV4/worker.py'
6571
templateContext:
6672
outputParentDirectory: $(Build.ArtifactStagingDirectory)
6773
outputs:
@@ -100,6 +106,9 @@ jobs:
100106
Python312V4:
101107
pythonVersion: '3.12'
102108
workerPath: 'python/prodV4/worker.py'
109+
Python313V4:
110+
pythonVersion: '3.13'
111+
workerPath: 'python/proxyV4/worker.py'
103112
templateContext:
104113
outputParentDirectory: $(Build.ArtifactStagingDirectory)
105114
outputs:
@@ -137,6 +146,9 @@ jobs:
137146
Python312V4:
138147
pythonVersion: '3.12'
139148
workerPath: 'python/prodV4/worker.py'
149+
Python313V4:
150+
pythonVersion: '3.13'
151+
workerPath: 'python/proxyV4/worker.py'
140152
templateContext:
141153
outputParentDirectory: $(Build.ArtifactStagingDirectory)
142154
outputs:
@@ -168,6 +180,9 @@ jobs:
168180
Python312V4:
169181
pythonVersion: '3.12'
170182
workerPath: 'python/prodV4/worker.py'
183+
Python313V4:
184+
pythonVersion: '3.13'
185+
workerPath: 'python/proxyV4/worker.py'
171186
templateContext:
172187
outputParentDirectory: $(Build.ArtifactStagingDirectory)
173188
outputs:

‎eng/templates/official/jobs/ci-e2e-tests.yml

+18-2
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,15 @@ jobs:
6363
SQL_CONNECTION: $(LinuxSqlConnectionString312)
6464
EVENTGRID_URI: $(LinuxEventGridTopicUriString312)
6565
EVENTGRID_CONNECTION: $(LinuxEventGridConnectionKeyString312)
66+
Python313:
67+
PYTHON_VERSION: '3.13'
68+
STORAGE_CONNECTION: $(LinuxStorageConnectionString312)
69+
COSMOSDB_CONNECTION: $(LinuxCosmosDBConnectionString312)
70+
EVENTHUB_CONNECTION: $(LinuxEventHubConnectionString312)
71+
SERVICEBUS_CONNECTION: $(LinuxServiceBusConnectionString312)
72+
SQL_CONNECTION: $(LinuxSqlConnectionString312)
73+
EVENTGRID_URI: $(LinuxEventGridTopicUriString312)
74+
EVENTGRID_CONNECTION: $(LinuxEventGridConnectionKeyString312)
6675
steps:
6776
- task: UsePythonVersion@0
6877
inputs:
@@ -120,7 +129,14 @@ jobs:
120129
Write-Host "pipelineVarSet: $pipelineVarSet"
121130
$branch = "$(Build.SourceBranch)"
122131
Write-Host "Branch: $branch"
123-
if($branch.StartsWith("refs/heads/sdk/") -or $pipelineVarSet -eq "true")
132+
133+
$PY_VER = "$(PYTHON_VERSION)"
134+
Write-Host "Python version: $PY_VER"
135+
# Extract minor version as integers
136+
$PY_MINO = "${PY_VER#*.}"
137+
Write-Host "Branch: PY_MINOR"
138+
139+
if($branch.StartsWith("refs/heads/sdk/") -or $pipelineVarSet -eq "true" -or $PY_MINOR -ge 13 )
124140
{
125141
Write-Host "##vso[task.setvariable variable=skipTest;]true"
126142
}
@@ -129,7 +145,6 @@ jobs:
129145
Write-Host "##vso[task.setvariable variable=skipTest;]false"
130146
}
131147
displayName: 'Set skipTest variable'
132-
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
133148
- powershell: |
134149
Write-Host "skipTest: $(skipTest)"
135150
displayName: 'Display skipTest variable'
@@ -145,4 +160,5 @@ jobs:
145160
AzureWebJobsEventGridTopicUri: $(EVENTGRID_URI)
146161
AzureWebJobsEventGridConnectionKey: $(EVENTGRID_CONNECTION)
147162
skipTest: $(skipTest)
163+
PYAZURE_WEBHOST_DEBUG: true
148164
displayName: "Running $(PYTHON_VERSION) Python E2E Tests"

‎pack/Microsoft.Azure.Functions.V4.PythonWorker.nuspec

+5
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,11 @@
3838
<file src="..\3.12_LINUX_X64\**" target="tools\3.12\LINUX\X64" />
3939
<file src="..\3.12_OSX_X64\**" target="tools\3.12\OSX\X64" />
4040
<file src="..\3.12_OSX_X64\**" target="tools\3.12\OSX\Arm64" />
41+
<file src="..\3.13_WINDOWS_X64\**" target="tools\3.13\WINDOWS\X64" />
42+
<file src="..\3.13_WINDOWS_X86\**" target="tools\3.13\WINDOWS\X86" />
43+
<file src="..\3.13_LINUX_X64\**" target="tools\3.13\LINUX\X64" />
44+
<file src="..\3.13_OSX_X64\**" target="tools\3.13\OSX\X64" />
45+
<file src="..\3.13_OSX_X64\**" target="tools\3.13\OSX\Arm64" />
4146
<file src="..\python\prodV4\worker.config.json" target="tools" />
4247
<file src="Microsoft.Azure.Functions.PythonWorker.targets" target="build" />
4348
<file src="..\_manifest\spdx_2.2\manifest.spdx.json" target="SBOM\manifest.spdx.json" />

‎pack/scripts/mac_arm64_deps.sh

+7-1
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,10 @@ python -m invoke -c test_setup build-protos
1313

1414
cd ..
1515
cp .artifactignore "$BUILD_SOURCESDIRECTORY/deps"
16-
cp -r azure_functions_worker/protos "$BUILD_SOURCESDIRECTORY/deps/azure_functions_worker"
16+
17+
version_minor=$(echo $1 | cut -d '.' -f 2)
18+
if [[ $version_minor -lt 13 ]]; then
19+
cp -r azure_functions_worker/protos "$BUILD_SOURCESDIRECTORY/deps/azure_functions_worker"
20+
else
21+
cp -r proxy_worker/protos "$BUILD_SOURCESDIRECTORY/deps/proxy_worker"
22+
fi

‎pack/scripts/nix_deps.sh

+7-1
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,10 @@ python -m invoke -c test_setup build-protos
1313

1414
cd ..
1515
cp .artifactignore "$BUILD_SOURCESDIRECTORY/deps"
16-
cp -r azure_functions_worker/protos "$BUILD_SOURCESDIRECTORY/deps/azure_functions_worker"
16+
17+
version_minor=$(echo $1 | cut -d '.' -f 2)
18+
if [[ $version_minor -lt 13 ]]; then
19+
cp -r azure_functions_worker/protos "$BUILD_SOURCESDIRECTORY/deps/azure_functions_worker"
20+
else
21+
cp -r proxy_worker/protos "$BUILD_SOURCESDIRECTORY/deps/proxy_worker"
22+
fi

‎pack/scripts/win_deps.ps1

+14-2
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,16 @@
1+
param (
2+
[string]$pythonVersion
3+
)
4+
$versionParts = $pythonVersion -split '\.' # Splitting by dot
5+
$versionMinor = [int]$versionParts[1]
6+
17
python -m venv .env
28
.env\Scripts\Activate.ps1
39
python -m pip install --upgrade pip
410

511
python -m pip install .
612

713
$depsPath = Join-Path -Path $env:BUILD_SOURCESDIRECTORY -ChildPath "deps"
8-
$protosPath = Join-Path -Path $depsPath -ChildPath "azure_functions_worker/protos"
914

1015
python -m pip install . azure-functions --no-compile --target $depsPath.ToString()
1116

@@ -15,4 +20,11 @@ python -m invoke -c test_setup build-protos
1520

1621
cd ..
1722
Copy-Item -Path ".artifactignore" -Destination $depsPath.ToString()
18-
Copy-Item -Path "azure_functions_worker/protos/*" -Destination $protosPath.ToString() -Recurse -Force
23+
24+
if ($versionMinor -lt 13) {
25+
$protosPath = Join-Path -Path $depsPath -ChildPath "azure_functions_worker/protos"
26+
Copy-Item -Path "azure_functions_worker/protos/*" -Destination $protosPath.ToString() -Recurse -Force
27+
} else {
28+
$protosPath = Join-Path -Path $depsPath -ChildPath "proxy_worker/protos"
29+
Copy-Item -Path "proxy_worker/protos/*" -Destination $protosPath.ToString() -Recurse -Force
30+
}

‎pack/templates/macos_64_env_gen.yml

+35
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,19 @@ steps:
88
inputs:
99
versionSpec: ${{ parameters.pythonVersion }}
1010
addToPath: true
11+
- bash: |
12+
major=$(echo $(pythonVersion) | cut -d. -f1)
13+
minor=$(echo $(pythonVersion) | cut -d. -f2)
14+
echo "##vso[task.setvariable variable=pythonMajor]$major"
15+
echo "##vso[task.setvariable variable=pythonMinor]$minor"
16+
echo $pythonMinor
17+
displayName: 'Parse pythonVersion'
1118
- task: ShellScript@2
1219
inputs:
1320
disableAutoCwd: true
1421
scriptPath: 'pack/scripts/mac_arm64_deps.sh'
22+
args: '${{ parameters.pythonVersion }}'
23+
displayName: 'Install Dependencies'
1524
- bash: |
1625
pip install pip-audit
1726
pip-audit -r requirements.txt
@@ -41,4 +50,30 @@ steps:
4150
!pkg_resources/**
4251
!*.dist-info/**
4352
!werkzeug/debug/shared/debugger.js
53+
!proxy_worker/**
54+
targetFolder: '$(Build.ArtifactStagingDirectory)'
55+
condition: in(variables['pythonMinor'], '7', '8', '9', '10', '11', '12')
56+
displayName: 'Copy azure_functions_worker files'
57+
- task: CopyFiles@2
58+
inputs:
59+
sourceFolder: '$(Build.SourcesDirectory)/deps'
60+
contents: |
61+
**
62+
!grpc_tools/**/*
63+
!grpcio_tools*/*
64+
!build/**
65+
!docs/**
66+
!pack/**
67+
!python/**
68+
!tests/**
69+
!setuptools*/**
70+
!_distutils_hack/**
71+
!distutils-precedence.pth
72+
!pkg_resources/**
73+
!*.dist-info/**
74+
!werkzeug/debug/shared/debugger.js
75+
!azure_functions_worker/**
76+
!dateutil/**
4477
targetFolder: '$(Build.ArtifactStagingDirectory)'
78+
condition: in(variables['pythonMinor'], '13')
79+
displayName: 'Copy proxy_worker files'

‎pack/templates/nix_env_gen.yml

+35
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,19 @@ steps:
88
inputs:
99
versionSpec: ${{ parameters.pythonVersion }}
1010
addToPath: true
11+
- bash: |
12+
major=$(echo $(pythonVersion) | cut -d. -f1)
13+
minor=$(echo $(pythonVersion) | cut -d. -f2)
14+
echo "##vso[task.setvariable variable=pythonMajor]$major"
15+
echo "##vso[task.setvariable variable=pythonMinor]$minor"
16+
echo $pythonMinor
17+
displayName: 'Parse pythonVersion'
1118
- task: ShellScript@2
1219
inputs:
1320
disableAutoCwd: true
1421
scriptPath: 'pack/scripts/nix_deps.sh'
22+
args: '${{ parameters.pythonVersion }}'
23+
displayName: 'Install Dependencies'
1524
- bash: |
1625
pip install pip-audit
1726
pip-audit -r requirements.txt
@@ -41,4 +50,30 @@ steps:
4150
!pkg_resources/**
4251
!*.dist-info/**
4352
!werkzeug/debug/shared/debugger.js
53+
!proxy_worker/**
54+
targetFolder: '$(Build.ArtifactStagingDirectory)'
55+
condition: in(variables['pythonMinor'], '7', '8', '9', '10', '11', '12')
56+
displayName: 'Copy azure_functions_worker files'
57+
- task: CopyFiles@2
58+
inputs:
59+
sourceFolder: '$(Build.SourcesDirectory)/deps'
60+
contents: |
61+
**
62+
!grpc_tools/**/*
63+
!grpcio_tools*/*
64+
!build/**
65+
!docs/**
66+
!pack/**
67+
!python/**
68+
!tests/**
69+
!setuptools*/**
70+
!_distutils_hack/**
71+
!distutils-precedence.pth
72+
!pkg_resources/**
73+
!*.dist-info/**
74+
!werkzeug/debug/shared/debugger.js
75+
!dateutil/**
76+
!azure_functions_worker/**
4477
targetFolder: '$(Build.ArtifactStagingDirectory)'
78+
condition: in(variables['pythonMinor'], '13')
79+
displayName: 'Copy proxy_worker files'

‎pack/templates/win_env_gen.yml

+34
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,17 @@ steps:
99
versionSpec: ${{ parameters.pythonVersion }}
1010
architecture: ${{ parameters.architecture }}
1111
addToPath: true
12+
- bash: |
13+
major=$(echo $(pythonVersion) | cut -d. -f1)
14+
minor=$(echo $(pythonVersion) | cut -d. -f2)
15+
echo "##vso[task.setvariable variable=pythonMajor]$major"
16+
echo "##vso[task.setvariable variable=pythonMinor]$minor"
17+
echo $pythonMinor
18+
displayName: 'Parse pythonVersion'
1219
- task: PowerShell@2
1320
inputs:
1421
filePath: 'pack\scripts\win_deps.ps1'
22+
arguments: '${{ parameters.pythonVersion }}'
1523
- bash: |
1624
pip install pip-audit
1725
pip-audit -r requirements.txt
@@ -41,4 +49,30 @@ steps:
4149
!pkg_resources\**
4250
!*.dist-info\**
4351
!werkzeug\debug\shared\debugger.js
52+
!proxy_worker\**
53+
targetFolder: '$(Build.ArtifactStagingDirectory)'
54+
condition: in(variables['pythonMinor'], '7', '8', '9', '10', '11', '12')
55+
displayName: 'Copy azure_functions_worker files'
56+
- task: CopyFiles@2
57+
inputs:
58+
sourceFolder: '$(Build.SourcesDirectory)\deps'
59+
contents: |
60+
**
61+
!grpc_tools\**\*
62+
!grpcio_tools*\*
63+
!build\**
64+
!docs\**
65+
!pack\**
66+
!python\**
67+
!tests\**
68+
!setuptools*\**
69+
!_distutils_hack\**
70+
!distutils-precedence.pth
71+
!pkg_resources\**
72+
!*.dist-info\**
73+
!werkzeug\debug\shared\debugger.js
74+
!dateutil\**
75+
!azure_functions_worker\**
4476
targetFolder: '$(Build.ArtifactStagingDirectory)'
77+
condition: in(variables['pythonMinor'], '13')
78+
displayName: 'Copy proxy_worker files'

‎proxy_worker/__init__.py

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Copyright (c) Microsoft Corporation. All rights reserved.
2+
# Licensed under the MIT License.

‎proxy_worker/__main__.py

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Copyright (c) Microsoft Corporation. All rights reserved.
2+
# Licensed under the MIT License.
3+
from proxy_worker import start_worker
4+
5+
if __name__ == '__main__':
6+
start_worker.start()

0 commit comments

Comments
 (0)