Skip to content

deps!: Drop support for Python 3.7 and 3.8 (AI Experiment) #337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.8"
python-version: "3.9"
- name: Install nox
run: |
python -m pip install --upgrade setuptools pip wheel
Expand Down
10 changes: 4 additions & 6 deletions .github/workflows/unittest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,11 @@ on:
name: unittest
jobs:
unit:
# TODO(https://github.com/googleapis/gapic-generator-python/issues/2303): use `ubuntu-latest` once this bug is fixed.
# Use ubuntu-22.04 until Python 3.7 is removed from the test matrix
# https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories
runs-on: ubuntu-22.04
# Use `ubuntu-latest` runner.
runs-on: ubuntu-latest
strategy:
matrix:
python: ['3.8', '3.9', '3.10', '3.11', '3.12', '3.13']
python: ['3.9', '3.10', '3.11', '3.12', '3.13']
steps:
- name: Checkout
uses: actions/checkout@v4
Expand Down Expand Up @@ -103,7 +101,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.8"
python-version: "3.9"
- name: Install coverage
run: |
python -m pip install --upgrade setuptools pip wheel
Expand Down
40 changes: 0 additions & 40 deletions .kokoro/samples/python3.7/common.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.7/continuous.cfg

This file was deleted.

11 changes: 0 additions & 11 deletions .kokoro/samples/python3.7/periodic-head.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.7/periodic.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.7/presubmit.cfg

This file was deleted.

40 changes: 0 additions & 40 deletions .kokoro/samples/python3.8/common.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.8/continuous.cfg

This file was deleted.

11 changes: 0 additions & 11 deletions .kokoro/samples/python3.8/periodic-head.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.8/periodic.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.8/presubmit.cfg

This file was deleted.

3 changes: 1 addition & 2 deletions .kokoro/test-samples-impl.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,7 @@ export PYTHONUNBUFFERED=1
env | grep KOKORO

# Install nox
# `virtualenv==20.26.6` is added for Python 3.7 compatibility
python3.9 -m pip install --upgrade --quiet nox virtualenv==20.26.6
python3.9 -m pip install --upgrade --quiet nox virtualenv

# Use secrets acessor service account to get secrets
if [[ -f "${KOKORO_GFILE_DIR}/secrets_viewer_service_account.json" ]]; then
Expand Down
14 changes: 5 additions & 9 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ In order to add a feature:
documentation.

- The feature must work fully on the following CPython versions:
3.7, 3.8, 3.9, 3.10, 3.11, 3.12 and 3.13 on both UNIX and Windows.
3.9, 3.10, 3.11, 3.12 and 3.13 on both UNIX and Windows.

- The feature must not add unnecessary dependencies (where
"unnecessary" is of course subjective, but new dependencies should
Expand Down Expand Up @@ -143,12 +143,12 @@ Running System Tests
$ nox -s system

# Run a single system test
$ nox -s system-3.8 -- -k <name of test>
$ nox -s system-3.9 -- -k <name of test>


.. note::

System tests are only configured to run under Python 3.8.
System tests are only configured to run under Python 3.9.
For expediency, we do not run them in older versions of Python 3.

This alone will not run the tests. You'll need to change some local
Expand Down Expand Up @@ -195,11 +195,11 @@ configure them just like the System Tests.

# Run all tests in a folder
$ cd samples/snippets
$ nox -s py-3.8
$ nox -s py-3.9

# Run a single sample test
$ cd samples/snippets
$ nox -s py-3.8 -- -k <name of test>
$ nox -s py-3.9 -- -k <name of test>

********************************************
Note About ``README`` as it pertains to PyPI
Expand All @@ -221,16 +221,12 @@ Supported Python Versions

We support:

- `Python 3.7`_
- `Python 3.8`_
- `Python 3.9`_
- `Python 3.10`_
- `Python 3.11`_
- `Python 3.12`_
- `Python 3.13`_

.. _Python 3.7: https://docs.python.org/3.7/
.. _Python 3.8: https://docs.python.org/3.8/
.. _Python 3.9: https://docs.python.org/3.9/
.. _Python 3.10: https://docs.python.org/3.10/
.. _Python 3.11: https://docs.python.org/3.11/
Expand Down
4 changes: 2 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,11 @@ dependencies.

Supported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^
Python >= 3.7
Python >= 3.9

Unsupported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Python <= 3.6.
Python <= 3.8.


Mac/Linux
Expand Down
81 changes: 46 additions & 35 deletions db_dtypes/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@
import warnings

import numpy
import packaging.version
import pandas
import pandas.api.extensions
from pandas.errors import OutOfBoundsDatetime
import pyarrow
import pyarrow.compute

from db_dtypes import core
from db_dtypes.json import JSONArray, JSONArrowType, JSONDtype
from db_dtypes.version import __version__

from . import _versions_helpers
Expand All @@ -47,15 +47,6 @@
_NP_BOX_DTYPE = "datetime64[us]"


# To use JSONArray and JSONDtype, you'll need Pandas 1.5.0 or later. With the removal
# of Python 3.7 compatibility, the minimum Pandas version will be updated to 1.5.0.
if packaging.version.Version(pandas.__version__) >= packaging.version.Version("1.5.0"):
from db_dtypes.json import JSONArray, JSONArrowType, JSONDtype
else:
JSONArray = None
JSONDtype = None


@pandas.api.extensions.register_extension_dtype
class TimeDtype(core.BaseDatetimeDtype):
"""
Expand Down Expand Up @@ -346,35 +337,55 @@ def __sub__(self, other):

return super().__sub__(other)

# Inside db_dtypes/__init__.py (TEMPORARY DEBUG)
print(f"DEBUG: Inside __init__, JSONArray type: {type(JSONArray)}, value: {repr(JSONArray)}")
print(f"DEBUG: Inside __init__, JSONDtype type: {type(JSONDtype)}, value: {repr(JSONDtype)}")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll want to remove these before merging.

Suggested change
# Inside db_dtypes/__init__.py (TEMPORARY DEBUG)
print(f"DEBUG: Inside __init__, JSONArray type: {type(JSONArray)}, value: {repr(JSONArray)}")
print(f"DEBUG: Inside __init__, JSONDtype type: {type(JSONDtype)}, value: {repr(JSONDtype)}")


sys_major, sys_minor, sys_micro = _versions_helpers.extract_runtime_version()
if sys_major == 3 and sys_minor in (7, 8):
warnings.warn(
"The python-bigquery library will stop supporting Python 3.7 "
"and Python 3.8 in a future major release expected in Q4 2024. "
f"Your Python version is {sys_major}.{sys_minor}.{sys_micro}. We "
"recommend that you update soon to ensure ongoing support. For "
"more details, see: [Google Cloud Client Libraries Supported Python Versions policy](https://cloud.google.com/python/docs/supported-python-versions)",
PendingDeprecationWarning,
)


if not JSONArray or not JSONDtype:
__all__ = [
def _determine_all(json_array_type, json_dtype_type):
"""Determines the list for __all__ based on JSON type availability."""
base_all = [
"__version__",
"DateArray",
"DateDtype",
"TimeArray",
"TimeDtype",
]
else:
__all__ = [
"__version__",
"DateArray",
"DateDtype",
"JSONDtype",
"JSONArray",
"JSONArrowType",
"TimeArray",
"TimeDtype",
]
# Check if both JSON types are available (truthy)
if json_array_type and json_dtype_type:
# print("DEBUG: Condition FALSE, including JSON in __all__") # Keep if needed
return base_all + ["JSONDtype", "JSONArray", "JSONArrowType"]
else:
# print("DEBUG: Condition TRUE, excluding JSON from __all__") # Keep if needed
return base_all

def _check_python_version():
"""Checks the runtime Python version and issues a warning if needed."""
sys_major, sys_minor, sys_micro = _versions_helpers.extract_runtime_version()
if sys_major == 3 and sys_minor in (7, 8):
warnings.warn(
"The python-bigquery library as well as the python-db-dtypes-pandas library no "
"longer supports Python 3.7 and Python 3.8. "
f"Your Python version is {sys_major}.{sys_minor}.{sys_micro}. We "
"recommend that you update soon to ensure ongoing support. For "
"more details, see: [Google Cloud Client Libraries Supported Python Versions policy](https://cloud.google.com/python/docs/supported-python-versions)",
FutureWarning,
stacklevel=2, # Point warning to the caller of __init__
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you use python_requires in setup.py, I'm having a hard time seeing how someone could end up in this state.

Do we want to issue this in a separate PR and release so folks see it before fully dropping?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can do that.



# sys_major, sys_minor, sys_micro = _versions_helpers.extract_runtime_version()
# if sys_major == 3 and sys_minor in (7, 8):
# warnings.warn(
# "The python-bigquery library as well as the python-db-dtypes-pandas library no "
# "longer supports Python 3.7 and Python 3.8. "
# f"Your Python version is {sys_major}.{sys_minor}.{sys_micro}. We "
# "recommend that you update soon to ensure ongoing support. For "
# "more details, see: [Google Cloud Client Libraries Supported Python Versions policy](https://cloud.google.com/python/docs/supported-python-versions)",
# FutureWarning,
# )

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this commented out code.

Suggested change
# sys_major, sys_minor, sys_micro = _versions_helpers.extract_runtime_version()
# if sys_major == 3 and sys_minor in (7, 8):
# warnings.warn(
# "The python-bigquery library as well as the python-db-dtypes-pandas library no "
# "longer supports Python 3.7 and Python 3.8. "
# f"Your Python version is {sys_major}.{sys_minor}.{sys_micro}. We "
# "recommend that you update soon to ensure ongoing support. For "
# "more details, see: [Google Cloud Client Libraries Supported Python Versions policy](https://cloud.google.com/python/docs/supported-python-versions)",
# FutureWarning,
# )

# Perform the version check
_check_python_version()

# Assign __all__ by calling the function
__all__ = _determine_all(JSONArray, JSONDtype)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems a bit too "magic" to me, but I suppose it could make testing easier?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code was originally just in the file with no function def. We added _check_python_version...

To test against supported and unsupported versions we attempted to mock the version but timing became an issue, partly because dependencies also rely on version checks so our mock ended up preventing error-free installs of some dependencies.

We tried half a dozen ways to alter when the mock happened but none hit that sweet spot of reliably allowing the dependencies to do their thing and slipping in to allow our test to run.

Similarly, we ran into a mess of problems with import timing and mocking whether JSONArray JSONDtype did or did not import as expected and so ...

putting them in functions made things much easier in the testing arena.

3 changes: 0 additions & 3 deletions db_dtypes/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -186,9 +186,6 @@ def median(
keepdims: bool = False,
skipna: bool = True,
):
if not hasattr(pandas_backports, "numpy_validate_median"):
raise NotImplementedError("Need pandas 1.3 or later to calculate median.")

pandas_backports.numpy_validate_median(
(),
{"out": out, "overwrite_input": overwrite_input, "keepdims": keepdims},
Expand Down
Loading
Loading