Unexpected SystemError in Python3.12.0a7: <method 'startswith' of 'str' objects> returned a result with an exception set

[abravalheri]

Bug report

I noticed recently that Setuptools CI started failing for Python 3.12.0a7 with a SystemError that is difficult to understand:

SystemError: <built-in method startswith of str object at 0x7fe399991298> returned a result with an exception set

At a first glance this error does not make much sense because str.startswith is a built-in method in a built-in data structure. I am struggling to see how it can have an exception set attached to the result.

I managed to create the following simplified reproducer:

docker run --rm -it python:3.12.0a7-bullseye
Python 3.12.0a7 (main, Apr 12 2023, 14:13:09) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Distribution:
...     @property
...     def version(self):
...         try:
...             return self._version
...         except AttributeError as e:
...             # ... complex code omitted for the sake of simplification ...
...             raise ValueError("Missing Version")
...     def __getattr__(self, attr):
...         if attr.startswith("_"):
...             raise AttributeError(attr)
...         # ... complex code omitted for the sake of simplification ...
...         return 42
...
>>> dist = Distribution()
>>> dist.version
Traceback (most recent call last):
  File "<stdin>", line 5, in version
  File "<stdin>", line 11, in __getattr__
AttributeError: _version. Did you mean: 'version'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 8, in version
ValueError: Missing Version

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 10, in __getattr__
SystemError: <method 'startswith' of 'str' objects> returned a result with an exception set
>>>

The error seems to be related to the combination of the following factors:

1. A property is being used¹
2. The property getter access an undefined attribute and triggers __getattr__
3. __getattr__ raises an AttributeError
4. The property getter is supposed to catch the AttributeError exception and raise a different exeception.

My expectation is that the SystemError never gets triggered with the confusing message. Instead the example should end up with a ValueError.

Please note that the example is very simplified. For the realistic implementation please consider pkg_resources.Distribution.

Your environment

• CPython versions tested on: 3.12.0a7
• Operating system and architecture:
  Reproducer tested on both Ubuntu 20.04.6 LTS machine and python:3.12.0a7-bullseye container

Problem also identified in GitHub Actions runners: ubuntu-latest, macos-latest, windows-latest (see https://github.com/pypa/setuptools/actions/runs/4710601389?pr=3893) for Python 3.12-dev.

Footnotes

1. If we replace the property with a regular method, the example works as expected. ↩

[sunmy2019]

It should be fixed by #103332

[sobolevn]

I agree that this should be fixed in main. Please, reopen if it is not fixed.

[abravalheri]

Thank you very much.

I run the test above with an adapted Dockerfile that gets the latest `main` and it seems to work

FROM buildpack-deps:bullseye

# ensure local python is preferred over distribution python
ENV PATH /usr/local/bin:$PATH

# http://bugs.python.org/issue19846
# > At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK.
ENV LANG C.UTF-8

# runtime dependencies
RUN set -eux; \
    apt-get update; \
    apt-get install -y --no-install-recommends \
        libbluetooth-dev \
        tk-dev \
        uuid-dev \
    ; \
    rm -rf /var/lib/apt/lists/*

ENV PYTHON_VERSION 3.12.0dev+3e0fec7


RUN set -eux; \
    \
    wget -O /tmp/python.tar.gz "https://github.com/python/cpython/tarball/3e0fec7e07a71bdeeab7554e980110fbc47763b9"; \
    mkdir -p /usr/src/python; \
    tar --extract --directory /usr/src/python --strip-components=1 --file /tmp/python.tar.gz; \
    rm /tmp/python.tar.gz; \
    \
    cd /usr/src/python; \
    gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"; \
    ./configure \
        --build="$gnuArch" \
        --enable-loadable-sqlite-extensions \
        --enable-optimizations \
        --enable-option-checking=fatal \
        --enable-shared \
        --with-lto \
        --with-system-expat \
        --without-ensurepip \
    ; \
    nproc="$(nproc)"; \
    EXTRA_CFLAGS="$(dpkg-buildflags --get CFLAGS)"; \
    LDFLAGS="$(dpkg-buildflags --get LDFLAGS)"; \
    make -j "$nproc" \
        "EXTRA_CFLAGS=${EXTRA_CFLAGS:-}" \
        "LDFLAGS=${LDFLAGS:-}" \
        "PROFILE_TASK=${PROFILE_TASK:-}" \
    ; \
# https://github.com/docker-library/python/issues/784
# prevent accidental usage of a system installed libpython of the same version
    rm python; \
    make -j "$nproc" \
        "EXTRA_CFLAGS=${EXTRA_CFLAGS:-}" \
        "LDFLAGS=${LDFLAGS:--Wl},-rpath='\$\$ORIGIN/../lib'" \
        "PROFILE_TASK=${PROFILE_TASK:-}" \
        python \
    ; \
    make install; \
    \
# enable GDB to load debugging data: https://github.com/docker-library/python/pull/701
    bin="$(readlink -ve /usr/local/bin/python3)"; \
    dir="$(dirname "$bin")"; \
    mkdir -p "/usr/share/gdb/auto-load/$dir"; \
    cp -vL Tools/gdb/libpython.py "/usr/share/gdb/auto-load/$bin-gdb.py"; \
    \
    cd /; \
    rm -rf /usr/src/python; \
    \
    find /usr/local -depth \
        \( \
            \( -type d -a \( -name test -o -name tests -o -name idle_test \) \) \
            -o \( -type f -a \( -name '*.pyc' -o -name '*.pyo' -o -name 'libpython*.a' \) \) \
        \) -exec rm -rf '{}' + \
    ; \
    \
    ldconfig; \
    \
    python3 --version

CMD ["python3"]

Ref: https://github.com/docker-library/python/blob/cb6ecdbcbc7e30a180481a9911f5f1f470b9dba1/3.12-rc/bullseye/Dockerfile