CI: warning from setuptools (Numpy Dev Build Failures)

[alimcmaster1]

   def warn_distutils_present():
        if 'distutils' not in sys.modules:
            return
        if is_pypy and sys.version_info < (3, 7):
            # PyPy for 3.6 unconditionally imports distutils, so bypass the warning
            # https://foss.heptapod.net/pypy/pypy/-/blob/be829135bc0d758997b3566062999ee8b23872b4/lib-python/3/site.py#L250
            return
        warnings.warn(
>           "Distutils was imported before Setuptools. This usage is discouraged "
            "and may exhibit undesirable behaviors or errors. Please use "
            "Setuptools' objects directly or at least import Setuptools first.")
E       UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.

e.g : https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=39110&view=logs&j=3a03f79d-0b41-5610-1aa4-b4a014d0bc70&t=4d05ed0e-1ed3-5bff-dd63-1e957f2766a9&l=195

[TomAugspurger]

To reproduce

In [1]: import pandas

In [2]: import setuptools
/Users/taugspurger/Envs/pandas-dev/lib/python3.7/site-packages/setuptools/distutils_patch.py:26: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
  "Distutils was imported before Setuptools. This usage is discouraged "

That's with setuptools 49.2.0.

I think this is because pandas uses distutils.version to parse the versions of optional dependencies. It's probably a good idea to not import distutils at the top level (it does more than we need) but I don't know of a better way off the top of my head (vendoring packinging.version.parse?).

But for now let's just fix CI by silencing that warning to show_versions.

[TomAugspurger]

Right now I think the two longer-term solutions are to either

1. Filter warnings in show_versions (perhaps just the one coming from setuptools)
2. Vendor packaging.version

If we go the vendoring route, we'd need

• https://github.com/pypa/packaging/blob/master/packaging/_structures.py
• https://github.com/pypa/packaging/blob/master/packaging/version.py

[jorisvandenbossche]

if it is only to avoid a warning in show_versions, vendoring might be a bit "overkill" ?

Also, if distutils is discouraged like that, it's a bit a pitty there is no better (stdlib) solution for parsing versions.

The setuptools issue where it was introduced: pypa/setuptools#2230

[TomAugspurger]

Also, if distutils is discouraged like that, it's a bit a pitty there is no better (stdlib) solution for parsing versions.

Agreed. Seems like the thing that should be built in :/

if it is only to avoid a warning in show_versions, vendoring might be a bit "overkill" ?

Probably. In theory this makes using pandas & setuptools together a bit fragile, but perhaps we just wait for complaints?

[alimcmaster1]

Does using packaging vs distutils help here? packaging is vendor'ed into setuptools ref: https://github.com/pypa/setuptools/tree/master/pkg_resources/_vendor/packaging

and think we can use packaging.version.parse ? I might be missing/oversimplifying things?

[jorisvandenbossche]

setuptools is not a runtime dependency (only an install dependency), so that does not directly help (except if we make packaging or setuptools a runtime dependency, but IMO I wouldn't do that just for the version checks)

[jbrockmendel]

if it is only to avoid a warning in show_versions, vendoring might be a bit "overkill" ?

Agreed.

The warning has been suppressed and I don't see "distutils" imported anywhere. Anything left to do here?

[mroeschke]

I don't see this error in our builds anymore (maybe the warning disappeared as setuptools is v 60ish now). Going to close for now but can reopen if we see this again