REF: use oldest-supported-numpy

[henryiii]

This code is currently hand-copied from SciPy, now there's a package that exists to do just that. As a bonus, NumPy will no longer need to build wheels on 3.10 (Linux 64-bit only currently)

• closes Use oldest-supported-numpy in pyproject.toml #40307
• tests added / passed
• Ensure all linting tests pass, see here for how to run them
• whatsnew entry

Two things to consider: first, for Python 3.7, this sets an older build time requirement for NumPy than is there currently. According to the NumPy devs, this is fine, older build time requirements than the run requirement is okay. But we could use selectors and manually set NumPy for this one case, it's not something that changes.

Second, we could pin here, but given that this just pins old versions, I don't think it needs pinning. We could set a minimum, though, >=0.10 for example.

[jreback]

looks fine, do we need to somehow add this to our build configuration on CI?

[henryiii]

do we need to somehow add this to our build configuration on CI

Well, it's already being built in most the jobs, but the important thing is what version of NumPy the PEP 517 build process downloads. It seems the build I checked (sdist / build) doesn't build verbosely, so you can't tell what version of NumPy it installed in the build environment, you can just see the runtime one. If there's a test that builds Pandas (either as a wheel or as an install such as pip install .) and then installs (if installing wheel) or rolls back (if installing directly) the version of NumPy to the oldest you support, then runs the tests, that would be perfect.

The reason this (the old list of NumPy versions and the package that maintains that list) exists is that you can't use newer runtime NumPy than you built with.

As long as the Python 3.7 tests pass and we can verify it is downloading a pinned NumPy version, I think it's pretty safe. I don't actually see Python 3.7 listed in the checks, though?

[henryiii]

Is actions-38.yaml, not slow and not network and not clipboard known to be flaky, perhaps? That had one memory address failure, but that seems to be a conda build, and conda doesn't use pyproject.toml, so don't think that could be affected?

FAILED pandas/tests/frame/methods/test_rename.py::TestRename::test_rename_inplace

Here,

pandas/ci/setup_env.sh

Line 112 in 8b90070

python -m pip install --no-build-isolation -e .

the pyproject.toml deps are disabled. So this shouldn't be affected at all by this patch?

[jreback]

just because the CI passes doesn't mean this change actually works. We don't build wheels in this repo. We are no longer on 3.7

[henryiii]

Ah, looks like master has dropped Python 3.7, so I guess 1.4 is planned for after December 26, which is the NEP 29 drop date for Python 3.7. However, there needs to be a Python 3.10 supporting release before December 26. Python 3.10 hits fully in October (and ideally packages are supposed to pick up support during the RC phase). Should this be backported to the 1.3 branch? I'm guessing that's where the Python 3.7 tests are.

(Sort of assuming Pandas is following NEP 29, seems to be in #29034 (comment) for example)

[jreback]

Ah, looks like master has dropped Python 3.7, so I guess 1.4 is planned for after December 26, which is the NEP 29 drop date for Python 3.7. However, there needs to be a Python 3.10 supporting release before December 26. Python 3.10 hits fully in October (and ideally packages are supposed to pick up support during the RC phase). Should this be backported to the 1.3 branch? I'm guessing that's where the Python 3.7 tests are.

(Sort of assuming Pandas is following NEP 29, seems to be in #29034 (comment) for example)

yes we can backport this

[henryiii]

pandas/.github/workflows/sdist.yml

Line 61 in 8b90070

python -m pip install dist/*.gz

This builds from SDist, which builds via PEP 517 with PEP 518 requirements (the comment in the pyproject.toml is wrong, by the way, using the implied default build-backend = "setuptools.build_meta:__legacy__" instead of build-backend = "setuptools.build_meta" does not disable PEP 517). That literally builds a wheel then installs the wheel.

What about slightly modifying that test, both to add "-v" for verbose output, as well as installing the oldest supported NumPy (not the package, just the version) after the build and then running at least one test that touches NumPy?

[henryiii]

I didn't add a test, the current job just does a simple import, which might not trigger the error. Any ideas for a simple test that would touch the NumPy portions?

[jreback]

this needs 3.7 as well (as this is going to be backported); will need to followup and then remove 3.7 on master. so this is complicated. let's just leave these additions to the sdist workflow for a followup PR instead. alt we can simply target master (maybe a better idea).

[github-actions]

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

[henryiii]

What's the status here? From what I understood, "we can simply target master (maybe a better idea)." is the current state, and there also was a suggestion to backport and add 3.7 to this list.

[jreback]

this PR overlaps with #43729 (now wouldn't mind adding the specific CI job that you are adding).

[henryiii]

Replaced by #43729