CI/TST: pip extras install

[mroeschke]

• closes ENH: CI check for new extras_require #48942 (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature

(c-)blosc is only available on conda, so removing it from the pip extras
https://www.pytables.org/usersguide/installation.html#prerequisites

[mroeschke]

blosc on pypi has a lower version than on conda-forge. Pending to see if blosc will be updated on pypi: Blosc/python-blosc#297

EDIT: Updated the OP to note that blosc is actually c-blosc which isn't available on pypi

[phofl]

This blocks quite a few runners for significant minutes (around 15 per build). Is there a way to avoid building pandas all the time?

[mroeschke]

Looks like pip install -e .[... will still run setup.py. Maybe I could patch it to avoid building the extensions?

[mroeschke]

@phofl found a hacky workaround to not build the extensions and now the jobs complete in a few seconds. Open to better ideas on how to achieve the same thing.

[phofl]

This works consistently? I'd be ok with this for the time being.

Alternative would be to change setup.py directly, correct?

[mroeschke]

This works consistently? I'd be ok with this for the time being.

Yeah this worked when I tried in in 05b6ef7.

Alternative would be to change setup.py directly, correct?

Yeah I think so, but my attempts at this were not successful. I tried creating a no-extensions flag to pass into setup.py that could be used with pip install --install-options (https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-install-option) but that flag is deprecated and will pass that option to other package's setup.py

[phofl]

Thanks, lets try the current solution and adjust if necessary

[lithomas1]

@mroeschke I'm not sure if this is adding anything useful to the test matrix. Taking a quick look, this just tests that installing each of the extras works(this is already covered by environment.yml/requirements-dev.txt, since all dependencies are installed when we install from there and these deps are just a subset of thoe), not that the extras are correct. (IMO, you'd need some sort of "golden" file to check this).

There's also coverage for this in the pre-commit checks which should be enough IMO.

So if it's fine with you, I would propose reverting this PR, since it relies on a setuptools hack that is blocking my meson PR.

[phofl]

Not really, this checks that pip install pandas[foo] works, this is different than the environment yml.

[mroeschke]

Yeah I think there still some use here smoke testing pip install pandas[foo] "works", but agreed there should be a stronger check that install dependencies match what the extras specify.

One concern here was build speed since most of the time was built the cython extensions, but I suppose we can live with some extra build time for now and remove my hack (modifying setup.py in the job) which should unblock your meson PR.

Alternatively do you have any ideas on how to disable building cython extensions when pip installing?

[lithomas1]

Not really, this checks that pip install pandas[foo] works, this is different than the environment yml.

Since it's a subset, if we know installing all the dependencies works, then installing only some of them should work too, right?

Yeah I think there still some use here smoke testing pip install pandas[foo] "works", but agreed there should be a stronger check that install dependencies match what the extras specify.

One concern here was build speed since most of the time was built the cython extensions, but I suppose we can live with some extra build time for now and remove my hack (modifying setup.py in the job) which should unblock your meson PR.

Alternatively do you have any ideas on how to disable building cython extensions when pip installing?

I came across a rejected issue pypa/pip#4783. It doesn't seem like it.

Why don't we restrict these builds to PRs tagged with "Build" for now? That should be a good compromise.

[mroeschke]

Why don't we restrict these builds to PRs tagged with "Build" for now? That should be a good compromise.

Sure that would be fine with me