Upload nightlies to new location

[jarrodmillman]

• closes #xxxx (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

[martinfleis]

A bit of context - the Scientific Python Developer Summit is happening this week. A part of the effort is to ensure that Scientific Python core packages follow the upcoming SPEC documents and support their implementation across the ecosystem. One of the specific changes is a move of the nightly wheels from pypi.anaconda.org/scipy-wheels-nightly which is planned to be closed to pypi.anaconda.org/scientific-python-nightly-wheels. That is what this PR does.

Once all of the packages that currently use scipy-wheels-nightly switch to scientific-python-nightly-wheels, we will use a bot to open PRs in downstream repositories to automatically change the index for nightlies across the ecosystem to minimise breakage.

One more step will need to happen here though. The upload requires an upload token stored as a repository secret (as you have right now). However, you will need to generate a new token for the new anaconda organisation. @jarrodmillman already added some people to the org but I don't think anyone from the pandas team is there right now. So let us know who is managing this at the moment so we can give you a proper access to the new repo so you can generate a new token.

The same change is happening in the other packages as well (e.g. numpy/numpy#23792).

[mroeschke]

Thanks for opening. It sounds like the numpy, scipy and scikit-learn are moving in this direction so +1 to this change. Would be great to coordinate the change around the same time the other libraries are moving

[ksunden]

Numpy has moved already, scipy is likely to move today, I think pandas is the next domino after scipy, then matplotlib can follow shortly thereafter (we are basically ready, but the PR includes our nightly tests, and we install pandas for that workflow)

xref numpy/numpy#23792 scipy/scipy#18504 matplotlib/matplotlib#25950

[mroeschke]

Thanks for the update. Are most projects doing a hard cutoff from uploading from the old to new index?

[martinfleis]

@mroeschke that is what scipy and numpy have done and it seems that matplotlib will do the same.

[pllim]

Hello! I am waiting for matplotlib to move wheel to new location and they say they are waiting for you. Looks like scipy wheel is at new location now. Thank you!

[tupui]

Hi folks,

So here are the latest information about the migration. NumPy, SciPy and Scikit-learn have now migrated (still some small things to adjust, but some wheels are there.)

To add a new package this needs to be done (instructions have been updated on the SPEC4):

1. Tell us who needs to be responsible for nighlies for your package. And give us the anaconda handles.
2. We download from PyPi a wheel and upload it using an admin token. This will create the package.
3. We link the package to your group.
4. You can now create a personal token and upload your package.

Let me know if you need any assistance, I am happy to make calls to speed up things or we can chat on Slack/Discord.

[ksunden]

@pllim I uploaded a set of mpl wheels yesterday, so if that was what was holding you up, you should be good to go, we do want this PR merged so we can merge matplotlib/matplotlib#25950 to keep our nightly installs working/start automatically uploading to the new location, but there is a recent nightly of mpl on the new location now.

[pllim]

@ksunden , yes, I saw, thank you very much! This is not a blocker, though I also do not want to rely on a one-off that could go stale, in case this drags on for months. I just want a feel of the timing. No worries here (for now).

[mroeschke]

@pandas-dev/pandas-core does anyone have a anaconda handle that would like to be associated with the new nightly wheels location?

[lithomas1]

aarch64 wheels are currently uploaded with a token from my personal account(lithomas1).

I believe @TomAugspurger has access to the scipy-wheels-nightly account that uploads the rest of the wheels.

I'm fine with continuing with the current setup, but its probably better if an admin of the pandas-dev org does the uploads.
(I can't add secrets to the Github Actions workflows, only an admin can do that I think)

[mroeschke]

I can work on creating an anaconda account early next week and migrate our wheel uploads to scientific-python-nightly-wheels

[mroeschke]

Sorry for the delay @tupui, I just created an anaconda account with username mroeschke. I can be responsible for the nightly package.

Let me know what the best next steps would be. Side note: I could also work with @betatim on the transition since we work together

[tupui]

@mroeschke all done 😃 Thanks!

https://anaconda.org/scientific-python-nightly-wheels/pandas

I uploaded the latest wheel for linux to create the package. You should now be able to administrate the package itself. I did not add for now Tim as he is already on another project. But up to you in the end.

Let me know if you have any questions, issues. Note that the policy is still in the making, so please allow us some time to adjust 😅 Moving forward, for transparency, please open an issue on the SPEC repo if you need anything.

[jarrodmillman]

@mroeschke I redid this PR based off main (bdfaca6). It should be ready to merge if it looks good. Let me know if I can do anything else to help with the transition.

[jarrodmillman]

What about actions-310-numpydev.yaml which has the line

"--extra-index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple"

It looks like that was already updated in #53508.

[mroeschke]

Great thanks all! Let's see if the nightly wheels get uploaded to the new index later today