Refactor CI #46493

jonashaag · 2022-03-24T07:54:08Z

This PR 1 from this list #46346 (comment)

closes #xxxx (Replace xxxx with the Github issue number)
Tests added and passed if fixing a bug or adding a new feature
All code checks passed.
Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

mroeschke · 2022-03-24T18:17:33Z

Looks like only 1 check ran

- Drop Azure (move to GHA). - Use mamba-org/provision-with-micromamba to set up Conda envs. - Cache Conda envs. - Use hendrikmuhs/ccache-action to cache C compiliation. - Add Windows 32 bit build. - Drop Ubuntu 32 bit build. - Factor out common GHA code into reusable actions.

jonashaag · 2022-03-24T20:13:53Z

GHA has serious problems right now

lithomas1 · 2022-03-24T21:21:49Z

Can we hold off on removing Azure pipelines for now? Right now, jobs queue for very long times >2 hours(sometimes up to 4 hours) when lots of PRs are merged. I'm trying to clear out some redundant builds right now, but that will probably take me a while before I get the builds down to a reasonable number(~20 jobs per build).

lithomas1 · 2022-03-24T21:20:26Z

.github/workflows/sdist.yml

-  build:
-    if: ${{ github.event.label.name == 'Build' || contains(github.event.pull_request.labels.*.name, 'Build') || github.event_name == 'push'}}
+  sdist:
+    #if: ${{ github.event.label.name == 'Build' || contains(github.event.pull_request.labels.*.name, 'Build') || github.event_name == 'push'}}


Why was this commented? sdists are not supposed to run on every PR as per #43745

Thanks, that's a debugging leftover

lithomas1 · 2022-03-24T21:23:10Z

.github/actions/setup-python/action.yml

+  using: composite
+  steps:
+    # TODO: GH#44980 https://github.com/pypa/setuptools/issues/2941
+    - name: Create temporary requirements.txt


Any reason for this? I think we can pin setuptools in the main file.

This is required to use setup-python's caching.

lithomas1

Thanks for the PR! Generally like the changes in this PR, however the diff still feels a bit too large and clunky to review. I left a few initial comments, but will have try to get a deeper review in later.

I would recommend splitting the Azure changes out pending discussion, and to only migrate a couple of the workflows at a time to keep the diffs smaller.

lithomas1 · 2022-03-24T21:40:35Z

.github/workflows/macos-windows.yml

@@ -0,0 +1,62 @@
+name: macOS and Windows


It might be better to pair together macOS and Ubuntu together in the future when we do drop azure.

mroeschke · 2022-03-25T02:57:21Z

Agreed that this PR seems to tackle too much still for a thorough review, especially for CI changes where we need to be careful about things passing. Generally it's easier if 1 PR tackles 1 general issue/theme

jonashaag · 2022-03-25T10:18:13Z

Right now, jobs queue for very long times >2 hours(sometimes up to 4 hours) when lots of PRs are merged

Is there any way to change to a large GHA plan? Maybe through some GH open source sponsoring program or some Pandas sponsor?

Re even more splitting, would it be a viable option to ship most of the shared code in .github/actions but only a single (or so) changed workflow? That would reduce the size of the changes substantially but the PR might include changes not relevant to the changed workflows (because only relevant to changes in workflows in a subsequent PR).

jonashaag · 2022-03-25T10:21:15Z

Right now, jobs queue for very long times >2 hours(sometimes up to 4 hours) when lots of PRs are merged

Is there any way to change to a large GHA plan? Maybe through some GH open source sponsoring program or some Pandas sponsor?

Also maybe we can bail out earlier if there are test failures. I thought that maybe w e could come up with a very small test of "pre tests" that we can run in eg. 60 s and that would exit the test run if failed.

mroeschke · 2022-03-26T01:00:00Z

Is there any way to change to a large GHA plan?

We could probably reach out to Github to see what's possible. Probably good to bring up at the next developer meeting. FWIW I don't think the long queue times are a huge deal. Minus the redundancy of testing which wastes compute resources, development is fairly asynchronous over longer time spans (especially reviewer availability) without major deadlines so the waiting is not major IMO.

Re even more splitting

Some of the changes I see that could be a single PR

pytest-asyncio>=0.17
Bumping actions/checkout
bash -el
Reordering the yaml of the include jobs
Refactoring sdist workflow
setup actions
build action
test action

jonashaag · 2022-03-28T06:57:35Z

Probably good to bring up at the next developer meeting.

Are you suggesting that I should join the meeting (where can I find info?) or are you going to bring it up?

FWIW I don't think the long queue times are a huge deal.

I think they are because they create a very long feedback loop for contributors. It's fine if tests take a long time but I think it's worth investing some effort into reducing the latency between a push and the first test being executed.

Some of the changes I see that could be a single PR

Working on a new PR!

jreback · 2022-03-28T12:42:32Z

@jonashaag the CI is very large on pandas and we are already using a large plan. it is unlikey we are going to be able to change anything here. The cycle times are what they are.

lithomas1 · 2022-03-28T14:35:25Z

Right now, jobs queue for very long times >2 hours(sometimes up to 4 hours) when lots of PRs are merged

Is there any way to change to a large GHA plan? Maybe through some GH open source sponsoring program or some Pandas sponsor?

Also maybe we can bail out earlier if there are test failures. I thought that maybe w e could come up with a very small test of "pre tests" that we can run in eg. 60 s and that would exit the test run if failed.

Going along a similar vein, maybe we can try making the slow CI depend on non slow CI and see if that helps(we can cache the built pandas from the non-slow CI, and install from that for the slow CI).

jonashaag · 2022-03-28T20:51:25Z

What's really curious is that the GHA machines are SO incredibly slow. On my M1 machine with no pytest-xdist the full test suite takes 8 minutes, while on GHA with 2 cores it takes ~ 45 minutes.

simonjayhawkins added the CI Continuous Integration label Mar 24, 2022

jonashaag force-pushed the refactor-ci branch from 15ccca5 to 20c6db3 Compare March 24, 2022 15:54

jonashaag added 4 commits March 24, 2022 21:13

Undo Ccache and Micromamba changes

7106b45

Remove dead code

ad5dd75

Fix

408e83e

jonashaag force-pushed the refactor-ci branch from 20c6db3 to 408e83e Compare March 24, 2022 20:14

lithomas1 reviewed Mar 24, 2022

View reviewed changes

jonashaag mentioned this pull request Mar 28, 2022

CI: Refactor sdist workflow and add setup-python action #46538

Closed

4 tasks

This was referenced Mar 28, 2022

Upgrade GitHub Actions versions #46540

Merged

GHA: Use bash -el #46541

Merged

Reorder YAML #46542

Merged

Pin pytest-asyncio>=0.17 #46543

Merged

datapythonista mentioned this pull request Apr 1, 2022

CI: Increase GitHub Actions resources #46600

Closed

jonashaag mentioned this pull request Apr 2, 2022

Add setup-conda action #46611

Merged

4 tasks

jonashaag closed this Apr 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor CI #46493

Refactor CI #46493

jonashaag commented Mar 24, 2022

mroeschke commented Mar 24, 2022 •

edited

Loading

jonashaag commented Mar 24, 2022

lithomas1 commented Mar 24, 2022

lithomas1 Mar 24, 2022

jonashaag Mar 25, 2022

lithomas1 Mar 24, 2022

jonashaag Mar 25, 2022

lithomas1 left a comment

lithomas1 Mar 24, 2022

mroeschke commented Mar 25, 2022

jonashaag commented Mar 25, 2022

jonashaag commented Mar 25, 2022

mroeschke commented Mar 26, 2022

jonashaag commented Mar 28, 2022

jreback commented Mar 28, 2022

lithomas1 commented Mar 28, 2022

jonashaag commented Mar 28, 2022

Refactor CI #46493

Refactor CI #46493

Conversation

jonashaag commented Mar 24, 2022

mroeschke commented Mar 24, 2022 • edited Loading

jonashaag commented Mar 24, 2022

lithomas1 commented Mar 24, 2022

lithomas1 Mar 24, 2022

Choose a reason for hiding this comment

jonashaag Mar 25, 2022

Choose a reason for hiding this comment

lithomas1 Mar 24, 2022

Choose a reason for hiding this comment

jonashaag Mar 25, 2022

Choose a reason for hiding this comment

lithomas1 left a comment

Choose a reason for hiding this comment

lithomas1 Mar 24, 2022

Choose a reason for hiding this comment

mroeschke commented Mar 25, 2022

jonashaag commented Mar 25, 2022

jonashaag commented Mar 25, 2022

mroeschke commented Mar 26, 2022

jonashaag commented Mar 28, 2022

jreback commented Mar 28, 2022

lithomas1 commented Mar 28, 2022

jonashaag commented Mar 28, 2022

mroeschke commented Mar 24, 2022 •

edited

Loading