CI: Run all code checks even if one fails

[mroeschke]

xref #47469 (comment)

Solution gleaned from: https://stackoverflow.com/a/58859404

Want to always() run all these steps If pandas was built correctly

Note: Cannot use continue-on-error because if a check fails the build will still pass which we don't want https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepscontinue-on-error=

[MarcoGorelli]

if I've understood the GHA docs, this looks good to me

[jonashaag]

If we have this does it make sense to split the steps into multiple jobs? And have one upstream job that builds pandas so that work isn't duplicated. We could use that upstream job for ASV and docs as well.

[mroeschke]

+1 to have one build pandas job for all the checks that share the environment.yml dependencies. I am slightly partial to the current job grouping though. Is there a max number of jobs that can run in parallel across workflows? I couldn't immediately find it in the docs

[jonashaag]

I monitored the load on the Pandas CI pipeline for the past 14 days (with the help of another GHA pipeline :)) and this is the quantiles

0.0: 0
5.0: 0
10.0: 0
15.0: 0
20.0: 0
25.0: 0
30.0: 0
35.0: 2
40.0: 2
45.0: 2
50.0: 3
55.0: 5
60.0: 5
65.0: 5
70.0: 7
75.0: 8
80.0: 10
85.0: 14
90.0: 20

Checked every 13 minutes. 65% of the time we have < 5 workflows running, for a maximum of 5 * 17 = 85 parallel jobs (we have 17 jobs in the Ubuntu workflow). So I'm not sure it's worth reducing the number of parallel jobs right now. We might want to optimize latency-to-first-valuable-feedback instead.