CI Builds: Conditional builds with path filtering

[benjaoming]

Benefits of conditional builds:

1. Tests are over 45 minutes, so they are valuable to skip them when it's easy and risk-free.
2. Documentation PRs (we have a lot of them) will have a much quicker feedback time.

Because of the many current documentation PRs, the timing for "conditional builds" will be really good.

• Step 1: Make path filtering work
• Step 2: Discuss and adjust the actual paths and filters!
• Step 3: Apply path filters (and other conditions that we might want)

[benjaoming]

These are essential bits to adjust. We want to run this stuff whenever something essential is changed.

2 considerations that we might have for later:

1. Breaking up checks and tests pipelines and give them separate path filters.
2. Branch rules: I think we should also consider an alternative pipeline that always runs on the main branch and sets build-code to true regardless of the path. This requires adding a separate pipeline that runs on main (and configuring the other pipeline to NOT run on main), calling the continuation Orb.

[humitos]

I think this is pretty low priority, unless there is an obvious benefit of investing time here that I'm not seeing yet.

That said, what are the exact cases where we want to avoid running the tests? I suppose the only one is when only modifying the docs/ directory since they will be run by Read the Docs.

[benjaoming]

That said, what are the exact cases where we want to avoid running the tests? I suppose the only one is when only modifying the docs/ directory since they will be run by Read the Docs.

That's what this is mostly about, yes 👍

There's 40 min+ test running time for no reason.

[benjaoming]

When we target branches different from main, there might be some false positives. I've opened up a feature request: CircleCI-Public/path-filtering-orb#65

[benjaoming]

This is now ready for review 👍

I imagine the following issues, which IMO are interesting for further study (which can indeed inform our own build logic!):

• Merge commits might trigger redundant builds. This is low-risk since it means building too much.
• A series of commits pushed at the same time aren't compared correctly to the base branch (main) AFAICT. This can get annoying. The workarounds are numerous and quite easy. So I wouldn't expect it to be a big problem.

Our safety nets are:

• main is always built

We can expand rules as we become more clever.

[humitos]

I'm 👍🏼 by closing this PR and continue using the workflow we have. The documentation build is skipped now with the latest changes in our code. On the other hand, I prefer to not deal with edge cases for the test cases and always be sure they are running. That gives me confidence about merging the PR.

Feel free to re-open if you consider it's important to continue with this work and you think there are some room for prioritization here.

[benjaoming]

Seems good to close it, since it won't stay ready without a lot of heavy maintenance. It was pretty hard to maintain this PR because it refactored the Circle CI configuration that was constantly changing a while back (new Tox etc).

I wanted this change because when working on documentation, I didn't need the whole platform CI to trigger and it became a huge distraction. Once others realize that the 1 hour build times also annoy them, you know where to find me 😄