Skip to content

Add additional triggers to integration test workflow #124

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 27, 2023
Merged

Add additional triggers to integration test workflow #124

merged 4 commits into from
Mar 27, 2023

Conversation

per1234
Copy link
Collaborator

@per1234 per1234 commented Mar 27, 2023

No description provided.

per1234 added 4 commits March 26, 2023 22:04
The project uses the standard GitHub Actions convention of providing a major version ref. A branch with the format
`v<major>` is used for that purpose.

These release branches may contain a subset of the history of the default branch. Although most often this would mirror
the history on the `main` branch, in some cases it might be beneficial to cherry-pick the commits ready for release to
the release branch, excluding other commits that are not yet ready for release.

The status of the GitHub Actions workflows should be evaluated before making a release. However, this is not so simple
as checking the status of the commit at the tip of the release branch. The reason is that, for the sake of efficiency,
the workflows are configured to run only when the processes are relevant to the trigger event (e.g., no need to run unit
tests for a change to the readme).

In the case of the default branch, you can simply set the workflow runs filter to that branch and then check the result
of the latest run of each workflow of interest. However, that was not possible to do with the release branch since it
might be that the workflow was never run in that branch. The status of the latest run of the workflow in the default
branch might not match the status for the release branch if the release branch does not contain the full history.

For this reason, it will be helpful to trigger all relevant workflows on the creation of a release branch. This will
ensure that each of those workflows will always have at least one run in the release branch. Subsequent commits pushed to
the branch can run based on their usual trigger filters and the status of the latest run of each workflow in the branch
will provide an accurate indication of the state of that branch.

Branches are created for purposes other than releases, most notably feature branches to stage work for a pull request.
Because the collection of workflows in a Tooling project are often very comprehensive, it would not be convenient or
efficient to run them on the creation of every feature branch.

Unfortunately, GitHub Actions does not support filters on the `create` event of branch creation like it does for the
`push` and `pull_request` events. There is support for a `branches` filter of the `push` event, but that filter is an AND
to the `paths` filter and this application requires an OR. For this reason, the workflows must be triggered by the
creation of any branch. The unwanted job runs are prevented by adding a `run-determination` job with the branch filter
handled by Bash commands. The other jobs of the workflow use this `run-determination` job as a dependency, only running
when it indicates they should via a job output. Because this minimal `run-determination` job runs very quickly, it is
roughly equivalent to the workflow having been skipped entirely for non-release branch creations. This approach has been
in use for some time already in the versioned "Deploy Website" workflows.

Previously the regular expression used to identify release branches used the `<major>.<minor>.x` format that is standard
in non-action tooling projects, but wrong for this project.
The project uses the standard GitHub Actions convention of providing a major version ref. A branch  is used for that
purpose.

These release branches may contain a subset of the history of the default branch. Although most often this would mirror
the history on the `main` branch, in some cases it might be beneficial to cherry-pick the commits ready for release to
the release branch, excluding other commits that are not yet ready for release.

The status of the GitHub Actions workflows should be evaluated before making a release. However, this is not so simple
as checking the status of the commit at the tip of the release branch. The reason is that, for the sake of efficiency,
the workflows are configured to run only when the processes are relevant to the trigger event (e.g., no need to run unit
tests for a change to the readme).

In the case of the default branch, you can simply set the workflow runs filter to that branch and then check the result
of the latest run of each workflow of interest. However, that was not possible to do with the release branch since it
might be that the workflow was never run in that branch. The status of the latest run of the workflow in the default
branch might not match the status for the release branch if the release branch does not contain the full history.

For this reason, it will be helpful to trigger all relevant workflows on the creation of a release branch. This will
ensure that each of those workflows will always have at least one run in the release branch. Subsequent commits pushed to
the branch can run based on their usual trigger filters and the status of the latest run of each workflow in the branch
will provide an accurate indication of the state of that branch.

Branches are created for purposes other than releases, most notably feature branches to stage work for a pull request.
Because the collection of workflows in a Tooling project are often very comprehensive, it would not be convenient or
efficient to run them on the creation of every feature branch.

Unfortunately, GitHub Actions does not support filters on the `create` event of branch creation like it does for the
`push` and `pull_request` events. There is support for a `branches` filter of the `push` event, but that filter is an AND
to the `paths` filter and this application requires an OR. For this reason, the workflows must be triggered by the
creation of any branch. The unwanted job runs are prevented by adding a `run-determination` job with the branch filter
handled by Bash commands. The other jobs of the workflow use this `run-determination` job as a dependency, only running
when it indicates they should via a job output. Because this minimal `run-determination` job runs very quickly, it is
roughly equivalent to the workflow having been skipped entirely for non-release branch creations.
The project infrastructure may be broken by changes to the external dependencies (e.g., GitHub Actions runner changes,
runner machine OS/environment updates, action releases, tool releases).

If this breakage is not caught and fixed quickly by the maintainer, it may cause confusion to the contributors whose
contributions trigger a spurious workflow run failure.

Monitoring for breakage can be done by periodically triggering a workflow run.
These allow the maintainer to trigger a workflow run after a relevant external change.
@per1234 per1234 added type: enhancement Proposed improvement topic: infrastructure Related to project infrastructure labels Mar 27, 2023
@per1234 per1234 self-assigned this Mar 27, 2023
@codecov-commenter
Copy link

Codecov Report

Patch and project coverage have no change.

Comparison is base (b3ab95f) 99.81% compared to head (e6e67e5) 99.81%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #124   +/-   ##
=======================================
  Coverage   99.81%   99.81%           
=======================================
  Files           2        2           
  Lines        1610     1610           
=======================================
  Hits         1607     1607           
  Misses          3        3           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@per1234 per1234 merged commit 1a36eaa into arduino:main Mar 27, 2023
@per1234 per1234 deleted the triggers branch March 27, 2023 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: infrastructure Related to project infrastructure type: enhancement Proposed improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants