Skip to content

Disable "fail fast" feature in "Test Go" workflow #1914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 10, 2022
Merged

Disable "fail fast" feature in "Test Go" workflow #1914

merged 1 commit into from
Oct 10, 2022

Conversation

per1234
Copy link
Contributor

@per1234 per1234 commented Oct 10, 2022

Please check if the PR fulfills these requirements

See how to contribute

  • The PR has no duplicates (please search among the Pull Requests
    before creating one)
  • The PR follows
    our contributing guidelines
  • [N/A] Tests for the changes have been added (for bug fixes / features)
  • [N/A] Docs have been added / updated (for bug fixes / features)
  • [N/A] UPGRADING.md has been updated with a migration guide (for breaking changes)

What kind of change does this PR introduce?

Infrastructure enhancement

What is the current behavior?

To ensure functionality on all platforms, the "Test Go" GitHub Actions workflow runs the tests on Windows, Linux, and macOS. This is done via a "job matrix", which runs the three jobs in parallel.

By default GitHub Actions uses a "fail fast" behavior for job matrixes, where all in progress or pending jobs of the matrix are immediately canceled upon failure of any job.

The benefit of this "fail fast" behavior is efficiency in the case where a failure is universal and those other jobs were certain to eventually fail. However, in other cases a failure is either specific to the configuration of a single job (e.g., a Windows-specific bug), or a spurious result of a transient network outage. In the latter case, canceling the other jobs is very harmful. Running the jobs for the other operating systems would highlight the nature of an OS-specific failure for the contributor. Canceling other jobs due to a transient failure means all those jobs must be reran instead of only the specific job that suffered the failure.

What is the new behavior?

The workflow is configured to disable the "fail fast" behavior. All the matrix jobs will run even if one of them has failed.

Does this PR introduce a breaking change, and is titled accordingly?

No breaking change

Other information

Reference:

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategyfail-fast

To ensure functionality on all platforms, the "Test Go" GitHub Actions workflow runs the tests on Windows, Linux, and
macOS. This is done via a "job matrix", which runs the three jobs in parallel.

By default GitHub Actions uses a "fail fast" behavior for job matrixes, where all in progress or pending jobs of the
matrix are immediately canceled upon failure of any job.

The benefit of this "fail fast" behavior is efficiency in the case where a failure is universal and those other jobs
were certain to eventually fail. However, in other cases a failure is either specific to the configuration of a single
job (e.g., a Windows-specific bug), or a spurious result of a transient network outage. In the latter case, canceling
the other jobs is very harmful. Running the jobs for the other operating systems would highlight the nature of an
OS-specific failure for the contributor. Canceling other jobs due to a transient failure means all those jobs must be
reran instead of only the specific job that suffered the failure.

The workflow is hereby configured to disable the "fail fast" behavior. All the matrix jobs will now run even if one of
them has failed.
@per1234 per1234 added type: enhancement Proposed improvement topic: infrastructure Related to project infrastructure labels Oct 10, 2022
@per1234 per1234 self-assigned this Oct 10, 2022
@codecov
Copy link

codecov bot commented Oct 10, 2022

Codecov Report

Base: 36.65% // Head: 36.84% // Increases project coverage by +0.19% 🎉

Coverage data is based on head (eb3a58e) compared to base (2480d93).
Patch coverage: 95.18% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1914      +/-   ##
==========================================
+ Coverage   36.65%   36.84%   +0.19%     
==========================================
  Files         231      235       +4     
  Lines       19683    19763      +80     
==========================================
+ Hits         7215     7282      +67     
- Misses      11640    11650      +10     
- Partials      828      831       +3     
Flag Coverage Δ
unit 36.84% <95.18%> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
internal/integrationtest/shared_directory.go 82.60% <82.60%> (ø)
internal/integrationtest/arduino-cli.go 85.91% <100.00%> (+0.25%) ⬆️
internal/integrationtest/environment.go 100.00% <100.00%> (ø)
internal/integrationtest/http_server.go 100.00% <100.00%> (ø)
arduino/monitor/monitor.go 41.05% <0.00%> (-4.74%) ⬇️
internal/integrationtest/subtests.go 100.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@per1234 per1234 merged commit 0a9b81a into arduino:master Oct 10, 2022
@per1234 per1234 deleted the no-fail-fast branch October 10, 2022 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: infrastructure Related to project infrastructure type: enhancement Proposed improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants