[skip changelog] Move upload of coverage data to dedicated job in test workflow #1915
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The code coverage data generated by running the unit tests in the "Test Go" GitHub Actions workflow is uploaded to Codecov. This upload is prone to occasional transient failures. With the previous workflow configuration this caused a failure of the Linux test run job. Contributors would need to review the logs to understand that these failures were not caused by a test failing. The failure also causes the ongoing jobs that run the tests on other operating systems to be immediately canceled, meaning their full test results are not available. The entire test suite must be reran just to attempt the upload again. Moving the upload to a dedicated workflow job makes it faster and easier for contributors to interpret the cause of a test failure. It also allows the test suite to complete for all operating systems, making their results immediately available to the contributor even when the coverage data upload fails. The coverage upload job can be reran, making recovery from a transient failure more efficient.
The code coverage data generated by running the unit tests in the "Test Go" GitHub Actions workflow is uploaded to Codecov. Arduino CLI's unit tests are split into two collections: those covering the modernized part of the codebase, and those covering the "legacy" code inherited from arduino-builder. A separate code coverage data file is produced for each of these collections. The `codecov/codecov-action` GitHub Actions action is used to perform this upload. Previously, a separate step was used for the upload of each of the code coverage files. The action provides the capability to specify multiple files for upload in a comma-separated list. This approach will make the workflow more efficient and reliable.
per1234
added
type: enhancement
MatteoPologruto
approved these changes
Oct 11, 2022
cmaglie
approved these changes
Oct 11, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please check if the PR fulfills these requirements
See how to contribute
before creating one)
our contributing guidelines
UPGRADING.mdhas been updated with a migration guide (for breaking changes)What kind of change does this PR introduce?
Infrastructure enhancement
What is the current behavior?
The code coverage data generated by running the unit tests in the "Test Go" GitHub Actions workflow is uploaded to Codecov.
This upload is prone to occasional transient failures. With the previous workflow configuration this caused a failure of the Linux test run job. Contributors would need to review the logs to understand that these failures were not caused by a test failing.
The failure also causes the ongoing jobs that run the tests on other operating systems to be immediately canceled, meaning their full test results are not available. The entire test suite must be reran just to attempt the upload again.
What is the new behavior?
Move the coverage data upload to a dedicated workflow job.
This makes it faster and easier for contributors to interpret the cause of a test failure.
It also allows the test suite to complete for all operating systems, making their results immediately available to the contributor even when the coverage data upload fails.
The coverage upload job can be reran, making recovery from a transient failure more efficient.
Does this PR introduce a breaking change, and is titled accordingly?
No breaking change