Skip to content

Increase nominal ccache cache size for coverage CI job #7492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 19, 2023

Conversation

tautschnig
Copy link
Collaborator

@tautschnig tautschnig commented Jan 19, 2023

Speeding up builds for coverage reporting is desirable for two reasons:

  1. Coverage-checking builds are approximately twice as slow compared to release builds, even in absence of any ccache-supported caching on either side.
  2. Test runs with coverage logging and coverage collection take approximately 46 minutes when regular test runs just take 17 minutes on a similar platform.

Therefore, effective caching during builds has the potential to considerably speed up the overall job. Caching using ccache, however, was not effective, because ccache cleanups kicked in: cached artifacts were being removed while the build was still in progress. The resulting cache archive, therefore, was necessarily incomplete. ccache initiates a cleanup when a subdirectory reaches max_size / 16. Increasing max_size from 4 GB to 7 GB (empirically devised value) avoids such automatic cleanup during builds while still keeping the overall cache size well below 5 GB.

  • Each commit message has a non-empty body, explaining why the change was made.
  • n/a Methods or procedures I have added are documented, following the guidelines provided in CODING_STANDARD.md.
  • n/a The feature or user visible behaviour I have added or modified has been documented in the User Guide in doc/cprover-manual/
  • Regression or unit tests are included, or existing tests cover the modified code (in this case I have detailed which ones those are in the commit message).
  • My commit message includes data points confirming performance improvements (if claimed).
  • My PR is restricted to a single feature or bugfix.
  • n/a White-space or formatting changes outside the feature-related changed lines are in commits of their own.

Speeding up builds for coverage reporting is desirable for two reasons:
1. Coverage-checking builds are approximately twice as slow compared to
release builds, even in absence of any ccache-supported caching on
either side.
2. Test runs with coverage logging and coverage collection take
approximately 46 minutes when regular test runs just take 17 minutes on
a similar platform.

Therefore, effective caching during builds has the potential to
considerably speed up the overall job. Caching using ccache, however,
was not effective, because ccache cleanups kicked in:cached artifacts
were being removed while the build was still in progress. The resulting
cache archive, therefore, was necessarily incomplete. ccache initiates a
cleanup when a subdirectory reaches max_size / 16. Increasing max_size
from 4 GB to 7 GB (empirically devised value) avoids such automatic
cleanup during builds while still keeping the overall cache size well
below 5 GB.
@tautschnig tautschnig merged commit fb65095 into diffblue:develop Jan 19, 2023
@tautschnig tautschnig deleted the feature/coverage-cache branch January 19, 2023 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants