Skip to content

Add github actions ubuntu-20.04 make build #5449

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 18, 2020

Conversation

thomasspriggs
Copy link
Contributor

@thomasspriggs thomasspriggs commented Aug 11, 2020

So that we have testing of the make build on Ubuntu-20.04.

  • Each commit message has a non-empty body, explaining why the change was made.
  • n/a No code modified - Methods or procedures I have added are documented, following the guidelines provided in CODING_STANDARD.md.
  • n/a No features or user visible behaviour changes - The feature or user visible behaviour I have added or modified has been documented in the User Guide in doc/cprover-manual/
  • n/a No code modified - Regression or unit tests are included, or existing tests cover the modified code (in this case I have detailed which ones those are in the commit message).
  • n/a None claimed - My commit message includes data points confirming performance improvements (if claimed).
  • My PR is restricted to a single feature or bugfix.
  • n/a White-space or formatting changes outside the feature-related changed lines are in commits of their own.

@codecov
Copy link

codecov bot commented Aug 11, 2020

Codecov Report

Merging #5449 into develop will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop    #5449   +/-   ##
========================================
  Coverage    68.22%   68.22%           
========================================
  Files         1178     1178           
  Lines        97585    97585           
========================================
  Hits         66581    66581           
  Misses       31004    31004           
Flag Coverage Δ
#cproversmt2 42.80% <ø> (ø)
#regression 65.40% <ø> (ø)
#unit 32.25% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7d8a6f7...fa9a9cc. Read the comment docs.

@codecov
Copy link

codecov bot commented Aug 11, 2020

Codecov Report

Merging #5449 into develop will increase coverage by 36.01%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##           develop    #5449       +/-   ##
============================================
+ Coverage    32.23%   68.25%   +36.01%     
============================================
  Files          927     1180      +253     
  Lines        81237    97722    +16485     
============================================
+ Hits         26187    66698    +40511     
+ Misses       55050    31024    -24026     
Flag Coverage Δ
#cproversmt2 42.81% <ø> (?)
#regression 65.42% <ø> (?)
#unit 32.23% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/pointer-analysis/value_set_analysis.h 20.00% <0.00%> (-13.34%) ⬇️
src/analyses/cfg_dominators.h 88.88% <0.00%> (-11.12%) ⬇️
src/analyses/natural_loops.h 92.30% <0.00%> (-7.70%) ⬇️
src/util/symbol.h 94.59% <0.00%> (-5.41%) ⬇️
src/goto-programs/goto_model.h 66.66% <0.00%> (-4.77%) ⬇️
src/analyses/static_analysis.h 54.16% <0.00%> (-2.36%) ⬇️
src/util/union_find.h 85.71% <0.00%> (-0.50%) ⬇️
src/analyses/ai.h 57.27% <0.00%> (-0.02%) ⬇️
src/analyses/is_threaded.h 100.00% <0.00%> (ø)
src/solvers/sat/cnf_clause_list.h 0.00% <0.00%> (ø)
... and 896 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 199f6ef...f58407f. Read the comment docs.

make -C jbmc/unit CXX='/usr/bin/g++' -j2
- name: Run unit tests
run: |
make -C unit test

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the unit tests, at least the jbmc ones, actually need to be run from the unit directory

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The -C unit argument to make changes into the unit directory and changes back before exiting make. This is the same way as the make build is implemented in Travis right now.

@thomasspriggs thomasspriggs force-pushed the tas/gha_ubuntu20_make branch 3 times, most recently from 5a49d8a to a8ebed7 Compare August 13, 2020 12:47
@thomasspriggs thomasspriggs marked this pull request as ready for review August 13, 2020 13:06
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with explanation

@hannes-steffenhagen-diffblue
Copy link
Contributor

So yeah, the TESTPL_JOBS optimisation here leads to a roughly 2x speedup in test runtime. This is expected, but good to have confirmed.

@thomasspriggs thomasspriggs force-pushed the tas/gha_ubuntu20_make branch from a8ebed7 to ad13e66 Compare August 17, 2020 12:44
TESTPL_JOBS: 4
run: |
sudo apt-get install -yq gcc gdb g++ maven jq flex bison libxml2-utils cpanminus
make -C src minisat2-download
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be more inclined to move this make invocation into the build section - it's part of the system-under-tests (i.e. the Makefiles) rather than general environment setup (the apt-get)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but afaik this doesn't really build minisat, does it? It only downloads the sources and applies a patch as far as I can recall?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of which is defined in the Makefile, which is what is being tested here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I recall correctly (https://github.com/diffblue/cbmc/blob/develop/src/Makefile#L120).

I would argue this is part of the configuration, as without minisat being downloaded, the build fails - but it's really because the sources are missing, not because of a compilation issue or anything else. And seeing as minisat is one of the SAT solvers we support, I would argue even more this is part of the configuration, even if it's just the default right now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a massive issue with this - so I'm happy to approve if everyone else is happy. My logic was:

  1. Cmake does the minisat download as part of its configure/build steps, so my suggestion would be more consistent with that
  2. If the apt-get fails, or the cpanm fails, its mostly just a case of saying "shrug, infrastructure failed - hit retry", but if the minisat-download fails you do have to double check whether our own stuff needs to be modified. If you have the "setup our CI host" as a separate section to "run our own stuff" then it makes a clear distinction between stuff we may need to manage/modify and stuff that is just pure environment setup.

Copy link
Contributor

@NlightNFotis NlightNFotis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@chrisr-diffblue chrisr-diffblue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A minor quibble, but not going to block the PR considering other approvals.

So that we have testing of the make build on Ubuntu-20.04.

Adding gdb installation is required for unit tests. Installation of
`libxml2-utils` providing `xmllint` is so that the tests for valid XML
output can run.
@thomasspriggs thomasspriggs force-pushed the tas/gha_ubuntu20_make branch from ad13e66 to f58407f Compare August 17, 2020 17:39
@thomasspriggs thomasspriggs merged commit 6301c3f into diffblue:develop Aug 18, 2020
@thomasspriggs thomasspriggs deleted the tas/gha_ubuntu20_make branch August 18, 2020 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants