Skip to content

enable --validate-goto-model for all cbmc regressions #3661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

kroening
Copy link
Member

@kroening kroening commented Jan 3, 2019

This will prevent the introduction of changes that violate the checks
in the goto-validation procedure.

This will need more work to actually pass CI; at the moment, basically all tests fail.

  • Each commit message has a non-empty body, explaining why the change was made.
  • n/a Methods or procedures I have added are documented, following the guidelines provided in CODING_STANDARD.md.
  • n/a The feature or user visible behaviour I have added or modified has been documented in the User Guide in doc/cprover-manual/
  • Regression or unit tests are included, or existing tests cover the modified code (in this case I have detailed which ones those are in the commit message).
  • n/a My commit message includes data points confirming performance improvements (if claimed).
  • My PR is restricted to a single feature or bugfix.
  • n/a White-space or formatting changes outside the feature-related changed lines are in commits of their own.

@@ -1,13 +1,12 @@
default: tests.log

test:
@../test.pl -p -c ../../../src/cbmc/cbmc -X smt-backend
test: tests.log
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is tempting, but I think it's wrong: we do not have proper dependency tracking here (tests should be re-run whenever either the binary or the tests themselves change).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'll simply make "test" the default target.

@chrisr-diffblue
Copy link
Contributor

Sounds great if we can turn this on. One issue we've discovered recently is with the symbol checks that form part of --validate-goto-model failing on the return symbol, e.g. the following example:

int main(){
    return 0;
}

currently fails if passed through --validate-goto-model:

cbmc return.c --validate-goto-model
CBMC version 5.9 (cbmc-5.9-381-g32d7a6b89) 64-bit x86_64 macos
Parsing return.c
Converting
Type-checking return
Generating GOTO Program
Symbol is malformed:  (at: )

<< EXTRA DIAGNOSTICS >>
return'
<< END EXTRA DIAGNOSTICS >>

which I suspect is accounting for a large number of the CI failures when this option is enabled....

@chrisr-diffblue
Copy link
Contributor

This particular symbol.is_well_formed() check was added in #3193

@chrisr-diffblue
Copy link
Contributor

So #3193 added a test that : has_suffix(id2string(name), id2string(base_name))) - and this seems to not be the case for a number of cases, such as the return' symbol created for the entry point function, compound symbols created by c_typecheck_baset::typecheck_compound_type and possibly other cases.

@kroening
Copy link
Member Author

kroening commented Jan 3, 2019

@chrisr-diffblue I can well imagine a few more cases where we might use suffixes on the identifier to disambiguate; say we are doing that for C++ and Java method overloading.

@kroening kroening force-pushed the enable-validate-goto-model branch from 26b9b4a to 74722af Compare January 3, 2019 17:29
@chrisr-diffblue
Copy link
Contributor

@kroening So can we enumerate what makes a symbol 'well formed' ? We can remove this suffix check if that's not something that we expect to hold, which would leave 'well formed' simply being "has a mode".

@kroening kroening force-pushed the enable-validate-goto-model branch from 74722af to 0f66da3 Compare January 3, 2019 17:55
@kroening kroening force-pushed the enable-validate-goto-model branch 4 times, most recently from 6e46bca to 745351e Compare January 4, 2019 08:09
Copy link
Contributor

@allredj allredj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫
This PR failed Diffblue compatibility checks (cbmc commit: 745351e).
Build URL: https://travis-ci.com/diffblue/test-gen/builds/96270244
Status will be re-evaluated on next push.
Please contact @peterschrammel, @thk123, or @allredj for support.

Common spurious failures:

  • the cbmc commit has disappeared in the mean time (e.g. in a force-push)
  • the author is not in the list of contributors (e.g. first-time contributors).

The incompatibility may have been introduced by an earlier PR. In that case merging this
PR should be avoided unless it fixes the current incompatibility.

@chrisr-diffblue
Copy link
Contributor

@kroening - I'm working at the moment on getting the symbol wellformedness checks working so that you won't have to revert df616e4 - I'll let you know when I have something that is useable.

This will prevent the introduction of changes that violate the checks
in the goto-validation procedure.
@kroening kroening force-pushed the enable-validate-goto-model branch from 745351e to 9849491 Compare January 17, 2019 08:18
@tautschnig
Copy link
Collaborator

This has been merged as part of #3767.

@tautschnig tautschnig closed this Jan 30, 2019
@tautschnig tautschnig deleted the enable-validate-goto-model branch January 30, 2019 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants