Skip to content

Property reporting utilities [blocks: 3584] #3583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jan 14, 2019
Merged

Property reporting utilities [blocks: 3584] #3583

merged 3 commits into from
Jan 14, 2019

Conversation

peterschrammel
Copy link
Member

@peterschrammel peterschrammel commented Dec 16, 2018 •
edited
Loading

Based on #3579, only review last 4 commits.

  • Each commit message has a non-empty body, explaining why the change was made.
  • Methods or procedures I have added are documented, following the guidelines provided in CODING_STANDARD.md.
  • The feature or user visible behaviour I have added or modified has been documented in the User Guide in doc/cprover-manual/
  • Regression or unit tests are included, or existing tests cover the modified code (in this case I have detailed which ones those are in the commit message).
  • [n/a] My commit message includes data points confirming performance improvements (if claimed).
  • My PR is restricted to a single feature or bugfix.
  • [n/a] White-space or formatting changes outside the feature-related changed lines are in commits of their own.

@peterschrammel peterschrammel changed the title Property reporting utilities [depends: 3579] Property reporting utilities [depends: 3579, 3580] Dec 16, 2018
@peterschrammel peterschrammel changed the title Property reporting utilities [depends: 3579, 3580] Property reporting utilities [depends: 3579, 3580, blocks: 3584] Dec 16, 2018
@peterschrammel peterschrammel force-pushed the property-reporting-utils branch from 67ad462 to 7943da0 Compare December 17, 2018 10:39
tautschnig
tautschnig reviewed Dec 17, 2018
View reviewed changes
src/goto-checker/goto_checker.h Outdated
@@ -59,6 +59,11 @@ class goto_checkert
/// This builds and returns the counterexample
virtual goto_tracet build_error_trace() const = 0;

/// This outputs an error witness
virtual void output_error_witness(const goto_tracet &) = 0;
/// This outputs an proof
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/an/a/

src/goto-checker/goto_checker.h Outdated
/// This outputs an error witness
virtual void output_error_witness(const goto_tracet &) = 0;
/// This outputs an proof
virtual void output_proof() = 0;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there actually no users of goto_checkert at this point (I've partly lost sight of which commits are or aren't currently on the stack)? Otherwise some implementation would have to be added to make sure things compile even with just this commit.

src/goto-checker/report_util.cpp
// collect properties in a vector
std::vector<propertiest::const_iterator> sorted_properties;
for(auto p_it = properties.begin(); p_it != properties.end(); p_it++)
sorted_properties.push_back(p_it);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should introduce sorted ranges (i.e., add a sort alongside concat, filter)?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm as discussed out of band: this was just an idea while reading this code, no need for changes in here.

@peterschrammel peterschrammel force-pushed the property-reporting-utils branch from 7943da0 to a312377 Compare January 13, 2019 15:00
@peterschrammel peterschrammel changed the title Property reporting utilities [depends: 3579, 3580, blocks: 3584] Property reporting utilities [blocks: 3584] Jan 13, 2019
@peterschrammel peterschrammel force-pushed the property-reporting-utils branch from a312377 to 3330164 Compare January 13, 2019 15:30
@peterschrammel
Add count_properties with given result
cc1a90e 
To count how many properties have a certain status.
This will be used in property reporting.
@peterschrammel
Add property output functions
4da669d 
Plain text, XML and JSON output for property results.
@peterschrammel
Add report_util for outputting verification results
519fb98 
Prints the property report in plain text, JSON or XML.
@peterschrammel peterschrammel force-pushed the property-reporting-utils branch from 3330164 to 519fb98 Compare January 13, 2019 16:37
allredj
allredj reviewed Jan 13, 2019
View reviewed changes
Copy link
Contributor

@allredj allredj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✔️
Passed Diffblue compatibility checks (cbmc commit: 519fb98).
Build URL: https://travis-ci.com/diffblue/test-gen/builds/97168198

tautschnig
tautschnig approved these changes Jan 13, 2019
View reviewed changes
src/goto-checker/report_util.cpp
}
log.status() << "\n** "
<< count_properties(properties, property_statust::FAIL)
<< " of " << properties.size() << " failed" << messaget::eom;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we really print this unconditionally?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what is happening at the moment: https://github.com/diffblue/cbmc/blob/develop/src/cbmc/all_properties.cpp#L246
Which condition do you have in mind?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes, I had missed that.

src/goto-checker/properties.cpp
{
xmlt xml_result("result");
xml_result.set_attribute("property", id2string(property_id));
xml_result.set_attribute("status", as_string(property_info.status));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing description

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what we are doing at the moment: https://github.com/diffblue/cbmc/blob/develop/src/cbmc/all_properties.cpp#L258

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It still seems wrong.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed as #3798.

@peterschrammel peterschrammel mentioned this pull request Jan 13, 2019
Merged
6 tasks
@tautschnig tautschnig mentioned this pull request Jan 14, 2019
Open
@tautschnig tautschnig merged commit dd52eec into diffblue:develop Jan 14, 2019
@peterschrammel peterschrammel deleted the property-reporting-utils branch January 14, 2019 09:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tautschnig tautschnig tautschnig approved these changes

@allredj allredj allredj left review comments

@cesaro cesaro Awaiting requested review from cesaro

@chrisr-diffblue chrisr-diffblue Awaiting requested review from chrisr-diffblue

@cristina-david cristina-david Awaiting requested review from cristina-david

@kroening kroening Awaiting requested review from kroening kroening is a code owner

@pkesseli pkesseli Awaiting requested review from pkesseli

@smowton smowton Awaiting requested review from smowton

Assignees

@peterschrammel peterschrammel

Labels
blocker Goto-checker
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@peterschrammel @tautschnig @allredj