-
Notifications
You must be signed in to change notification settings - Fork 273
[TG-1523] fix removed required virtual calls #1666
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[TG-1523] fix removed required virtual calls #1666
Conversation
Current state is as follows, the dispatch table is analyzed for entries that have a empty For
that is for none of the candidate classes there's an implementing function. Calling In the GOTO this results in the following code
That is, for the class_id in question, the parent classes' implementation is called and the non-existence of another implementation is marked in the comment of the generated @smowton as you certainly have more insight into how this should work, is this a viable solution? Unfortunately I failed to construct a simple regression test for this. Building something where a child class implements the interface and its parent class actually implements the interface's method, CBMC created a non-det version of the function for the interface. |
@mgudemann Would you be able to provide a complicated regression test (you said you failed to build a simple one)? I think it would be great if anything vaguely resembling a bugfix also came with a regression test. |
|
||
bool class_hierarchyt::is_abstract_class(const irep_idt &c) const | ||
{ | ||
return abstract_classes.count(c) > 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand that using count
requires less typing, but then it does require looking up the container type to figure out what's going on. Would you mind using abstract_classes.find(c) != abstract_classes.end()
instead?
bool class_hierarchyt::is_abstract_class(const irep_idt &c) const | ||
{ | ||
return abstract_classes.count(c) > 0; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I might not have a sufficient understanding of this code, but it seems the approach taken here has potential for problems: this function may return different answers depending on when it is called.
I'd strongly recommend doing lookups in the symbol_table
instead, thus avoiding the need for abstract_classes
altogether.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tautschnig definitely right with both comments
src/goto-programs/class_hierarchy.h
Outdated
|
||
protected: | ||
void get_children_trans_rec(const irep_idt &, idst &) const; | ||
void get_parents_trans_rec(const irep_idt &, idst &) const; | ||
bool has_abstract_bases(const irep_idt &c) const; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is this implemented?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will go away, isn't used currently
@tautschnig the known case where this problem appears uses proprietary extensions unfortunately, the basic structure is the one outlined above, a class that implements an interface where the necessary methods actually come from a base class. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is currently incorrect. Recommend simplifying and using existing code to check for inherited methods when a method is abstract at a particular point in the class hierarchy; beyond that no structural change should be necessary.
class_id, | ||
root_function.symbol_expr, | ||
component_name, | ||
functions, | ||
visited); | ||
|
||
const auto find_empty = [&](const dispatch_table_entryt &entry) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just capture this
?
return entry.symbol_expr == symbol_exprt(); | ||
}; | ||
bool empty_found = | ||
std::any_of(functions.begin(), functions.end(), find_empty); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Name the lambda is_empty
std::set<irep_idt> &visited) const; | ||
std::set<irep_idt> &visited, | ||
bool recurse=true) const; | ||
void get_parent_functions( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This reads like a duplicate of resolve_concrete_function_callt
(see https://github.com/diffblue/cbmc/blob/develop/src/goto-programs/remove_virtual_functions.cpp#L300 for example)
|
||
if(recurse) | ||
{ | ||
get_child_functions( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why would you ever want to not recurse over child types?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had abused this to check only direct parents, but this is now the recursive get_parent_function
method (and probably soon the one you pointed out)
visited); | ||
if(recurse) | ||
{ | ||
get_parent_functions( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As above, get_parent_functions is probably a duplicate of the existing resolve-concrete method.
std::cout << f.class_id << ":" << f.symbol_expr.pretty() << " "; | ||
std::cout << std::endl; | ||
|
||
// with a non-empty virtual dispatch table but only empty function |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The condition says any class fails to provide a definition, but the comment says all
// method. Therefore we try finding it in the set of parent classes of a | ||
// class in the dispatch table. | ||
|
||
std::map<irep_idt, symbol_exprt> new_entries; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we doing this as a fixup step, not getting it right the first time? During the walk over children I could simply ask at each node, if there is no definition at this child, use resolve-concrete to search for an inherited definition
// } | ||
} | ||
|
||
// collect sets of children classes for each class in the new additional |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is incorrect. If we have an interface Interface
that prototypes a method foo
, and we have types A
, B
and C
, such that (writing <:
for subtyping) A <: C, B <: C, A <: Interface
, and C
and B
define concrete definitions of foo
, then a call ((Interface)something).foo()
may target an A
object, and thus call C.foo
, but may not target a B
since that doesn't inherit Interface
. This loop would appear to gather B
as a candidate callee because it is a child of C
.
In short, it should suffice to try to resolve a concrete function call at each child node where the method is abstract, without need for this further search over child types.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@smowton thanks for your comments, I now use resolve-concrete-function which does this in a better way. For each child, I check for such an implementation if get_child_functions was not successful in finding one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it should check for one even if get-child-functions finds one. Example:
C extends B extends A. B implements Interface. C and A provide implementations of f. Then a call to ((Interface)x).f() should consider B.f() aka A.f() as a target, even though its child C provides an implementation.
5977238
to
b88a1eb
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As already suggested, it would be great to have a test for this. If not a regression test, how about a unit test? Chris mentioned few class/interface hierarchies that should be caught correctly in his comments, can they be used to see is the search finds the correct function?
{ | ||
function.class_id = resolved_call.get_class_identifier(); | ||
const symbolt &called_symbol = | ||
*symbol_table.lookup(resolved_call.get_virtual_method_name()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you use lookup_ref
instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do a re-review once existing comments are addressed.
const irep_idt &class_id, | ||
const irep_idt &component_name) const; | ||
std::set<irep_idt> &visited, | ||
const std::function< |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider typedef'ing this
3bc3929
to
619da88
Compare
619da88
to
7a559f6
Compare
@smowton @svorenova @tautschnig I added a regression test for this. Interestingly, when using the orignal version, there's a call to |
Is the "Do not merge" still accurate? (The linter failure will certainly need addressing, but otherwise it sounded like this was ready for another review by @smowton.) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Patch looks reasonable, I think the is_abstract_class
stuff is unused though?
@@ -31,6 +31,11 @@ void class_hierarchyt::operator()(const symbol_tablet &symbol_table) | |||
const struct_typet &struct_type= | |||
to_struct_type(it->second.type); | |||
|
|||
if(to_class_type(struct_type).is_abstract()) | |||
{ | |||
abstract_classes.insert(it->first); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unused?
src/goto-programs/class_hierarchy.h
Outdated
@@ -32,6 +33,7 @@ class class_hierarchyt | |||
|
|||
typedef std::map<irep_idt, entryt> class_mapt; | |||
class_mapt class_map; | |||
std::set<irep_idt> abstract_classes; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again, unused?
576dd0d
to
6a3f87b
Compare
@tautschnig @smowton adressed your comments, linter should be happy now and the |
It seems the linter isn't yet perfectly happy, and I'm still wondering about do-not-merge? |
@tautschnig we switched to clang-format and there are cases where clan-format and cpplint do not agree, e.g., in this case of where to put braces for a lambda function clang-format says
while cpplint would like the opening brace |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One line of probably unnecessary change; otherwise LGTM, no need for re-review.
src/goto-programs/class_hierarchy.h
Outdated
@@ -16,6 +16,7 @@ Date: April 2016 | |||
|
|||
#include <iosfwd> | |||
#include <map> | |||
#include <set> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unnecessary
That's news to me, and this PR seems proof of that: the "Cpp Linter" job certainly failed and is blocking.
As far as I recall a similar discussion has happened on other PRs; either someone needs to sit down and adjust the linter, or else (much easier) just add |
@tautschnig and the do not merge just went away and this could be merged once reviewers are ok |
@tautschnig cpplint failures do not prevent merging, in contrast to clang-format failures which are part of Travis CI runs |
6a3f87b
to
f394d01
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thank you for adding the test.
}; | ||
|
||
const resolve_concrete_function_callt::concrete_function_callt | ||
&resolved_call = resolve_function_call(class_id, component_name); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not just call get_virtual_call_target
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
function.symbol_expr = called_symbol.symbol_expr(); | ||
function.symbol_expr.set( | ||
ID_C_class, resolved_call.get_class_identifier()); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does an invalid resolved_call
mean? Should this be an invariant or is this reasonable behaviour? If reasonable, would be good to have a to have a test that explores this path.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
an unresolved call should not simply remove the function call, but t least call a non-deterministic function, like it seems to be done when not using --lazy-methods
f394d01
to
3365054
Compare
@tautschnig the difference between clang-format and cpplint isn't ideal, I agree. But then, these cases seem to be quite limited in number. |
I think it's any use of a lambda and the colon in class declarations/inheritance, as far as I can tell. But then I don't know what to deduce from that comment? |
d190fd8 Merge remote-tracking branch 'upstream/develop' into pull-support-20180112 5bd5962 Merge pull request diffblue#1667 from romainbrenguier/feature/type_cast 8ecb55a Merge pull request diffblue#1717 from smowton/smowton/feature/remove_virtual_functions_per_function 0d7310a Merge pull request diffblue#1691 from romainbrenguier/bugfix/getClass#TG-1245 2064849 Use type_checked_cast in boolbv_width c5fc351 Validate data in pointer_typet in to_pointer_typet 35905c6 Add can_cast_type, validate_type for pointer_typet b72bacf Define type_try_dynamic_cast and type_checked_cast 23c3561 Unit test for string symbol resolution c13e602 Adding only needed equations in symbol resolution ae4deff Debug information for string equations 912828d JBMC: Run remove-virtual-functions as each function is converted a711c64 Introduce mechanism for renumbering an individual GOTO program e308a32 Merge pull request diffblue#1679 from NlightNFotis/nondet_extra_test e6ceb91 Merge pull request diffblue#1724 from tautschnig/fix-visitor ea74bed Add extra test for nondet-static flag and arrays 4f74896 Use irept API, not implementation-level API c55b4a5 Merge pull request diffblue#1682 from martin-cs/fix/dependence-graph-namespace-lifespan 1a2c14b Merge pull request diffblue#1722 from diffblue/unsafe_type_constructors 957a568 Merge pull request diffblue#1677 from NlightNFotis/pb4_develop 9c5add4 remove deprecated constructors for three bitvector types c96e02a no longer use deprecated constructors for some bitvector types 954060e Add unit test for has_subtype 3dd3877 Refactor has_char_pointer_subtype with has_subtype 4699c13 Extend symbol resolution to string_typet 74144fc Handle if_exprt in add_axioms_for_string_literal c6c1b3f Add an optional guard to add_axioms_for_constant 933d635 Merge pull request diffblue#1716 from mgudemann/fix/null_check_for_java_instanceof 1659314 Merge pull request diffblue#1715 from smowton/smowton/cleanup/jbmc_unused_passes 9c457b7 Add regression test for null instanceof. 2080cd3 Complete instanceof for Java. d4300d0 Merge pull request diffblue#1697 from diffblue/nondet_symbol_exprt 1c68dd4 Merge pull request diffblue#1714 from tautschnig/c-library-strcat 44b5bae Merge pull request diffblue#1698 from thomasspriggs/tg1633 c4304ba JBMC: Remove C-only passes bb8cfaa C library: Check upper bounds in memset, memcpy, memmove 7d4984f C library: Implement strcat, strncat 2a5cea2 This introduces nondet_symbol_exprt, which is generated by symbolic execution in response to side_effect_expr_nondett 85193a0 Merge pull request diffblue#1694 from NathanJPhillips/feature/add-raw-lhs-to-trace d9122dc Merge pull request diffblue#1710 from NathanJPhillips/feature/remove_instanceof_per_function 092df69 Switch from custom file / path routines to Boost-filesystem c8821b2 Allow to remove instanceof when remove exceptions 94b7658 Don't pass iterators into function calls a9c4e4f Added regression tests 76318ce Protect extended trace behind a command line option 69b0ff1 Added base_name in comments for all symbols e86080a Add raw LHS irep field to trace output ddd1b7a Add remove_instanceof overload to remove from a particular instruction 1c227b7 Merge pull request diffblue#1660 from smowton/smowton/fix/lazy_methods_array_parameters ae89c94 Lazy loading: assume concrete generic parameter types are needed 80eb6a6 TG-1877: Include array pointer types in needed classes 1053e5f Fix for [TG-1633] Inner generic types have incorrect type identifier e2cda1a Merge pull request diffblue#1704 from tautschnig/fix-copy-paste ef4a65e Fix op1/op0 copy&paste typo 21ea31f Merge pull request diffblue#1702 from peterschrammel/goto-diff-java c4bc953 Merge pull request diffblue#1701 from peterschrammel/allow-instrument-jdk 2811363 Java regression test for goto-diff 43d2e09 Also reset fresh temporary symbol counter 9ef28f4 Compare relative goto target offsets eaf3a7d Get source location from symbol table ab59659 Allow instrumentation of java.* and org.cprover.* 6fbd59c Merge pull request diffblue#1631 from tautschnig/fix-pointer-minus 7c04b5c Merge pull request diffblue#1699 from NathanJPhillips/feature/reset-main-in-tests 5e0f186 Pointer difference over void* is difference over char* faf8f00 Merge commit 'a83b52cddbed22304372c276512c63701eb3aedb' into pull-support-20180104 8236db4 Merge pull request diffblue#1419 from peterschrammel/refactor/cover-instrument a580e27 Merge pull request diffblue#1689 from smowton/smowton/feature/get_this 591511a Allow callers of load_java_class to pass the name of the main function 1b86b27 Merge pull request diffblue#1687 from smowton/smowton/feature/class-hierarchy-dot fd2bf6a Merge pull request diffblue#1688 from smowton/smowton/feature/parameter_indices f570ce5 Merge pull request diffblue#1696 from smowton/smowton/fix/identical_struct_equality 61b0d6d Merge pull request diffblue#1666 from mgudemann/bugfix/removed_required_virtual_calls 3365054 Add regression test 2b6dc8b Resolve concrete function call if no implementation is found 3f1fd64 Add code_typet::get_parameter_indices 42cf61a Fix testing for empty line in test desc file 2090000 Fix missing newline at end of desc file e448d5f Fix unsatisfiable test line f7f033d String smoke tests: ensure no type mismatches are seen b627c3d Replace unsound struct-cast simplification 8fa42b3 Class hierarchy: add DOT output, unit tests 04f2faf Mark GOTO instructions with unresolved virtual calls aac181f Pass command line options via optionst b6fa3e8 Factorize source location initialization 8da5395 Document cover functions a7f0c3d Introduce cover instrumenter 873627a Split cover into several files 0fc08f3 Replace cover-function-only by cover-include-pattern 1f2102c Add code_typet::get_this 2801f0f Avoid crashing when --dependence-graph is used by correcting namespace scoping. acac776 Add a test for the same-named static functions crashing dependence graph in the goto-analyser 05f46a9 Fix the problem where two static functions with the same name would cause the dependency graph to fail. git-subtree-dir: cbmc git-subtree-split: d190fd8
first version of approach to solve the problem of parent classes implementing an interface