Replace unsound struct-cast simplification #1696

smowton · 2018-01-03T18:14:09Z

Previously the simplifier would simplify ((struct A)x) into (x) if x.type() and A
were structurally compatible (i.e. had the same named and typed fields). However this
isn't really a sound simplification, as it changes the type of the expression. I therefore
replace the rule with one that only applies within the common-case member expression, i.e.
simplifiying ((struct A)x).y -> x.y, which does not change the type of the outer expression.

Notes: @allredj @romainbrenguier initially I simply disabled the unsound simplification, however this caused two string-smoke-tests to fail because a hard java.lang.String -> java.lang.CharSequence cast went unsimplified, preventing constant propagation from completing and causing the contains and append_string tests to fail because they needed to use their general case rather than the special case where a parameter is known to be a constant. If you try disabling the new simplification introduced here you should be able to reproduce the problem -- this might be an interesting bug, or could just be a known existing weakness in the solver.

This solution could be generalised by permitting this simplification only for inner expressions, i.e. casts which do not occur at the root of the expression tree. @tautschnig do we want to try that, or is handling this special case enough?

tautschnig · 2018-01-03T18:37:58Z

We had that discussion elsewhere - would you happen to have pointers to that?

tautschnig · 2018-01-03T18:46:28Z

src/util/simplify_expr_struct.cpp

+    if(base_type_eq(op_type, op.op0().type(), ns))
+    {
+      expr.op0() = op.op0();
+      simplify_rec(expr);


I don't think it's necessary to simplify recursively, I'd say a call to simplify_member should do the trick.

I think this is the only bit keeping me from approving.

tautschnig · 2018-01-03T18:47:52Z

It would be great to have some regression tests that show that the previous one was unsound. Otherwise I'm fine with this special case.

smowton · 2018-01-03T21:16:07Z

@tautschnig for reference the previous discussion of the same problem is here: #1499 (comment)

Previously the simplifier would simplify ((struct A)x) into (x) if x.type() and A were structurally compatible (i.e. had the same named and typed fields). However this isn't really a sound simplification, as it changes the type of the expression. I therefore replace the rule with one that only applies within the common-case member expression, i.e. simplifiying ((struct A)x).y -> x.y, which does not change the type of the outer expression.

smowton · 2018-01-04T10:34:05Z

@tautschnig the string smoke tests were exposing the problem, so I've added an extra commit to verify they don't any longer. Tagging string developers to check this is reasonable.

I've also implemented your comment above.

tautschnig

Looks good, but the *.desc files need some (mostly trivial) work.

tautschnig · 2018-01-04T10:47:00Z

regression/strings-smoke-tests/java_append_object/test.desc

 ^EXIT=0$
 ^SIGNAL=0$
 ^VERIFICATION SUCCESSFUL$
 --
+non equal types
 Issue: diffblue/test-gen#82


There is a problem in the original test description file: this comment "Issue: ..." should have a preceding -- line to separate strings-not-to-be-found-in-output from comments.

tautschnig · 2018-01-04T10:47:29Z

regression/strings-smoke-tests/java_char_array_init/test.desc

 ^EXIT=0$
 ^SIGNAL=0$
 ^VERIFICATION SUCCESSFUL$
 --
+non equal types
 cbmc/test-gen#259


As above, the comment needs a preceding -- line.

tautschnig · 2018-01-04T10:48:12Z

regression/strings-smoke-tests/java_equal/test_2.desc

 ^EXIT=0$
 ^SIGNAL=0$
 --
+non equal types
 https://github.com/diffblue/test-gen/issues/856


As above: -- required.

tautschnig · 2018-01-04T10:49:10Z

regression/strings-smoke-tests/java_if/test.desc

 ^EXIT=10$
 ^SIGNAL=0$
 ^\[.*assertion.1\].* line 11.* SUCCESS$
 ^\[.*assertion.2\].* line 13.* FAILURE$
 --
+non equal types
 $ignoring\s*char\s*array


Why does this pattern start with a $?

tautschnig · 2018-01-04T10:49:22Z

regression/strings-smoke-tests/java_index_of_char/test.desc

 ^EXIT=0$
 ^SIGNAL=0$
 ^VERIFICATION SUCCESSFUL$
 --
+non equal types
 Issue: cbmc/test-gen#77


As above: -- required.

tautschnig · 2018-01-04T10:49:40Z

regression/strings-smoke-tests/java_int_to_string/test2_bug.desc

 ^EXIT=10$
 ^SIGNAL=0$
 assertion.* line 7 .* SUCCESS$
 assertion.* line 10 .* FAILURE$
 --
+non equal types
 --
 When string-max-length is not set, the solver can run out of memory in
 an unpredictable way.


Missing newline at end-of-file.

tautschnig · 2018-01-04T10:50:11Z

regression/strings-smoke-tests/java_length2/test.desc

 ^EXIT=0$
 ^SIGNAL=0$
 ^VERIFICATION SUCCESSFUL$
 --
+non equal types
 This test currently fails because we do not handle the case where the argument


As above: -- needed

tautschnig · 2018-01-04T10:50:46Z

regression/strings-smoke-tests/java_replace/test.desc

 ^EXIT=0$
 ^SIGNAL=0$
 ^VERIFICATION SUCCESSFUL$
 --
+non equal types
 diffblue/test-gen#256


As above: -- required

Type mismatches previously resulted from the expression simplifier removing casts from one struct type to another, so e.g. (struct A)b would be simplified to b, changing its type and leading to an equality with a type mismatch. Now that the simplifier has been fixed this should no longer happen -- this change ensures that none of the smoke tests generate the warning any longer.

smowton · 2018-01-04T11:05:27Z

@tautschnig changes addressed. Looks like my script to add the rejected lines wasn't quite smart enough :)

romainbrenguier

this looks fine

d190fd8 Merge remote-tracking branch 'upstream/develop' into pull-support-20180112 5bd5962 Merge pull request diffblue#1667 from romainbrenguier/feature/type_cast 8ecb55a Merge pull request diffblue#1717 from smowton/smowton/feature/remove_virtual_functions_per_function 0d7310a Merge pull request diffblue#1691 from romainbrenguier/bugfix/getClass#TG-1245 2064849 Use type_checked_cast in boolbv_width c5fc351 Validate data in pointer_typet in to_pointer_typet 35905c6 Add can_cast_type, validate_type for pointer_typet b72bacf Define type_try_dynamic_cast and type_checked_cast 23c3561 Unit test for string symbol resolution c13e602 Adding only needed equations in symbol resolution ae4deff Debug information for string equations 912828d JBMC: Run remove-virtual-functions as each function is converted a711c64 Introduce mechanism for renumbering an individual GOTO program e308a32 Merge pull request diffblue#1679 from NlightNFotis/nondet_extra_test e6ceb91 Merge pull request diffblue#1724 from tautschnig/fix-visitor ea74bed Add extra test for nondet-static flag and arrays 4f74896 Use irept API, not implementation-level API c55b4a5 Merge pull request diffblue#1682 from martin-cs/fix/dependence-graph-namespace-lifespan 1a2c14b Merge pull request diffblue#1722 from diffblue/unsafe_type_constructors 957a568 Merge pull request diffblue#1677 from NlightNFotis/pb4_develop 9c5add4 remove deprecated constructors for three bitvector types c96e02a no longer use deprecated constructors for some bitvector types 954060e Add unit test for has_subtype 3dd3877 Refactor has_char_pointer_subtype with has_subtype 4699c13 Extend symbol resolution to string_typet 74144fc Handle if_exprt in add_axioms_for_string_literal c6c1b3f Add an optional guard to add_axioms_for_constant 933d635 Merge pull request diffblue#1716 from mgudemann/fix/null_check_for_java_instanceof 1659314 Merge pull request diffblue#1715 from smowton/smowton/cleanup/jbmc_unused_passes 9c457b7 Add regression test for null instanceof. 2080cd3 Complete instanceof for Java. d4300d0 Merge pull request diffblue#1697 from diffblue/nondet_symbol_exprt 1c68dd4 Merge pull request diffblue#1714 from tautschnig/c-library-strcat 44b5bae Merge pull request diffblue#1698 from thomasspriggs/tg1633 c4304ba JBMC: Remove C-only passes bb8cfaa C library: Check upper bounds in memset, memcpy, memmove 7d4984f C library: Implement strcat, strncat 2a5cea2 This introduces nondet_symbol_exprt, which is generated by symbolic execution in response to side_effect_expr_nondett 85193a0 Merge pull request diffblue#1694 from NathanJPhillips/feature/add-raw-lhs-to-trace d9122dc Merge pull request diffblue#1710 from NathanJPhillips/feature/remove_instanceof_per_function 092df69 Switch from custom file / path routines to Boost-filesystem c8821b2 Allow to remove instanceof when remove exceptions 94b7658 Don't pass iterators into function calls a9c4e4f Added regression tests 76318ce Protect extended trace behind a command line option 69b0ff1 Added base_name in comments for all symbols e86080a Add raw LHS irep field to trace output ddd1b7a Add remove_instanceof overload to remove from a particular instruction 1c227b7 Merge pull request diffblue#1660 from smowton/smowton/fix/lazy_methods_array_parameters ae89c94 Lazy loading: assume concrete generic parameter types are needed 80eb6a6 TG-1877: Include array pointer types in needed classes 1053e5f Fix for [TG-1633] Inner generic types have incorrect type identifier e2cda1a Merge pull request diffblue#1704 from tautschnig/fix-copy-paste ef4a65e Fix op1/op0 copy&paste typo 21ea31f Merge pull request diffblue#1702 from peterschrammel/goto-diff-java c4bc953 Merge pull request diffblue#1701 from peterschrammel/allow-instrument-jdk 2811363 Java regression test for goto-diff 43d2e09 Also reset fresh temporary symbol counter 9ef28f4 Compare relative goto target offsets eaf3a7d Get source location from symbol table ab59659 Allow instrumentation of java.* and org.cprover.* 6fbd59c Merge pull request diffblue#1631 from tautschnig/fix-pointer-minus 7c04b5c Merge pull request diffblue#1699 from NathanJPhillips/feature/reset-main-in-tests 5e0f186 Pointer difference over void* is difference over char* faf8f00 Merge commit 'a83b52cddbed22304372c276512c63701eb3aedb' into pull-support-20180104 8236db4 Merge pull request diffblue#1419 from peterschrammel/refactor/cover-instrument a580e27 Merge pull request diffblue#1689 from smowton/smowton/feature/get_this 591511a Allow callers of load_java_class to pass the name of the main function 1b86b27 Merge pull request diffblue#1687 from smowton/smowton/feature/class-hierarchy-dot fd2bf6a Merge pull request diffblue#1688 from smowton/smowton/feature/parameter_indices f570ce5 Merge pull request diffblue#1696 from smowton/smowton/fix/identical_struct_equality 61b0d6d Merge pull request diffblue#1666 from mgudemann/bugfix/removed_required_virtual_calls 3365054 Add regression test 2b6dc8b Resolve concrete function call if no implementation is found 3f1fd64 Add code_typet::get_parameter_indices 42cf61a Fix testing for empty line in test desc file 2090000 Fix missing newline at end of desc file e448d5f Fix unsatisfiable test line f7f033d String smoke tests: ensure no type mismatches are seen b627c3d Replace unsound struct-cast simplification 8fa42b3 Class hierarchy: add DOT output, unit tests 04f2faf Mark GOTO instructions with unresolved virtual calls aac181f Pass command line options via optionst b6fa3e8 Factorize source location initialization 8da5395 Document cover functions a7f0c3d Introduce cover instrumenter 873627a Split cover into several files 0fc08f3 Replace cover-function-only by cover-include-pattern 1f2102c Add code_typet::get_this 2801f0f Avoid crashing when --dependence-graph is used by correcting namespace scoping. acac776 Add a test for the same-named static functions crashing dependence graph in the goto-analyser 05f46a9 Fix the problem where two static functions with the same name would cause the dependency graph to fail. git-subtree-dir: cbmc git-subtree-split: d190fd8

smowton requested review from kroening, peterschrammel and tautschnig as code owners January 3, 2018 18:14

tautschnig reviewed Jan 3, 2018

View reviewed changes

smowton requested review from allredj and romainbrenguier January 4, 2018 10:34

smowton force-pushed the smowton/fix/identical_struct_equality branch from 3cdcf04 to 8727f7b Compare January 4, 2018 10:34

tautschnig approved these changes Jan 4, 2018

View reviewed changes

smowton added 4 commits January 4, 2018 11:03

Fix unsatisfiable test line

e448d5f

Fix missing newline at end of desc file

2090000

Fix testing for empty line in test desc file

42cf61a

smowton force-pushed the smowton/fix/identical_struct_equality branch from 8727f7b to 42cf61a Compare January 4, 2018 11:04

romainbrenguier approved these changes Jan 4, 2018

View reviewed changes

tautschnig merged commit f570ce5 into diffblue:develop Jan 4, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace unsound struct-cast simplification #1696

Replace unsound struct-cast simplification #1696

smowton commented Jan 3, 2018 •

edited

Loading

tautschnig commented Jan 3, 2018

tautschnig Jan 3, 2018

tautschnig Jan 3, 2018

tautschnig commented Jan 3, 2018

smowton commented Jan 3, 2018 •

edited

Loading

smowton commented Jan 4, 2018 •

edited

Loading

tautschnig left a comment

tautschnig Jan 4, 2018

tautschnig Jan 4, 2018

tautschnig Jan 4, 2018

tautschnig Jan 4, 2018

tautschnig Jan 4, 2018

tautschnig Jan 4, 2018

tautschnig Jan 4, 2018

tautschnig Jan 4, 2018

smowton commented Jan 4, 2018 •

edited

Loading

romainbrenguier left a comment

Replace unsound struct-cast simplification #1696

Replace unsound struct-cast simplification #1696

Conversation

smowton commented Jan 3, 2018 • edited Loading

tautschnig commented Jan 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tautschnig commented Jan 3, 2018

smowton commented Jan 3, 2018 • edited Loading

smowton commented Jan 4, 2018 • edited Loading

tautschnig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smowton commented Jan 4, 2018 • edited Loading

romainbrenguier left a comment

Choose a reason for hiding this comment

smowton commented Jan 3, 2018 •

edited

Loading

smowton commented Jan 3, 2018 •

edited

Loading

smowton commented Jan 4, 2018 •

edited

Loading

smowton commented Jan 4, 2018 •

edited

Loading