Skip to content

goto-instrument 5.84 crash when trying to use __CPROVER_pointer_in_range_dfcc contract #7760

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rod-chapman opened this issue Jun 9, 2023 · 12 comments · Fixed by #7763
Closed
Assignees

Comments

@rod-chapman
Copy link
Collaborator

I am using CBMC 5.84 on Apple M1 macOS.

I'm trying to verify a simple algorithms that loops over an array of data that is passed in as a parameter.
I think I need to use __CPROVER_pointer_in_range_dfcc to verify that my pointer(s) remain valid for the array of data that has been passed in to my function.

See https://github.com/rod-chapman/cbmc-examples/tree/main/arrays

In that directory,

make TARGET=arsum

yields

rodchap@f4d4889dcf6d arrays % make TARGET=arsum
rm -f *.goto
goto-cc --function arsum_harness -DCBMC -o ar.goto ar.c
goto-instrument --bounds-check --pointer-check --memory-cleanup-check --div-by-zero-check --signed-overflow-check --pointer-overflow-check --conversion-check --undefined-shift-check --enum-range-check --pointer-primitive-check --dfcc arsum_harness --apply-loop-contracts --enforce-contract arsum ar.goto ar_contracts.goto
Reading GOTO program from 'ar.goto'
Function Pointer Removal
Virtual function removal
Cleaning inline assembler statements
Loading CPROVER C library (arm64)
Adding the cprover_contracts library (arm64)
file <builtin-library-malloc> line 6: symbol '__CPROVER_malloc_is_new_array' already has an initial value
symbol '__CPROVER_alloca_object' already has an initial value
symbol '__CPROVER_new_object' already has an initial value
file <builtin-library-free> line 11: symbol '__CPROVER_malloc_is_new_array' already has an initial value
Instrumenting memory predicate 'arsum'
Instrumenting memory predicate 'arsum_harness'
Instrumenting harness function 'arsum_harness'
Wrapping 'arsum' with contract 'arsum' in CHECK mode
--- begin invariant violation report ---
Invariant check failed
File: /tmp/cbmc-20230525-15049-1i7eyc/src/util/std_expr.cpp:173 function: instantiate
Condition: variables.size() == values.size()
Reason: Precondition
Backtrace:
0   goto-instrument                     0x0000000100b9bd4c _Z15print_backtraceRNSt3__113basic_ostreamIcNS_11char_traitsIcEEEE + 120
1   goto-instrument                     0x0000000100b9c09c _Z13get_backtracev + 44
2   goto-instrument                     0x000000010076a224 _Z29invariant_violated_structuredI17invariant_failedtJRKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEENS1_9enable_ifIXsr3std10is_base_ofIS0_T_EE5valueEvE4typeES9_S9_iS9_DpOT0_ + 52
3   goto-instrument                     0x000000010076a1f0 _Z29invariant_violated_structuredI17invariant_failedtJRKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEENS1_9enable_ifIXsr3std10is_base_ofIS0_T_EE5valueEvE4typeES9_S9_iS9_DpOT0_ + 0
4   goto-instrument                     0x0000000100c0d7d8 _ZNK13binding_exprt11instantiateERKNSt3__16vectorI5exprtNS0_9allocatorIS2_EEEE + 804
5   goto-instrument                     0x00000001007c4450 _ZN21dfcc_wrapper_programt23encode_requires_clausesEv + 220
6   goto-instrument                     0x00000001007c26c4 _ZN21dfcc_wrapper_programtC2E19dfcc_contract_modetRK7symboltS3_RK24dfcc_contract_functionstS3_R11goto_modeltR16message_handlertR13dfcc_librarytR16dfcc_instrumenttR28dfcc_lift_memory_predicatest + 1868
7   goto-instrument                     0x00000001007a214c _ZN22dfcc_contract_handlert25add_contract_instructionsE19dfcc_contract_modetRK8dstringtS3_S3_RK7symboltR13goto_programtRNSt3__13setIS1_NS9_4lessIS1_EENS9_9allocatorIS1_EEEE + 176
8   goto-instrument                     0x00000001007bdd00 _ZN19dfcc_swap_and_wrapt14check_contractE24dfcc_loop_contract_modetRK8dstringtS3_RNSt3__13setIS1_NS4_4lessIS1_EENS4_9allocatorIS1_EEEEb + 1604
9   goto-instrument                     0x00000001007bd3a0 _ZN19dfcc_swap_and_wrapt13swap_and_wrapE19dfcc_contract_modet24dfcc_loop_contract_modetRK8dstringtS4_RNSt3__13setIS2_NS5_4lessIS2_EENS5_9allocatorIS2_EEEEb + 148
10  goto-instrument                     0x0000000100798560 _ZN5dfcct21wrap_checked_functionEv + 180
11  goto-instrument                     0x0000000100796fa0 _ZN5dfcct20transform_goto_modelEv + 56
12  goto-instrument                     0x0000000100796e78 _ZN5dfcctC2ERK8optionstR11goto_modeltRK8dstringtRKN6nonstd13optional_lite8optionalINSt3__14pairIS5_S5_EEEEbRKNSB_3mapIS5_S5_NSB_4lessIS5_EENSB_9allocatorINSC_IS6_S5_EEEEEE24dfcc_loop_contract_modetR16message_handlertRKNSB_3setINSB_12basic_stringIcNSB_11char_traitsIcEENSK_IcEEEENSI_ISY_EENSK_ISY_EEEE + 404
13  goto-instrument                     0x0000000100796c50 _Z4dfccRK8optionstR11goto_modeltRK8dstringtRKN6nonstd13optional_lite8optionalINSt3__14pairIS4_S4_EEEEbRKNSA_3mapIS4_S4_NSA_4lessIS4_EENSA_9allocatorINSB_IS5_S4_EEEEEEbbRKNSA_3setINSA_12basic_stringIcNSA_11char_traitsIcEENSJ_IcEEEENSH_ISU_EENSJ_ISU_EEEER16message_handlert + 112
14  goto-instrument                     0x0000000100796564 _Z4dfccRK8optionstR11goto_modeltRK8dstringtRKN6nonstd13optional_lite8optionalIS4_EEbRKNSt3__13setIS4_NSD_4lessIS4_EENSD_9allocatorIS4_EEEEbbRKNSE_INSD_12basic_stringIcNSD_11char_traitsIcEENSH_IcEEEENSF_ISQ_EENSH_ISQ_EEEER16message_handlert + 304
15  goto-instrument                     0x00000001007ffbc4 _ZN30goto_instrument_parse_optionst23instrument_goto_programEv + 5092
16  goto-instrument                     0x00000001007fc0dc _ZN30goto_instrument_parse_optionst4doitEv + 552
17  goto-instrument                     0x0000000100bc43e8 _ZN19parse_options_baset4mainEv + 180
18  goto-instrument                     0x000000010076999c main + 48
19  dyld                                0x0000000194847f28 start + 2236


--- end invariant violation report ---
make: *** [ar_contracts.goto] Abort trap: 6

Any ideas where I went wrong?

@remi-delmas-3000 remi-delmas-3000 self-assigned this Jun 9, 2023
@remi-delmas-3000
Copy link
Collaborator

Thanks, I'll take a look at the problem to diagnose it.

@remi-delmas-3000
Copy link
Collaborator

Update: There's a superposition of issues:

  1. __CPROVER_pointer_in_range_dfcc can only be used in contract clauses, and not in the body of the function.

  2. __CPROVER_pointer_in_range should be used instead of __CPROVER_pointer_in_range_dfcc inside the function and the loop contract. In pre and post conditions __CPROVER_pointer_in_range_dfcc must be used instead of __CPROVER_pointer_in_range

  3. is being worked on at the moment, __CPROVER_pointer_in_range_dfcc was temporary workaround for function contracts and will ultimately be replaced by __CPROVER_pointer_in_range.

In any case we should not trigger an assertion failure by using __CPROVER_pointer_in_range_dfcc outside of contract clauses, will propose a fix nextweek.

@rod-chapman
Copy link
Collaborator Author

rod-chapman commented Jun 12, 2023

Thanks for the advice - I will update my code and try again. If the use of these contracts are specific to a particular context that can be automatically decided, then it seems just one contract will do, not two.

Another piece of advice: Huffman code your name-space! e.g. the most frequently used contracts should have the shortest names, so are less typing for the user...

@rod-chapman
Copy link
Collaborator Author

OK... making progress now. Chaning to use __CPROVER_pointer_in_range fixes the crash, and I realized I had to tighten the upper-bound on my current_byte_ptr variable. See the latest commit to ar.c in the repo. above.

I am now left with 2 FAILUREs:
[arsum.loop_assigns.2] line 54 Check assigns clause inclusion for loop arsum.0: FAILURE
[arsum.loop_invariant_step.1] line 54 Check invariant after step for loop arsum.0: FAILURE

I don't understand the first one for sure... What is being checked? What's wrong? The loop body modifies 3 variables, and my "assigns" clause mentions all three of them. What did I do wrong?

@qinheping qinheping self-assigned this Jun 12, 2023
@qinheping
Copy link
Collaborator

I don't understand the first one for sure... What is being checked? What's wrong? The loop body modifies 3 variables, and my "assigns" clause mentions all three of them. What did I do wrong?

It checks if the assigns clause of the loop is included in the assigns clause of the function. That is, the loop should not write to any target that is not allowed to be written in the function.

In this case, all three targets in the loop assigns are local targets and should not be checked in the inclusion check. It is a bug that we need to fix.

Here is the minimal implementation of arsum to reproduce the bug

uint32_t arsum(const uint8_t *data, size_t num_blocks)
{
    uint32_t sum = 0;


    for(; num_blocks--;)
    __CPROVER_assigns(num_blocks)
    __CPROVER_loop_invariant(1==1)
    {
    }
    return sum;
}

@remi-delmas-3000
Copy link
Collaborator

@rod-chapman

  • due to the loop guard side effect happening after the guard check it seems it is possible to enter the loop with num_blocks==1, have it decremented, and reach a state post loop where num_blocks == 0 which violates the invariant, similarly, it seems like post loop step current_byte_ptr can point one byte past last_block_ptr.

  • The failed assigns clause inclusion check is because right now we don't allow loops to assign function parameters, this will be fixed shortly. A temporary workaround is to declare a proxy variable for num_blocks in the body of the function and use that one instead of the parameter.

@remi-delmas-3000
Copy link
Collaborator

remi-delmas-3000 commented Jun 12, 2023

Fix for the write set inclusion check #7763

@rod-chapman
Copy link
Collaborator Author

OK... I switched to using a local variable for the loop counter rather than modifying the formal parameter. Good idea anyway - changing a scalar parameter is an abomination anyway... I now have 1 FAILURE left... on the loop invariant... but which one?!?!

What is the exact sequencing of the loop invariant preservation check with respect to decrementing the loop counter the the evaluation of the condition that the loop exit? I agree that current_byte_ptr does get incremented once too many times, but that value is never dereferenced, so is that OK or not?

@remi-delmas-3000
Copy link
Collaborator

remi-delmas-3000 commented Jun 12, 2023

Side effects of the guard are considered to be part of the loop (they are part of the natural loop in the control flow graph at the GOTO instruction level).

The base case is checked before any of the guard's side effect take place,

The step case and loop variant are checked after the very last instruction of the loop body.

In the current form the invariant is too weak. To become inductive it needs to relate current_byte_pointer, num_block, and the initial value of num_block :

uint32_t arsum(const uint8_t *data, size_t num_blocks)
__CPROVER_requires(num_blocks >= 1)
__CPROVER_requires(__CPROVER_is_fresh(data, num_blocks * BLOCK_SIZE))
{
    // Form uint32_t sum of bytes denoted by data.
    // Counting in blocks of 4 bytes at a time.
    uint8_t *current_byte_ptr = data;
    const uint8_t *last_block_ptr = data + ((num_blocks - 1) * BLOCK_SIZE);
    uint32_t sum = 0;

    // current_byte_ptr should always be between data and last_block_ptr, so we
    // should be able to assert...
    __CPROVER_assert(__CPROVER_pointer_in_range(data, current_byte_ptr, last_block_ptr),
                     "Check current_byte_ptr initially in block range");
    size_t num_blocks_old = num_blocks;
    for(; num_blocks; num_blocks--)
    __CPROVER_assigns(num_blocks, sum, current_byte_ptr)
    __CPROVER_loop_invariant(num_blocks <= __CPROVER_loop_entry(num_blocks))
    __CPROVER_loop_invariant(current_byte_ptr == (data + (__CPROVER_loop_entry(num_blocks) - num_blocks) * BLOCK_SIZE))
      {
        sum += *current_byte_ptr;
        current_byte_ptr++;
        sum += *current_byte_ptr;
        current_byte_ptr++;
        sum += *current_byte_ptr;
        current_byte_ptr++;
        sum += *current_byte_ptr;
        current_byte_ptr++;
    }
    return sum;
}

void arsum_harness()
{
    uint8_t *data;
    uint32_t num_blocks;
    uint32_t retval = arsum(data, num_blocks);
}

You might need extra assumptions about the data array to avoid overflows on sum:

[arsum.overflow.19] line 59 arithmetic overflow on unsigned + in sum + (unsigned int)*current_byte_ptr: FAILURE
[arsum.overflow.20] line 61 arithmetic overflow on unsigned + in sum + (unsigned int)*current_byte_ptr: FAILURE
[arsum.overflow.21] line 63 arithmetic overflow on unsigned + in sum + (unsigned int)*current_byte_ptr: FAILURE
[arsum.overflow.22] line 65 arithmetic overflow on unsigned + in sum + (unsigned int)*current_byte_ptr: FAILURE

@rod-chapman
Copy link
Collaborator Author

OK...I now have a version that proves OK. I have no idea what's going on with __CPROVER_loop_entry() there and the variable num_block_old .. it looks wrong to me!

One thing also confused me... I need to have
__CPROVER_loop_invariant(current_byte_ptr == (data + (num_blocks - blocks_to_go) * BLOCK_SIZE))

Why do I need the " * BLOCK_SIZE" there? I thought an addition of a pointer and an integer was automatically scaled in C? (Unless... num_blocks and blocks_to_go are both of type size_t, and automatic scaling is NOT applied when the RHS of the addition is size_t? Yuk!)

I will commit and push my latest code now...

@rod-chapman
Copy link
Collaborator Author

Note - no need to worrying about overflow of the summation, since I am not using --unsigned-overflow-check. (I find this option strange anyway - unsigned types are supposed to overflow!)

@remi-delmas-3000
Copy link
Collaborator

remi-delmas-3000 commented Jun 13, 2023

Sorry the size_t num_blocks_old = num_blocks; was a remnant of my own manual instrumentation.
__CPROVER_loop_entry is a history operator that holds the value of its argument at the first loop entry. So here it stores the initial value of num_blocks.

data is a uint8_t * so the stride with pointer arithmetic is sizeof(uint8_t) which is 1, hence the need for * BLOCK_SIZE to make it move by BLOCK_SIZE bytes at every iteration.

The unsigned overflow check is still worth using in general, as it catches cases where the result of an unsigned operation would wrap around unexpectedly, resulting for example in an allocation that is too small, an array index that is too large and would go out of bounds, etc.

We usually keep all checks active but selectively disable them around operations using a pragma (cf https://diffblue.github.io/cbmc/cprover-manual/md_properties.html):

#pragma CPROVER check push
#pragma CPROVER check disable "unsigned-overflow"
        sum += *current_byte_ptr;
        current_byte_ptr++;
        sum += *current_byte_ptr;
        current_byte_ptr++;
        sum += *current_byte_ptr;
        current_byte_ptr++;
        sum += *current_byte_ptr;
        current_byte_ptr++;
#pragma CPROVER pop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants