Skip to content

Commit aa65eb9

Browse files
author
bors-servo
authored
Auto merge of #237 - fitzgen:using-creduce-with-bindgen, r=emilio
Describe how to use `creduce` with `bindgen` This commit extends CONTRIBUTING.md with information on how to use `creduce` to minimize input header test cases that cause `bindgen` to panic, generate bad bindings, or any other incorrectness. r? @emilio
2 parents b3322dd + 267d440 commit aa65eb9

File tree

1 file changed

+86
-0
lines changed

1 file changed

+86
-0
lines changed

CONTRIBUTING.md

+86
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ yourself.
1414
* [Authoring New Tests](#tests-new)
1515
* [Automatic Code Formatting](#formatting)
1616
* [Debug Logging](#logs)
17+
* [Using `creduce` to Minimize Test Cases](#creduce)
1718

1819
## Code of Conduct <span id="coc"/>
1920

@@ -138,3 +139,88 @@ This logging can also be used when debugging failing tests under
138139
```
139140
$ RUST_LOG=bindgen ./tests/tools/run-bindgen.py ./target/debug/bindgen tests/headers/whatever.h
140141
```
142+
143+
## Using `creduce` to Minimize Test Cases <span id="creduce"/>
144+
145+
If you are hacking on `bindgen` and find a test case that causes an unexpected
146+
panic, results in bad Rust bindings, or some other incorrectness in `bindgen`,
147+
then using `creduce` can help reduce the test case to a minimal one.
148+
149+
[Follow these instructions for building and/or installing `creduce`.](https://github.com/csmith-project/creduce/blob/master/INSTALL)
150+
151+
Running `creduce` requires two things:
152+
153+
1. Your isolated test case, and
154+
155+
2. A script to act as a predicate script describing whether the behavior you're
156+
trying to isolate occurred.
157+
158+
With those two things in hand, running `creduce` looks like this:
159+
160+
$ creduce ./predicate.sh ./isolated_test_case.h
161+
162+
### Isolating Your Test Case
163+
164+
Use the `-save-temps` flag to make Clang spit out its intermediate
165+
representations when compiling the test case into an object file.
166+
167+
$ clang[++ -x c++ --std=c++14] -save-temps -c my_test_case.h
168+
169+
There should now be a `my_test_case.ii` file, which is the results after the C
170+
pre-processor has processed all the `#include`s, `#define`s, and `#ifdef`s. This
171+
is generally what we're looking for.
172+
173+
### Writing a Predicate Script
174+
175+
Writing a `predicate.sh` script for a `bindgen` test case is fairly
176+
straightforward. One potential gotcha is that `creduce` can and will attempt to
177+
reduce test cases into invalid C/C++ code. That might be useful for C/C++
178+
compilers, but we generally only care about valid C/C++ input headers.
179+
180+
Here is a skeleton predicate script:
181+
182+
```bash
183+
#!/usr/bin/env bash
184+
185+
# Exit the script with a nonzero exit code if:
186+
# * any individual command finishes with a nonzero exit code, or
187+
# * we access any undefined variable.
188+
set -eu
189+
190+
# Print out Rust backtraces on panic. Useful for minimizing a particular panic.
191+
export RUST_BACKTRACE=1
192+
193+
# If the `libclang.so` you're using for `bindgen` isn't the system
194+
# `libclang.so`, let the linker find it.
195+
export LD_LIBRARY_PATH=~/path/to/your/directory/containing/libclang
196+
197+
# Make sure that the reduced test case is valid C/C++ by compiling it. If it
198+
# isn't valid C/C++, this command will exit with a nonzero exit code and cause
199+
# the whole script to do the same.
200+
clang[++ --std=c++14] -c ./pre_processed_header.hpp
201+
202+
# Run `bindgen` and `grep` for the thing your hunting down! Make sure to include
203+
# `2>&1` to get at stderr if you're hunting down a panic.
204+
~/src/rust-bindgen/target/debug/bindgen \
205+
./pre_processed_header.hpp \
206+
[ <extra flags> ] \
207+
2>&1 \
208+
| grep "<pattern in generated bindings or a panic string or ...>"
209+
```
210+
211+
When hunting down a panic, I `grep`ed like this:
212+
213+
... | grep "thread main panicked at '<panic error message here>'"
214+
215+
When hunting down bad codegen for a base member, I `grep`ed like this:
216+
217+
... | grep "pub _base: MyInvalidBaseTypeThatShouldntBeHere"
218+
219+
That's pretty much it! I want to impress upon you that `creduce` is *really*
220+
helpful and has enabled me to reduce 30k lines of test case into 5 lines. And it
221+
works pretty quickly too. Super valuable tool to have in your belt when hacking
222+
on `bindgen`!
223+
224+
Happy bug hunting and test case reducing!
225+
226+
[More information on using `creduce`.](https://embed.cs.utah.edu/creduce/using/)

0 commit comments

Comments
 (0)