Auto merge of #1159 - snewt:feat/970-quickcheck-fuzzing, r=fitzgen

Property testing with quickcheck

This PR represents an attempt to address issue #970. It also represents a portion of the meta issue for fuzzing #972.

The code base reflected here uses quickcheck to generate C headers that
include a variety of types including basic types, structs, unions,
function prototypes and function pointers. The headers generated by quickcheck
are passed to the `csmith-fuzzing/predicate.py` script. Examples of headers
generated by this iteration of the tooling can be viewed
[here](https://gist.github.com/snewt/03ce934f35c5b085807d2d5cf11d1d5c).

At the top of each header are two simple struct definitions,
`whitelistable` and `blacklistable`. Those types are present in the vector that
represents otherwise primitive types used to generate. They represent a naive
approach to exposing custom types without having to intuit generated type names like
`struct_21_8` though _any actual whitelisting logic isn't implemented
here_.

Test success is measured by the success of the
`csmith-fuzzing/predicate.py`
script. This means that for a test to pass the following must be true:
- bindgen doesn't panic
- the resulting bindings compile
- the resulting bindings layout tests pass

#### Usage
```bash
cd tests/property_test
cargo test
```

Some things I'm unsure of:
#### Where should this feature live?
At the moment it lives in `tests/property_test` but isn't run when
`cargo test` is invoked from bindgen's cargo manifest directory.

#### What's an acceptable ammount of time for these tests to take?
At this point, the source is genereated in ~1 second but the files are
large enough that it takes the `predicate.py` script ~30 seconds to run
through each one. In order for the tests to run in under a minute only 2 are
generated by quickcheck by default. This can be changed in the `test_bindgen`
function of the `tests/property_test/tests/fuzzed-c-headers.rs` file.

#### How do we expose the generated code for easy inspection?
For now the `run_predicate_script` function in the
`tests/property_test/tests/fuzzed-c-headers.rs` file contains a
commented block that will copy generated source in the `tests/property_test/tests`
directory. Should it be easier?

#### Special casing
There is some logic in the fuzzer that disallows 0 sized arrays because
tests will regulary fail due to issues documented in #684 and #1153. Should
this be special casing?

#### Does the fuzzer warrant its own crate?
After any iterations the reviewers are interested in required to make
this a functional testing tool, should/could the fuzzing library be made into
its own crate? I didn't move in that direction yet because having it all in one
place seemed like the best way to figure out what works an doesn't but I'm
interested in whether it might be useful as a standalone library.

#### What does it look like to expose more useful functionality?
I'm looking forward to feedback on how to make this a more useful tool
and one that provides the right configurability.

Thanks!

r? @fitzgen

Author: bors-servo

.travis.yml

@@ -34,6 +34,7 @@ env:
     - LLVM_VERSION="4.0.0" BINDGEN_JOB="expectations" BINDGEN_PROFILE=
     - LLVM_VERSION="4.0.0" BINDGEN_JOB="expectations" BINDGEN_PROFILE="--release"
     - LLVM_VERSION="4.0.0" BINDGEN_JOB="misc"
+    - LLVM_VERSION="4.0.0" BINDGEN_JOB="quickchecking"
 
 matrix:
   fast_finish: true

ci/script.sh

@@ -38,6 +38,11 @@ case "$BINDGEN_JOB" in
         # ./ci/assert-rustfmt.sh
         ;;
 
+    "quickchecking")
+        cd ./tests/quickchecking
+        # TODO: Actually run quickchecks once `bindgen` is reliable enough.
+        cargo check
+        ;;
     *)
         echo "Error! Unknown \$BINDGEN_JOB: '$BINDGEN_JOB'"
         exit 1

tests/quickchecking/Cargo.toml

@@ -0,0 +1,10 @@
+[package]
+name = "quickchecking"
+description = "Bindgen property tests with quickcheck. Generate random valid C code and pass it to the csmith/predicate.py script"
+version = "0.1.0"
+authors = ["Shea Newton <[email protected]>"]
+
+[dependencies]
+quickcheck = "0.4"
+tempdir = "0.3"
+rand = "0.3"