Skip to content

Crangler #6367

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 17, 2021
Merged

Crangler #6367

merged 3 commits into from
Nov 17, 2021

Conversation

kroening
Copy link
Member

@kroening kroening commented Sep 28, 2021

This adds a new command-line tool, which replaces current flows that enact changes to the program that is analysed. The new flow is to make these changes before parsing, which enables more far-reaching changes and changes that depend on scoped identifiers.

  • Each commit message has a non-empty body, explaining why the change was made.
  • Methods or procedures I have added are documented, following the guidelines provided in CODING_STANDARD.md.
  • The feature or user visible behaviour I have added or modified has been documented in the User Guide in doc/cprover-manual/
  • Regression or unit tests are included, or existing tests cover the modified code (in this case I have detailed which ones those are in the commit message).
  • n/a My commit message includes data points confirming performance improvements (if claimed).
  • My PR is restricted to a single feature or bugfix.
  • n/a White-space or formatting changes outside the feature-related changed lines are in commits of their own.

@kroening kroening added the aws Bugs or features of importance to AWS CBMC users label Sep 28, 2021
@tautschnig
Copy link
Collaborator

Two high-level comments:

  1. Do we really need YAML? Couldn't we use JSON instead and have people use tools like yq to convert?
  2. Even if not a particularly elaborate regression test, could we at least have an example of an application?

@kroening kroening force-pushed the crangler branch 3 times, most recently from 1c86b88 to df0c3f4 Compare September 29, 2021 19:50
@codecov
Copy link

codecov bot commented Sep 29, 2021

Codecov Report

Merging #6367 (9412ce5) into develop (e5db980) will decrease coverage by 0.04%.
The diff coverage is 62.84%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #6367      +/-   ##
===========================================
- Coverage    76.00%   75.96%   -0.05%     
===========================================
  Files         1527     1541      +14     
  Lines       164454   165017     +563     
===========================================
+ Hits        125000   125349     +349     
- Misses       39454    39668     +214     
Impacted Files Coverage Δ
src/crangler/ctokenit.cpp 0.00% <0.00%> (ø)
src/crangler/ctokenit.h 0.00% <0.00%> (ø)
src/solvers/smt2_incremental/smt_commands.h 100.00% <ø> (ø)
src/crangler/c_defines.cpp 41.66% <41.66%> (ø)
src/crangler/crangler_parse_options.cpp 56.25% <56.25%> (ø)
src/crangler/c_wrangler.cpp 58.29% <58.29%> (ø)
src/crangler/scanner.l 63.63% <63.63%> (ø)
src/crangler/mini_c_parser.cpp 72.78% <72.78%> (ø)
src/goto-symex/symex_dead.cpp 95.65% <94.44%> (-4.35%) ⬇️
src/crangler/c_defines.h 100.00% <100.00%> (ø)
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce3fcaa...9412ce5. Read the comment docs.

@kroening kroening assigned tautschnig and unassigned kroening Oct 1, 2021
@kroening kroening marked this pull request as ready for review October 1, 2021 12:46
{
if(cmdline.args.empty())
{
std::cerr << "please give a configuration file\n";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use error logging via the message handler instead?

Comment on lines +35 to +37
auto id = line.substr(8, space_pos - 8);
auto value = line.substr(space_pos + 1, std::string::npos);
map[id].value = value;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to care about #undef and a possible re-definition?

Comment on lines +50 to +64
for(auto &t : tokens)
{
if(is_identifier(t))
{
auto m_it = map.find(t.text);
if(m_it != map.end())
{
out << m_it->second.value;
}
else
out << t.text;
}
else
out << t.text;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about macros that use other macros? What about macros that take parameters (see getc_unlocked(fp) example in the comment)?

public:
struct definet
{
optionalt<std::vector<std::string>> parameters;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never used (although they'd need to be taken into account).

Comment on lines +271 to +340
std::vector<std::string> argv = {"cc", "-E", source_file};

for(const auto &include : c_wrangler.includes)
{
argv.push_back("-I");
argv.push_back(include);
}

for(const auto &define : c_wrangler.defines)
argv.push_back(std::string("-D") + define);

std::ostringstream result;

auto run_result = run("cc", argv, "", result, "");
if(run_result != 0)
throw system_exceptiont("preprocessing " + source_file + " has failed");

return result.str();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have a Windows/MSVC version thereof?

Comment on lines +294 to +362
std::vector<std::string> argv = {"cc", "-E", "-dM", source_file};

for(const auto &include : config.includes)
{
argv.push_back("-I");
argv.push_back(include);
}

std::ostringstream result;

auto run_result = run("cc", argv, "", result, "");
if(run_result != 0)
throw system_exceptiont("preprocessing " + source_file + " has failed");

c_definest defines;
defines.parse(result.str());
return defines;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Windows/MSVC?

Copy link
Collaborator

@tautschnig tautschnig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving after some further out-of-band discussion. In addition to my earlier comments: some further tests and/or documentation would be really helpful.

@tautschnig tautschnig removed their assignment Oct 7, 2021
@kroening kroening force-pushed the crangler branch 2 times, most recently from 84c6a3b to daacc1f Compare October 21, 2021 09:28
@kroening kroening requested a review from a team as a code owner October 21, 2021 09:28
@kroening kroening force-pushed the crangler branch 3 times, most recently from 9a8ddb6 to 39462d6 Compare October 21, 2021 12:50
@tautschnig tautschnig force-pushed the crangler branch 6 times, most recently from c4b4049 to 6816e48 Compare November 2, 2021 20:52
kroening and others added 3 commits November 3, 2021 14:32
This is a command-line utility that makes changes to a preprocessed C file
that are prescribed in a JSON configuration file.

The initial set of transformations is as follows.
 * add a contract (pre/post/assigns) to a named C function
 * add a loop invariant to a loop identified by the name
   of the function its in and a loop number
 * remove the 'static' storage classifier from a function

The resulting source file is written to standard output or to file
specified in the JSON configuration.
To avoid having to list large numbers of functions for mangling, permit
regular expressions as function name entries in the JSON configuration
file.
Just like functions, objects can be file-local. To support accessing
them, permit removing "static" from objects as well.
@tautschnig
Copy link
Collaborator

We might still want to add a capability to perform name mangling, but once done we should be able to get rid of the export-file-local-symbols cross-cutting changes.

Copy link
Contributor

@TGWDB TGWDB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an approval for the Cmake/make aspects as these were blocked by OpenSource/Codeowner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
aws Bugs or features of importance to AWS CBMC users
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants