Introduce operand offset (C++ and Java) #6

arvi18 · 2025-04-21T05:51:40Z

This PR introduces a new pre-defined symbol named operand_offset to expose the relative byte offset of the current operand from inst_start. This is required to correctly compute PC-relative operand addresses for DEC VAX.

See NationalSecurityAgency#4606 about my attempts of solving this (and failing) with existing methods.

While the diff looks relatively large, there's no new computation going on. It merely exposes the offset which parserWalker already has. The rest of the code is just "symbol ceremony" copied from inst_start.

There are two commits, the first one covers the "C++" side, the second covers the "Java" side.

Fixes NationalSecurityAgency#4606

Signed-off-by: Klaus Kämpf <[email protected]>

arvi18 · 2025-04-21T05:51:41Z

Did you forget to
git add Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/context/ParserWalker.java ?

arvi18 · 2025-04-21T05:51:42Z

Did you forget to git add Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/context/ParserWalker.java ?

🤔 - I don't have this file (and I don't need it either).

The two commits are the two patches I use to build Ghidra 10.1.5. (Update to 10.2.2 is in progress 😉).

arvi18 · 2025-04-21T05:51:43Z

Thanks for the hint. 39baf3a removed ParserWalker.
Current squashed patch to build against "master":
https://github.com/jobermayr/ghidra-staging/blob/master/4812-Introduce-operand-offset-C-and-Java.patch

arvi18 · 2025-04-21T05:51:44Z

Rebased to head

arvi18 · 2025-04-21T05:51:44Z

You did not take my comment from 12/12/2022 into account:

> Task :Decompiler:compileSleighLinux_x86_64ExecutableSleighCpp
/tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/slghscan.cc: In function ???ghidra::int4 ghidra::find_symbol()???:
/tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/slghscan.cc:1578:5: error: ???yylval??? was not declared in this scope
 1578 |     yylval.offsetsym = (OffsetSymbol *)sym;
      |     ^~~~~~
/tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/slghscan.cc: At global scope:
/tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/slghscan.cc:2965:17: warning: ???void yyunput(int, char*)??? defined but not used [-Wunused-function]
 2965 |     static void yyunput (int c, char * yy_bp )
      |                 ^~~~~~~

/tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.cc:2929:7: error: no declaration matches ???ghidra::uintb ghidra::PcodeSnippet::allocateTemp()???
 2929 | uintb PcodeSnippet::allocateTemp(void)
      |       ^~~~~~~~~~~~
In file included from /tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.cc:88:
/tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.hh:80:17: note: candidate is: ???virtual ghidra::uint4 ghidra::PcodeSnippet::allocateTemp()???
   80 |   virtual uint4 allocateTemp(void);
      |                 ^~~~~~~~~~~~
/tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.hh:72:7: note: ???class ghidra::PcodeSnippet??? defined here
   72 | class PcodeSnippet : public PcodeCompile {
      |       ^~~~~~~~~~~~


> Task :SoftwareModeling:compileJava FAILED
/tmp/ghidra/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghsymbol/OffsetSymbol.java:20: error: cannot find symbol
import ghidra.pcodeCPort.context.ParserWalker;
                                ^
  symbol:   class ParserWalker
  location: package ghidra.pcodeCPort.context
/tmp/ghidra/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghsymbol/OffsetSymbol.java:76: error: cannot find symbol
        public void getFixedHandle(FixedHandle hand, ParserWalker pos) {
                                                     ^
  symbol:   class ParserWalker
  location: class OffsetSymbol
/tmp/ghidra/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghsymbol/OffsetSymbol.java:84: error: cannot find symbol
        public void print(PrintStream s, ParserWalker pos) {
                                         ^
  symbol:   class ParserWalker
  location: class OffsetSymbol
/tmp/ghidra/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghpatexpress/OffsetInstructionValue.java:20: error: cannot find symbol
import ghidra.pcodeCPort.context.ParserWalker;
                                ^
  symbol:   class ParserWalker
  location: package ghidra.pcodeCPort.context
/tmp/ghidra/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghpatexpress/OffsetInstructionValue.java:35: error: cannot find symbol
        public long getValue(ParserWalker pos) {
                             ^
  symbol:   class ParserWalker
  location: class OffsetInstructionValue
/tmp/ghidra/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghsymbol/OffsetSymbol.java:75: error: method does not override or implement a method from a supertype
        @Override
        ^
/tmp/ghidra/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghsymbol/OffsetSymbol.java:83: error: method does not override or implement a method from a supertype
        @Override
        ^
/tmp/ghidra/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghpatexpress/OffsetInstructionValue.java:34: error: method does not override or implement a method from a supertype
        @Override
        ^
8 errors

> Task :Decompiler:compileSleighLinux_x86_64ExecutableSleighCpp FAILED

arvi18 · 2025-04-21T05:51:45Z

Sorry, @jobermayr, indeed ParserWalker moved and I didn't properly test after rebasing.

Fixed now.

arvi18 · 2025-04-21T05:51:46Z

> Task :Decompiler:compileSleighLinux_x86_64ExecutableSleighCpp
/tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.cc:2929:7: error: no declaration matches ???ghidra::uintb ghidra::PcodeSnippet::allocateTemp()???
 2929 | uintb PcodeSnippet::allocateTemp(void)
      |       ^~~~~~~~~~~~
In file included from /tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.cc:88:
/tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.hh:80:17: note: candidate is: ???virtual ghidra::uint4 ghidra::PcodeSnippet::allocateTemp()???
   80 |   virtual uint4 allocateTemp(void);
      |                 ^~~~~~~~~~~~
/tmp/ghidra/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.hh:72:7: note: ???class ghidra::PcodeSnippet??? defined here
   72 | class PcodeSnippet : public PcodeCompile {
      |       ^~~~~~~~~~~~


> Task :Decompiler:compileSleighLinux_x86_64ExecutableSleighCpp FAILED

Fix (edit: previously it was to generated pcodeparse.cc):

diff --git a/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.y b/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.y
index 1eee53d8f..b7d8288c8 100644
--- a/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.y
+++ b/Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.y
@@ -650,7 +650,7 @@ void PcodeLexer::initialize(istream *t)
   }
 }
 
-uintb PcodeSnippet::allocateTemp(void)
+uint4 PcodeSnippet::allocateTemp(void)
 
 { // Allocate a variable in the unique space and return the offset
   uint4 res = tempbase;

coderabbitai · 2025-04-21T05:51:47Z

Warning

Rate limit exceeded

@visz11 has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 27 minutes and 31 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 2abfd4e and 607ae4a.

📒 Files selected for processing (25)

Ghidra/Features/Decompiler/src/decompile/cpp/pcodeparse.y (4 hunks)
Ghidra/Features/Decompiler/src/decompile/cpp/semantics.cc (3 hunks)
Ghidra/Features/Decompiler/src/decompile/cpp/semantics.hh (1 hunks)
Ghidra/Features/Decompiler/src/decompile/cpp/slgh_compile.cc (1 hunks)
Ghidra/Features/Decompiler/src/decompile/cpp/slghparse.y (4 hunks)
Ghidra/Features/Decompiler/src/decompile/cpp/slghpatexpress.cc (1 hunks)
Ghidra/Features/Decompiler/src/decompile/cpp/slghpatexpress.hh (1 hunks)
Ghidra/Features/Decompiler/src/decompile/cpp/slghscan.l (1 hunks)
Ghidra/Features/Decompiler/src/decompile/cpp/slghsymbol.cc (2 hunks)
Ghidra/Features/Decompiler/src/decompile/cpp/slghsymbol.hh (2 hunks)
Ghidra/Framework/SoftwareModeling/src/main/antlr/ghidra/sleigh/grammar/SleighCompiler.g (6 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/app/plugin/assembler/sleigh/SleighAssemblerBuilder.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/app/plugin/processors/sleigh/expression/OffsetInstructionValue.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/app/plugin/processors/sleigh/expression/PatternExpression.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/app/plugin/processors/sleigh/symbol/OffsetSymbol.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/app/plugin/processors/sleigh/symbol/SymbolTable.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/semantics/ConstTpl.java (3 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slgh_compile/SleighCompile.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slgh_compile/Yylval.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghpatexpress/OffsetInstructionValue.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghpatexpress/PatternExpression.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghsymbol/OffsetSymbol.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghsymbol/SymbolTable.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcodeCPort/slghsymbol/symbol_type.java (1 hunks)
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/lang/PcodeParser.java (1 hunks)

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR.
@coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

arvi18 · 2025-04-21T05:51:47Z

Thanks @jobermayr - I had fixed this in the generated file only 🤦🏻

Can you point me to your build system ? It seems as if you've extended "make realclean" within Ghidra/Features/Decompiler/src/decompile/cpp 🤔

arvi18 · 2025-04-21T05:51:48Z

Can you point me to your build system ? It seems as if you've extended "make realclean" within Ghidra/Features/Decompiler/src/decompile/cpp

Not really. I only created a makeBison.sh to quickly regenerate files after modifications to .l or .y files.
I never do a make realclean or so. I only "clone" and apply patches from ghidra-staging after restarting my PC and RAM (64 GB :) :) ) gets cleared then.

On ghidra-staging I try to keep all pull requests (and their updates) building.
Sometimes I also find runtime errors, try to fix them and report back :)

arvi18 · 2025-04-21T05:51:49Z

To apply and build after 8fbd171 / ae6f7b4: https://github.com/jobermayr/ghidra-staging/blob/master/4812-Introduce-operand-offset-C-and-Java.patch
@kkaempf Can you please check it?

arvi18 · 2025-04-21T05:51:49Z

To apply and build after 8fbd171 / ae6f7b4:

I'd assume these commits are included in Ghidra_11.0.2_build ?! 🤔

https://github.com/jobermayr/ghidra-staging/blob/master/4812-Introduce-operand-offset-C-and-Java.patch @kkaempf Can you please check it?

It looks wrong to me. For example:

Your patch:

 .../Decompiler/src/decompile/cpp/pcodeparse.y |   1 +

mine (also here):

 .../Decompiler/src/decompile/cpp/pcodeparse.y |  6 ++

I carry my commits along at my own fork, currently at (resp. on top of) Ghidra_11.0.2_build.

See https://build.opensuse.org/package/show/security:forensics/ghidra for the patches.

arvi18 · 2025-04-21T05:51:50Z

To apply and build after 8fbd171 / ae6f7b4:

I'd assume these commits are included in Ghidra_11.0.2_build ?! 🤔

No. master which leads currently to 11.1

https://github.com/jobermayr/ghidra-staging/blob/master/4812-Introduce-operand-offset-C-and-Java.patch @kkaempf Can you please check it?

It looks wrong to me. For example:

Your patch:
 .../Decompiler/src/decompile/cpp/pcodeparse.y |   1 +
mine (also here):
 .../Decompiler/src/decompile/cpp/pcodeparse.y |  6 ++

I try to collapse it like ae6f7b4:

Doesn't it make sense?

arvi18 · 2025-04-21T05:51:51Z

I will need to take a closer look. Not sure I can manage it this week. Asking for your patience 😉

arvi18 · 2025-04-21T05:51:52Z

I usually rebase my branch only on major releases.
I'll need to do a test-rebase to main and check the results.

kkaempf · 2025-04-22T07:57:23Z

Please see https://github.com/kkaempf/ghidra-vintage for an up-to-date version of this PR.
(And, yes, I still need to close the original PR and open a new one based on Ghidra 11.2 - a lot has changed internally)

visz11 · 2025-04-23T07:47:14Z

@coderabbitai full review

coderabbitai · 2025-04-23T07:47:19Z

✅ Actions performed

Full review triggered.

kkaempf added 2 commits May 14, 2023 11:26

Introduce operand_offset

17c875e

Signed-off-by: Klaus Kämpf <[email protected]>

Add operand_offset to sleigh Java

607ae4a

Signed-off-by: Klaus Kämpf <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce operand offset (C++ and Java) #6

Introduce operand offset (C++ and Java) #6

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

coderabbitai bot commented Apr 21, 2025 •

edited

Loading

Rate limit exceeded

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Documentation and Community

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

kkaempf commented Apr 22, 2025

visz11 commented Apr 23, 2025

coderabbitai bot commented Apr 23, 2025

Introduce operand offset (C++ and Java) #6

Are you sure you want to change the base?

Introduce operand offset (C++ and Java) #6

Conversation

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

coderabbitai bot commented Apr 21, 2025 • edited Loading

Rate limit exceeded

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Documentation and Community

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

arvi18 commented Apr 21, 2025

kkaempf commented Apr 22, 2025

visz11 commented Apr 23, 2025

coderabbitai bot commented Apr 23, 2025

coderabbitai bot commented Apr 21, 2025 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)