Indirect Offset Handling#

Overview#

Indirect offsets dereference a pointer stored in the file buffer: the evaluator reads a numeric value at a base position, optionally adjusts it, and uses the result as the final file offset. The canonical example is PE header detection — read the 4-byte pointer at 0x3c, then check that location for PE\0\0.

The implementation lives in src/evaluator/offset/indirect.rs and is invoked from src/evaluator/offset/mod.rs via resolve_indirect_offset_with_anchor.

AST Representation#

Indirect offsets are represented as OffsetSpec::Indirect with the following fields:

Subtraction is encoded as Add with a negative adjustment; there is no separate Sub variant 1.

Supported pointer types are Byte (.b/.B), Short (.s/.S), Long (.l/.L), and Quad (.q/.Q). Passing any non-numeric TypeKind returns EvaluationError::UnsupportedType 2.

Evaluation Pipeline#

resolve_indirect_offset_with_anchor executes four steps:

1. Resolve base — Convert base_offset to an absolute position. If base_relative, add the anchor first 3.
2. Read pointer — Call the appropriate typed reader (read_byte/read_short/read_long/read_quad) at the resolved base. Signed values are reinterpreted as raw u64 4.
3. Apply adjustment — Combine the pointer value with adjustment and adjustment_op using checked arithmetic. Div and Mod reject a zero operand 5.
4. Validate bounds — Return EvaluationError::BufferOverrun if the final offset is >= buffer.len() 6.

base_offset and Subroutines#

Inside a use subroutine, the evaluator applies a base_offset bias to OffsetSpec::Absolute values so that >0 string ABC inside a subroutine invoked at file offset 64 reads from byte 64, not byte 0.

Indirect offsets bypass this bias by design. The call site in src/evaluator/offset/mod.rs passes Some(last_match_end) as the anchor but does not add context.base_offset() 7. The assumption is that the pointer value read from the buffer is already an absolute file position. A pinned test (test_resolve_offset_with_base_does_not_bias_indirect) documents and enforces this 8.

This is confirmed in GOTCHAS.md §3.10:

Negative absolute offsets (FromEnd-style), Indirect pointer reads, and Relative(&N) offsets are unaffected — they already have well-defined reference frames.

Open Bug: Mach-O Universal Binary Misclassification (Issue #378)#

Despite the intentional design above, there is a confirmed bug when indirect offsets appear inside use subroutines called at a non-zero base offset: the indirect pointer-read base is not shifted by the subroutine's invocation offset (issue #378).

Concrete failure — Mach-O fat binary:

The mach-o magic subroutine contains:

>(8.L) indirect x \b:

When the subroutine is invoked at file offset 8 (arch[0] entry), (8.L) should read a long from base(8) + 8 = 16 — the arch[0].offset field pointing to the Mach-O header. Instead, rmagic reads from absolute offset 8, resolves 0x01000007 (CPU_TYPE_X86_64), and generates a buffer overrun at offset 16777223 9.

Symptoms:

• Direct reads within the subroutine correctly apply the base offset (e.g., >0 belong resolves to file offset 8).
• Indirect pointer-read offsets do not — they read from the raw base_offset value without adding the subroutine's invocation point.
• Consequence: the inner per-arch classification (Mach-O 64-bit executable x86_64) is never emitted; rmagic produces [ x86_64] instead of [x86_64:Mach-O 64-bit executable x86_64] 9.

Fix needed: Thread the subroutine's base_offset into the indirect pointer-read step in evaluator/offset/indirect.rs. This is the primary blocker tracked in issue #378 alongside three additional Mach-O gaps 9.

Additional Parse Limitations (Issue #378)#

Issue #378 also identifies two parser gaps that cause silent rule drops for adjacent Mach-O rules 9:

• Bare (N) indirect syntax (no .type specifier, e.g., >>(56) indirect x): the parser requires an explicit type character after . and fails, dropping the rule silently.
• indirect/r flag (e.g., >>12 indirect/r x): not recognized by the grammar; dropped silently.

Neither is yet implemented. Both are lower-priority than the base_offset threading fix.

Key Files#