DBAL Decompilation and dbal.py - krcenov/RNS510-Map-Viewer GitHub Wiki

DBAL.OUT Full Decompilation and research/dbal.py

A still later session installed a real decompiler (Ghidra, NSA's free/ open-source tool β€” OpenJDK 21 + Ghidra 12.1.4, both installed locally) and ran it headless against dbal/V010_015/DBAL.OUT, producing a complete, single-file decompilation: 1,915 functions, 0 failures, ~2.6MB of C-like pseudocode. This is a different, smaller-scale effort than the existing Hex-Rays/capstone work on the main firmware image (FHDD6.FLI, see Firmware Reversing) β€” DBAL.OUT is a much smaller, unlinked object file with no disassembly wall, so a full automated decompile was straightforward once a decompiler existed to run.

Not committed to the repo (the Ghidra install + decompiled output are 570MB+; tools/ is gitignored). Reproducible: tools/ghidra_scripts/ DecompileAll.java (a GhidraScript using DecompInterface) run via analyzeHeadless with -processor PowerPC:BE:32:default against any dbal/V0NN_0MM/DBAL.OUT.

Spot-checked against this project's own earlier manual disassembly β€” exact match

Before trusting the decompiler's output, db_get_minor_road_list_offset's decompiled C was compared against this project's own earlier, independent, by-hand capstone disassembly of the same function (see eeu.iof's own page) β€” identical logic, confirming the decompiler is trustworthy for this binary.

CORRECTION (a later session): the generic field-reader is NOT external β€” it's a decompiler artifact, resolvable via ELF relocations

The "DEFINITIVELY confirmed external... proven, permanent limit" claim below (this section's original text) was wrong. It was reached by reading Ghidra's pseudocode at face value: both eeu.iof's unpack_intersection_offset_rec and eeu.rd's read_current_record_from_database call every field through an indirect pointer β€” (**(code **)(*(int *)(obj + 0xc) + 0x164))(...) β€” which Ghidra can't statically resolve to a name (that's just what a C++ virtual/indirect call always looks like once decompiled), and that "can't resolve" was misread as "code doesn't exist." It does.

Proven directly, this session: DBAL.OUT's .symtab has 621 DBF*-prefixed symbols β€” a full, real, compiled generic database framework (DBFView, DBFViewmodel, DBFDataItem, DBFInteger, DBFIntegerSpecial, DBFBitComponent, DBFOffset, DBFString, DBFCounter, DBFList, DBFIndex, and more) β€” almost all of it shndx=1 (i.e. real code sitting in .text, not SHN_UNDEF). The indirect calls above are virtual dispatch through one of these classes' own vtables (_vt$<ClassName> symbols, also real and present), and .rela.text's relocation entries populate each vtable slot with the REAL target function β€” reading them resolves any given call exactly:

unpack_intersection_offset_rec's 3 field reads, resolved via
_vt$17DBFIntegerSpecial's own relocations (vtable value=642168):
  offset+0x154 -> getValueUnsignedLong__C17DBFIntegerSpecial (reloc @ +340)
  offset+0x15c -> (a different class's own vtable -- NOT the same
                   offset-to-method mapping across classes, see caveat)
  offset+0x164 -> getValueLong__C17DBFIntegerSpecial          (reloc @ +356)

The actual generic integer codec is real, fully decompiled code: setValPtr__17DBFIntegerSpecialPcT1b (@0x86a3c, 1008 bytes) is a 9-case switch implementing every integer encoding this disc uses β€” fixed 1/2/3/4-byte (with 24-bit sign-extension for the 3-byte case), and 2 distinct variable-length (base-128 "varint") schemes, one with an extra sign bit packed into the first byte, in both byte orders. The decoded value is cached at a fixed object offset; the various getValueLong/getValueUnsignedLong/getValueShort/.../getValueChar accessors just read back different widths/offsets of that same cached value (confirmed by reading their own 1-line bodies β€” NOT all of them return the full value: getValueUnsignedShort reads only its low 16 bits, for instance, so which accessor a given field actually calls through DOES matter). setValPtr__15DBFBitComponentPcT1b/ getValueLong__C15DBFBitComponent confirm DBFBitComponent is a thin wrapper around one of these integer fields, doing exactly (underlying_value >> bit_offset) & ((1 << bit_width) - 1) β€” independently matching eeu.mod's own bit_width/bit_offset descriptor fields, and exactly the model this project already used (empirically) for kd_info's bits.

Honest caveat β€” this doesn't mean every field is now trivially decodable: a given call site's indirect pointer resolves through WHICHEVER concrete DBF* class getDataItem() returned for that field name at runtime (schema-driven, not fixed at compile time) β€” so "vtable offset 0x164" means a different method on DBFInteger than on DBFIntegerSpecial than on DBFBitComponent, and each call site needs its own class identified before its relocations can be read. What's now established beyond doubt: the code for ALL of it is real and present in DBAL.OUT, nothing here is an external/missing-code wall, and the method (vtable symbol β†’ .rela.text/.rela.data relocations in its byte range β†’ real function name) is proven and repeatable.

Follow-up (same session): the field→class mapping question turned out to be moot

Chased the "not yet done" item above β€” concretely resolved 2 real call sites end to end, picking the right _vt$<ClassName> by matching each call's own vtable-slot values against known candidates, not by assumption:

  • eeu.iof's offset_data (unpack_intersection_offset_rec): offset_o_527 resolves (vtable offset 0x154) to getValueUnsignedLong__C17DBFIntegerSpecial β€” a plain cached 32-bit value. type_o_528/count_o_529 BOTH use vtable offset 0x164, but that offset means something different on each class's own vtable (DBFIntegerSpecial's own layout has getValueLong there; DBFBitComponent's has getValueShort, confirmed by dumping its _vt$15DBFBitComponent relocations directly) β€” these 2 are DBFBitComponent, matching eeu.mod's own countAndType schema EXACTLY: a single 2-byte field split into count (15 bits, offset 0)
    • type (1 bit, offset 15) β€” i.e. DBFBitComponent::getValueShort computing (countAndType >> bit_offset) & ((1<<bit_width)-1) for each, cross-confirming this project's own earlier, independently- reached type/count semantics (README/eeu.iof's own page) from a 3rd, completely different angle.
  • eeu.rd's roadInfo: eeu.mod names exactly 6 sub-fields β€” highway(bit 0), preferred(1), explicate(2), ramp(3), name type(bits 4-6, 3 bits wide), interchange(7) β€” an EXACT, bit-for- bit match to this project's own already-shipped, independently- derived decode_roadinfo_byte() layout (research/dbal.py). Same DBFBitComponent mechanism.

The practical upshot: the "map every field to its DBF class" task isn't actually necessary for decoding values. Scanned every one of eeu.mod's ~1,600 field descriptors across all 36 known tables for anything that DOESN'T fit the 2 patterns research/mod_reader.py already extracts (byte_width N>0 = plain fixed-width int; byte_width 0 + sane bit_width/bit_offset = a DBFBitComponent-style bit field) — zero anomalies found. Every real field on this disc is one of those 2 shapes; none exercise the generic codec's variable-length ("varint") cases found in setValPtr__17DBFIntegerSpecialPcT1b above — real, present, and now fully understood code, just apparently unused by this particular disc's own schema. Concretely: eeu.mod's existing, already-used byte_width/bit_width/bit_offset descriptors were ALREADY a complete, correct recipe for decoding any field on this disc — this session's DBAL.OUT work explains, at the real code level, WHY that's true (down to the exact shift/mask/sign-extension performed), rather than unlocking previously-inaccessible data. The one open question from the original "not yet done" (an explicit field→DBF*- class table) is answered: not needed, and dropped as a goal.

research/dbal.py β€” a faithful Python port of what IS resolvable

Given the above, dbal.py doesn't (and can't) port "the extraction" β€” instead it ports the real logic DBAL.OUT applies on TOP of already- extracted fields, confirmed fully self-contained (no external calls):

  • FeaCoordReceiverInternal β€” eeuz.fea's real point-chain decoder (get_coordinates()) and its real Douglas- Peucker-style line-simplification pass (thinout()/ _thinout_recursive()), ported instruction-by-instruction. The core delta-accumulation loop was validated against a synthetic chain ((5,3),(2,-1),(4,0) β†’ correctly cumulative-sums to (5,3),(7,2), (11,2), with the real (0,0)-after-first-point terminator rule also confirmed). The thinning path's real per-level distance-table constants were NOT successfully extracted from the binary (a quick attempt landed on an address that reads as all-zero, almost certainly a wrong file-offset mapping, not a real value) β€” documented as unresolved in the code rather than silently guessed.
  • apply_rd_record_transform() β€” the real rdtype & 0x7fff masking and maxLong = minLong + deltaLong / maxLat = minLat + deltaLat conversion read_current_record_from_database performs.
  • gate_minor_road_list_offset()/gate_minor_road_count() β€” eeu.iof's real validity-gating decision logic (type must be exactly 0 for offset to be valid; type==1 means the on-disk count is provisional), already documented on eeu.iof's own page from the earlier capstone-only session, now re-confirmed against the decompiled C.
  • decode_roadinfo_byte() β€” NOT a DBAL.OUT port (that extraction is external, see above) β€” this project's own already-validated bit layout, centralized here as the one place every caller reads it from.

Wired into the live tool

rns510_core.RoadRecord now exposes max_lon/max_lat, computed via dbal.apply_rd_record_transform() (previously only delta_lon/ delta_lat were exposed). road_naming.RdCache's highway/ highway_names() now goes through dbal.decode_roadinfo_byte() instead of inline bit math. Verified end-to-end against the real ISO: identical results to the pre-refactor values (578,946/8,809,081 highway records, 42,505 distinct names) β€” confirming the refactor changed provenance, not behavior.

Reader code

research/dbal.py β€” see its own module docstring for the full honesty accounting of what's a faithful port vs. this project's own independent crack. tools/ghidra_scripts/DecompileAll.java (not committed) for reproducing the decompilation itself.

⚠️ **GitHub.com Fallback** ⚠️