Extraction Pipeline - thethiny/NRS-Asset-Manager GitHub Wiki

Extraction Pipeline

All NRS games follow a three-stage extraction pipeline.

Stage 1: Compressed Archive (.xxx)

The .xxx file is a compressed UE3 package. Its structure:

┌─────────────────────────────┐
│ Asset Header                │  Game-specific (100-104 bytes)
├─────────────────────────────┤
│ Compressed Chunk Table      │  Array of offset/size pairs
├─────────────────────────────┤
│ Padding (0x18 bytes)        │
├─────────────────────────────┤
│ Filename (length-prefixed)  │
├─────────────────────────────┤
│ [File Tables - MK11 only]   │  PSF/Bulk table references
├─────────────────────────────┤
│ Compressed Blocks (Oodle)   │  Each block: header + chunks
└─────────────────────────────┘

Key classes: MK11UE3Asset, IJ2UE3Asset

Each compressed block uses the Oodle codec:

Block Header (32 bytes):
  magic = 0x9E2A83C1
  chunk_size = 0x20000 (128KB)
  compressed_size, decompressed_size

Chunk Headers (16 bytes each):
  compressed_size, decompressed_size
  [compressed_size bytes of Oodle data]

Stage 2: Midway Format (Decompressed)

The archive is decompressed and reassembled into the internal "Midway" format (.upk):

┌─────────────────────────────┐
│ Header (compression = 0)    │
├─────────────────────────────┤
│ Filename                    │
├─────────────────────────────┤
│ [File Tables - MK11 only]   │
├─────────────────────────────┤
│ Name Table                  │  String pool for all names
├─────────────────────────────┤
│ Import Table                │  External object references
├─────────────────────────────┤
│ Export Table                │  Internal object definitions
├─────────────────────────────┤
│ Export Data                 │  Raw object data blobs
└─────────────────────────────┘

Key classes: MidwayAsset (MK11), IJ2MidwayAsset (IJ2)

Key methods:

  • parse_summary() — Read header + validate magic/four_cc
  • parse_name_table() — Build string lookup table
  • parse_uobject_table() — Parse import/export structs
  • resolve_table_info() — Resolve cross-references between tables

Stage 3: Export Processing

Each export is a game object (Texture2D, Database, Package, etc.). Type-specific handlers parse the raw data:

for export in midway.export_table:
    handler = handlers.get(export.class_.name)
    if handler:
        handler_class = handler["handler_class"]
        midway.parse_and_save_export(export, handler_class, output_dir)

Current handlers:

Handler Input Output Games
DatabaseHandler UE3 properties .json MK11, IJ2
Texture2DHandler Mip data + TFC/PSF .dds, .png, .json MK11, IJ2
LocalizationParser Coalesced.ENG/.ini Text files MK11, IJ2

See Database Handler and Texture2D Handler for details.

External Bulk Data

Large data (textures) is stored in separate companion files:

Game Extension Lookup Mechanism
MK11 .psf Centralized PSF tables in header, keyed by CookedBulkDataOwnerKey
IJ2 .tfc Per-texture TextureFileCacheName property + inline mip offsets

See Format Comparison for the architectural differences.