CHUNK_GEOMETRY - themeldingwars/Documentation GitHub Wiki

The CHUNK_GEOMETRY (0x040210) and the slightly different CHUNK_GEOMETRY2 (0x40203) nodes contain the visual data for static objects in the chunk.

Parsing for these nodes is available here: https://github.com/themeldingwars/Documentation/blob/master/010%20Templates/_common_gtlayer.bt#L708

The layer format can be described as

ushort countEntries
[10 bytes] unknown (always the same sequence)
<entries>

The number of entries depends on the level of detail that we are on and allows the CHUNK_GEOMETRY to be sectioned into pieces separately from the SubChunks.

The format of each of these entries described as

uint index
uint count1 (geometry data)
<count1 entries>
uint count2 (???)
<count2 entries>

Whilst we have parsing for count2 entries we have not really explored how that works, so it will not be described here.

The count 1 entries are the main path of interest, with each representing a group of viusal data.

The format of the count 1 entries can be descirbed as:

Vec3 min
Vec3 max
uint64 unknown (only for CHUNK_GEOMETRY)
uint count inner
<inner entries>

Here, we have bounds defined by two Vec3, creating a bounding box for the geometry inside of the inner entries. There's an extra 8 bytes of data that is only present in the CHUNK_GEOMETRY version of the layer.

The format of the inner entries can be described as:

uint materialId
uint textureId1
uint textureId2
uint textureId3
Vec3 center (only for CHUNK_GEOMETRY2)
float extra (only for CHUNK_GEOMETRY2)
Vec3 min
Vec3 max
[4 bytes] (some form of node/leaf count for the remaining data)
uint unk2
uint numTransforms
<Matrix4x4 transform entries>
<Lz77 compressed uniqueIndex>
<Lz77 compressed transformIndex>
uint numVerts
<Lz77 compressed repeatIndex>
<Lz77 compressed vertIndex>
<Lz77 compressed repTable>
uint unk10

For each of the Lz77 compressed blocks, there is a num value indicating how many entries you'll find in the data, and the compressed size. For the repeatIndex only, there is an additional ushort to parse before the compressed data, which is an offset value needed after decompression.

uint num
uint size
byte[size] compressedData

The game uses the "FastLZ" algorithm for the lz77 compression. You need to use a matching implementation to get the correct output. After decompressing, some operations need to be performed to get the data to the correct state. Below is some code to showcase how to prepare the data:

function parseGTLz77_UniqueIndex (lz77_data_1, lz77_num_1) {
  const parsed = new Array(lz77_num_1)
  const decompressed = Lz77Util.Decompress(lz77_data_1)
  for (let n = 0; n < lz77_num_1; n++) {
    parsed[n] = decompressed.readInt32LE(n * 4)
  }
  if (lz77_num_1 > 1) {
    for (let n = 1; n < lz77_num_1; n++) {
      parsed[n] = parsed[n] + parsed[n - 1]
    }
  }
  return parsed.map((value) => {
    return {
      vertIdx: value & 0x7ff,
      pageIdx: value >> 0xb
    }
  })
}

function parseGTLz77_MatrixIndex (lz77_data_2, lz77_num_2) {
  const parsed = new Array(lz77_num_2)
  const decompressed = Lz77Util.Decompress(lz77_data_2)
  for (let n = 0; n < lz77_num_2; n++) {
    parsed[n] = decompressed.readInt32LE(n * 4)
  }
  return parsed
}

function parseGTLz77_RepeatIndex (lz77_data_3, lz77_num_3, lz77_data_3_offset) {
  const parsed = new Array(lz77_num_3)
  if (lz77_num_3 > 0) {
    const decompressed = Lz77Util.Decompress(lz77_data_3)
    for (let n = 0; n < lz77_num_3; n++) {
      parsed[n] = (decompressed.readInt16LE(n * 2) + lz77_data_3_offset)
    }
    if (lz77_num_3 > 1) {
      for (let n = 1; n < lz77_num_3; n++) {
        parsed[n] = parsed[n] + parsed[n - 1]
      }
    }
  }
  return parsed
}

function parseGTLz77_SourceIndex (lz77_data_4, lz77_num_4) {
  const parsed = new Array(lz77_num_4)
  const decompressed = Lz77Util.Decompress(lz77_data_4)
  for (let n = 0; n < lz77_num_4; n++) {
    parsed[n] = decompressed.readInt8(n)
  }
  return parsed
}

function parseGTLz77_TriStripIndex (lz77_data_5, lz77_num_5_repTblNum) {
  const parsed = new Array(lz77_num_5_repTblNum)
  const decompressed = Lz77Util.Decompress(lz77_data_5)
  for (let n = 0; n < lz77_num_5_repTblNum; n++) {
    parsed[n] = decompressed.readInt8(n)
  }
  return parsed
}

The data can be summarized like this:

1. The repTable gives instructions on how to form triangles in a tristrip format. Each entry is 1 byte that describes how to form the triangle in the strip sequence. This means you need verts to form the triangle.
2. vertIndex contains as many entries as there are verts necessary to complete the repTable. Each entry is 1 byte that describes whether the vert comes from the uniqueIndex or through the repeatIndex.
3. uniqueIndex and transformIndex will have the same number of entries and are used jointly. The uniqueIndex entry is a uint value that references a vert in the VG data. At the same index in the transformIndex, you find a byte value that indexes into the transforms array, selecting the transform matrix to apply to this vert data.
4. repeatIndex references verts in the unique/transformIndex.

A simple approach to get started would be to first process and prepare all vertices in a single array, and then process each instruction from the repTable to produce the appropriate indices.

To do this, the process would be as follows:

1. Iterate the uniqueIndex with an index pointer i, and fetch the referenced vert data from the VG file using the page and vert index. Use the same i in transformIndex to find the transform to apply to the vert.
2. Iterate the vertIndex with an index pointer i whilst also keeping two index pointers into uniqueIndex and repeatIndex. When the vertIndex[i] value is 1, retrieve vert data from uniqueIndex and increment the respective pointer. When the vertIndex[i] value is 0, retrieve the value from the repeatIndex, incrementing that pointer, and use the retrieved value as the index into the uniqueIndex to get the vert data. Put the vert data in a new outputVerts[i] array.
3. Iterate the repTable and use the repTable instruction conversion below to produce the indices. Supply your renderer with the outputVerts and indices.

let previous = new Array(3)
let vrt = 0
for (const instruction of repTable) {
  const tri = new Array(3)

  if (instruction === 3) {
    tri[0] = vrt++
    tri[1] = vrt++
    tri[2] = vrt++
  } else if (instruction === 0) {
    tri[0] = previous[0]
    tri[1] = previous[2]
    tri[2] = vrt++
  } else if (instruction === 1) {
    tri[0] = previous[2]
    tri[1] = previous[1]
    tri[2] = vrt++
  } else if (instruction === 2) {
    tri[0] = previous[1]
    tri[1] = previous[0]
    tri[2] = vrt++
  }

  previous = tri
  indices.push(tri[0], tri[1], tri[2])
}