toc - Tkachov/ALERT GitHub Wiki
toc.py | sections/toc/ | Overstrike: TOC.cs | Sections/TOC/ | rivet's format doc
Lists all archives and assets, storing where to find those assets in those archives. DAT1 format, sometimes zlib-compressed. In modding, is tweaked to include mods archives.
Has ArchiveTOC string in it, and TOC obviously stands for "table of contents", as it lists all of the content.
Magic is 0x77AF12AF (compressed, SO/MSMR/MM) or 0x34E89035 (not compressed, RCRA/i30/i33).
File defines spans, in which assets are spread. Every asset is defined by its id (unique number that is usually a hash of asset's original name), index of an archive where this asset's at, offset in that archive and size of the asset. Archives are defined by their filename.
Additionally, there could be "key assets" list, and a few less known properties like archive's bucket and chunkmap numbers.
Since RCRA, it also contains some of the assets' headers, textures list and their metadata.
In Sunset Overdrive, the format is pretty similar to what we have in MSMR and MM (i20), with a couple of small differences. First, assets' ids are 32-bit. Second, archives' names in the structure are only 16 characters long.
Game comes with 'archive_input.json', 'chunkmap.txt', 'layout.csv' and 'scan.csv' files, which seem to be leftovers that are not used by the game itself. Those were useful to find assets' names and the terms used in those were probably used by developers in their SO scripts. Some of the other works, following their steps, picked up the same terms.
As for the contents of toc, SO has only 114 spans (while later games use 256) and contains duplicates of assets. Asset's id is only mentioned once, but sizes section marks how many copies of it exist, and there are this many records in the offsets section. This was supposedly used for faster access on HDDs, because the system could know which archives are currently loaded or even which area of the disk is closer right now, and choose to read the asset from one place or the other. Data in all of such copies is exactly the same.
Sections
| Tag | Name | Short description |
|---|---|---|
EDE8ADA9
|
"Archive TOC Header" | Array of structs describing spans |
506D7B8A
|
"Archive TOC Asset IDs" | Array of 64-bit asset ids |
6D921D7B
|
"Archive TOC Key Asset IDs" | Array of "key" asset ids |
65BCF461
|
"Archive TOC Asset Metadata" | Array of assets' size/count records |
DCD720B5
|
"Archive TOC Asset Dupe Metadata" | Array of assets' archive/offset records |
398ABFF0
|
"Archive TOC File Metadata" | Array of structs describing archives |
File defines 256 spans. Every 8 of them form a locale, and within those 8 spans with even index (0, 2, 4 and 6) are for DAT1 assets, and odd ones are for raw assets. Languages that correspond to locales depend on the game.
Assets ids are listed by span, and are sorted within the span in ascending order.
List of key assets is of unclear use, and typically is not modified. In MSMR, it contains only .zone files, 12162 of them (which is ~99% of all .zones that are in game).
Archives that are listed last are filenames (up to 64 characters long) with "install bucket" and "chunkmap" numbers. Those are not exactly known, and using 0 as their values seems to work just fine.
Assets' offsets in the archive are given for the uncompressed stream, even if archive itself is DSAR-compressed.
Sections
| Tag | Name | Short description |
|---|---|---|
EDE8ADA9
|
"Archive TOC Header" | Array of structs describing spans |
506D7B8A
|
"Archive TOC Asset IDs" | Array of 64-bit asset ids |
65BCF461
|
"Archive TOC Asset Metadata" | Array of assets' size/archive/offset records |
398ABFF0
|
"Archive TOC File Metadata" | Array of structs describing archives |
36A6C8CC
|
"Archive TOC Texture Asset Ids" | Array of .texture asset ids |
C9FB9DDA
|
"Archive TOC Texture Meta" | Array of .texture metadata |
62297090
|
"Archive TOC Texture Header" | One int32 telling how many textures there are |
654BDED9
|
"Archive TOC Asset Header Data" | Array of assets' headers |
Format is similar to previous games. However, assets no longer duplicate, and there is no "key" assets list anymore. But, there is now a list of textures, with their metadata put directly in this file. Similarly, assets' headers are also here. Archives' struct has also changed slightly, seemingly no longer includes "install bucket" and "chunkmap", and instead has timestamp and version numbers.
Typically, only assets from 0th span (in every 8, aka locale) have headers. Overstrike .stage format allows to specify whether asset's first 36 bytes need to go into toc or not. Adding new textures through .stage is not supported as 72 bytes of texture meta cannot (yet?) be determined by file contents, and there is no way to specify if asset is texture. Some research for it was done in Ripped Apart.
i16 i20 i29 | SpansSection
Array of span structs. Struct contains two 32-bit integers: index of first asset in the span and count of assets in it.
i16 i20 i29 | AssetIdsSection
Array of assets' ids (32-bit in i16, 64-bit later).
i16 i20 | KeyAssetsSection
Array of "key" assets' ids (32-bit in i16, 64-bit later).
i16 i20 i29 | SizesSection
Array of structs, one struct per asset.
Before i29, struct consists of three 32-bit integers: number of duplicates, asset size and index of first duplicate in
Since i29, struct contains four 32-bit integers: asset size, index of archive asset is in, offset in that archive, and, optionally, offset to asset header in
i16 i20 | OffsetsSection
Array of structs, one struct per asset duplicate. Struct contains two 32-bit integers: index of archive asset duplicate is in and its offset in that archive.
i16 i20 i29 | ArchivesSection
Array of structs describing archives.
Struct is 24 bytes long in i16, 72 bytes in i20 and 66 in i29.
Before i29, it starts with two 32-bit integers, that seem to be "install bucket" and "chunkmap" numbers (according to Sunset Overdrive leftover files). The rest is 0-terminated string, filename of archive, relative to toc file.
In i29, first 40 bytes are 0-terminated filename string. The rest are unknown numbers, some of which are possibly timestamp and version of archive.
i29 | TexturesSection
Array of texture assets' ids.
i29
Array of structs per texture. Each struct is 72 bytes long.
i29
One 32-bit integer, number of textures in
i29 | AssetHeadersSection
Array of assets' headers. Each header is 36 bytes long. Offset to the header, if asset has one, is specified in