XWA Texture formats and compression - morallo/xwa_ddraw_d3d11 GitHub Wiki

Current XWA supported formats

Uncompressed

  • Format 24: Format: 8-bit indexed colors and 8-bit alpha. Use: in-flight.

  • Format 25: Format: 32-bit ARGB. Use: in-flight.

Compressed on disk

  • Format 7: Format: 8-bit indexed colors and 1-bit alpha, run-length-encoding compressed. Use: concourse or in-flight.

  • Format 23: Format: 8-bit indexed colors and 8-bit alpha, run-length-encoding compressed. Use: concourse.

  • Format 25C: Format: 32-bit ARGB, LZMA compressed. Use: in-flight

  • Zipped PNG, JPG, GIF (effects ddraw)

    • https://www.xwaupgrade.com/phpBB3/viewtopic.php?p=184518&hilit=Side#p184518
    • Cons:
      • Only supported for side-loaded images, not for the native DATs like planet.dat
      • Not supported by JeremyA's Golden ddraw (to be confirmed)
      • creates temporary unzipped files in the XWA folder
      • Redundant compression? If the base image format is already compressed like jpg and png, the zip will not reduce size much, but incur higher loading times.

Potential improvements

Use more balanced speed-to-size algorithms in DAT

LZMA is slow to decompress which means long loading times. Something like LZ4 could be a better compromise.

After some benchmarks created by JeremyAnsel, compressing BMP, PNG, JPG images it seems the situation is in fact the opposite, at least with the implementations used. LZMA is faster to decompress, LZ4 is faster to compress ¯(°_o)/¯

This would need small changes in the 32bit hook and DAT editor.

Use GPU native compression formats

BC7, BC5, DXT etc are supported directly by the GPU, which means no runtime decompression needed and better performance (lower memory bandwidth needed).

This would probably need profound changes in ddraw (resources format, shaders?). Currently, the performance bottleneck does not seem to be memory bandwidth so this would not be the best optimization.

Optimize DAT loading?

Is the game engine only loading the minimum required information from a DAT? (Only the required image instead of the full package

Uncompressed (tar?) image packages instead of zip

If ZIP is indeed redundant when images are already in lossy/lossless compressed formats like JPG and PNG, use some other file storage format that is faster to load.

References