Brute Forcing KIF Archives - trigger-segfault/TriggersTools.CatSystem2 GitHub Wiki

Brute Forcing KIF Archive Encryption

🚧 This page is a work in progress

Some knowledge about Kifint Archives as described on other pages is assumed to be known when explaining the topics on this page.

Terminology

Overview

KIF Archives use two independent modes of encryption:

Affects Parameters Post-calculations
File Name CRC-32 of the game's V_CODE2 (external) 1. CRC + Index
2. Mersenne Twister
3. Sum of MT bytes & 0xFF
4. Beaufort cipher [A-z]
File Data Mersenne Twister of archive key's Length (internal) 1. Blowfish cipher

Because these two types of information are independently encrypted, you can easily perform operations or tests on just one set without running the extra calculations on the other.


This is not a necessary task to ever perform, but when you have an archive without the game, you may want to see what's inside.

First, we're going to throw out the idea of just using a list of known V_CODE2s, that's no fun, and isn't a foolproof solution.

BUT, if you test a known V_CODE2 that results in 100% expected deciphered filenames, then you can save yourself from the trouble below.

Realistic use-cases

Mangled filenames: Some older patches created for CatSystem2 games contain incorrectly deciphered filenames, and as such, can be unmangled through two applications of the Beaufort cipher. The resulting keys only require the same difference as that of the original two keys, so the first transform can just use a key of zero.

Notes

  • CatSystem2 treats archive file names as case-insensitive, but what about extensions?
  • In the end, each Beaufort key must be a value between 0-51.
  • Because of the Mersenne Twister transform requiring the entry index, you cannot reliably find the key once and have it work for every entry. However, the this allows us to determine the original key value.
  • We can make many assumptions about what the resulting fileName will look like.
  • There are only a handful of common file extensions per each archive type.
  • The inclusion of Upper and lowercase letters simplifies things, because an extension with upper and lowercase characters is almost always unexpected.
  • Numbers and other symbols are constant and unchanged in the Beaufort cipher, so they can be used to guess the original filename.

Scoring accuracy

There is no guaranteed solution to determining the the validity of the resulting filename, but there are many assumptions that can be worked off of in most scenarios.

Patterns

  • Comparing the extension.
  • Comparing the name.
  • Non alpha characters are important giveaways.

The biggest hack

  • Compare resulting order to alpha-numeric (win32) order of each item in the archive.

Machine learning (why not)

Using known sets of expected file names, extensions, and archive names as a model, Machine Learning could be a viable, albeit over-complicated, approach to identifying what constitutes a "correct deciphered filename."

Properties of this Beaufort cipher

  • Symmetrical transform (the same key works both ways)
  • Reciprocal transform (the same algorithm works both ways)
  • Reversed cipherset (input A-z -> output z-A)
  • Shifted index ("AAAB" -> "ABCE", before reversing)
  • More than one transformation will not obscure the text anymore.
  • An even number of transforms will result in a fixed cipher index offset for all characters in the text.
  • No character in Text ciphered with different keys can ever match (will fail at index 0 or match)

Decipher by known file-extensions

If you know exactly what extensions to expect in an archive, then brute forcing each filename without Mersenne Twister should be the easiest method.

For example, if we have image.int, we know to expect anm, hg3, and possibly hg2 file extensions. Running guesses isn't even required because each extension is identifiable by the number at the end. anm ends in no number, hg3 ends in 3, and hg2 ends in 2. These will never be enciphered.

For each file:

  • Get the length of the name, then compare the last letter.
  • From here we can then calculate the desired output's key through this statement:
key = 51 - ((Cipher.IndexOf(known) + Cipher.IndexOf(input[index]) + index) % 52);

Where Cipher.IndexOf(char c) refers to this expression:

cipherIndex = (c - (c<'a' ? 'A' : 'a'-Half));

Example code

Here is an example function that finds the key of an enciphered filename with a known character, and then deciphers it.

public static class Beaufort {
    const int Length   = 52;
    const int Half     = 26;

    public static bool TryFindCipher(byte[] input, byte[] output, int length,
                                     int knownChar, int atIndex)
    {
        byte ci = input[atIndex];
        byte co = knownChar;
        // Make sure our input is as we expect, and contains decipherable characters
        if ((ci >= 'A' && ((ci <= 'Z') || (ci >= 'a' && ci <= 'z'))) &&
            (co >= 'A' && ((co <= 'Z') || (co >= 'a' && co <= 'z'))))
        {
            // Key = 51 - ((CIndex(known) + CIndex(input[index]) + index) % 52)
            int outputCipherIndex = (co - (co<'a' ? 'A' : 'a'-Half));
            int inputCipherIndex  = (ci - (ci<'a' ? 'A' : 'a'-Half));
            int decipherIndex = (outputCipherIndex + inputCipherIndex + atIndex) % Length;

            uint cipherKey = (uint) ((Length-1) - decipherIndex);
            Cipher(input, output, length, cipherKey);
            return true;
        }
        return false;
    }

    public static void Cipher(byte[] input, byte[] output, int length, uint key) {
        // ...
    }
}

Example usage

An example usage of the above for Tama02s_4.hg3 would be:

class Program {
    static void Main(string[] args) {
        //SETUP:
        const string OriginalName = "Tama02s_4.hg3";
        byte[] original = Encoding.ASCII.GetBytes(OriginalName);
        byte[] enciphered = new byte[original.Length];
        byte[] deciphered = new byte[original.Length];

        uint randKey = (uint) new Random().Next(0, 52);
        Beaufort.Cipher(original, enciphered, original.Length, randKey);
        string EncipheredName = Encoding.ASCII.GetString(enciphered);

        //RUN:
        int index_g = Array.LastIndexOf(original, (byte)'g');
        Beaufort.TryFindCipher(enciphered, deciphered, original.Length, (byte)'g', index_g);
        string DecipheredName = Encoding.ASCII.GetString(deciphered);
        Console.WriteLine($"Original   = \"{OriginalName}\"");
        Console.WriteLine($"Enciphered = \"{EncipheredName}\"");
        Console.WriteLine($"Deciphered = \"{DecipheredName}\"");
    }
}

Known file extensions

Kind CatSystem2 Formats Common Formats
Image anm hg2 hg3 bmp cur otf ttf
Media tag avi mov mpg ogg wav wmv
Config dat tbl cvs txt xml
Script cst cstl fes js
Graphics kx2 kx3 mdl ppi shd
Data qdb zt
Unknown cnf kcs psz
Common
Unarchived
dat dnf hg3 id int
lst ppi
bin bmp cur dll exe ico
otf ttf txt
⚠️ **GitHub.com Fallback** ⚠️