OTR Encrypted File Transfer Specification - Gordin/cryptocat GitHub Wiki

Authors: Daniel Faucon, Nadim Kobeissi.

Summary

This is a specification for the encryption of files using the OTR protocol, and their subsequent transfer using an out-of-band medium such as XMPP.

Key Derivation

For file transfers, a dedicated key is generated for each file transmitted. This is done through the Extra Symmetric Key agreement defined in the OTRv3 protocol: The sender generates a type 8 TLV and with a 4 byte indication of 0001 (arbitrary value for our protocol) indicating its use for a file transfer. Its contents is the name of the file to transfer.

Once a 256 bit symmetric key is obtained, it is run through SHA512. The first 256 bits are used as encryption key for the AES-256 primitive in CTR mode, while the last 256 bits will be used for message authentication through HMAC-SHA512.

Chunk Size and Counter Information

The file is segmented into "chunks" of 63kB (64511 bytes). Because AES256 has a 16 bytes output, the encryption primitive has to be run approximately 4032 times per full chunk, each time with an incremented counter value.

The counter is 16 bytes: the top 8 bytes act as a nonce that start at 0 and are incremented in the application once per chunk. The bottom half also start at 0 and are incremented independently by the AES-CTR. The bottom half of the counter is reset for each chunk, while the top 8 bytes are incremented.

Our cryptographic library uses a 4 bytes counter for its AES256-CTR implementation, so the counter will max out at (2^31)-1 when encrypting a single chunk. This means the absolute limit for chunk encryption beyond which the counter runs the risk of being reset is of (2^31)-1 [counter increments] * 16 [bytes encrypted per counter increment] = 32768 kbytes or 32 megabytes per chunk. As we use a 8 bytes counter incremented for each chunk on top of this, which will max out at (2^53) -1, it is actually possible to encrypt files large enough in the future that rekeying becomes a cryptographic necessity.

However, XEP-0047 limits transfers to 65535 bytes, so a 64 kbytes maximum chunk size has been decided to stay well within the limit, and make room for the MAC.

Sending a file

The file is split into chunks. Before encryption, each chunk is prefixed with 16 bytes, the top 8 bytes representing the current chunk number, and the last 8 bytes representing the total number of chunk for the file transfer. The first chunk number is 0 and last one is the total number -1.

The prefixed current chunk is encrypted with the file-specific encryption key and a counter. The first chunk is encrypted with the counter as described in the "Chunk Size and Counter Information" section.

The result of the encryption is run through HMAC-SHA512 with the MAC key mentioned in the key derivation process. The MAC is concatenated to the end of the ciphertext, and the resulting chunk is sent via XMPP's In Band Bytestream channel (XEP-0047) inside a "File Transfer" stream initiation (XEP-0096).
It is recommended that the sending implementation be programmed to randomize the filename of the file being sent, in order to prevent the actual filename from being sent as cleartext over XEP-0096

Following chunks are processed similarly, with the counter incremented according to the procedure described in the "Chunk Size and Counter Information" section.

Receiving a File

The reception of a file is carried out chunk by chunk, first separating the last 88 characters of the received chunk to isolate the MAC from the ciphertext. This is because the MAC is base64-encoded before being appended to the message. Therefore, the 64 byte HMAC becomes 88 bytes after the encoding.

The MAC is then checked against the HMAC-SHA256 of the remaining message with the MAC key specific to the file being transfered.

If the received MAC does not match the computed MAC, the transfer is aborted.

If the MAC matches, the decryption of the chunk is carried out with the symmetric key agreed upon for that specific file transfer. The counter iteration process is identical to what is described in the "Sending a file" section. Once the plaintext of the chunk is obtained, it is assembled with the previous ones until the entire file has been reconstructed.