file compression - wrongbaud/sf-cabinet GitHub Wiki
Notes on File Compression for EPOS Images
Example File: shell.zgj
First we run file and binwalk like good reverse engineers ...
pi@voidstar:~/projects/sf-cabinet/test-output/rootfs $ file shell.zgj
shell.zgj: data
pi@voidstar:~/projects/sf-cabinet/test-output/rootfs $ binwalk shell.zgj
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
pi@voidstar:~/projects/sf-cabinet/test-output/rootfs $
Unfortunately that yields little - so let's focus on what we do know from the minfs parsing script.
The metadata for the file is as follows:
INFO:root:Extracting file: b'shell.zgj'
Offset: 2560
Raw Size: 27976
origSize: 121372
Entry Length: 160
Flags: 6
Name Length: 9
Extra Length: 128
Name : b'shell.zgj'
It look like we have two size fields (I pulled these from an old minfs fuse driver and named them as they were named in the driver). And since the data has a relatively high entropy - we can probably assume that it's compressed.
If we look at the first 0x20 bytes of the file we see the following:
pi@voidstar:~/projects/sf-cabinet/test-output/rootfs $ hexdump -n32 -C shell.zgj
00000000 5d 00 80 00 00 00 78 10 41 ae 90 01 08 ef c5 cb |].....x.A.......|
00000010 ac d0 0e fb 70 31 69 ba 7c 2d ba e6 93 bf c4 51 |....p1i.|-.....Q|
The first byte sequence 0x5D is something that might stand out to some of you as it's a common sequence in an LZMA header. As a matter of fact, if we look a little deeper we can see this exact sequence here:
-1 5d 00 00 01 00
-2 5d 00 00 10 00
-3 5d 00 00 08 00
-4 5d 00 00 10 00
-5 5d 00 00 20 00
-6 5d 00 00 40 00
-7 [5d 00 00 80 00] --- This is us!
-8 5d 00 00 00 01
-9 5d 00 00 00 02
According to the specification information here the next 4 bytes after that sequence are suppose to be the uncompressed size. Let's see what happens if we change the header of the binary to include the uncompressed size and try to extract it with 7zip
When performing the above steps, the data is indeed decoded and look like valid ARM code (see output below) but we will get a decode error from 7z. There may be some interesting information available to us in the documents provided here. For now I am going to move onto a simpler (hopefully) file format.
Example File: res/bootui/logo.bmp
If we look at the header for this file we can see the following:
pi@voidstar:~/projects/sf-cabinet/mainfs/rootfs/res/boot_ui $ hexdump -C -n32 logo.bmp
00000000 41 5a 31 30 38 60 09 00 5d 00 80 00 00 00 21 13 |AZ108`..].....!.|
00000010 43 04 3e ca 2f 41 e5 d4 a6 b8 ef 08 bd 73 a2 0a |C.>./A.......s..|
Note that we see the same LZMA-ish 0x5D at the beginning of the header, preceded by AZ108 and another DWORD.
| Data | Size | Purpose |
|---|---|---|
41 5a 31 30 |
4 bytes | Header (?) |
38 60 09 00 |
4 Bytes | Size (?) |
5d |
1 Byte | LZMA Compression Parameters |
00 80 00 00 |
4 Bytes | LZMA Dict Size |
This is somewhat similar to the standard LZMA header which contains:
| Data | Size | Purpose |
|---|---|---|
5d |
1 Byte | LZMA Compression Parameters |
00 80 00 00 |
4 Bytes | LZMA Dict Size |
| XXX | 8 Bytes | Decompressed Size |
So if we take the existing header for the logo.bmp file and modify it to be as such:
5D 00 80 00 00 38 60 09 00 00 00 00 00 00 21 13
43 04 3E CA 2F 41 E5 D4 A6 B8 EF 08 BD 73 A2 0A
Is properly decompresses to the following!
TODO: Insert Image
Now that we have a small understanding of how the compression works, let's see if we can generate a new boot image file.
The decompressed boot image file had the following characteristics:
pi@voidstar:~ $ file logo.bmp
logo.bmp: PC bitmap, Windows 3.x format, 320 x 480 x 24
So our new logo must at the very least match these characteristics and also be smaller or the same size as the previous one. This is due to the fact that we don't want to clobber the next file entry in the partition.
I've generated a very simple bitmap file, that seems to match the BMP attributes:
pi@voidstar:~ $ file new-logo.bmp
new-logo.bmp: PC bitmap, Windows 3.x format, 320 x 480 x 24
And have compressed it with the following:
pi@voidstar:~ $ lzma_alone e new-logo.bmp logo.bmp.lzma -d15 -lc3 -pb2
The header of the newly compressed file can be seen below:
pi@voidstar:~ $ hexdump -C -n32 logo.bmp.lzma
00000000 5d 00 80 00 00 36 08 07 00 00 00 00 00 00 21 13 |]....6........!.|
00000010 42 be d0 19 a6 35 c6 ec 5b 92 17 53 63 41 34 de |B....5..[..ScA4.|
00000020
To fix up this file we need to do the following:
- Add the 4 byte header
41 5a 31 30 - Add the 32 bit decompressed size
36 08 07 00 - Remove the 64 bit decompressed sequence after the dictionary size
Doing this results in the following header:
pi@voidstar:~ $ hexdump -C -n32 logo.bmp.lzma
00000000 41 5a 31 30 36 08 07 00 5d 00 80 00 00 21 13 42 |AZ106...]....!.B|
00000010 be d0 19 a6 35 c6 ec 5b 92 17 53 63 41 34 de 46 |....5..[..ScA4.F|
Now all that is left is to modify the FS entry for this file to contain our newly compressed data, if we look at the stock file entry for this file we have the following attributes (pulled from the minfs parser)
668 INFO:root:Parsing File Entry for file at 6096
669 INFO:root:File located in directory test-output/rootfs/res/boot_ui named b'logo.bmp'
670 INFO:root:Offset: 7662440
671 Raw Size: 13326
672 origSize: 13326
673 Entry Length: 28
674 Flags: 0
675 Name Length: 8
676 Extra Length: 0
677 Name : b'logo.bmp'
The file table entry is:
2:5BD0h: 68 EB 74 00 0E 34 00 00 0E 34 00 00 1C 00 00 00 hët..4...4......
2:5BE0h: 08 00 00 00 6C 6F 67 6F 2E 62 6D 70 78 1F 75 ....logo.bmpx.u
After editing the size to match our new binary we have:
2:5BD0h: 68 EB 74 00 C8 05 00 00 C8 05 00 00 1C 00 00 00 hët.È...È.......
2:5BE0h: 08 00 00 00 6C 6F 67 6F 2E 62 6D 70 ....logo.bmp
Now all that remains is that we need to modify the data entry pointed to by 68 EB 74 00 in the flash to contain our newly compressed bmp
Success ? -->