Extracting data from stock ROM - sailfishos-sony-nagara/main GitHub Wiki

As described below, it is simple to extract needed from the stock ROM by using firmware images downloaded by XperiFirm.

To extract the data, you would need:

To get the firmware, start XperiFirm and download stock ROM you want to get the data from.

To examine content of sin-files:

  • Use unsin from UnSIN tool to convert it to raw image file. Ex: unsin super_X-FLASH-ALL-6348.sin
  • If it is a simpler image file (not "super"), you can mount it directly using Linux loop device. Ex: mount -o ro modem_X-FLASH-ALL-6348.img explore (here "explore" is a folder made to look into content)
  • For "super" image, extract its components using lpunpack. lpunpack can be either packaged in your distribution (dev-util/android-tools in Gentoo) or you have to either compile it from AOSP sources or find pre-compiled binary. Ex: 
    mkdir -p super
lpunpack super_X-FLASH-ALL-6348.img super
sudo mount -o ro super/vendor_a.img explore

Extracting examples below are written assuming that unsin tool has been applied already and you have raw image file for that partition.

Mounting partitions

To simplify work with stock ROM, there are several scripts in this repository that allow you to extract and mount partitions from downloaded stock ROM in order to reproduce environment on your device. Scripts are in https://github.com/sailfishos-sony-nagara/main/tree/main/scripts .

First, unsin all sin-files from the ROM using unsin-all.sh script. This script requires unsin tool to be in $PATH and takes the source folder with ROM and target directory for writing img files as arguments. Example:

unsin-all.sh \ 
  XQ-CT54_Customized\ EU-UK_64.2.A.2.185 \
  XQ-CT54_Customized\ EU-UK_64.2.A.2.185\ Unpacked

Second, unpack super partition:

unpack-super.sh \
  XQ-CT54_Customized\ EU-UK_64.2.A.2.185\ Unpacked

This script takes folder with img files as an argument and creates super sub-folder in it with the images of partitions from super-partition unpacked into it.

To mount the images into a tree, use mount-stock.sh script:

mount-stock.sh XQ-CT54_Customized\ EU-UK_64.2.A.2.185\ Unpacked root

It takes two arguments: folder with images and the name of the folder where you want to mount the images. In the example above, folder root is created (if missing) and images are mounted in that folder. Now you can explore contents of the files in the images.

To unmount all images, run umount-stock.sh:

umount-stock.sh root

Argument to the script is the name of the folder with the mounted images.

Extracting boot

Ramdisk and kernel can be extracted from boot image. For that, use AOSP sources ($AOSP in the commands below):

mkdir boot
cd boot
$AOSP/system/tools/mkbootimg/unpack_bootimg.py --boot_img ../ROM/boot_X-FLASH-ALL-6348.img --out .
mv ramdisk ramdisk.lz4
mkdir rdisk
cd rdisk
lz4cat ../ramdisk.lz4| cpio -i

Extracting DTBO

# activate your python venv
pip install extract-dtb
mkdir dtbo-extracted
cd dtbo-extracted
extract-dtb -o . ../ROM/dtbo_X-FLASH-ALL-6348.img
for file in *.dtb; do; dtc -I dtb -O dts -o "${file%.dtb}.dts" "$file"; done

Extracting kernel config

  • Grab extract-ikconfig script.
  • Extract kernel from boot image as described above
  • Extract kernel config by running extract-ikconfig kernel

Packing kernel

To pack kernel, we have to find out first information from regular boot images

To get offsets, the easiest is to check AOSP build logs (out/verbose.gz) and see how the boot image is assembled. Example log:

out/host/linux-x86/bin/mkbootimg \
 --kernel out/target/product/pdx223/kernel \
 --ramdisk out/target/product/pdx223/ramdisk.img \
 --os_version 13 --os_patch_level 2023-09-01 \
 --header_version 4 \
 --ramdisk_offset 0x01000000 \
 --tags_offset 0x00000100 \
 --output out/target/product/pdx223/boot.img

Notice the offsets from the command above, those will be used below.

Next, check out stock boot image parameters:

# out/host/linux-x86/bin/avbtool info_image --image images/boot_X-FLASH-ALL-6348.img 
Footer version:           1.0
Image size:               100663296 bytes
Original image size:      48234496 bytes
VBMeta offset:            48234496
VBMeta size:              832 bytes
--
Minimum libavb version:   1.0
Header Block:             256 bytes
Authentication Block:     0 bytes
Auxiliary Block:          576 bytes
Algorithm:                NONE
Rollback Index:           0
Flags:                    0
Rollback Index Location:  0
Release String:           'avbtool 1.2.0'
Descriptors:
    Hash descriptor:
      Image Size:            48234496 bytes
      Hash Algorithm:        sha256
      Partition Name:        boot
      Salt:                  549e4e9560dc2d558ca0a42f87a7358efab56a34e5a9585bfdd511d01be3a641
      Digest:                ba8f0e2b3b5bcf0aef7b0cbfa9fb14338c412abcc0a5bd2871c699d8bf35ee6e
      Flags:                 0
    Prop: com.android.build.boot.fingerprint -> 'Sony/pdx223/pdx223:12/SKQ1.220714.001/1:user/release-keys'
    Prop: com.android.build.boot.os_version -> '12'
    Prop: com.android.build.boot.security_patch -> '2024-07-01'
    Prop: com.android.build.boot.security_patch -> '2024-07-01'

Finally,

# out/host/linux-x86/bin/unpack_bootimg --boot_img XQ-CT54_Customized\ EU-UK_64.2.A.2.185/boot_X-FLASH-ALL-6348.img --out boot-full
boot magic: ANDROID!
kernel_size: 46842948
ramdisk size: 1380091
os version: 12.0.0
os patch level: 2024-05
boot image header version: 4
command line args:
boot.img signature size: 4096

Now, assuming that we have kernel and ramdisk (lz4 compressed) ready, we can assemble it all together:

out/host/linux-x86/bin/mkbootimg \
  --header_version 4 --os_version 12.0.0 --os_patch_level 2024-05 \
  --kernel boot-full/kernel --ramdisk boot-full/ramdisk \
  --ramdisk_offset 0x01000000 --tags_offset 0x00000100 --cmdline '' \
  --output new_boot.img

out/host/linux-x86/bin/avbtool add_hash_footer \
  --image new_boot.img \
  --partition_name boot --partition_size 100663296 \
  --prop com.android.build.boot.fingerprint:'Sony/kernelsu_xqct54/pdx223:13/TQ3A.230901.001/root11192255:userdebug/test-keys' \
  --prop com.android.build.boot.os_version:'12' \
  --prop com.android.build.boot.security_patch:'2024-07-01' \
  --hash_algorithm sha256

Using this approach, it is possible to boot GKI kernels, such as KernelSU (https://github.com/tiann/KernelSU/releases/download/v1.0.2/android12-5.10.205_2024-03-boot.img.gz).

Building Stock kernel

Stock kernel is distributed as 2 kernels - one is GKI and the second has modules adjusted by Sony. To build, follow https://source.android.com/docs/setup/build/building-kernels to setup the sources. Then add external kernel, for example into external/kernel.

Optionally, add KernelSU to GKI kernel (under custom) by running the following script in custom folder:

curl -LSs "https://raw.githubusercontent.com/tiann/KernelSU/main/kernel/setup.sh" | bash -

With that set, you can build as follows

export BUILD_CONFIG=external/kernel/build.config.msm.waipio
export VARIANT=gki
#export VARIANT=consolidate
export LTO=thin

echo Building with external modules
echo $EXT_MODULES
echo

build/build.sh

# strip all modules
find out/msm-waipio-waipio*/dist/ -iname "*.ko" -exec ./prebuilts-master/clang/host/linux-x86/clang-r399163b/bin/llvm-strip --strip-debug {} \;

In addition, external modules are needed. The ones packaged for LineageOS can be used and added to external/sm8450-modules. Then definition of EXT_MODULES before build/build.sh would be sufficient to build them:

export EXT_MODULES="
    external/sm8450-modules/qcom/opensource/mmrm-driver
    external/sm8450-modules/qcom/opensource/audio-kernel
    external/sm8450-modules/qcom/opensource/camera-kernel
    external/sm8450-modules/qcom/opensource/display-drivers/msm
    external/sm8450-modules/qcom/opensource/cvp-kernel \
    external/sm8450-modules/qcom/opensource/dataipa/drivers/platform/msm \
    external/sm8450-modules/qcom/opensource/datarmnet/core \
    external/sm8450-modules/qcom/opensource/datarmnet-ext/aps \
    external/sm8450-modules/qcom/opensource/datarmnet-ext/offload \
    external/sm8450-modules/qcom/opensource/datarmnet-ext/shs \
    external/sm8450-modules/qcom/opensource/datarmnet-ext/perf \
    external/sm8450-modules/qcom/opensource/datarmnet-ext/perf_tether \
    external/sm8450-modules/qcom/opensource/datarmnet-ext/sch \
    external/sm8450-modules/qcom/opensource/datarmnet-ext/wlan \
    external/sm8450-modules/qcom/opensource/eva-kernel \
    external/sm8450-modules/qcom/opensource/video-driver \
    external/sm8450-modules/cirrus/kernel-modules/cs35l41/sound/soc/codecs \
    external/sm8450-modules/cirrus/kernel-modules/cs40l25/drivers/misc \
    external/sm8450-modules/cirrus/kernel-modules/cs40l25/sound/soc/codecs \
    external/sm8450-modules/semc/hardware/camera-kernel-module/camera_sync \
    external/sm8450-modules/semc/hardware/camera-kernel-module/sony_camera \
    external/sm8450-modules/semc/hardware/camera-kernel-module/tcs3490 \
    external/sm8450-modules/semc/hardware/camera-kernel-module/slg51000_regulator \
    external/sm8450-modules/semc/hardware/charge/kernel-modules/battman_dbg \
    external/sm8450-modules/semc/hardware/charge/kernel-modules/battchg_ext \
    external/sm8450-modules/semc/hardware/kernel-modules/misc/bu520x1nvx \
    external/sm8450-modules/semc/hardware/kernel-modules/misc/et6xx \
    external/sm8450-modules/semc/hardware/kernel-modules/misc/last_logs \
    external/sm8450-modules/semc/hardware/kernel-modules/misc/ldo_vibrator \
    external/sm8450-modules/semc/hardware/kernel-modules/misc/powerkey_forcecrash \
    external/sm8450-modules/semc/hardware/kernel-modules/msm/sec_ts \
    external/sm8450-modules/semc/hardware/nfc/drivers/sn1x0_i2c \
    external/sm8450-modules/semc/hardware/nfc/drivers/sn1x0_spi
"