LuksWrapMbrPartition - hpaluch/hpaluch.github.io GitHub Wiki

Wrap MBR partition around LUKS volume

Here is my life story: how to use Linux Device Mapper to create fake MBR partition table around LUKS volume (which is by its nature always without partition table).

NOTE: there is same issue when you use LVM: Logical Volume (LV) contains directly filesystem without partition table which may sometimes be problem...

My scenario:

I use USB Disk with LUKS partition (supported on Linux only) and exFAT filesystem (supported by both Linux and Windows)

While I was copying data to that disk, my XHCI controller completely froze (on Cubi), causing grave errors (visible with dmesg):

Buffer I/O error on dev dm-1, logical block 1095864064, lost async page write
I/O error, dev sdb, sector 1095871200 op 0x1:(WRITE) flags 0x100000 phys_seg 800 prio class 0

xhci_hcd 0000:00:14.0: Abort failed to stop command ring: -110
xhci_hcd 0000:00:14.0: xHCI host controller not responding, assume dead
xhci_hcd 0000:00:14.0: HC died; cleaning up
xhci_hcd 0000:00:14.0: Timeout while waiting for setup device command
usb 4-3: device not accepting address 5, error -22

scsi 2:0:0:0: rejecting I/O to dead devi

fortunately I have configured AC Power button to perform shutdown on Press, but ...
On Linux fsck.exfat is in various shape (on openSUSE LEAP 15.6 unusable - actually only detect errors, on Arch there is better version, but still not sure about its quality)
So I needed to pass decrypted LUKS device to Windows VM (which has chkdsk that officially supports exFAT).
But there is a problem! Windows requires Disk device with partition table! (LUKS contains just filesystem not partition table)

So result:

I have to create separate image file with fake MBR partition table
I have to setup Loopback block device
I have to concatenate both to single fake disk with:
1. fake MBR partition table
2. decrypted LUKS partition with my (broken but still there) exFAT filesystem
and finally attach it to VM with Windows (to run chkdsk) - tested Windows Server 2019.

How I did it

This guide would not be possible without important hints from: https://superuser.com/a/1167639

So what I did:

attached USB disk with LUKS partition to PC - using rock stable dual-core Opteron with PCIe/USB 3 adapter card (to avoid above XHCI controller crash while performing data recovery).

find LUKS partition name with:

$ blkid | grep crypto_LUKS

/dev/sdb1: UUID="24372d0c-a869-4826-ad91-9fa9cd6c8545" TYPE="crypto_LUKS" PARTUUID="693726e6-98d5-0346-92ef-d32a47d6f27c"

so /dev/sdb1 is encrypted LUKS partition (containing fractured exFAT filesystem)

open (unlock) encrypted LUKS partition with:

$ udisksctl unlock -b /dev/sdb1

Passphrase:
Unlocked /dev/sdb1 as /dev/dm-0.

Verify that there is exFAT filesystem with expected label (WD4TB_LUKS in my case):

$ blkid /dev/dm-0

/dev/dm-0: LABEL="WD4TB_LUKS" UUID="FDB3-F9CB" BLOCK_SIZE="512" TYPE="exfat"

now find size of exFAT partition in 512 byte sectors:

bdev=/dev/dm-0
exfat_size_sectors=$(blockdev --getsz $bdev)
echo "exfat_size_sectors=$exfat_size_sectors"

# output: exfat_size_sectors=2147479552

Now we will create completely fake disk (with sparse zeroes) that will contains both:

1MB with MBR partition table

full size of exFAT partition - so we can allocate proper partition with fdisk

# 1 MB in sectors = 2048 * 512
mb_sectors=2048
fake_disk_size_sectors=$(( $exfat_size_sectors + $mb_sectors ))
echo "fake_disk_size_sectors=$fake_disk_size_sectors"
fake_disk_size_bytes=$(( $fake_disk_size_sectors * 512 ))
echo "fake_disk_size_bytes=$fake_disk_size_bytes"

# output:
fake_disk_size_sectors=2147481600
  fake_disk_size_bytes=1099510579200

Now create empty disk with composite size:

truncate -s $fake_disk_size_bytes fake_disk_mbr.img
# actual command: truncate -s 1099510579200 fake_disk_mbr.img

Now we need to create fake exFAT partition table starting at 1MB offset (there will be mapped real LUKS exFAT partition) and default end offset (it will automatically match our values because our sparse fake_disk_mbr.img matches its size):

fdisk ./fake_disk_mbr.img

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-2147481599, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-2147481599, default 2147481599):

Created a new partition 1 of type 'Linux' and of size 1024 GiB.

Command (m for help): t
Selected partition 1

Hex code or alias (type L to list all): 7
Changed type of partition 'Linux' to 'HPFS/NTFS/exFAT'.

Command (m for help): p
Disk ./fake_disk_mbr.img: 1024 GiB, 1099510579200 bytes, 2147481600 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x76f23205

Device               Boot Start        End    Sectors  Size Id Type
./fake_disk_mbr.img1       2048 2147481599 2147479552 1024G  7 HPFS/NTFS/exFAT

Command (m for help): v
No errors detected.

Command (m for help): w
The partition table has been altered.

Now we will extract just first 1MB of this fake disk with MBR partition table:

dd if=fake_disk_mbr.img of=fake_mbr_only.img bs=512 count=2048

We have to create real block-device from that file fake_mbr_only of 1MB size using:

$ losetup -f --show ./fake_mbr_only.img

/dev/loop0

So we basically have to concatenate 2 block devices together (also known as "linear" mapping):

/dev/loop0 1MB starting with fake MBR partition table
/dev/dm-0 decrypted LUKS partition with exFAT filesystem (unlocked LUKS device)

We need to create "map table" for device mapper with:

cat > whole.map <<EOF
0 $mb_sectors linear $mbr_dev 0
$mb_sectors $exfat_size_sectors linear $bdev 0
EOF

Actual content of whole.map:

0 2048 linear /dev/loop0 0
2048 2147479552 linear /dev/dm-0 0

All numbers are in 512 byte sectors. NOTE: 2nd number is length in sectors which may NOT exceed real backing disk size (the 2147479552 is exactly size of decrypted LUKS exFAT partition).

And finally:

dmsetup create mbr-luks.raw < whole.map

The whole disk with fake partition table is now:

block device /dev/dm-1
symlink: /dev/mapper/mbr-luks.raw

Now we can start Windows VM (using libvirt).

Once it is running we can attach our composite MBR partitioned disk using:

virsh attach-disk --sourcetype block win2019-chkdsk /dev/dm-1 vdc

In Windows VM you have to:

start Computer Management -> Disk Administrator
if disk is marked "Offline" - you have to click on "left area" with disk name and select "Bring Online"
you should see that disk will automatically get assigned letter (in my case E:)

WARNING! In my case Windows did NOT recognize that broken exFAT as valid filesystem. However chkdsk identified it correctly. Finally I did:

chkdsk E:
REM interrupted with Ctrl-C
REM run in fix mode - will modify data!
chkdsk E: /f

Cleanup:

remember to first shutdown Windows VM
find and remove composite device-mapper device:
```
dmsetup ls
dmsetup remove /dev/dm-1
```
deactivate/unlock LUKS partition:
```
udisksctl lock -b /dev/sdb1
```
deactivate USB disk - for safe removal:
```
udisksctl power-off -b /dev/sdb
```

That's all!

Again - most important part of this article is based on: https://superuser.com/a/1167639

Troubleshooting

Directly access partitions on Fake disk

When we created /dev/dm-1 composite disk there are NOT automatically accessible partitions devices (we have 1 partition there). We have to create them with:

kpartx -a /dev/dm-1

In my case it created additional /dev/dm-2 for my only fake MBR partition with exFAT filesystem. So you can directly test for example:

fsck.exfat -n /dev/dm-2 # decrypted exFAT on faked disk partition

# or examing boot sector of exFAT with:
file -s /dev/dm-2
dd if=/dev/dm-2 bs=512 count=1 of=x.bin
hexdump -C x.bin

When finished we can remove partition mappings using:

kpartx -d /dev/mapper/mbr-luks.raw # or /dev/dm-1 in my example

Map QEMU disk to Host

I used it to examine partition ID and "real" exFAT formatted from Windows OS Virtual machine:

I created new empty 16GB disk in LibVirt
attached it to Windows VM
on Windows I setup MBR partition table in Disk Administrator and formatted it with exFAT with command: format e: /fs:exfat /q
WARNING! Windows GUI hides exFAT as option to format such device (!) - you need to use CLI.

Once such disk is prepared we can:

shutdown Windows VM (important!)
map qcow2 to real block device on host with:

modprobe nbd
qemu-nbd -c /dev/nbd0 /var/lib/libvirt/images/win2019-chkdsk-1.qcow2
# now /dev/nbd0 is whole disk
# /dev/nbd0pX are individual partitions
fdisk /dev/nbd0
# ...

When done I can delete mapped disk with:

qemu-nbd -d /dev/nbd0

Resources

Important tip how to concatenate disks in Linux using Device mapper:

https://superuser.com/a/1167639

This XHCI problem is strikingly similar to several resources:

https://bbs.archlinux.org/viewtopic.php?id=196734

Or maybe?

https://bugzilla.kernel.org/show_bug.cgi?id=220069

--hp