I ‐ Preparing the build - Nimpoo/ft_linux GitHub Wiki

Introduction

Link to of where I am in the process - II. Preparing for the Build

We will procede step by step to setup the virtual disk image for LFS. In this section we will create the disk image lfs.qcow2 which will be used as a secondary disk in our Ubuntu VM, partition it, and format it, etc...

Element Description
Physical Machine Host machine where the disks lfs.qcow2 and ubuntu-proud.qcow2 are created.
lfs.qcow2 Empty virtual disk of 18G, will be used as a second disk in the Ubuntu VM.
ubuntu-proud.qcow2 Virtual disk containing Ubuntu 24.04 LTS, used as the primary disk for the VM.
Ubuntu VM (hda) Virtual machine launched with ubuntu-proud.qcow2 as the primary disk.
Second Disk (hdb) In the Ubuntu VM, lfs.qcow2 is attached as a second disk.
Working Environment All work will be done inside this Ubuntu VM.

1 - Creation of our virtual disk image for LFS

On the physical machine, we will create the virtual disk image lfs.qcow2, this is the really start of our Linux From Scratch adventure.

Two ways to create the disk image:

  1. Using the qemu-img command :
qemu-img create -f qcow2 /path/to/lfs.qcow2 -o preallocation=off 18G
  • qemu-img : the command to create a disk image, part of the QEMU package.
  • create : the action to create a new disk image
  • -f qcow2 : the format of the disk image, here we use qcow2 for its features like compression and snapshots, but you can use other formats.
  • /path/to/lfs.qcow2 : the path where the disk image will be created
  • -o preallocation=off : this option is used to disable preallocation, which means the disk image will grow as needed, rather than being allocated all at once.
  • 18G : the size of the disk image, here we use 18GB which is a little bit more than the recommended minimal size of 10GB for Linux From Scratch (source).
  1. Using the dd command :
dd if=/dev/zero of=lfs.iso bs=4M count=4500 conv=fdatasync status=progress
  • dd : the command to create a disk image, part of the GNU coreutils package.
  • if=/dev/zero : the input file, here we use /dev/zero which is a special file that produces null bytes.
  • of=lfs.iso : the output file, here we use lfs.iso which is the name of the disk image.
  • bs=4M : the block size, here we use 4MB which is a good size for performance.
  • count=4500 : the number of blocks to write, here we use 4500 which is equivalent to 18GB (4500 * 4MB = 18GB).
  • conv=fdatasync : this option is used to ensure that the data is written to the disk before the command returns.
  • status=progress : this option is used to display the progress of the command.

Okay, now we have our disk image of 18GB created, but it is still empty, we need to partition it and format it, and all that inside the Ubuntu VM.

2 - Creation and setup of our Ubuntu VM

This VM will be our working environment, it's important to setup correctly our working environment.

2.1 - Creation of the Ubuntu VM

Same here, creation of our disk image ubuntu-proud.qcow2 which will be used as the primary disk in our Ubuntu VM.

qemu-img create -f qcow2 /path/to/ubuntu-proud.qcow2 -o preallocation=off 128G

Then download the Ubuntu 24.04 LTS ISO image from the official website:

curl -O https://mirror.tutosfaciles48.fr/ubuntu/24.04.2/ubuntu-24.04.2-desktop-amd64.iso

And launch the VM with the Ubuntu ISO image:

qemu-system-x86_64 \
	-enable-kvm \
	-m 28G \
	-smp cores=16,threads=2,sockets=1 \
	-cpu host \
	-net nic,model=virtio \
	-net user \
	-device virtio-balloon \
	-vga virtio \
	-full-screen \
	-daemonize \
	-hda ubuntu-proud.qcow2 \
	-cdrom ubuntu-24.04.2-desktop-amd64.iso
  • qemu-system-x86_64 : the command to launch a QEMU virtual machine
  • -enable-kvm : enables KVM (Kernel-based Virtual Machine) for better performance
  • -m 28G : allocates 28GB of RAM to the VM
  • -smp cores=16,threads=2,sockets=1 : configures the VM to use 16 CPU cores with 2 threads per core and 1 socket
  • -cpu host : uses the host CPU model for better compatibility
  • -net nic,model=virtio : creates a virtual network interface card (can be removed if not needed)
  • -net user : sets up user-mode networking for the VM (same)
  • -device virtio-balloon : adds a balloon device for memory management
  • -vga virtio : uses the VirtIO graphics driver for better performance
  • -full-screen : starts the VM in full-screen mode
  • -daemonize : runs the VM in the background
  • -hda ubuntu-proud.qcow2 : specifies the primary disk image for the VM
  • -cdrom ubuntu-24.04.2-desktop-amd64.iso : specifies the Ubuntu ISO image to boot from

And now, you install Ubuntu 24.04 LTS on the ubuntu-proud.qcow2 disk image, just like you would do on a physical machine, shutdown the VM when the installation is done, remove the ISO image from the VM and add the lfs.qcow2 disk image as a secondary disk before starting the VM again:

qemu-system-x86_64 \
	-enable-kvm \
	-m 28G \
	-smp cores=16,threads=2,sockets=1 \
	-cpu host \
	-net nic,model=virtio \
	-net user \
	-device virtio-balloon \
	-vga virtio \
	-full-screen \
	-daemonize \
	-hda ubuntu-proud.qcow2 \
	-hdb lfs.qcow2

(Obviously, you can use another distro than Ubuntu.)

2.2 - Check the host requirements

In the book of LFS, Gerard Beekmans created a bash script to check if all the tools, compilatiors, libraries and theis versions are installed and configured correctly. Run this script to check if everything is good, I pushed it in the repository : version-checker.sh. And here the page of the book.

3 - Partitioning and formatting the disk image lfs.qcow2

Now we are inside the Ubuntu VM, and its time to partition the disk image lfs.qcow2.

3.1 - What is partitioning and formatting?

Partitioning is the process of dividing a disk into smaller, manageable sections called partitions. Each partition can be used for different purposes, such as storing data, installing an operating system, or creating a swap space. Formatting is the process of preparing a partition for use by an operating system. It involves creating a file system on the partition, which allows the operating system to read and write data to it.

For exemple, we will create a partition for the root filesystem (/) for the operating system files, the boot (/boot) for boot files, and another one for the swap space. The root filesystem will contain the operating system files, while the swap space will be used as virtual memory when the physical RAM is full.

3.2 - Partitioning the disk image lfs.qcow2

To partition the disk image lfs.qcow2, we will use the cfdisk command, which is a powerful tool for managing disk partitions (man cfdisk(8)). It provides a user-friendly interface to create, delete, and modify partitions on a disk.

You can run cfdisk with the following command:

sudo cfdisk /dev/sdb

cfdisk detects if the disk to be partitioned is empty or not. If it is empty, it displays this :

bios

You need to select a label type for the disk.

For all step talking about UEFI, I refer to the part of the Beyond Linux From Scratch (BLFS) book, which is a continuation of the LFS book and provides more advanced topics and features. You can find it here : Beyond Linux® From Scratch (systemd Edition) - Version r12.3-1115 Chapter 5. File Systems and Disk Management.

3.3 - What is a label type?

A label type is a way to organize the partitions on a disk. There are two main types of labels:

  • GPT (GUID Partition Table) : a modern partitioning scheme that supports larger disks and more partitions. It is recommended for disks larger than 2TB or for systems using UEFI firmware.
  • DOS (Disk Operating System) : an older partitioning scheme that is compatible with legacy BIOS systems. It supports a maximum of 4 primary partitions and is limited to disks smaller than 2TB.
  • SGI (Silicon Graphics International) : a partitioning scheme used by SGI systems, not commonly used in modern systems.
  • SUN (Sun Microsystems) : a partitioning scheme used by Sun systems, not commonly used in modern systems.

3.4 - BIOS, UEFI, what is that?

BIOS (Basic Input/Output System) and UEFI (Unified Extensible Firmware Interface) are two types of firmware interfaces that initialize hardware components and boot the operating system. They are responsible for the early stages of the boot process and provide a bridge between the hardware and the operating system. Without a firmware interface, the operating system cannot communicate with the hardware.

  • BIOS is the traditional firmware interface that has been used for decades. It is limited in terms of features and capabilities, such as supporting only MBR (Master Boot Record) partitioning scheme and having a maximum disk size limit of 2TB.
  • UEFI is a modern firmware interface that replaces BIOS. It supports GPT partitioning scheme, which allows for larger disks and more partitions. UEFI also provides additional features such as secure boot, faster boot times, and better compatibility with modern hardware. It's modern and more flexible than BIOS, and it is recommended for new systems.

3.5 - Partitioning

Okay, if we want to use the lfs.qcow2 disk image with UEFI, we need to select the GPT label type. If you want to use it with BIOS, you can select the DOS label type. Here, we will use the GPT label type. After selecting the label type, you can proceed to create the necessary partitions:

  • EFI System Partition (/boot/efi) : this partition is used to store the bootloader and other files needed to boot the system in UEFI mode. It should be at least 512MB in size and formatted with the vfat file system.
[   New  ] -> partition size: 512M -> [  Type  ] -> select `EFI System`
  • Boot Partition (/boot) : this partition is used to store the kernel and other boot files. It's here where grub is installed. I chos 1GB where it is really enough, but you can use a smaller size if you want.
[   New  ] -> partition size: 1G -> [  Type  ] -> select `Linux filesystem`
  • Root Partition (/) : this partition is used to store the root filesystem, which contains the operating system files. It should be the rest of the your enable disk image minus the size you want to allocate for the swap partition. If I want 2GB for the swap partition, I will use 15GB for the root partition.
[   New  ] -> partition size: 15G -> [  Type  ] -> select `Linux filesystem`
  • Swap Partition : this partition is used as virtual memory when the physical RAM is full. It should be at least 2GB in size.
[   New  ] -> partition size: 2G -> [  Type  ] -> select `Linux swap`

If you chose the DOS label type, you can create the same partitions, but you will not be able to use the EFI System Partition (/boot/efi) because it is only used in UEFI mode. Instead, you can create a boot partition (/boot) and a root partition (/), and a swap partition. AND you need to specified what is the boot partition, so you need to select the boot partition and set it as the bootable partition:

[Bootable] -> select the boot partition

After that :

[  Write ] -> "Are you sure you want to write the partition table to disk?" -> yes
The partition table has been altered.
[  Quit  ]

Our LFS is now partitioned, we can format the partitions with the appropriate file systems.

4 - Formatting the partitions

To format the partitions, we will use the mkfs command, which is used to create a file system on a partition. The command syntax is as follows:

4.1 - What is a file system?

A file system is a way to organize and store files on a disk. It defines how files are named, stored, and accessed. Different file systems have different features and capabilities, such as support for large files, file permissions, and journaling.

More easily, a file system is the thing that creates HOW we create a directory structure, how we name files, how we access files, etc... Without that, we cannot store files on a disk.

For exemple:

  • ext4 : a widely used file system in Linux, it supports large files, journaling, and file permissions. It is the default file system for many Linux distributions.
  • ext2/ext3 : older versions of the ext file system, they are still used in some systems but are less common than ext4.
  • vfat : a file system used for the EFI System Partition, it is compatible with UEFI firmware and is used to store the bootloader and other files needed to boot the system
  • ntfs : a file system used by Windows, it is not commonly used in Linux systems but can be used to access files on Windows partitions.

And the mkfs command is used to create a file system on a partition (man mkfs(8)). For more informations : https://fr.wikipedia.org/wiki/Syst%C3%A8me_de_fichiers .

4.2 - Formatting the partitions using mkfs

We have three different partitions to format, and each one will have its own file system:

  • EFI System Partition (/boot/efi) : we will format it with the vfat file system, which is compatible with UEFI firmware.
sudo mkfs.vfat -F32 /dev/sdb1
  • Boot Partition (/boot) and Root Partition (/) : we will format them with the ext4 file system.
sudo mkfs -v -t ext4 /dev/sdb2
sudo mkfs -v -t ext4 /dev/sdb3
  • Swap Partition : we will format it with the swap file system.
sudo mkswap /dev/sdb3

Now, we have our partitions formatted with the appropriate file systems.

5 - Mounting the partitions

To use the partitions we just created, we need to mount them.

5.1 - What is mounting?

Mounting is the process of making a file system accessible to the operating system by attaching it to a specific directory in the file system hierarchy. When a partition is mounted, the files and directories on that partition become available for use. For example, when we mount the root partition (/), we can access the files and directories on that partition as if they were part of the main file system. And for that, we use the mount command, which is used to attach a file system to a directory in the file system hierarchy (man mount(8)).

We call the directory where we mount a partition a mount point. For example, when we mount the root partition (/), the mount point is /, and when we mount the boot partition (/boot), the mount point is /boot.

5.2 - Setting up the working environment

Obviously, we don't "enter on our LFS" and do some things or compile some packets, because it's COMPLETLY EMPTY. So we need to create a working environment on our Ubuntu VM.

What we can do ? On our Ubuntu VM, we are going to create a reserved directory where our LFS will be mounted. For more clarity, when you boot a PC, your hard disk has partitions, and each partition has a mount point. For example, the root partition (/) is mounted on /, the boot partition (/boot) is mounted on /boot, etc... And all of that are mounted automatically through a file called /etc/fstab (if you use a Linux distribution), which is used to define how file systems are mounted at boot time. We see more closely about this file later.

So for your physical machine, you have a hard disk, and here, for LFS we use the virtual disk of our Ubuntu VM for creating mount points and accessible files to create our LFS system. ALL the system will be stored in the virtual disk lfs.qcow2, and we will mount it on a directory in our Ubuntu VM, our VM will be the host system for our LFS. If we do that, it's because we have all the necessary requirements on our Ubuntu VM.

Here's what we will do:

+---------------------+
| Physical Machine    |
|                     |
|   +-------------+   |
|   | Ubuntu VM   |   |
|   | (Host)      |   |
|   |             |   |
|   |   +-------+ |   |
|   |   | LFS   | |   |
|   |   | Disk  | |   |
|   |   | (Mount| |   |
|   |   | Point)| |   |
|   |   +-------+ |   |
|   +-------------+   |
+---------------------+

And after finishing the LFS system, we will be able to boot it and use it as a normal Linux system.

+---------------------+
| Physical Machine    |
|                     |
|   +-------------+   |
|   | LFS System  |   |
|   |(Independent)|   |
|   +-------------+   |
|   +-------------+   |
|   | Ubuntu VM   |   |
|   | (Host for   |   |
|   |  building)  |   |
|   +-------------+   |
+---------------------+

And even if you want, boot your LFS directly on your physical hard disk instead of your usual OS.

AND HERE WE ARE, after knowing all that, we can create the folders of where we will mount our partitions, and we will mount them. Before that, the book INSISTS to create an environment variable called LFS which will be used to store the path of the mount point of our LFS system. This mandatory for following the book, and it will be used in ALL the next chapters, so it's really important to do it.

DISCLAIMER: FROM HERE ON, RUN ALL COMMANDS AS USER root EXCEPT WHEN USER CHANGE IS SPECIFIED

Let's create the environment variable:

export LFS=/mnt/lfs

And set umask to 022 to ensure that the files created in the LFS environment have the correct permissions:

umask 022

(Personally, I set another environment variable called LFS_TGT that is not set for the root user, but it will be used later in the book.)

export LFS_TGT=$(uname -m)-lfs-linux-gnu

And write these commands in your ~/.bashrc file to make them persistent, it's REALLY IMPORTANT.

And now, we can create the mount point directory:

mkdir -pv $LFS
mkdir -pv $LFS/boot
mkdir -pv $LFS/boot/efi

5.3 - Mounting the partitions using mount

Only the swap partition should be mounted using a different command who is swapon. The other partitions will be mounted using the mount command.

IMPORTANT ! You need to create the mount points in a specific order, because the /boot/efi partition is mounted inside the /boot partition, and the /boot partition is mounted inside the $LFS directory. So we need to mount them in this order. So first we mount the root partition (/), then the boot partition (/boot), and finally the EFI System Partition (/boot/efi) (the swap partition will be mounted whenever needed).

mount -v -t ext4 /dev/sdb3 $LFS/
mount -v -t ext4 /dev/sdb2 $LFS/boot
mount -v -t vfat /dev/sdb1 -o codepage=437,iocharset=iso8859-1 $LFS/boot/efi
/sbin/swapon -v /dev/sdb4

6 - Checks before continue

From now, this is what you supposed to have:

cfdisk

cfdisk

lsblk

lsblk

If something is not like that, you need to check your steps and make sure everything is correct (retry from the beginning is more simple and quick).

7 - Sources and final preparation

Now we created the directory where all the sources will be downloaded, and create a limited directory layout for security and not destroy our VM.

7.1 - Create the sources directory

All our sources will be downloaded in the $LFS/sources directory, we download all the tarball here, extract them here, and compile them.

mkdir -pv $LFS/sources

Make this directory writable and sticky.

chmod -v a+wt $LFS/sources

Gerard Beekmans created a file where all the necessary sources will be downloaded, you can found them here with more explications. And download a file who check the integrity of the sources by comparing the checksums of the downloaded files with the checksums provided in the file.

wget --input-file=wget-list-systemd --continue --directory-prefix=$LFS/sources
wget --input-file=md5sums --continue --directory-prefix=$LFS/sources

And now, check the integrity of the downloaded sources:

pushd $LFS/sources
	md5sum -c md5sums
popd

If you downloaded all the sources with a non-root user, you need to change the ownership of the $LFS/sources directory to root:

chown root:root $LFS/sources/*

You can find the details of all the sources here: Linux From Scratch - Version r12.3-86-systemd Chapter 3. Packages. And all the needed patches here : Linux From Scratch - Version r12.3-86-systemd Chapter 3. Patches.

7.2 - Creating a Limited Directory Layout in the LFS Filesystem

Create the required directory layout by issuing the following commands as root:

mkdir -pv $LFS/{etc,var} $LFS/usr/{bin,lib,sbin}

for i in bin lib sbin; do
  ln -sv usr/$i $LFS/$i
done

case $(uname -m) in
  x86_64) mkdir -pv $LFS/lib64 ;;
esac

Later, some packets need to be cross-compiled, so we prepare the field for that part:

mkdir -pv $LFS/tools

7.3 - Creating the lfs user and group

This step is very not necessary, if you want to rush into the pile, do all in root and skip this step at your own risk and destroy your VM.

So we create a limited directory layout for the lfs user and group.

groupadd lfs
useradd -s /bin/bash -g lfs -m -k /dev/null lfs
passwd lfs

And grant lfs user full access to all the directories under $LFS by making lfs the owner:

chown -v lfs $LFS/{usr{,/*},var,etc,tools}
case $(uname -m) in
  x86_64) chown -v lfs $LFS/lib64 ;;
esac

Now log in as the lfs user:

su - lfs

Create a .bash_profile file in the home directory of the lfs user:

cat > ~/.bash_profile << "EOF"
exec env -i HOME=$HOME TERM=$TERM PS1='\u:\w\$ ' /bin/bash
EOF

And set all the necessary environment variables for the lfs user (for informations about these variables, see the LFS book : Linux From Scratch - Version r12.3-86-systemd Chapter 4. Final Preparations):

cat > ~/.bashrc << "EOF"
set +h
umask 022
LFS=/mnt/lfs
LC_ALL=POSIX
LFS_TGT=$(uname -m)-lfs-linux-gnu
PATH=/usr/bin
if [ ! -L /bin ]; then PATH=/bin:$PATH; fi
PATH=$LFS/tools/bin:$PATH
CONFIG_SITE=$LFS/usr/share/config.site
export LFS LC_ALL LFS_TGT PATH CONFIG_SITE
EOF

If you want to compile with a fixed number of logical cores (for example, 16 cores, you can compile withe flag -j[CORE_NBR] like make -j16), you can set the MAKEFLAGS variable:

export MAKEFLAGS=-j16

Or with all your logical cores:

export MAKEFLAGS=$(nproc)

Write this in your ~/.bashrc file to make it persistent, it's REALLY IMPORTANT. Then source the file to apply the changes:

cat >> ~/.bashrc << "EOF"
export MAKEFLAGS=-j$(nproc) # Or export MAKEFLAGS=-j16
EOF
source ~/.bashrc

7.4 - SBU, the unit measure

If you want to know about the unit of measurement to calculate the time of compilation of a package, you can read the Linux From Scratch - Version r12.3-86-systemd Chapter 4. Final Preparations.