Link to of where I am in the process - III. Building the LFS Cross Toolchain and Temporary Tools

It's really important to know what is cross-compilation. In this section, we will build a cross toolchain that will be used to compile the LFS system. This toolchain will allow us to compile software for the LFS system while running on the host system (Ubuntu in our case).

I redirect you to the LFS book for more details on the cross toolchain and temporary tools it's explained in detail there : Linux From Scratch - Version r12.3-86-systemd Important Preliminary Material

In this section, I will quotes Gerard Beekmans directly from the LFS book several times when it's necessary, as he explains very well what we are going to do.

"This chapter shows how to build a cross-compiler and its associated tools. Although here cross-compilation is faked, the principles are the same as for a real cross-toolchain. The libraries, on the other hand, are installed into their final place, since they pertain to the system we want to build." Gerard Beekmans, LFS book.

Before continuing, check these important points:

• bash is the shell in use.
• sh is a symbolic link to bash.
• /usr/bin/awk is a symbolic link to gawk.
• /usr/bin/yacc is a symbolic link to bison, or to a small script that executes bison.
• The environment variable LFS and LFS_TGT are set correctly.
• Ensure that all the sources tarballs are in the /mnt/lfs/sources/ directory.
• The directory /mnt/lfs/tools/ exists and is empty.

You have 2 options here :

• Option 1: Use the LFS book to build the cross toolchain. This is the recommended way as it ensures that you follow the steps correctly and have a working toolchain. Linux From Scratch - Version r12.3-86-systemd Important Preliminary Material At this link you found all the general instructions for compilation on your own. It says you change to the /mnt/lfs/sources/ and for each package you will compile following these steps:

  1. Using the tar program, extract the package to be built
  2. Change to the directory created when the package was extracted
  3. Follow the instructions for building the package
  4. Change back to the sources directory when the build is complete
  5. Delete the extracted source directory unless instructed otherwise

• Option 2: Use the script provided in this repository to build the cross toolchain. This is a simplified version of the LFS book instructions and will automate the process for you. The script will handle the extraction, building, and installation of the cross toolchain packages BUT you need to confirm to continue before processing each package. This is useful for checking any errors during the compilation process. The script is located in the scripts/ directory of this repository : compiling_cross-toolchain.sh. You can run it with the following command:

curl -sL https://raw.githubusercontent.com/Nimpoo/ft_linux/refs/heads/main/scripts/packages_compilation/compiling_cross-toolchain.sh | bash

After assuring that the cross toolchain is built correctly, we can proceed to the next step of building the temporary tools.

"This chapter shows how to cross-compile basic utilities using the just built cross-toolchain. Those utilities are installed into their final location, but cannot be used yet. Basic tasks still rely on the host's tools. Nevertheless, the installed libraries are used when linking.

Using the utilities will be possible in the next chapter after entering the “chroot” environment. But all the packages built in the present chapter need to be built before we do that. Therefore we cannot be independent of the host system yet.

Once again, let us recall that improper setting of LFS together with building as root, may render your computer unusable. This whole chapter must be done as user lfs, with the environment." Gerard Beekmans, LFS book.

Same here, 2 options:

• Option 1: Use the LFS book to build the temporary tools. This is the recommended way as it ensures that you follow the steps correctly and have a working temporary tools. Again, process to decompile as described before Linux From Scratch - Version r12.3-86-systemd Chapter 6. Cross Compiling Temporary Tools

• Option 2: Use my script that works like the previous one, it will automate the process for you and you confirm before continue. The script is located in the scripts/ directory of this repository : cross_compiling_temporary_tools-tools.sh

Last reminder before compiling anything :

• Ensure that the environment variable LFS is set correctly to /mnt/lfs.
• Ensure that the environment variable LFS_TGT is set correctly to your target architecture

echo $LFS
echo $LFS_TGT

You can run it with the following command:

curl -sL https://raw.githubusercontent.com/Nimpoo/ft_linux/refs/heads/main/scripts/packages_compilation/cross_compiling_temporary_tools.sh | bash

Ensure that all compiled packages are installed correctly and that the environment variables are set correctly. After this step, we will be able to use the temporary tools in the next chapter, If you have any issues, please refer to the LFS book for troubleshooting (or retry from the beginning if it's very critical).

"This chapter shows how to build the last missing bits of the temporary system: the tools needed to build the various packages. Now that all circular dependencies have been resolved, a “chroot” environment, completely isolated from the host operating system (except for the running kernel), can be used for the build.

For proper operation of the isolated environment, some communication with the running kernel must be established. This is done via the so-called Virtual Kernel File Systems, which will be mounted before entering the chroot environment. You may want to verify that they are mounted by issuing the findmnt command.

Until Section “Entering the Chroot Environment”, the commands must be run as root, with the LFS variable set. After entering chroot, all commands are run as root, fortunately without access to the OS of the computer you built LFS on. Be careful anyway, as it is easy to destroy the whole LFS system with bad commands." Gerard Beekmans, LFS book.

NOW, the commands must be performed while logged in as user root and no longer as user lfs. Also, double check that $LFS and $LFS_TGT is set in root's environment.

First of all, change the ownership of $LFS/* directories to user root by running the following command:

chown --from lfs -R root:root $LFS/{usr,var,etc,tools}
case $(uname -m) in
  x86_64) chown --from lfs -R root:root $LFS/lib64 ;;
esac

"Applications running in userspace utilize various file systems created by the kernel to communicate with the kernel itself. These file systems are virtual: no disk space is used for them. The content of these file systems resides in memory. These file systems must be mounted in the $LFS directory tree so the applications can find them in the chroot environment." Gerard Beekmans, LFS book.

As Gerard Beekmans said, we need to mount the virtual kernel file systems in the $LFS directory tree. Run the following commands as root:

mkdir -pv $LFS/{dev,proc,sys,run}

"During a normal boot of an LFS system, the kernel automatically mounts the devtmpfs file system on the /dev directory; the kernel creates device nodes on that virtual file system during the boot process, or when a device is first detected or accessed. The udev daemon may change the ownership or permissions of the device nodes created by the kernel, and create new device nodes or symlinks, to ease the work of distro maintainers and system administrators. (See Section 9.3.2.2, “Device Node Creation” for details.) If the host kernel supports devtmpfs, we can simply mount a devtmpfs at $LFS/dev and rely on the kernel to populate it.

But some host kernels lack devtmpfs support; these host distros use different methods to create the content of /dev. So the only host-agnostic way to populate the $LFS/dev directory is by bind mounting the host system's /dev directory. A bind mount is a special type of mount that makes a directory subtree or a file visible at some other location. Use the following command to do this." Gerard Beekmans, LFS book.

mount -v --bind /dev $LFS/dev

And now, we can mount the remaining virtual kernel file systems:

mount -vt devpts devpts -o gid=5,mode=0620 $LFS/dev/pts
mount -vt proc proc $LFS/proc
mount -vt sysfs sysfs $LFS/sys
mount -vt tmpfs tmpfs $LFS/run

• gid=5: "This ensures that all devpts-created device nodes are owned by group ID 5. This is the ID we will use later on for the tty group. We use the group ID instead of a name, since the host system might use a different ID for its tty group."
• mode=0620: "This ensures that the device nodes created in the devpts file system are readable and writable by the owner and group, but not accessible to other users."

"In some host systems, /dev/shm is a symbolic link to run/shm. The /run tmpfs was mounted above so in this case only a directory needs to be created with the correct permissions. In other host systems /dev/shm is a mount point for a tmpfs. In that case the mount of /dev above will only create /dev/shm as a directory in the chroot environment. In this situation we must explicitly mount a tmpfs."

if [ -h $LFS/dev/shm ]; then
  install -v -d -m 1777 $LFS$(realpath /dev/shm)
else
  mount -vt tmpfs -o nosuid,nodev tmpfs $LFS/dev/shm
fi

What is a chroot environment ? You can begin by this : chroot(2).

chroot for "change root" is a command to create a chroot environment. This environment can be used and host a separate virtualized copy of the software system. It can be used for several purposes: testing, dependency control, security, recovery, etc... chroot is powerful because it allows changing the root directory of the current running process and its children. This means that the process will only see the files and directories under the new root directory, effectively isolating it from the rest of the system. With that, we can litteraly change our system.

By creating partitions, mount points, filesystems, and mounting them, we can create a new system that is completely isolated from the host system: it's called the chroot jail. It's ensure we install the packages in the right place with the right filesystem of our LFS and permissions ownerships, this is why we need to enter the chroot environment.

Now, it's time to enter the chroot environment, in our chroot jail. Run the following command as root:

chroot "$LFS" /usr/bin/env -i   \
    HOME=/root                  \
    TERM="$TERM"                \
    PS1='(lfs chroot) \u:\w\$ ' \
    PATH=/usr/bin:/usr/sbin     \
    MAKEFLAGS="-j$(nproc)"      \
    TESTSUITEFLAGS="-j$(nproc)" \
    /bin/bash --login

The flag -i is used to clear all the environment variables and create a new environment with only the specified variables.

Remove MAKEFLAGS and TESTSUITEFLAGS if you don't want to use parallel compilation.

Now if you understand, you don't need the environment variable LFS anymore, as you are now in the chroot environment where $LFS is set to / (the root directory of the chroot jail).

If the bash prompt say "I have no name!", it means that the chroot environment is working correctly, and it's normal if "you have no name", it's because the /etc/passwd file has not been created yet.

If you enter exit in the chroot environment or go out of the chroot environment, you can enter the previous command to enter the chroot environment again. But if you reboot your system, you will need to remount the virtual kernel file systems and enter the chroot environment again.

From now, i consider that you are in the chroot environment.

Create some root-level directories that are not in the limited set required in the previous chapters by issuing the following command:

mkdir -pv /{boot,home,mnt,opt,srv}

And create the required set of subdirectories below the root-level by issuing the following commands:

mkdir -pv /etc/{opt,sysconfig}
mkdir -pv /lib/firmware
mkdir -pv /media/{floppy,cdrom}
mkdir -pv /usr/{,local/}{include,src}
mkdir -pv /usr/lib/locale
mkdir -pv /usr/local/{bin,lib,sbin}
mkdir -pv /usr/{,local/}share/{color,dict,doc,info,locale,man}
mkdir -pv /usr/{,local/}share/{misc,terminfo,zoneinfo}
mkdir -pv /usr/{,local/}share/man/man{1..8}
mkdir -pv /var/{cache,local,log,mail,opt,spool}
mkdir -pv /var/lib/{color,misc,locate}

ln -sfv /run /var/run
ln -sfv /run/lock /var/lock

install -dv -m 0750 /root
install -dv -m 1777 /tmp /var/tmp

"This directory tree is based on the Filesystem Hierarchy Standard (FHS) (available at https://refspecs.linuxfoundation.org/fhs.shtml). The FHS also specifies the optional existence of additional directories such as /usr/local/games and /usr/share/games. In LFS, we create only the directories that are really necessary. However, feel free to create more directories, if you wish." Gerard Beekmans, LFS book.

"Historically, Linux maintained a list of the mounted file systems in the file /etc/mtab. Modern kernels maintain this list internally and expose it to the user via the /proc filesystem." Gerard Beekmans, LFS book.

To satisfy utilities that expect to find /etc/mtab, create the following symbolic link:

ln -sv /proc/self/mounts /etc/mtab

Create a basic /etc/hosts file:

cat > /etc/hosts << EOF
127.0.0.1  localhost $(hostname)
::1        localhost
EOF

Create the /etc/passwd file by running the following command:

cat > /etc/passwd << "EOF"
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/dev/null:/usr/bin/false
daemon:x:6:6:Daemon User:/dev/null:/usr/bin/false
messagebus:x:18:18:D-Bus Message Daemon User:/run/dbus:/usr/bin/false
systemd-journal-gateway:x:73:73:systemd Journal Gateway:/:/usr/bin/false
systemd-journal-remote:x:74:74:systemd Journal Remote:/:/usr/bin/false
systemd-journal-upload:x:75:75:systemd Journal Upload:/:/usr/bin/false
systemd-network:x:76:76:systemd Network Management:/:/usr/bin/false
systemd-resolve:x:77:77:systemd Resolver:/:/usr/bin/false
systemd-timesync:x:78:78:systemd Time Synchronization:/:/usr/bin/false
systemd-coredump:x:79:79:systemd Core Dumper:/:/usr/bin/false
uuidd:x:80:80:UUID Generation Daemon User:/dev/null:/usr/bin/false
systemd-oom:x:81:81:systemd Out Of Memory Daemon:/:/usr/bin/false
nobody:x:65534:65534:Unprivileged User:/dev/null:/usr/bin/false
EOF

The actual password for root will be set later.

Create the /etc/group file by running the following command:

cat > /etc/group << "EOF"
root:x:0:
bin:x:1:daemon
sys:x:2:
kmem:x:3:
tape:x:4:
tty:x:5:
daemon:x:6:
floppy:x:7:
disk:x:8:
lp:x:9:
dialout:x:10:
audio:x:11:
video:x:12:
utmp:x:13:
cdrom:x:15:
adm:x:16:
messagebus:x:18:
systemd-journal:x:23:
input:x:24:
mail:x:34:
kvm:x:61:
systemd-journal-gateway:x:73:
systemd-journal-remote:x:74:
systemd-journal-upload:x:75:
systemd-network:x:76:
systemd-resolve:x:77:
systemd-timesync:x:78:
systemd-coredump:x:79:
uuidd:x:80:
systemd-oom:x:81:
wheel:x:97:
users:x:999:
nogroup:x:65534:
EOF

Later, some tests require a regular user, let's create it:

echo "tester:x:101:101::/home/tester:/bin/bash" >> /etc/passwd
echo "tester:x:101:" >> /etc/group
install -o tester -d /home/tester

And for remove the "I have no name!" prompt, restart a new shell, the /etc/passwd file is now created:

exec /usr/bin/bash --login

Some programs required a log file. If they aren't present, the programs never create them and write to them, so let's create them:

touch /var/log/{btmp,lastlog,faillog,wtmp}
chgrp -v utmp /var/log/lastlog
chmod -v 664  /var/log/lastlog
chmod -v 600  /var/log/btmp

Compilation time again ! 2 options again:

• Option 1: Use the LFS book to build the additional temporary tools. It begin here : Linux From Scratch - Version r12.3-95-systemd Chapter 7. Entering Chroot and Building Additional Temporary Tools . Don't forget to follow the instructions for each package, and to change back to the /mnt/lfs/sources/ directory after each package is compiled (/sources here because you'r in chroot environment).

• Option 2: Use my script that works like the previous one, it will automate the process for you and you confirm before continue. The script is located in the scripts/ directory of this repository : adding_temporary_tools.sh. And you can't run it because you curl is not installed in your LFS.

For each package, the explanation of how the binary configure is run is given in the LFS book.

After successfully building all the additional temporary tools, you can clean up the sources directory (ensure all the packages are installed correctly), create a backup of your current LFS system state, and restore it later if needed. The backup is optional but very useful if you encounter issues later in the process.

Remove the currently installed documentation files :

rm -rf /usr/share/{info,man,doc}/*

Remove the libtool .la files are only useful for libltdl. No libraries in LFS are loaded by libltdl, and it's known that some .la files can cause BLFS package failures.

find /usr/{lib,libexec} -name \*.la -delete

And remove all the /tools directory, as it is no longer needed, you free about 1G of space:

rm -rf /tools

"At this point the essential programs and libraries have been created and your current LFS system is in a good state. Your system can now be backed up for later reuse. In case of fatal failures in the subsequent chapters, it often turns out that removing everything and starting over (more carefully) is the best way to recover. Unfortunately, all the temporary files will be removed, too. To avoid spending extra time to redo something which has been done successfully, creating a backup of the current LFS system may prove useful." Gerard Beekmans, LFS book.

If you're sure that the packages are installed correctly, you can create a backup of your current LFS system state. Obviously, create it only if you want, it's optional.

First you can exit from the chroot environment:

exit

From now, you need to run the commands as root, check the following environment variables and umask

echo $LFS
echo $LFS_TGT
umask

umask should be 0022, if it's not, run the following command:

umask 0022

After ensuring all that (for not destroy the VM), unmount the virtual kernel file systems:

mountpoint -q $LFS/dev/shm && umount $LFS/dev/shm
umount $LFS/dev/pts
umount $LFS/{sys,proc,run,dev}

And create the tarball of your current LFS system state:

cd $LFS
tar -cJpf $HOME/lfs-temp-tools-r12.3-95-systemd.tar.xz .

You can count 10 minutes for the tarball creation.

echo $LFS
echo $LFS_TGT
umask

umask should be 0022, if it's not, run the following command:

umask 0022

After ensuring all that, you can restore the backup by running the following commands:

cd $LFS
rm -rf ./*
tar -xpf $HOME/lfs-temp-tools-r12.3-95-systemd.tar.xz

And your LFS system state is restored. Don't hesitate to create other backup during the next chapters, it will very long to install all the needed packages.

And let's go for the next chapter, the longer one. Check of all your packages are installed correctly, if one is not, you can encounter issues that are horrible to debug, and always finished by a "ok let's start over". So ensure that all the packages are installed correctly, your backup is intact if an error occurs, and you can proceed to the next chapter.