cross compiler - bunnyamin/bunnix GitHub Wiki
In the context of Binutils and GCC:
- Architecture: CPU
- Build: The system that compiles the cross-compiler
- Host: The system that uses the cross-compiler to compile binaries
- Target: The system that executes the binaries compiled by the host
- Sys-root: A directory representation of the target system root
In the context of glibc:
- Build: The system that compiles the glibc.
- Host: The system that uses the compiled files.
Reference to AMD execution 64-bit state.
- The official term is AArch64, and is used in GNU projects (
aarch64
) - The ARM64 is more informal, and used by the Linux kernel source code (
arm64
)
The technical environment is expected to have a fully functional version of GCC. The other main components for building a cross-compiler:
- binutils
- gcc
- glibc headers
- linux kernel headers
There are different versions of each component and
- source codes can require specific versions of them
- specific versions of them can require a specific combinations of the other components.
The build-flow of the components:
- Build Linux headers - Required for step 3.
- Build Binutils
- Build preparatory GCC (bootstrap compiler)
- Use Binutils to build a minimal GCC-compiler for target system.
- Build preparatory
glibc
(bootstrap library)- Use the preparatory GCC-compiler binaries to build a preparatory C standard library files for target system.
- Build
libgcc
for cross-compiling- Use the preparatory C standard library files to build
libgcc
for target system.
- Use the preparatory C standard library files to build
- Build the
glibc
(complete)- Build the full C standard library
libc
for the target system with- the Binutils
- preparatory GCC
- preparatory
glibc
- the
libgcc
.
- Build the full C standard library
- Build GCC (complete)
- Use the full C standard library
libc
to build GCC.
- Use the full C standard library
An example of a directory structure:
linux/
+-- 5.10.235/
+-- .compile/arm64-defconfig/
+-- arm64-defconfig/
| +-- include/
+-- src/
gnu/
+-- binutils/
| +-- 2.36.1/
| +-- x86-64+generic+linux-6.x+glibc-2.41/
| +-- .compile/
| | +-- armv8.2-a-cortex-a76/
| +-- armv8.2-a-cortex-a76/
| +-- bin/
| +-- lib/
| | +-- bfd-plugins/
| | +-- ldscripts/
| +-- share/
| +-- src/
+-- gcc/
+-- 11.2.0/
+-- x86-64+generic+linux-6.x+glibc-2.41/
| +-- .compile/
| | +-- linux-5.10.235+glibc-2.33+armv8.2-a-cortex-a76/
| +-- linux-5.10.235+glibc-2.33+armv8.2-a-cortex-a76/
| +-- bootstrap/
| | +-- bin/
| | +-- include/ (symlink to usr/include)
| | +-- lib/ (symlink to usr/lib)
| | +-- lib64/ (symlink to usr/lib)
| | +-- libexec/
| | | +-- gcc/
| | | | +-- aarch64-linux-gnu/
| | | | +-- 11.2.0/
| | | +-- getconf/
| | +-- sbin/
| | +-- share/
| | +-- usr/
| | | +-- include/ (does not seem to be used)
| | | +-- lib/
| | +-- var/
| +-- sysroot
| +-- bin/
| +-- etc/
| +-- include/ (symlink to usr/include)
| +-- lib/ (symlink to usr/lib)
| +-- lib64/ (symlink to usr/lib)
| +-- libexec/
| | +-- gcc/
| | | +-- aarch64-linux-gnu/
| | | +--11.2.0/
| | +-- getconf/
| +-- sbin/
| +-- share/
| +-- usr/
| | +-- include/ (Linux kernel headers)
| | +-- lib/
| | +-- lib64/ (symlink to usr/lib)
| +-- var/
+-- src/
The main cause of errors I encountered when compiling the cross-compiler could be traced back to a failure to define paths to binaries and system root.
The consequence of such a failure resulted in the compilation throwing one of two types of errors:
Most other errors were resolved when the paths were correctly defined.
Note that the make
has been downgraded because of a bug when compiling glibc
.
- Basically, if glibc <2.41 then
make
<4.4
config_binutils-2.36.1+gcc-11.2.0_glibc-2.33+linux-5.10.235+aarch64.sh
ARCH_GNU='aarch64'
ARCH_LNX='arm64'
GNU_ROOT_DIR='/obj/g/gnu'
OS='linux'
OS_V='5.10.235'
OS_ROOT_DIR='/obj/l/linux'
OS_CFG='defconfig'
BINUTILS_V='2.36.1'
GCC_V='11.2.0'
GLIB_V='2.33'
MAKE_V='4.3'
# Define as "generic" if no specific ISA and,or CPU is the intended target.
# * ISA is more generic than CPU (specific implementation)
# * Cortex-a76 support since GCC 9
ISA='armv8.2-a'
CPU='cortex-a76'
# E.O echo $MACHTYPE -> 'x86_64-pc-linux-gnu'
# E.O echo $($GLIBC_SRC_DIR/scripts/config.guess) -> 'x86_64-pc-linux-gnu'
CHOST_LNX='6.x'
CHOST_GLIBC='2.41'
CHOST_ISA='x86-64'
CHOST_CPU='generic'
CHOST=$MACHTYPE
BHOST_ISA='x86-64'
BHOST_CPU='generic'
BIN_DIRNAME='bin'
INC_DIRNAME='include'
LIB_DIRNAME='lib'
LIB64_DIRNAME='lib64'
USR_DIRNAME='usr'
BIN_PATH="$BIN_DIRNAME/"
INC_PATH="$USR_DIRNAME/$INC_DIRNAME/"
LIB_PATH="$USR_DIRNAME/$LIB_DIRNAME/"
LIB64_PATH="$USR_DIRNAME/$LIB64_DIRNAME/"
USR_PATH="$USR_DIRNAME/"
A basic configuration without any pre-post tests or directory creation. For a more complete script:
TARGET="$ARCH_GNU-$OS-gnu"
KERNEL_HEADERS="$OS_ROOT_DIR/$OS_V/$ARCH_LNX-$OS_CFG/include/"
if [[ "$CPU" == 'generic' ]]; then
ARCH="${ISA}-generic"
else
ARCH="${ISA}-${CPU}"
fi
TARGET_OS="$OS-$OS_V+glibc-$GLIB_V"
CHOST_OS="linux-${CHOST_LNX}+glibc-${CHOST_GLIBC}"
CHOST_ARCH="${CHOST_ISA}+${CHOST_CPU}+${CHOST_OS}"
TOOLCHAIN_V="$TARGET_OS+$TARGET_ARCH"
TOOLCHAIN_PATH="$CHOST_ARCH/$TOOLCHAIN_V/"
MAKE="$GNU_ROOT_DIR/make/$MAKE_V/$ARCH_GNU/bin/make"
BUILDROOT_DIR='/'
BOOTSTRAP_PATH="$TOOLCHAIN_PATH/bootstrap/"
SYSROOT_PATH="$TOOLCHAIN_PATH/sysroot/"
GCC_BASE_DIR="$GNU_ROOT_DIR/gcc/$GCC_V/"
GCC_SRC_DIR="$GCC_BASE_DIR/src/"
GCC_COMP_DIR="$GCC_BASE_DIR/$CHOST_ARCH/.compile/$TOOLCHAIN_V/"
GCC_BOOTSTRAP_DIR="$GCC_BASE_DIR/$BOOTSTRAP_PATH"
GCC_BUILD_DIR="$GCC_BASE_DIR/$TOOLCHAIN_PATH"
GCC_BIN_DIR="$GCC_BUILD_DIR/bin"
BOOTSTRAP_DIR="$GCC_BASE_DIR/$BOOTSTRAP_PATH"
BOOTSTRAP_BIN_DIR="$BOOTSTRAP_DIR/bin/"
BOOTSTRAP_USR_DIR="$BOOTSTRAP/$USR_PATH/"
BOOTSTRAP_INC_DIR="$BOOTSTRAP_DIR/$INC_PATH/"
BOOTSTRAP_LIB_DIR="$BOOTSTRAP_DIR/$LIB_PATH/"
BOOTSTRAP_LIB64_DIR="$BOOTSTRAP_DIR/$LIB64_PATH/"
SYSROOT_DIR="$GCC_BASE_DIR/$SYSROOT_PATH"
SYSROOT_BIN_DIR="$SYSROOT_DIR/$BIN_PATH/"
SYSROOT_USR_DIR="$SYSROOT_DIR/$USR_PATH/"
SYSROOT_INC_DIR="$SYSROOT_DIR/$INC_PATH/"
SYSROOT_LIB_DIR="$SYSROOT_DIR/$LIB_PATH/"
SYSROOT_LIB64_DIR="$SYSROOT_DIR/$LIB64_PATH/"
BINUTILS_BASE_DIR="$GNU_ROOT_DIR/binutils/$BINUTILS_V/"
BINUTILS_SRC_DIR="$BINUTILS_BASE_DIR/src/"
BINUTILS_COMP_DIR="$BINUTILS_BASE_DIR/$CHOST_ARCH/.compile/$TARGET_ARCH/"
BINUTILS_BUILD_DIR="$BINUTILS_BASE_DIR/$CHOST_ARCH/$TARGET_ARCH/"
BINUTILS_BIN_DIR="$BINUTILS_BUILD_DIR/bin/"
GLIBC_BASE_DIR="$GNU_ROOT_DIR/glibc/$GLIB_V/"
GLIBC_SRC_DIR="$GLIBC_BASE_DIR/src/"
GLIBC_COMP_DIR="$GLIBC_BASE_DIR/$CHOST_ARCH/.compile/$TARGET_ARCH/"
GLIBC_BUILD_DIR="$BOOTSTRAP_DIR/"
The Linux headers must be built for target architecture. For example aarch64 (arm64).
- A cross compiler is not required; the available GNU compiler for the host used to build the cross compiler should suffice.
- A configuration is required; the
defconfig
should suffice.
A basic build without any pre-post tests or directory creation.
#!/bin/bash
LNX_VERSION='5.10.235'
LNX_ROOT_DIR='/obj/l/linux/'
LNX_BASE_DIR="$LNX_ROOT_DIR/$lnx_version/"
LNX_BUILD_DIR="$LNX_BASE_DIR/$ARCH_LNX/"
LNX_COMP_DIR="$LNX_BASE_DIR/.compile/$ARCH_LNX/"
LNX_SRC_DIR="$LNX_BASE_DIR/src/"
$MAKE defconfig
HEADERS_DIR="$LNX_BUILD_DIR/"
make \
O="$LNX_COMP_DIR" \
ARCH="$ARCH_LNX" \
INSTALL_HDR_PATH="$KERNEL_HEADERS" headers_install
I could not find any other official documentation than the "brief manual".
A basic configuration without any pre-post tests or directory creation. For a more complete script:
$BINUTILS_SRC_DIR/configure \
--srcdir="$BINUTILS_SRC_DIR" \
--prefix="$BINUTILS_BUILD_DIR" \
--build="$MACHTYPE" \
--host="$CHOST" \
--target="$TARGET" \
--with-arch="$ISA" \
--with-cpu="$SUPPORTED_CPU" \
--with-sysroot="$SYSROOT_DIR" \
--with-build-sysroot="$BUILDROOT_DIR" \
--disable-multilib \
--disable-nls \
--disable-shared \
--disable-werror
$MAKE \
tooldir=$BINUTILS_BUILD_DIR \
all install
For a list of documented parameters:
$BINUTILS_SRC_DIR/configure --help
- GCC
The example documents those parameters used to compile a working cross-compiler.
-
Note the absence of the
configure
parameter--root-sysroot
. -
--build="$MACHTYPE"
defines the architecture on which the compilation is done. -
--host="$CHOST"
defines the architecture on which the cross-compiler is executed. -
--target="$TARGET"
defines the architecture for which the cross-compiler is used to compile binaries. -
--with-arch
defines the supported generic CPU architecture for which the cross-compiler is used to compile binaries; at leastaarch64
, verify more specific ARM-architectures such asarmv8.2-a
. -
--with-cpu
defines the supported specific CPU architecture for which the cross-compiler is used to compile binaries; for exampleCortex-a76
support since GCC 9. -
--with-sysroot
defines the system root for toolchain. -
--disable-multilib
disables support for executing binaries for other CPU architectures on the target CPU architecture. -
--disable-nls
disables NLS (native language support) for GCC output diagnostics for any other human scripts than American English. -
--disable-shared
disables build of shared libraries; binutils is built only with static libraries. -
--disable-werror
disables warning errors.
Parameters encountered in other documentation.
-
--disable-bootstrap
Could not find any documentation. -
--enable-gold=default
enables "gold" linker and defines it as the default linker; it should not be used: The gold linker is deprecated -
--enable-interwork
Could not find any documentation.
The parameter --with-sysroot="$SYSROOT_DIR"
locks the build to a specific toolchain.
If the parameter is not defined then it should be possible to build binutils
independent of specific toolchain combination of GCC and glibc
.
- Emphasizes because I have not confirmed it; given the problems with defining the search paths I installed everything together.
The system root can then be defined with variables. For example:
- Specify system root in environment variables
AS="$BINUTILS_BUILD_DIR/bin/aarch64-linux-gnu-as --sysroot=$SYSROOT_DIR"
LD="$BINUTILS_BUILD_DIR/bin/aarch64-linux-gnu-ld --sysroot=$SYSROOT_DIR"
The tooldir
can be used to define the path to the architecture specific
directory. For example, the directory aarch64-linux-gnu
is placed as default:
binutils/
+-- aarch64-linux-gnu/
+-- aarch64-linux-gnu <-- !
| +-- bin/
| +-- lib/
+-- bin/
+-- share/
If tooldir=/aarch64-linux-gnu
then:
binutils/
+-- aarch64-linux-gnu/ <-- !
+-- bin/
+-- lib/
+-- share/
If the directories are merged then the "bin" directory will contained both the files named
- with architecture specific prefix
aarch64-linux-gnu-*
, for exampleaarch64-linux-gnu-gcc
andaarch64-linux-gnu-ld
, and - without architecture specific prefix, for example
gcc
andld
.
This causes a problem during cross-compilation of the Linux kernel:
- The path variable is required to be
GCC_BIN_DIR:$BINUTILS_BIN_DIR:$PATH
- The binaries required for the host first searches the
GCC_BIN_DIR
, then$BINUTILS_BUILD_DIR
- Normally the binaries required for the host would not be found and the search
moves on to find the correct binaries in
$PATH
- However, because the generic binary file names exist in the
$BINUTILS_BIN_DIR
they will be used, causing errors such asas: unrecognized option '--64'
The implemented solution is to define path as GCC_BIN_DIR:$PATH
and create
symbolic links from aarch64-linux-gnu-*
in $BINUTILS_BIN_DIR
to GCC_BIN_DIR
.
Use binutils to build target compiler. The host and build compilers should be the current system.
A basic configuration without any pre-post tests or directory creation. For a more complete script:
$GCC_SRC_DIR/configure \
--srcdir="$GCC_SRC_DIR" \
--prefix="$GCC_BOOTSTRAP_DIR" \
--build="$MACHTYPE" \
--host="$CHOST" \
--target="$TARGET" \
--with-sysroot="$SYSROOT_DIR" \
--with-glibc-version="$GLIB_V" \
--with-arch="$ISA" \
--with-cpu="$SUPPORTED_CPU" \
--with-gnu-as \
--with-as="$BINUTILS_BIN_DIR/$TARGET-as" \
--with-gnu-ld \
--with-ld="$BINUTILS_BIN_DIR/$TARGET-ld" \
--with-newlib \
--enable-checking=release \
--enable-default-pie \
--enable-languages=c,c++ \
--enable-libbacktrace \
--enable-tls \
--enable-static \
--enable-threads='posix' \
--disable-default-ssp \
--disable-libatomic \
--disable-libsanitizer \
--disable-libstdcxx \
--disable-libgomp \
--disable-libquadmath \
--disable-libssp \
--disable-libvtv \
--disable-profile \
--disable-lto \
--disable-multiarch \
--disable-multilib \
--disable-nls \
--disable-profile \
--disable-shared \
--without-headers
$MAKE --debug=v \
all-gcc install-gcc
For a list of documented parameters:
Note:
- If disabling c++ then all c++ dependencies must be disabled otherwise errors will occur during the 5. Build glibc (complete). For example:
aarch64-linux-gnu-ld: cannot find -lstdc++
aarch64-linux-gnu-ld: cannot find -lgcc_s
collect2: error: ld returned 1 exit status
- GNU C Library Reference Manual (latest) C.1 Configuring and compiling the GNU C Library
Use the GCC binaries to build libgcc for the target architecture.
A basic configuration without any pre-post tests or directory creation. For a more complete script:
Recommendation:
- Test for existence of Linux header files before configuration.
export PATH="$BOOTSTRAP_BIN_DIR:$PATH"
$GLIBC_SRC_DIR/configure \
--srcdir="$GLIBC_SRC_DIR" \
--prefix="$SYSROOT_DIR" \
--build="$MACHTYPE" \
--host="$TARGET" \
--with-binutils="$BINUTILS_BIN_DIR" \
--with-headers="$SYSROOT_INC_DIR" \
--enable-kernel="$OS_V" \
--enable-shared \
--enable-static \
--enable-threads='posix' \
--enable-tls \
--enable-default-pie \
--disable-profile \
--disable-multiarch \
--disable-multilib \
--disable-werror \
libc_cv_forced_unwind=yes \
libc_cv_c_cleanup=yes
$MAKE --debug=v \
install_root="$SYSROOT_DIR" prefix="/" \
install-bootstrap-headers=yes \
install-headers
$MAKE csu/subdir_lib
install csu/crt1.o csu/crti.o csu/crtn.o csu/Scrt1.o "$SYSROOT_LIB_DIR"
touch "$SYSROOT_INC_DIR/gnu/stubs.h"
$BINUTILS_BIN_DIR/$TARGET-gcc \
--sysroot="$SYSROOT_DIR" \
-B "$BINUTILS_BIN_DIR" \
-shared \
-nostdlib \
-nostartfiles \
-x c /dev/null \
-o "$SYSROOT_LIB_DIR/libc.so"
For a list of documented parameters:
$GLIBC_SRC_DIR/configure --help
- See also GCC
Use the glibc libraries to build libgcc for the target architecture.
A basic configuration without any pre-post tests or directory creation. For a more complete script:
$MAKE --debug=v \
C_INCLUDE_PATH="$SYSROOT_INC_DIR" \
all-target-libgcc \
install-target-libgcc
Building glibc for the target with the binutils and GCC binaries compiled for the target.
A basic configuration without any pre-post tests or directory creation. For a more complete script:
export PATH="$BOOTSTRAP_BIN_DIR:$PATH"
$GLIBC_SRC_DIR/configure \
--srcdir="$GLIBC_SRC_DIR" \
--prefix="/" \
--libdir="/$LIB_PATH" \
--includedir="/$INC_PATH" \
--build="$MACHTYPE" \
--host="$TARGET" \
--with-binutils="$BINUTILS_BIN_DIR" \
--with-headers="$SYSROOT_INC_DIR" \
--enable-kernel="$OS_V" \
--enable-add-ons \
--enable-default-pie \
--enable-shared \
--enable-static \
--enable-threads='posix' \
--enable-tls \
--disable-debug \
--disable-profile \
--disable-multiarch \
--disable-multilib \
--disable-werror \
libc_cv_forced_unwind=yes \
libc_cv_c_cleanup=yes
$MAKE --debug=v \
DESTDIR="$SYSROOT_DIR" \
all install
For a list of documented parameters:
$GLIBC_SRC_DIR/configure --help
- See also GCC
Use the complete glibc to build GCC for the target architecture.
A basic configuration without any pre-post tests or directory creation. For a more complete script:
# Note that host and build path comes before bootstrap path.
export PATH=$PATH:$BOOTSTRAP_BIN_DIR
$GCC_SRC_DIR/configure \
AS_FOR_TARGET="$BINUTILS_BIN_DIR/$TARGET-as" \
LD_FOR_TARGET="$BINUTILS_BIN_DIR/$TARGET-ld" \
CC_FOR_TARGET="$BOOTSTRAP_BIN_DIR/$TARGET-gcc" \
CXX_FOR_TARGET="$BOOTSTRAP_BIN_DIR/$TARGET-g++" \
--srcdir="$GCC_SRC_DIR" \
--prefix="$GCC_BUILD_DIR" \
--build="$MACHTYPE" \
--host="$CHOST" \
--target="$TARGET" \
--with-sysroot="$SYSROOT_DIR" \
--with-build-sysroot="$SYSROOT_DIR" \
--with-native-system-header-dir="/$INC_PATH" \
--with-arch="$ISA" \
--with-cpu="$SUPPORTED_CPU" \
--with-gnu-as \
--with-as="$BINUTILS_BIN_DIR/$TARGET-as" \
--with-gnu-ld \
--with-ld="$BINUTILS_BIN_DIR/$TARGET-ld" \
--enable-languages=c,c++ \
--enable-deterministic-build \
--enable-shared \
--enable-threads='posix' \
--disable-multiarch \
--disable-multilib \
--disable-werror
make --debug=v all install
For a list of documented parameters:
Debugging can be difficult as an incorrect setting at an early stage may not cause an error until a later stage.
The GGC C++ compiler (or "driver") can be invoked with aarch64-linux-gnu-c++
or aarch64-linux-gnu-g++
.
- As far as I can tell, they are equivalent. For example, their SHA-2 256 digest is the same.
The error is not caused by the compiler but could occur during compilation if the temporary directory is full.
- Verify that the
/tmp
mount is not at 100 % usedf -h
.
checking for suffix of object files... configure: error: in `gnu/glibc/2.31/compile-linux-5.5-aarch64':
configure: error: cannot compute suffix of object files: cannot compile
See `config.log' for more details
The CCFLAG
has the invalid value of -02
(zero two) instead of -O2
(O-two).
Errors such as
[LD_LIBRARY_PATH, LIBRARY_PATH, PATH] shouldn't contain the directory that glibc is
installed to...
occurs if an empty path exists in environment variable. For example
-
/lib:
should be/lib
.
In file included from ../sysdeps/unix/sysv/linux/x86_64/sysdep.h:24,
from <stdin>:1:
../sysdeps/x86_64/nptl/tls.h:23:11: fatal error: asm/prctl.h: No such file or directory
23 | # include <asm/prctl.h> /* For ARCH_SET_FS. */
| ^~~~~~~~~~~~~
compilation terminated.
The compilation of glibc is using x86_64-pc-linux-gcc
instead of aarch64-pc-linux-gcc
.
In file included from ../sysdeps/aarch64/nptl/tls.h:37:
../sysdeps/unix/sysv/linux/aarch64/sysdep.h:203:17: error: invalid register name for โ_x0โ
203 | register long _x0 asm ("x0");
...
../sysdeps/unix/sysv/linux/aarch64/sysdep.h:180:22: error: invalid register name for โ_x8โ
180 | register long _x8 asm ("x8") = (name);
The compilation of glibc is not using aarch64-pc-linux-gcc
.
The compilation of glibc
results in an endless re-execution. For example:
...
Re-executing[18482]: make subdir=stdio-common -C stdio-common ..=../ others
...
Re-executing[18483]: make subdir=stdio-common -C stdio-common ..=../ others
...
Re-executing[18484]: make subdir=stdio-common -C stdio-common ..=../ others
...
The remedy is to either downgrade make
or upgrade glibc
. For example,
glibc
2.31 (2020-02-01) and
- make 4.3 (2020-01-09) works, while
- make 4.4.1 (2020-01-09) results in endless "re-executing"
The problem seems to arise with make
>=4.4 (2022-10-31) and GLIBC <2.41.