This guide documents the complete steps to create rootfs/sysroot so that you can create a cross compile environment for any Raspberry Pi(running 64-bit OS such as Raspbian Buster 64-bit OS) on a Linux machine using rsync from the Raspberry Pi hardware you're already running. After that we will cross-compile a working Software Binaries (Hello-World CMAKE example in this case) with CMAKE using only the Raspberry Pi GCC Toolchains available within our project.

• Host [PC/Laptop]: Any x86/x86_64 AMD/Intel machine
• Target [Raspberry Pi]: Raspberry Pi any variant/module

• Host: Any Linux machine (💡 Our Cross-Compiler toolchains also works out-of-the-box on any Linux distro via WSL2 on Windows 10 Machines. 💯)
• Target: Any Raspberry Pi Linux 64-bit OS (Tested raspios_lite_arm64-2021-04-09)

• Networking: Your Target Machine (Raspberry Pi) and Host Machine (where you cross-compiling) both MUST have Internet Access, and MUST be on SAME Network to follow these instructions.

⚠️ Important: If you just brought a new Raspberry Pi or wanted to start from scratch just follow along. Otherwise, if you already has your Raspberry Pi setup, running, and Network Ready, then just skip to step 2.

Note: This section assume you have atleast 10GB SDcard for installing Raspbian Buster OS and a Laptop/PC for uploading it.

• Download the latest version of Raspbian (Buster) from here on your laptop/pc.
• You will be needing an image writer to write the downloaded OS into the SD card (micro SD card in our case). So download the open-source "win32 disk imager" from here, OR you can also use Balena Etcher instead.
• Insert the SD card into the laptop/pc and run the image writer. Once open, browse and select the downloaded Raspbian image file. Select the correct device, that is the drive representing the SD card. Note: If the drive (or device) selected is different from the SD card then the other selected drive will become corrupted. SO BE CAREFUL!
• Once the write is complete, eject the SD card and insert it into the Raspberry Pi and turn it on. It should start booting up.
• Please remember that after booting the Pi, there might be situations when the user credentials like the "username" and password will be asked. Raspberry Pi comes with a default username pi and password raspberry and so use it whenever it is being asked.

Now the you have your Raspberry Pi up and Running, its time to connect it your network with one of following ways:

• If you have Monitor: Connect it to your raspberry pi along with a keyboard and mouse to navigate, and follow this guide.
• If you don't have Monitor: Follow this guide
• Any other way

• If you have Monitor: On the Raspberry Pi terminal, type: sudo raspi-config and menu should pop up on your terminal. To enable SSH, go to: Interfacing Options -> SSH -> Yes, and Click OK to enable it. Choose Finish finally and exit.

• If you don't have Monitor: After setting up the network, if you don't have monitor or you operating it remotely. Then, enable SSH by just taking out your SD card, and hook it your computer, and simply create an empty file called ssh in the /boot/parition path inside SD card. Now insert back SD card into the Raspberry Pi.

• From another Laptop/PC using SSH: To connect to your Pi from a different computer, copy and paste the following command into the terminal window but replace 192.160.1.47 with the IP address of the Raspberry Pi. Use Ctrl + Shift + V to paste in the terminal.

  ssh pi@192.168.1.47 

  It will ask for password, and if not changed, it is default (raspberry), and so use it whenever it is being asked.

  Note: It is possible to configure your Raspberry Pi to allow access from another computer without needing to provide a password each time you connect. For more details, see here.

• On Raspberry Pi directly with a Monitor: Just search "Terminal" and click on it.

You need to edit your sources list to enable development sources. To do this, enter the following command into pi terminal:

sudo nano /etc/apt/sources.list

In the nano text editor, uncomment the following line by removing the # character from following line:

deb-src http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi

When done, press Ctrl+O and then ENTER to quit.

Run the following commands in terminal to update the system

sudo apt update
sudo apt dist-upgrade
sudo apt install rsync

Later in this guide, we will be using the rsync command to sync files between the Host PC/Laptop and the Raspberry Pi. For some of these files, root rights (i.e. sudo) is required internally.

You can do this with a single terminal command as follows:

echo "$USER ALL=NOPASSWD:$(which rsync)" | sudo tee --append /etc/sudoers

That's it. Now rsync should be setup to run with sudo if needed.

Run the following commands in Raspberry Pi terminal to install the required packages:

sudo apt install build-essential cmake unzip pkg-config gfortran gcc g++ gperf flex texinfo gawk bison openssl pigz libncurses-dev autoconf automake tar figlet

Our toolchains requires few additional symbolic links to work properly. Therefore, to create all required symbolic link reliably, we need to download SSymlinker bash script as follows:

wget https://raw.githubusercontent.com/abhiTronix/raspberry-pi-cross-compilers/master/utils/SSymlinker

Once it is downloaded, you just need to make it executable, and then run it for each path manually using the following commands:

sudo chmod +x SSymlinker
./SSymlinker -s /usr/include/aarch64-linux-gnu/asm -d /usr/include
./SSymlinker -s /usr/include/aarch64-linux-gnu/gnu -d /usr/include
./SSymlinker -s /usr/include/aarch64-linux-gnu/bits -d /usr/include
./SSymlinker -s /usr/include/aarch64-linux-gnu/sys -d /usr/include
./SSymlinker -s /usr/include/aarch64-linux-gnu/openssl -d /usr/include
./SSymlinker -s /usr/lib/aarch64-linux-gnu/crtn.o -d /usr/lib/crtn.o
./SSymlinker -s /usr/lib/aarch64-linux-gnu/crt1.o -d /usr/lib/crt1.o
./SSymlinker -s /usr/lib/aarch64-linux-gnu/crti.o -d /usr/lib/crti.o

Note: Make sure to install additional packages based on software you cross-compiling with CMAKE.

That's it for Raspberry Pi setup.

Now Raspberry Pi Side all setup, Let's focus on commands for our Host Machine, i.e. PC/Laptop, where you going to cross-compile software binaries (Hello world application in this case) for your Raspberry Pi.

Important: Make sure your Raspberry Pi and this Host machine (where you cross-compiling) MUST be on the SAME Network.

First of all, Run the following commands to update your system and install important dependancies:

sudo apt update
sudo apt dist-upgrade
sudo apt install build-essential cmake unzip gfortran
sudo apt install gcc g++ git bison python gperf pkg-config gdb-multiarch wget rsync
sudo apt install gperf flex texinfo gawk bison openssl pigz libncurses-dev autoconf automake tar figlet

You can use these following commands to create "cmake-test" to use as workspace for the project:

sudo mkdir ~/cmake-test
sudo mkdir ~/cmake-test/tools
sudo mkdir ~/cmake-test/build
sudo mkdir ~/cmake-test/rootfs
sudo mkdir ~/cmake-test/rootfs/usr
sudo mkdir ~/cmake-test/rootfs/opt
sudo chown -R 1000:1000 ~/cmake-test
cd ~/cmake-test

Note: Ensure the last command should have changed your current directory to ~/cmake-test. If not, run the last line again to make sure you are inside it, as the next steps assume you're running your commands from this directory.

Let's first change into tools directory for downloading our Precompiled Cross-compiler with the following command:

cd ~/cmake-test/tools

Note: Ensure the last command should have changed your current directory to ~/cmake-test/tools now. If not, run it again.

Copy URL from one of following Precompiled Compressed Base-Toolchain (for maximum compatability) based on your Raspberry Pi Variant and OS you installed on it, from below:

Note: You can also use the latest cross-compiler binaries instead. But they are not tested.

After that, paste your copied URL and run the following command to download the Cross-compiler:

wget <Copied Binary URL goes here> #for e.g. => wget https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Bonus%20Raspberry%20Pi%20GCC%2064-Bit%20Toolchains/Raspberry%20Pi%20GCC%2064-Bit%20Cross-Compiler%20Toolchains/Buster/GCC%208.3.0/cross-gcc-8.3.0-pi_64.tar.gz

Once it is downloaded, we can extract it using the following command:

tar xf cross-gcc-*.tar.gz

First, let's move back into the cmake-test folder as needed for the next sections:

cd ~/cmake-test

Now, we need to sync up our rootfs folder with the system files from the Raspberry Pi. We will be using rsync that let us sync (i.e. copy) files from the Raspberry Pi with appropriate permission onto your Host Machine, potentially saving you alot of time.

To do this, enter the following commands one by one into your terminal (change 192.168.1.47 with the IP address of your Raspberry Pi present on the same network):

• Command 1: rsync -avz --rsync-path="sudo rsync" --delete pi@192.168.1.47:/lib rootfs
• Command 2: rsync -avz --rsync-path="sudo rsync" --delete pi@192.168.1.47:/usr/include rootfs/usr
• Command 3: rsync -avz --rsync-path="sudo rsync" --delete pi@192.168.1.47:/usr/lib rootfs/usr
• Command 4: rsync -avz --rsync-path="sudo rsync" --delete pi@192.168.1.47:/opt/vc rootfs/opt

Note: Double check after each of the above commands that all the files have been copied to ~/cmake-test/rootfs folder. There will be an information message if there were any issues.

The files we copied in the previous step still have symbolic links pointing to the file system on the Raspberry Pi. We need to alter this so that they become relative links from the new rootfs directory on the host machine. We can do this with a python script, which we can download as follows:

wget https://raw.githubusercontent.com/abhiTronix/rpi_rootfs/master/scripts/sysroot-relativelinks.py

Once it is downloaded, you just need to make it executable and run it, using the following commands:

sudo chmod +x sysroot-relativelinks.py
./sysroot-relativelinks.py rootfs

First, let's move back into the cmake-test folder as needed for the next sections:

cd ~/cmake-test

Now we need to create suitable files for our CMAKE project.

Let’s try to compile and run a C++17 code that uses an if block with init-statement (the example is a bit simple, but will show you how to compile C++17 programs):

#include <iostream>

int main() {
  // if block with init-statement:
  if (int a = 5; a < 8) {
    std::cout << "Local variable a is < 8\n";
  } else {
    std::cout << "Local variable a is >= 8\n";
  }
  return 0;
}

Let's create a typical cross-compiling toolchain that has content such as:

⚠️ Remember to replace Cross-Compiler Toolchain absolute path with yours in following CMAKE file.

set(CMAKE_VERBOSE_MAKEFILE ON)
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR aarch64)

set(tools <path to toolchain. for e.g $ENV{HOME}/cmake-test/tools/cross-pi-gcc-8.3.0-64>) # warning change toolchain path here.
set(rootfs_dir $ENV{HOME}/cmake-test/rootfs)

set(CMAKE_FIND_ROOT_PATH ${rootfs_dir})
set(CMAKE_SYSROOT ${rootfs_dir})

set(CMAKE_LIBRARY_ARCHITECTURE aarch64-linux-gnu)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fPIC -Wl,-rpath-link,${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE} -L${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE}")
set(CMAKE_C_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -Wl,-rpath-link,${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE} -L${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -Wl,-rpath-link,${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE} -L${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE}")

## Compiler Binary 
SET(BIN_PREFIX ${tools}/bin/aarch64-linux-gnu)

SET (CMAKE_C_COMPILER ${BIN_PREFIX}-gcc)
SET (CMAKE_CXX_COMPILER ${BIN_PREFIX}-g++ )
SET (CMAKE_LINKER ${BIN_PREFIX}-ld 
            CACHE STRING "Set the cross-compiler tool LD" FORCE)
SET (CMAKE_AR ${BIN_PREFIX}-ar 
            CACHE STRING "Set the cross-compiler tool AR" FORCE)
SET (CMAKE_NM {BIN_PREFIX}-nm 
            CACHE STRING "Set the cross-compiler tool NM" FORCE)
SET (CMAKE_OBJCOPY ${BIN_PREFIX}-objcopy 
            CACHE STRING "Set the cross-compiler tool OBJCOPY" FORCE)
SET (CMAKE_OBJDUMP ${BIN_PREFIX}-objdump 
            CACHE STRING "Set the cross-compiler tool OBJDUMP" FORCE)
SET (CMAKE_RANLIB ${BIN_PREFIX}-ranlib 
            CACHE STRING "Set the cross-compiler tool RANLIB" FORCE)
SET (CMAKE_STRIP {BIN_PREFIX}-strip 
            CACHE STRING "Set the cross-compiler tool RANLIB" FORCE)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

Finally create a suitable CMakeLists.txt file:

cmake_minimum_required(VERSION 3.10)
project(cmake_test)
set(CMAKE_VERBOSE_MAKEFILE ON)
set(CMAKE_CXX_STANDARD 17)
add_executable(cmake_test main.cpp)

Let's move into the build directory for further steps, as we don't want to build within that source directory as its crowded, so we will access it from within this this directory:

cd ~/cmake-test/build

Important: Ensure you are still in the ~/cmake-test/build directory.

Finally, Now we can configure our Hello-World CMAKE Project as follows:

cmake -DCMAKE_TOOLCHAIN_FILE=~/cmake-test/PI.cmake  -DCMAKE_BUILD_TYPE=Debug ..

Our build has been configured now, and it is time to actually build the source files, and run the following command:

make -j$(nproc)

Note: -j$(nproc) option indicates that the job should be spread into mutliple threads and run in parallel on available cores.

and your compiled files will be available at ~/cmake-test/build.