Set Up your RPi as a distcc Client - sakaki-/gentoo-on-rpi-64bit GitHub Wiki
Let your RPi4 (or RPi3) invoke remote cross-compilation on your PC, via distcc
, and massively speed up the build process!
This guide assumes that you have already set up your PC as a distcc
server as detailed here. So, if you haven't yet, please do this first, before proceeding.
In what follows, I'm going to assume your PC is at the fixed address 192.168.1.116; obviously, adapt the instructions as appropriate to match your setup.
As we are using a stable-branch compiler on the PC, but the RPi image uses the 'testing' (aka ~arm64
) branch by default, begin by downgrading your gcc
compiler (and binutils
, for consistency) on the RPi to the 'stable' version (this version is also available on the binhost, so the following process shouldn't take long). Issue:
pi64 ~ # echo "sys-devel/binutils -~arm64" >> /etc/portage/package.accept_keywords/binutils
pi64 ~ # echo "sys-devel/gcc -~arm64" >> /etc/portage/package.accept_keywords/gcc
pi64 ~ # emerge -u sys-devel/binutils sys-devel/gcc
pi64 ~ # gcc-config --list-profiles
Take a note of the index number of the 'stable branch' version of the compiler returned by the last command above (this will probably be 1
), and then ensure the profile is set (substitute it for 1
in the below, if different):
pi64 ~ # gcc-config 1
pi64 ~ # env-update && source /etc/profile
pi64 ~ # emerge --oneshot sys-devel/libtool
Next, if you haven't already got distcc
on your RPi (it is pre-installed on this project's image), begin by emerging it now. Issue:
pi64 ~ # emerge --ask --verbose sys-devel/distcc
Then, tell distcc
where the server is; edit the file /etc/distcc/hosts
(on the RPi) and append the line:
192.168.1.116/10,cpp,lzo
Note - you can of course set up multiple servers here, but that's beyond the scope of this short guide; see the
distcc
homepage for further details. Note also that with a relatively slow system like the RPi3, I would not recommend addinglocalhost
to the hosts list. It will fall back to local compilation where necessary in any event. By contrast, on an RPi4, particularly if you have the 2GiB or 4GiB variant (and sufficient cooling),localhost
may safely be used.
Note - the
/10
is nothing to do with the IP address (confusingly); it is the maximum number of simultaneous jobs to send to that server (here, 10). The default is 4 if unspecified. Adjust to suit your particulardistcc
server's capabilities.
Next, edit the file /etc/portage/make.conf
(on the RPi4/3), and append (or uncomment, if already present) the following line:
FEATURES="${FEATURES} distcc"
That's it! You're now ready to roll.
Now you have everything set up, using distributed cross-compilation from your RPi4 (or RPi3) is really easy. In fact, with the FEATURES
variable set as above, emerge
will automatically make use of it, so the next time you build a package or update your system, you should notice a significant speed-up, without having to do anything further.
To use distcc
when performing a normal make
, simply prefix the command with pump
. So for example:
pi64 mycproject # pump make -j10
Remember to specify a reasonably large job number with using make
with distcc
as shown above, to maximize the remote processing capacity. That's it!
You can check on distcc's operation at the client end with:
pi64 ~ # watch DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text
and at the server, most simply, by
gentoo_pc ~ # journalctl -f --unit=distccd
if using systemd
, or
gentoo_pc ~ # tail -f /var/log/messages | grep distccd --line-buffered
if using OpenRC
.
Distributed cross-compilation with distcc
/crossdev
is neat, but it is not a magic bullet. For example:
- some parts of the build process (such as autotools
./configure
) must still run locally on the RPi (and are considerably time-consuming); - some packages (like
sys-devel/gcc
) rely on running tools that were created during the build, as part of the build; such operations cannot be distributed withdistcc
; and - languages other than C and C++ are not supported by
distcc
.
In addition, you will occasionally encounter packages that do not build correctly under distcc
(although this is quite rare). You can simply build locally in such cases (and force this to happen in future by adding an entry to /etc/portage/package.env/<pkgname>
- see e.g. /etc/portage/package.env/numpy
on the image, for an example of this).
As to other issues, this is only a short guide, and you should read the notes on the distcc
site and Gentoo wiki if you experience problems. One important point that is worth mentioning here though - you must ensure that gcc
is the same version on your RPi and in your crossdev
environment on the PC; bad things are almost guaranteed to happen if not ><