Yocto - hpaluch/hpaluch.github.io GitHub Wiki

Yocto

Yocto Project (YP) is set of tools to build embedded Linux. Yocto itself provides Linux distribution called Poky.

It is basically alternative to Buildroot, however Yocto uses its own Python based build tool called bitbake (shortcut "BB"), while Buildroot uses Makefiles adapted from Linux kernel build system)

Quickstart

We will more or less follow https://docs.yoctoproject.org/brief-yoctoprojectqs/index.html

WARNING! Because bitbake is written in Python and Python 3 maintainers enjoy breaking API - you should always use supported Host distribution. In my case I checked list is on https://docs.yoctoproject.org/ref-manual/system-requirements.html#supported-linux-distributions and decided to use Debian 12

First we have to install required packages following above guide:

# requirements on bare-bone Debian
sudo apt install locales
echo 'en_US.UTF-8 UTF-8' >> /etc/locale.gen
locale-gen

# standard requirements + parted (required for "wic ls image.wic"
sudo apt install gawk wget git diffstat unzip texinfo \
     gcc build-essential chrpath socat cpio python3 \
     python3-pip python3-pexpect xz-utils debianutils \
     iputils-ping python3-git python3-jinja2 python3-subunit \
     zstd liblz4-tool file locales libacl1 parted bmap-tools

Now as user that will build Poky:

cd
git clone https://git.yoctoproject.org/poky
cd poky
git checkout -t origin/scarthgap -b my-scarthgap
git branch -v

    master       39de8c6549 selftest: add test_product_match
  * my-scarthgap a39380d9c9 u-boot.inc: Refactor do_* steps into functions that can be overridden

source oe-init-build-env

NOTE: In my case I set DL_DIR = "/opt/builder/downloads" in ~/poky/build/conf/local.conf to have downloads outside Git directory....

I decided to fetch all downloads before build, to make easier backup of everything, see https://docs.yoctoproject.org/dev-manual/efficiently-fetching-sources.html#getting-source-files-and-suppressing-the-build for details:

NOTE: Original manual uses image "core-image-sato" - but it is full GUI appliance (for mobile platform) which take lot of space and lot of CPU time to build. Therefore I rather use smaller image core-image-minimal.

# it will run around 540 tasks, stores 4.6G in /opt/builder/downloads
# longest one is linux-yocto-6.6.35+git-r0 do_fetch - kernel source
bitbake core-image-minimal --runall=fetch

Where core-image-minimal is name of Image I plan to build in next step.

Now it is good opportunity to backup ~/poky before there will be many build and state files...

And finally build individual image files (but not HDD image - for this exists wic tool) suitable to run directly under QEMU. This task may take even hours - so it is recommended to run it inside tmux or screen to avoid interruption on connection drop.

To build image core-image-minimal simply run bitbake with its name in ~/poky/build directory:

# will run 4099 tasks in my case
time bitbake core-image-minimal
# warning! llvm-native may take several hours...

Native vs. Target:

  • you can see that there will be (still) build some GUI packages - however with native suffix. Native means that it will run on build Host. In our case QEMU is normally build with various graphics outputs so it needs support libraries.

TODO: Run image directly in QEMU

Following: https://docs.yoctoproject.org/brief-yoctoprojectqs/index.html

# TODO: verify
runqemu kvm serial nographic core-image-minimal

WARNING! You should pass "simplified options" first - add image name as last (see runqemu --help output) otherwise runqemu could be confused and interpret parameter name as image or filename.

TODO: build full disk images with Wic

Following: https://docs.yoctoproject.org/dev-manual/wic.html

TODO:

# TODO: verify
bitbake wic-tools
wic list-images
wic create qemux86-directdisk -e core-image-minimal

Building Poky for Raspberry PI 1.B

Work in Progress - awaiting for results of 1st build...

There is some README.md but a bit incomplete on:

Install system packages (same as above) and then:

mkdir -p ~/projects
cd ~/projects
git clone https://git.yoctoproject.org/poky.git
git clone https://git.yoctoproject.org/meta-raspberrypi.git

Totally undocumented - we need to find supported Poky branch:

$ fgrep LAYERSERIES meta-raspberrypi/conf/layer.conf

LAYERSERIES_COMPAT_raspberrypi = "styhead"

So next we need to checkout bracnh origin/styhead of Poky Linux:

cd poky
git checkout -t origin/styhead -b my-styhead
git branch -v

  master       185fd2b28c recipes-bsp: usbutils: Fix usb-devices command using busybox
* my-styhead   185fd2b28c recipes-bsp: usbutils: Fix usb-devices command using busybox

source ./oe-init-build-env rpi-build
vim conf/bblayers.conf

  # append /home/USERNAME/projects/meta-raspberrypi \
  # to BBLAYERS ?= ... variable

# ls ls ../../meta-raspberrypi/conf/machine/
# append to conf/local.conf:
# Raspberry PI 1.B
MACHINE = "raspberrypi"

# now try build (still in ~/projects/poky/rpi-build directory):
bitbake core-image-base

Build notes:

  • brief build time can be found with command like: 
    ls -lrt ~/projects/poky/rpi-build/tmp/buildstats/20240907140738

-rw-r--r-- 1 builder builder    142 Sep  7 14:07 build_stats
drwxr-xr-x 2 builder builder   4096 Sep  7 14:07 reduced_proc_pressure
drwxr-xr-x 2 builder builder   4096 Sep  7 14:07 quilt-native-0.68-r0
drwxr-xr-x 2 builder builder   4096 Sep  7 14:09 texinfo-dummy-native-1.0-r0
...
  • one of most consuming task is gcc-cross-arm-14.2.0-r0 do_compile (cross compiler for target ARM CPU)
    • build time can be found (once is build complete) on:
      • ~/projects/poky/rpi-build/tmp/buildstats/20240907140738/gcc-cross-arm-14.2.0-r0/do_compile
    • on Azure VM Standard E4bds v5 (4 vcpus, 32 GiB memory) Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz using local disk storage it takes 19 minutes. Complete build time is 3 hours 12 minutes.
    • on old AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ at 2 GHz (8 GiB memory) it takes around 1 hour 19 minutes.

No wwe need both wic file and bmap file to write it to SD card on host computer:

find tmp/deploy/ -name '*.wic*' -exec ls -l {} \;
-rw-r--r-- 2 builder builder 65710778 Sep  7 17:19 \
  tmp/deploy/images/raspberrypi/core-image-base-raspberrypi.rootfs-20240907140738.wic.bz2
-rw-r--r-- 2 builder builder 3502 Sep  7 17:19 \
  tmp/deploy/images/raspberrypi/core-image-base-raspberrypi.rootfs-20240907140738.wic.bmap
lrwxrwxrwx 2 builder builder 58 Sep  7 17:19 \
  tmp/deploy/images/raspberrypi/core-image-base-raspberrypi.rootfs.wic.bmap -> core-image-base-raspberrypi.rootfs-20240907140738.wic.bmap
lrwxrwxrwx 2 builder builder 57 Sep  7 17:19 \
  tmp/deploy/images/raspberrypi/core-image-base-raspberrypi.rootfs.wic.bz2 -> core-image-base-raspberrypi.rootfs-20240907140738.wic.bz2

I copied these two (symlinks Dereferencing them to target directory:

$ ls -gGh ~/backups/rpi-1b/image1/

total 308M
-rw-r--r-- 1 308M Sep  7 17:19 core-image-base-raspberrypi.rootfs.wic
-rw-r--r-- 1 3.5K Sep  7 17:19 core-image-base-raspberrypi.rootfs.wic.bmap

$ wic ls /home/azureuser/backups/rpi-1b/image1/core-image-base-raspberrypi.rootfs.wic

Num     Start        End          Size      Fstype
 1       4194304    140509183    136314880  fat16
 2     142606336    322541567    179935232  ext4

Looks good (there must bi FAT boot partition + ext4 rootfs partition)

Now we need bmaptool on Host OS (where we will write it to SD card). In openSUSE LEAP 15.6 we can install:

sudo zypper in python3-bmap-tools

To write image we will try - replace /dev/sdX with your SD card device:

lsblk -S # find device for SD card reader
sudo bmaptool copy core-image-base-raspberrypi.rootfs.wic /dev/sdX

bmaptool: info: discovered bmap file 'core-image-base-raspberrypi.rootfs.wic.bmap'
bmaptool: info: block map format version 2.0
bmaptool: info: 78746 blocks of size 4096 (307.6 MiB), mapped 31782 blocks (124.1 MiB or 40.4%)
bmaptool: info: copying image 'core-image-base-raspberrypi.rootfs.wic' to block device '/dev/sdb' using bmap file 'core-image-base-raspberrypi.rootfs.wic.bmap'
bmaptool: info: 100% copied
bmaptool: info: synchronizing '/dev/sdb'
bmaptool: info: copying time: 45.7s, copying speed 2.7 MiB/sec

In my case I mounted 1st FAT partition and appended console=ttyAMA0 to have working serial console.

And boot up (use root as login without password). Here are example messages:

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 6.6.36 (oe-user@oe-host) (arm-poky-linux-gnueabi-gcc (GCC) 14.2.0, GNU ld (GNU Binutils) 24
[    0.000000] CPU: ARMv6-compatible processor [410fb767] revision 7 (ARMv7), cr=00c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
[    0.000000] OF: fdt: Machine model: Raspberry Pi Model B Rev 2
...
[    0.000000] Kernel command line: coherent_pool=1M snd_bcm2835.enable_headphones=0 snd_bcm2835.enable_hdmi=0  vc_mem.0
...
Fri Mar  9 12:34:56 UTC 2018
ALSA: Restoring mixer settings...
alsa-lib /usr/src/debug/alsa-lib/1.2.12/src/ucm/main.c:1554:(snd_use_case_mgr_open) error: failed to import hw:0 use ca2
No state is present for card vc4hdmi
alsa-lib /usr/src/debug/alsa-lib/1.2.12/src/ucm/main.c:1554:(snd_use_case_mgr_open) erro
INIT: Entering runlevel: 5
Hardware is initialized using a generic method
No state is present for card vc4hdmi
...
Poky (Yocto Project Reference Distro) 5.0+snapshot-185fd2b28c98c3e1abe6494e03001f4a196d122e raspberrypi /dev/ttyAMA0

raspberrypi login: root

WARNING: Poky is a reference Yocto Project distribution that should be used for
testing and development purposes only. It is recommended that you create your
own distribution for production use.

details from /proc/cpuinfo

model name      : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS        : 697.95
Features        : half thumb fastmult vfp edsp java tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xb76
CPU revision    : 7

Hardware        : BCM2835
Revision        : 000e
Model           : Raspberry Pi Model B Rev 2

df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root               160.7M     88.7M     60.0M  60% /
devtmpfs                 85.0M         0     85.0M   0% /dev
tmpfs                   213.3M    208.0K    213.1M   0% /run
tmpfs                   213.3M     68.0K    213.2M   0% /var/volatile
/dev/mmcblk0p1          129.8M     29.7M    100.2M  23% /boot

However I don't like that there run many services that I don't consider minimal:

  • avahi-daemon (mDNS)
  • /usr/sbin/rpcbind - required only when using NFS client or other old RPC services
  • /usr/libexec/bluetooth/bluetoothd - why?
  • /usr/sbin/ofonod
    • from ~/projects/poky/meta/recipes-connectivity/ofono/ofono_2.10.bb
    • "oFono is a stack for mobile telephony devices on Linux. oFono supports speaking to telephony devices through specific drivers, or with generic AT commands"
    • why is that on RPi ?
  • /usr/libexec/nfc/neard
    • from ~/projects/poky/meta/recipes-connectivity/neard/neard_0.19.bb
    • Linux NFC daemon

Finding why some package is build?

$ bitbake -g core-image-base  # generates huge file task-depends.dot
$ oe-depends-dot -k bluez5 -w task-depends.dot

Because: core-image-base neard ofono packagegroup-base
core-image-base -> packagegroup-base -> neard -> bluez5
core-image-base -> packagegroup-base -> ofono -> bluez5

To get only package dependencies (because task-depends.dot is HUGE):

$ oe-depends-dot -r ./task-depends.dot

Saving reduced dot file to ./task-depends-reduced.dot

$ ls -lh *.dot

-rw-r--r-- 1 builder builder 2.2M Sep  8 14:20 task-depends.dot
-rw-r--r-- 1 builder builder  26K Sep  8 14:50 task-depends-reduced.dot

Generally you can use dot command from package graphviz to view .dot graph files. Although there exist graphviz-gnome package (supporting -Tgtk) backend it is weird (zoomed out image - unusable). I rather convert graph to SVG using:

dot -Tsvg task-depends-reduced.dot > t.svg

And viewing it in Firefox (it is zoomed-in and there are scrollbars). Anyway oe-depends-dot seems to be best solution to find why is package build.

In our example it seems that bluetooth is induced in file ~/projects/poky/meta/recipes-core/packagegroups/packagegroup-base.bb with:

PACKAGES = ' \
  ${@bb.utils.contains("DISTRO_FEATURES", "nfc", "packagegroup-base-nfc", "", d)} \
'
SUMMARY:packagegroup-base-nfc = "Near Field Communication support"
RDEPENDS:packagegroup-base-nfc = "\
    neard"

So we have to look how is DISTRO_FEATURES defined. Quick look is using environment (-e) output:

$ bitbake -e core-image-base | grep '^DISTRO_FEATURES='

DISTRO_FEATURES="acl alsa bluetooth debuginfod ext2 ipv4 ipv6 \
  pcmcia usbgadget usbhost wifi xattr nfs zeroconf pci 3g nfc \
   x11 vfat seccomp opengl ptest multiarch wayland vulkan \
   sysvinit pulseaudio gobject-introspection-data ldconfig"

Resources