fedora systems development - animeshtrivedi/notes GitHub Wiki
$ sudo dnf install gflags-devel
$ git checkout v6.3.1
$ PREFIX=/home/atr/local make -j4 static_lib db_bench install
This builds the static library and installs the binary in the local/. OK, so far so good.
$ rpm2cpio ../hoge.src.rpm | cpio -i
https://wiki.ubuntu.com/Kernel/BuildYourOwnKernel
sudo apt install libncurses-dev gawk flex bison openssl libssl-dev dkms libelf-dev libudev-dev \
libpci-dev libiberty-dev autoconf llvm htop sysstat net-tools autoconf automake
command on fedora looks like:
sudo dnf install ncurses-devel gawk flex bison openssl-devel openssl dkms libgudev-devel \
libpciaccess-devel binutils-devel autoconf llvm elfutils-libelf autoconf automake
sudo dnf install kernel-cross-headers kernel-headers
Further utilities:
sudo dnf install cmake autoconf g++ gcc make python3-ninja.noarch libaio-devel \
libslirp-devel pixman-devel numactl libtool autoconf automake
building QEMU:
$ mkdir build && cd ./build
$ ../configure --target-list=x86_64-softmmu --enable-kvm --enable-linux-aio \
--enable-trace-backends=log --disable-werror --disable-gtk \
--prefix=/home/atr/local/ --enable-slirp --enable-docs --enable-vnc \
--enable-linux-io-uring
$ make -j
made the images, so far ok, when starting qemu then we get:
qemu-system-x86_64: -net user,hostfwd=tcp::7777-:22: \
network backend 'user' is not compiled into this binary
You need to enable --enable-slirp, https://stackoverflow.com/questions/75641274/network-backend-user-is-not-compiled-into-this-binary and install libslirp-devel.
After this the QEMU setup works out of the box. No issues there.
meson building tool: https://mesonbuild.com/Commands.html
meson setup --prefix=/home/atr/local/ .build
Linking to a non-standard path issue
atr@cordova:~/src/nvme-cli$ meson setup --debug .build
The Meson build system
Version: 1.4.1
Source dir: /home/atr/src/nvme-cli
Build dir: /home/atr/src/nvme-cli/.build
Build type: native build
Project name: nvme-cli
Project version: 2.9.1
C compiler for the host machine: cc (gcc 14.1.1 "cc (GCC) 14.1.1 20240701 (Red Hat 14.1.1-7)")
C linker for the host machine: cc ld.bfd 2.41-37
Host machine cpu family: x86_64
Host machine cpu: x86_64
Found pkg-config: YES (/usr/bin/pkg-config) 2.1.1
Found CMake: /usr/bin/cmake (3.28.2)
Run-time dependency libnvme found: NO (tried pkgconfig and cmake)
Not looking for a fallback subproject for the dependency libnvme because:
Use of fallback dependencies is disabled.
meson.build:51:14: ERROR: Dependency 'libnvme' is required but not found.
A full log can be found at /home/atr/src/nvme-cli/.build/meson-logs/meson-log.txt
ld -lzlib --verbose
for others, meson and cmake, if you dont want to modify the make file, then put the path in the $LIBRARY_PATH
Furthermore, Fedora have lib64 variant in the library path, so the new bashrc looks like:
export TERM=xterm-color
export GO_HOME=/home/atr/src/go/
export JAVA_HOME=/home/atr/sw/jdk1.8.0_221/
export PATH=$GO_HOME/bin:$JAVA_HOME/bin/:/home/atr/local/bin/:$PATH
export LD_LIBRARY_PATH=/home/atr/local/lib/:/home/atr/local/usr/local/lib/:/home/atr/local/lib64/:$LD_LIBRARY_PATH
export LIBRARY_PATH=/home/atr/local/lib/:/home/atr/local/usr/local/lib/:/home/atr/local/lib64/:$LD_LIBRARY_PATH
export CPATH=:/home/atr/local/include/:/home/atr/local/usr/local/include/:
export PKG_CONFIG_PATH=/home/atr/local/usr/local/lib/pkgconfig/:/home/atr/local/lib/pkgconfig:/home/atr/local/lib64/pkgconfig/:$PKG_CONFIG_PATH
with that, to compile and setup nvme-cli package:
meson setup --wrap-mode nofallback --prefix=/home/atr/local/ .build
meson compile -C .build
meson install -C .build
make tools/all deb-pkg
pfrut.c:24:10: fatal error: uuid/uuid.h: No such file or directory
24 | #include <uuid/uuid.h>
| ^~~~~~~~~~~~~
sudo dnf install libuuid libuuid-devel
utils/helpers/amd.c:8:10: fatal error: pci/pci.h: No such file or directory
8 | #include <pci/pci.h>
| ^~~~~~~~~~~
compilation terminated.
sudo dnf install pciutils-devel
nosy-dump.c:12:10: fatal error: popt.h: No such file or directory
12 | #include <popt.h>
| ^~~~~~~~
compilation terminated.
sudo dnf install popt-devel
Makefile.config:455: No libdw DWARF unwind found, Please install elfutils-devel/libdw-dev >= 0.158 and/or set LIBDW_DIR
Makefile.config:460: No libdw.h found or old libdw.h found or elfutils is older than 0.138, disables dwarf support. Please install new elfutils-devel/libdw-dev
Makefile.config:602: No sys/sdt.h found, no SDT events are defined, please install systemtap-sdt-devel or systemtap-sdt-dev
Makefile.config:650: No libunwind found. Please install libunwind-dev[el] >= 1.1 and/or set LIBUNWIND_DIR
Makefile.config:688: Warning: Disabled BPF skeletons as clang (clang) is missing
Makefile.config:720: Disabling post unwind, no support found.
Makefile.config:800: slang not found, disables TUI support. Please install slang-devel, libslang-dev or libslang2-dev
Makefile.config:847: Missing perl devel files. Disabling perl scripting support, please install perl-ExtUtils-Embed/libperl-dev
Makefile.config:987: No liblzma found, disables xz kernel module decompression, please install xz-devel/liblzma-dev
Makefile.config:1011: No libcap found, disables capability support, please install libcap-devel/libcap-dev
Makefile.config:1024: No numa.h found, disables 'perf bench numa mem' benchmark, please install numactl-devel/libnuma-devel/libnuma-dev
Makefile.config:1083: No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-dev[el]/libbabeltrace-ctf-dev
Makefile.config:1095: No libcapstone found, disables disasm engine support for 'perf script', please install libcapstone-dev/capstone-devel
Makefile.config:1147: No openjdk development package found, please install JDK package, e.g. openjdk-8-jdk, java-1.8.0-openjdk-devel
Makefile.config:1160: libpfm4 not found, disables libpfm4 support. Please install libpfm4-dev
Makefile.config:1178: *** ERROR: libtraceevent is missing. Please install libtraceevent-dev/libtraceevent-devel or build with NO_LIBTRACEEVENT=1. Stop.
sudo dnf install elfutils-devel systemtap-sdt-devel libunwind-devel slang-devel perl-ExtUtils-Embed libcap-devel numactl-devel libbabeltrace-devel capstone-devel libtraceevent-devel
Building a custom kernel on Fedroa, https://fedoraproject.org/wiki/Building_a_custom_kernel or https://docs.fedoraproject.org/en-US/quick-docs/kernel-build-custom/
So the current issue is that deb-pkg compilation does not work on Fedora
atr@cordova:~/src/linux$ make -j $(getconf _NPROCESSORS_ONLN) deb-pkg LOCALVERSION=-atr-2024-07-05
UPD include/config/kernel.release
GEN debian
UPD .tmp_HEAD
ARCHIVE linux.tar.gz
Using default distribution of 'unstable' in the changelog
Install lsb-release or set $KDEB_CHANGELOG_DIST explicitly
dpkg-buildpackage --build=source,binary --no-pre-clean --unsigned-changes --unsigned-source --compression=gzip -R'make -f debian/rules' -j1 -a$(cat debian/arch)
dpkg-buildpackage: info: source package linux-upstream
dpkg-buildpackage: info: source version 6.10.0-rc6-1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by atr <[email protected]>
dpkg-architecture: warning: specified GNU system type x86_64-linux-gnu does not match CC system type x86_64-redhat-linux, try setting a correct CC environment variable
dpkg-buildpackage: info: host architecture amd64
dpkg-source --compression=gzip --before-build .
dpkg-source: info: using options from linux/debian/source/local-options: --diff-ignore --extend-diff-ignore=.*
dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper-compat (= 12) bc bison cpio flex kmod libelf-dev:native libssl-dev:native rsync
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
make[2]: *** [scripts/Makefile.package:121: deb-pkg] Error 3
make[1]: *** [/home/atr/src/linux/Makefile:1555: deb-pkg] Error 2
make: *** [Makefile:240: __sub-make] Error 2
Broken with:
atr@u24clean:~/src/linux-6.10-rc6$ make -j $(getconf _NPROCESSORS_ONLN) deb-pkg LOCALVERSION=-atr-2024-07-05
UPD include/config/kernel.release
GEN debian
error: creating source package requires git repository
make[2]: *** [scripts/Makefile.package:17: check-git] Error 1
make[2]: *** Waiting for unfinished jobs....
warning: Not a git repository. Use --no-index to compare two paths outside a working tree
usage: git diff --no-index [<options>] <path> <path>
Diff output format options
-p, --patch generate patch
<...>
Reason being that the build process is new now ("Kernel 6.3 custom compile error") : https://www.spinics.net/lists/kernel/msg4780486.html
With a clean kernel git checkout on ubuntu, it works with the right package installed:
atr@u24clean:~/src/linux$ make -j $(getconf _NPROCESSORS_ONLN) deb-pkg LOCALVERSION=-atr-2024-07-05
UPD include/config/kernel.release
GEN debian
UPD .tmp_HEAD
ARCHIVE linux.tar.gz
dpkg-buildpackage --build=source,binary --no-pre-clean --unsigned-changes --unsigned-source --compression=gzip -R'make -f debian/rules' -j1 -a$(cat debian/arch)
dpkg-buildpackage: info: source package linux-upstream
dpkg-buildpackage: info: source version 6.10.0-rc6-1
dpkg-buildpackage: info: source distribution noble
dpkg-buildpackage: info: source changed by atr <atr@u24clean>
dpkg-buildpackage: info: host architecture amd64
dpkg-source --compression=gzip --before-build .
dpkg-source: info: using options from linux/debian/source/local-options: --diff-ignore --extend-diff-ignore=.*
dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper-compat (= 12)
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
make[2]: *** [scripts/Makefile.package:121: deb-pkg] Error 3
make[1]: *** [/home/atr/src/linux/Makefile:1555: deb-pkg] Error 2
make: *** [Makefile:240: __sub-make] Error 2
atr@u24clean:~/src/linux$ apt show debhelper
Package: debhelper
Version: 13.14.1ubuntu5
Priority: optional
Section: devel
Origin: Ubuntu
Maintainer: Ubuntu Developers <[email protected]>
Original-Maintainer: Debhelper Maintainers <[email protected]>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 1,441 kB
Provides: debhelper-compat (= 9), debhelper-compat (= 10), debhelper-compat (= 11), debhelper-compat (= 12), debhelper-compat (= 13), dh-sequence-dwz, dh-sequence-elf-tools, dh-sequence-installinitramfs, dh-sequence-installsysusers, dh-sequence-movetousr, dh-sequence-single-binary, dh-sequence-systemd
Depends: autotools-dev, dh-autoreconf (>= 17~), dh-strip-nondeterminism (>= 0.028~), dpkg (>= 1.18.0~), dpkg-dev (>= 1.18.2~), debugedit, dwz (>= 0.12.20190711), file (>= 3.23), libdebhelper-perl (= 13.14.1ubuntu5), libdpkg-perl (>= 1.17.14), man-db, po-debconf, awk, perl:any
Suggests: dh-make
Breaks: cmake (<< 3.9~), dh-debputy (<< 0.1.18~), meson (<< 0.40.0~)
Download-Size: 869 kB
APT-Sources: http://ch.archive.ubuntu.com/ubuntu noble/main amd64 Packages
Description: helper programs for debian/rules
A collection of programs that can be used in a debian/rules file to
automate common tasks related to building Debian packages. Programs
are included to install various files into your package, compress
files, fix file permissions, integrate your package with the Debian
menu system, debconf, doc-base, etc. Most Debian packages use debhelper
as part of their build process.
atr@u24clean:~/src/linux$ sudo apt-get install debhelper
Reading package lists... Done
...
<works from here>
Logs for fedora system
Using default distribution of 'unstable' in the changelog
Install lsb-release or set $KDEB_CHANGELOG_DIST explicitly
make[2]: dpkg-parsechangelog: No such file or directory
/bin/sh: line 1: dpkg-parsechangelog: command not found
make[2]: dpkg-parsechangelog: No such file or directory
/bin/sh: line 1: dpkg-parsechangelog: command not found
dpkg-buildpackage --build=source,binary --no-pre-clean --unsigned-changes --unsigned-source --compression=gzip -R'make -f debian/rules' -j1 -a$(cat debian/arch)
/bin/sh: line 1: dpkg-buildpackage: command not found
make[2]: *** [scripts/Makefile.package:121: deb-pkg] Error 127
make[1]: *** [/home/atr/src/linux/Makefile:1555: deb-pkg] Error 2
make: *** [Makefile:240: __sub-make] Error 2
<install dpkg-devel package - twice, it did not get the commands >
Also for parsing install autopoint: [gettext-devel](https://stackoverflow.com/questions/72555674/failed-to-run-autopoint-no-such-file-or-directory)
sudo dnf install debhelper strace debconf dpkg-devel
How to debug the make file: how to debug the make: make -j $(getconf _NPROCESSORS_ONLN) deb-pkg LOCALVERSION=-atr-2024-07-05 -d`
all new packages: sudo dnf install debhelper strace debconf dpkg-devel bison-devel kernel-headers-uname -r.x86_64 dh-make cdbs
Even after installing all these packages, there is the issue such as :
dpkg-buildpackage: info: host architecture amd64
dpkg-source --before-build .
dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper-compat (= 12) bc bison cpio flex kmod libelf-dev:native libssl-dev:native rsync
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
make[2]: *** [scripts/Makefile.package:121: bindeb-pkg] Error 3
make[1]: *** [/home/atr/src/linux/Makefile:1555: bindeb-pkg] Error 2
make: *** [Makefile:240: __sub-make] Error 2
So I modified the make script to disable warnings (-d), and then start building:
atr@cordova:~/src/linux$ git diff
diff --git a/scripts/Makefile.package b/scripts/Makefile.package
index 38653f3e8108..8a9b5e3448b2 100644
--- a/scripts/Makefile.package
+++ b/scripts/Makefile.package
@@ -118,7 +118,7 @@ bindeb-pkg: private build-type := binary
deb-pkg srcdeb-pkg: debian-orig
bindeb-pkg: debian
deb-pkg srcdeb-pkg bindeb-pkg:
- +$(strip dpkg-buildpackage \
+ +$(strip dpkg-buildpackage -d \
--build=$(build-type) --no-pre-clean --unsigned-changes \
$(if $(findstring source, $(build-type)), \
--unsigned-source --compression=$(KDEB_SOURCE_COMPRESS)) \
atr@cordova:~/src/linux$
After fixing this:
CALL scripts/checksyscalls.sh
DESCEND objtool
INSTALL libsubcmd_headers
DESCEND bpf/resolve_btfids
INSTALL libsubcmd_headers
UPD init/utsversion-tmp.h
CC init/version.o
AR init/built-in.a
CHK kernel/kheaders_data.tar.xz
AR built-in.a
AR vmlinux.a
LD vmlinux.o
vmlinux.o: warning: objtool: .export_symbol+0x34678: data relocation to !ENDBR: stpcpy+0x0
OBJCOPY modules.builtin.modinfo
GEN modules.builtin
MODPOST Module.symvers
UPD include/generated/utsversion.h
CC init/version-timestamp.o
BTF: .tmp_vmlinux.btf: pahole (pahole) is not available
Failed to generate BTF for vmlinux
Try to disable CONFIG_DEBUG_INFO_BTF
make[5]: *** [scripts/Makefile.vmlinux:37: vmlinux] Error 1
make[4]: *** [Makefile:1160: vmlinux] Error 2
make[3]: *** [debian/rules:74: build-arch] Error 2
dpkg-buildpackage: error: make -f debian/rules binary subprocess returned exit status 2
make[2]: *** [scripts/Makefile.package:121: deb-pkg] Error 2
make[1]: *** [/home/atr/src/linux/Makefile:1541: deb-pkg] Error 2
make: *** [Makefile:240: __sub-make] Error 2
Solution is to install: sudo dnf install dwarves (source: https://stackoverflow.com/questions/61657707/btf-tmp-vmlinux-btf-pahole-pahole-is-not-available)
Success then the final complication works.
-rw-r--r--. 1 atr atr 8.9M Jul 9 11:46 linux-headers-6.9.0-atr-2024-07-05_6.9.0-13_amd64.deb
-rw-r--r--. 1 atr atr 97M Jul 9 11:48 linux-image-6.9.0-atr-2024-07-05_6.9.0-13_amd64.deb
-rw-r--r--. 1 atr atr 994M Jul 9 11:58 linux-image-6.9.0-atr-2024-07-05-dbg_6.9.0-13_amd64.deb
-rw-r--r--. 1 atr atr 1.3M Jul 9 11:46 linux-libc-dev_6.9.0-13_amd64.deb
-rw-r--r--. 1 atr atr 2.3K Jul 9 11:58 linux-upstream_6.9.0-13_amd64.buildinfo
-rw-r--r--. 1 atr atr 3.3K Jul 9 11:58 linux-upstream_6.9.0-13_amd64.changes
-rw-r--r--. 1 atr atr 69K Jul 9 10:53 linux-upstream_6.9.0-13.debian.tar.gz
-rw-r--r--. 1 atr atr 1.3K Jul 9 10:53 linux-upstream_6.9.0-13.dsc
-rw-r--r--. 2 atr atr 227M Jul 5 15:52 linux-upstream_6.9.0.orig.tar.gz
Installing these files inside the VM works and it boots.
https://docs.github.com/en/authentication/troubleshooting-ssh/error-permission-denied-publickey
Public and private keys both need read-only access, plus add them in the config file as ~/.ssh/config
Host github.com
IdentityFile ~/.ssh/github_rsa