Building Cilium

The instructions specify the steps to build Cilium version 1.19.4 on Linux on IBM Z for following distributions:

• RHEL (9.6, 9.8, 10.0, 10.2)
• SLES (15 SP7, 16)
• Ubuntu (22.04, 24.04)

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.
• A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.
• Docker with BuildKit support is required. Ensure Docker is installed and running before proceeding.
• All major components (LLVM, Bazel, Envoy, AWS-LC) are compiled inside Docker containers. The host only needs Docker, Go, git, and basic OS packages.
• The build process requires approximately 50GB of disk space and 16GB+ RAM.
• The system should have at least 8GB of swap space configured.

Prerequisites

• Docker packages can be installed by following the instructions here.

• Docker BuildKit must be configured with host networking. Create /etc/buildkit/buildkitd.toml with the following content:

  [worker.oci]
    networkMode = "host"
  

1. Build using script

If you want to build Cilium using manual steps, go to STEP 2.

Use the following commands to build Cilium using the build script. Please make sure you have wget installed.

wget https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Cilium/1.19.4/build_cilium.sh

# Build Cilium
bash build_cilium.sh -y    [Provide -t option for executing build with tests]

If the build and tests complete successfully, go to STEP 7. In case of error, check logs at <source_root>/logs/ for more details or go to STEP 2 to follow manual build steps.

2. Install Dependencies

export SOURCE_ROOT=/<source_root>/
export PATCH_URL="https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Cilium/1.19.4/patch"

2.1. Install Basic Dependencies

• RHEL (9.6, 9.8, 10.0, 10.2)

  sudo dnf install -y --allowerasing wget curl git make rsync file \
      ca-certificates gnupg yum-utils device-mapper-persistent-data lvm2
  

• SLES (15 SP7, 16)

  sudo zypper install -y wget curl git make rsync file patch \
      ca-certificates
  

• Ubuntu (22.04, 24.04)

  # Ensure universe repo is enabled
  # Ubuntu 24.04:
  sudo sed -i 's/Components: main$/Components: main universe/' /etc/apt/sources.list.d/ubuntu.sources
  # Ubuntu 22.04:
  sudo sed -i 's/^\(deb .* main\)$/\1 universe/' /etc/apt/sources.list
  
  sudo apt-get update
  sudo apt-get install -y wget curl git make rsync file \
      ca-certificates gnupg lsb-release apt-transport-https
  

3. Install Go 1.26.2

cd $SOURCE_ROOT
export GO_VERSION="1.26.2"
wget -q https://go.dev/dl/go${GO_VERSION}.linux-s390x.tar.gz
sudo mkdir -p /usr/local/go-${GO_VERSION}
sudo tar -C /usr/local/go-${GO_VERSION} --strip-components=1 -xzf go${GO_VERSION}.linux-s390x.tar.gz
export GOROOT="/usr/local/go-${GO_VERSION}"
export PATH="$GOROOT/bin:$PATH"
export GOPATH=$SOURCE_ROOT/go
mkdir -p $GOPATH
go version

4. Build container images

4.1. Build image-tools

cd $SOURCE_ROOT
git clone https://github.com/cilium/image-tools.git
cd image-tools
git checkout e8fd56563d1738f3cf72190c6ed54080d8215fc4
curl -sSL $PATCH_URL/image-tools-s390x.patch | git apply -

export REGISTRIES=local
export PLATFORMS=linux/s390x
export PUSH=false

docker buildx use default
echo "default" > .buildx_builder

export MAKER_IMAGE=local/image-maker:$(scripts/make-image-tag.sh images/maker)
export TESTER_IMAGE=local/image-tester:$(scripts/make-image-tag.sh images/tester)
export COMPILERS_IMAGE=local/image-compilers:$(scripts/make-image-tag.sh images/compilers)
export CILIUM_LLVM_IMAGE=local/cilium-llvm:$(scripts/make-image-tag.sh images/llvm)
export CILIUM_BPFTOOL_IMAGE=local/cilium-bpftool:$(scripts/make-image-tag.sh images/bpftool)
export CILIUM_IPTABLES_IMAGE=local/iptables:$(scripts/make-image-tag.sh images/iptables)

make maker-image
make tester-image
make iptables-image
make compilers-image
make bpftool-image
make llvm-image

Note: The LLVM image build compiles LLVM from source inside Docker.

4.2. Build Cilium proxy (Envoy)

cd $SOURCE_ROOT
git clone https://github.com/cilium/proxy.git
cd proxy
git checkout b87d1e32
curl -sSL $PATCH_URL/proxy-s390x-aws-lc.patch | git apply -

# Clone and patch AWS-LC (used by the Docker build)
cd $SOURCE_ROOT
git clone https://github.com/aws/aws-lc.git
cd aws-lc
git checkout ec37c27edb7b5002955b68a5c834164a43ddcb14
curl -sSL $PATCH_URL/aws-lc-s390x.patch | git apply -
git add -A && git commit -m "Apply s390x patches"

export AWS_LC_LOCAL_COMMIT="$(git rev-parse --short=9 HEAD)"
export AWS_LC_LOCAL_SRC="$PWD"

cd $SOURCE_ROOT/proxy
export DOCKER_DEV_ACCOUNT=local
export ARCH=s390x
export AWS_LC_FIPS=1
export DOCKER_BUILD_OPTS="--load"

docker buildx use default

make docker-image-builder

export IMAGE_PUSH=false
export CARGO_BAZEL_REPIN=true
export ENVOY_IP_TEST_VERSIONS=v4only

make \
    AWS_LC_LOCAL_SRC="${AWS_LC_LOCAL_SRC}" \
    AWS_LC_LOCAL_COMMIT="${AWS_LC_LOCAL_COMMIT}" \
    EXTRA_BAZEL_BUILD_OPTS="--config=aws-lc-fips-http3-exp --jobs=3 --local_ram_resources=8192 --local_cpu_resources=3 --action_env=AWS_LC_FIPS=1 --verbose_failures --sandbox_debug" \
    docker-image-envoy

export CILIUM_ENVOY_IMAGE="local/cilium-envoy-dev:$(git rev-parse HEAD)-s390x"

4.3. Build Cilium images

cd $SOURCE_ROOT
git clone --depth 1 -b v1.19.4 https://github.com/cilium/cilium.git
cd cilium
curl -sSL $PATCH_URL/cilium-s390x.patch | git apply -
images/scripts/update-cni-version.sh 1.9.1
git add -A && git commit -m "Apply s390x patches"

export PUSH=false
export REGISTRIES=local
export OUTPUT=--load
export PLATFORMS=linux/s390x
export CILIUM_RUNTIME_IMAGE="local/cilium-runtime-dev:$(images/scripts/make-image-tag.sh images/runtime)"
export CILIUM_BUILDER_IMAGE="local/cilium-builder-dev:$(images/scripts/make-image-tag.sh images/builder)"

docker buildx use default
echo "default" > images/.buildx_builder

make -C images runtime-image
make -C images builder-image
make -C images cilium-image
OPERATOR_VARIANT=operator-generic make -C images operator-image
make -C images hubble-relay-image

4.4. Build ancillary images

# Certgen
cd $SOURCE_ROOT
git clone -b v0.4.3 https://github.com/cilium/certgen.git
cd certgen
DOCKER_IMAGE=local/certgen:latest DOCKER_IMAGE_TAG=latest make docker-image

# Hubble UI
cd $SOURCE_ROOT
git clone -b v0.13.5 https://github.com/cilium/hubble-ui.git
cd hubble-ui
curl -sSL $PATCH_URL/hubble-ui-byteorder.patch | git apply -
docker buildx build --platform linux/s390x --load -t local/hubble-ui:latest .
docker buildx build --platform linux/s390x --load -f backend/Dockerfile -t local/hubble-ui-backend:latest ./backend

# Alpine-curl
cd $SOURCE_ROOT
git clone -b v1.10.0 https://github.com/cilium/alpine-curl.git
cd alpine-curl
docker buildx build --platform linux/s390x --load -t local/alpine-curl:v1.10.0 .

# JSON-mock
cd $SOURCE_ROOT
git clone -b v1.3.8 https://github.com/cilium/json-mock.git
cd json-mock
docker buildx build --platform linux/s390x --load -t local/json-mock:v1.3.8 .

5. Deploy and test on Kind

This section deploys the locally built Cilium images on a Kind cluster to verify end-to-end functionality. Kind does not ship s390x binaries, so it must be built from source.

5.1. Build Kind

cd $SOURCE_ROOT
export KIND_PATCH_URL="https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kind/0.32.0/patch"

git clone -b v0.32.0 https://github.com/kubernetes-sigs/kind.git
cd kind
curl -sSL $KIND_PATCH_URL/kind.patch | git apply --ignore-whitespace -
make build
sudo cp bin/kind /usr/local/bin/kind

make -C images/base quick REGISTRY=kindest TAG=v20260601-995e8fa5
make -C images/kindnetd REGISTRY=kindest TAG=v20260528-9350166c quick
make -C images/local-path-provisioner REGISTRY=kindest TAG=v20260521-9fb22683 quick
make -C images/local-path-helper REGISTRY=kindest TAG=v20260131-7181c60a quick

5.2. Build kube-cross and kindest/node images

cd $SOURCE_ROOT
git clone -b v0.19.0 https://github.com/kubernetes/release.git
cd release
curl -sSL $KIND_PATCH_URL/release.patch | { cat; echo; } | git apply --ignore-whitespace -
cd images/build/cross
REGISTRY=local TARGETPLATFORM=s390x make container

cd $SOURCE_ROOT
git clone --depth 1 -b v1.36.1 https://github.com/kubernetes/kubernetes.git
cd kubernetes
sed -i 's,v1.36.0-go1.26.2-bullseye.0,v1.36.0-go1.26.2-trixie.0,g' build/build-image/cross/VERSION
KUBE_CROSS_IMAGE=local/kube-cross-s390x kind build node-image --image kindest/node:v1.36.1

5.3. Install kubectl and Helm

cd $SOURCE_ROOT
wget -q "https://dl.k8s.io/release/$(wget -qO- https://dl.k8s.io/release/stable-1.36.txt)/bin/linux/s390x/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin/kubectl

wget -q https://get.helm.sh/helm-v3.18.4-linux-s390x.tar.gz
tar -xzf helm-v3.18.4-linux-s390x.tar.gz
sudo mv linux-s390x/helm /usr/local/bin/helm
rm -rf linux-s390x helm-v3.18.4-linux-s390x.tar.gz

5.4. Create Kind cluster and deploy Cilium

cat <<EOF | kind create cluster --name cilium-test --image kindest/node:v1.36.1 --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  disableDefaultCNI: true
  podSubnet: "10.244.0.0/16"
nodes:
  - role: control-plane
  - role: worker
EOF

Determine the image tags and load images into the cluster:

cd $SOURCE_ROOT/cilium
CILIUM_TAG=$(docker images --format '{{.Tag}}' --filter "reference=local/cilium-dev" | head -1)

cd $SOURCE_ROOT/proxy
ENVOY_TAG=$(docker images --format '{{.Tag}}' --filter "reference=local/cilium-envoy-dev" | grep -v HEAD | head -1)

# Helm chart expects the -generic suffix on the operator image
docker tag "local/operator-dev:${CILIUM_TAG}" "local/operator-dev-generic:${CILIUM_TAG}"

docker pull --platform linux/s390x docker.io/coredns/coredns:1.12.0

# Load images into Kind nodes using ctr directly — kind load uses --all-platforms
# which fails when Docker's containerd store preserves multi-arch manifest indexes
NODES=$(kind get nodes --name cilium-test)
for img in \
    "local/cilium-dev:${CILIUM_TAG}" \
    "local/operator-dev-generic:${CILIUM_TAG}" \
    "local/hubble-relay-dev:${CILIUM_TAG}" \
    "local/cilium-envoy-dev:${ENVOY_TAG}" \
    "local/certgen:latest" \
    "local/alpine-curl:v1.10.0" \
    "local/json-mock:v1.3.8" \
    "docker.io/coredns/coredns:1.12.0"; do
    for node in $NODES; do
        docker save "$img" | docker exec -i "$node" \
            ctr --namespace=k8s.io images import --digests --snapshotter=overlayfs -
    done
done

Install Cilium with Helm using the local images:

helm repo add cilium https://helm.cilium.io/
helm repo update

helm install cilium cilium/cilium --version "1.19.4" \
    --namespace kube-system \
    --set image.repository="local/cilium-dev" \
    --set image.tag="${CILIUM_TAG}" \
    --set image.useDigest=false \
    --set image.pullPolicy=Never \
    --set operator.image.repository="local/operator-dev" \
    --set operator.image.tag="${CILIUM_TAG}" \
    --set operator.image.useDigest=false \
    --set operator.image.pullPolicy=Never \
    --set hubble.relay.image.repository="local/hubble-relay-dev" \
    --set hubble.relay.image.tag="${CILIUM_TAG}" \
    --set hubble.relay.image.useDigest=false \
    --set hubble.relay.image.pullPolicy=Never \
    --set envoy.image.repository="local/cilium-envoy-dev" \
    --set envoy.image.tag="${ENVOY_TAG}" \
    --set envoy.image.useDigest=false \
    --set envoy.image.pullPolicy=Never \
    --set certgen.image.repository="local/certgen" \
    --set certgen.image.tag="latest" \
    --set certgen.image.useDigest=false \
    --set certgen.image.pullPolicy=Never \
    --set hubble.enabled=true \
    --set hubble.relay.enabled=true \
    --set hubble.ui.enabled=false \
    --set ipam.operator.clusterPoolIPv4PodCIDRList="{10.244.0.0/16}"

Wait for Cilium to become ready:

kubectl -n kube-system rollout status deployment/cilium-operator --timeout=300s
kubectl -n kube-system rollout status daemonset/cilium --timeout=300s
kubectl wait --for=condition=Ready nodes --all --timeout=120s

5.5. Build Cilium CLI and run connectivity test

Build the Cilium CLI from source (no s390x binary is published):

cd $SOURCE_ROOT
git clone -b v0.18.5 https://github.com/cilium/cilium-cli.git
cd cilium-cli
sed -i 's/armbe || arm64be || mips || mips64 || ppc64/armbe || arm64be || mips || mips64 || ppc64 || s390x/' \
    vendor/github.com/cilium/cilium/pkg/byteorder/byteorder_bigendian.go
go build -o cilium ./cmd/cilium
sudo cp cilium /usr/local/bin/cilium
cilium version --client

Wait for the cluster to be fully ready, then run the official Cilium connectivity test suite:

cilium status --wait

cilium connectivity test --test-concurrency=1 \
    --external-target www.ibm.com. \
    --external-other-target k8s.io. \
    --curl-image "local/alpine-curl:v1.10.0" \
    --json-mock-image "local/json-mock:v1.3.8" \
    --dns-test-server-image "docker.io/coredns/coredns:1.12.0"

5.6. Cleanup

kind delete cluster --name cilium-test

6. Integration

For deploying Cilium on a Kubernetes cluster, refer to the Cilium Quick Installation Guide.

7. Cleanup

cd $SOURCE_ROOT
rm -rf go${GO_VERSION}.linux-s390x.tar.gz
rm -rf aws-lc/build-fips-s390x

References:

• https://github.com/cilium/cilium
• https://docs.cilium.io
• https://docs.cilium.io/en/stable/operations/system_requirements/