Building Envoy - linux-on-ibm-z/docs GitHub Wiki

Building Envoy

The instructions provided below specify the steps to build Envoy version 1.37.1 on Linux on IBM Z for the following distributions:

  • RHEL (8.10, 9.6, 9.7, 10.0, 10.1)
  • SLES (15 SP7, 16)
  • Ubuntu (22.04, 24.04, 25.10)

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Build and Install Envoy

1. Build using script

If you want to build Envoy using manual steps, go to Step 2.

Use the following commands to build Envoy using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Envoy/1.37.1/build_envoy.sh

# Build Envoy
bash build_envoy.sh   [Provide -t option for executing build with tests]

In case of error, check logs for more details or go to STEP 2 to follow manual build steps.

2. Install Dependencies

export SOURCE_ROOT=/<source_root>/
export PATCH_URL="https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Envoy/1.37.1/patch/"

2.1. Install Basic Dependencies

  • RHEL (8.10, 9.6, 9.7)

    sudo yum install -y wget curl zip unzip patch gcc-toolset-12-gcc gcc-toolset-12-gcc-c++ gcc-toolset-12-libstdc++-devel gcc-toolset-12-binutils-devel gcc-toolset-12-annobin-plugin-gcc gcc-toolset-12-libatomic-devel pkgconf-pkg-config openssl-devel java-21-openjdk-devel python3.11 file

# set gcc 12 as default
source /opt/rh/gcc-toolset-12/enable

# set JAVA_HOME location
export JAVA_HOME=/usr/lib/jvm/java-21-openjdk
export PATH=$JAVA_HOME/bin:$PATH

  • RHEL (10.0, 10.1)

    sudo yum install -y wget curl zip unzip patch gcc gcc-c++ libatomic libstdc++-devel pkgconf-pkg-config openssl-devel java-21-openjdk-devel python3 file diffutils

# set JAVA_HOME location
export JAVA_HOME=/usr/lib/jvm/java-21-openjdk
export PATH=$JAVA_HOME/bin:$PATH

  • SLES 15 SP7

    sudo zypper install -y awk autoconf curl libtool patch pkg-config wget git gcc-c++ gzip make cmake python311 java-21-openjdk-devel unzip zip tar xz libopenssl-devel zlib-devel which

# set JAVA_HOME location
export JAVA_HOME=/usr/lib64/jvm/java-21-openjdk
export PATH=$JAVA_HOME/bin:$PATH

  • SLES 16

    sudo zypper install -y awk autoconf curl libtool patch pkg-config wget git gcc13-c++ gzip make cmake python3 java-21-openjdk-devel unzip zip tar xz libopenssl-devel zlib-devel which

sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-13 12
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-13 12
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-13 12
sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-13 12

# set JAVA_HOME location
export JAVA_HOME=/usr/lib64/jvm/java-21-openjdk
export PATH=$JAVA_HOME/bin:$PATH

  • Ubuntu 22.04

    sudo apt-get update
sudo apt-get install -y autoconf curl wget git libtool patch python3-pip unzip virtualenv pkg-config locales libssl-dev build-essential openjdk-21-jdk-headless python2 python2-dev python3 python3-dev zip

# Install GCC 12 from repo
sudo apt-get install -y gcc-12 g++-12

sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-12 12
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-12 12
sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-12 12

export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-s390x
export PATH=$JAVA_HOME/bin:$PATH

  • Ubuntu 24.04

    sudo apt-get update
sudo apt-get install -y autoconf curl wget git libtool patch python3-pip virtualenv pkg-config locales gcc g++ openssl libssl-dev build-essential openjdk-21-jdk-headless python3 zip unzip

export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-s390x
export PATH=$JAVA_HOME/bin:$PATH

  • Ubuntu 25.10

    sudo apt-get update
sudo apt-get install -y autoconf curl wget git libtool patch python3-pip virtualenv pkg-config locales openssl libssl-dev build-essential openjdk-21-jdk-headless python3 zip unzip | tee -a "${LOG_FILE}"

sudo apt-get install -y gcc-13 g++-13
sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-13 12
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-13 12
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-13 12
sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-13 12

export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-s390x
export PATH=$JAVA_HOME/bin:$PATH

2.2. Build and Install Clang

2.2.1 Install Dependencies
  • RHEL (8.10, 9.6, 9.7, 10.0, 10.1)
sudo dnf install -y "https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm" # Only for RHEL 8.*
sudo dnf install -y "https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm" # Only for RHEL 9.*
sudo dnf install -y "https://dl.fedoraproject.org/pub/epel/epel-release-latest-10.noarch.rpm" # Only for RHEL 10.*
sudo yum install -y gcc-c++ curl git cmake ninja-build chrpath elfutils-libelf-devel libffi-devel patchutils xz xz-libs python3 libedit-devel ncurses-devel binutils-devel libxml2-devel jsoncpp-devel pkg-config procps zlib-devel libzstd-devel libpfm-devel
  • SLES 15 SP7
sudo zypper install -y gcc-c++ curl git cmake ninja chrpath libelf-devel libffi-devel patchutils xz xz-devel python3 libedit-devel ncurses-devel binutils-devel libxml2-devel jsoncpp-devel pkg-config procps zlib-devel libzstd-devel libpfm-devel
  • SLES 16
sudo zypper install -y gcc13-c++ curl git cmake ninja chrpath libelf-devel libffi-devel patchutils xz xz-devel python3 libedit-devel ncurses-devel binutils-devel libxml2-devel jsoncpp-devel pkg-config procps zlib-devel libzstd-devel libpfm-devel

Note: GCC 13 is required for SLES 16 because GCC 15 (the default) cannot compile LLVM/Clang 18.1.8 stage 2. The build configuration will automatically use GCC 13 via CMake BOOTSTRAP flags.

  • Ubuntu (22.04, 24.04)
sudo apt-get update
sudo apt-get install -y g++ curl git cmake ninja-build chrpath libelf-dev libffi-dev patchutils xz-utils python3 libedit-dev libncurses-dev binutils-dev libxml2-dev libjsoncpp-dev pkg-config procps zlib1g-dev libzstd-dev libpfm4-dev
  • Ubuntu 25.10
sudo apt-get update
sudo apt-get install -y g++-13 curl git cmake ninja-build chrpath libelf-dev libffi-dev patchutils xz-utils python3 libedit-dev libncurses-dev binutils-dev libxml2-dev libjsoncpp-dev pkg-config procps zlib1g-dev libzstd-dev libpfm4-dev

Note: GCC 13 is required for Ubuntu 25.10 because GCC 15 (the default) cannot compile LLVM/Clang 18.1.8 stage 2. The build configuration will automatically use GCC 13 via CMake BOOTSTRAP flags.

2.2.2 Configure Build
mkdir -p "$SOURCE_ROOT/clang-build"
cd "$SOURCE_ROOT/clang-build"
cat << 'EOF' > Release-s390x.cmake
set_instrument_and_final_stage_var(LLVM_TARGETS_TO_BUILD "Native" STRING)

set(COMPILER_RT_USE_BUILTINS_LIBRARY OFF CACHE BOOL "")
set_instrument_and_final_stage_var(COMPILER_RT_USE_BUILTINS_LIBRARY "OFF" BOOL)
set(COMPILER_RT_BUILD_BUILTINS OFF CACHE BOOL "")
set_instrument_and_final_stage_var(COMPILER_RT_BUILD_BUILTINS "OFF" BOOL)
set(LIBCXX_USE_COMPILER_RT OFF CACHE BOOL "")
set_instrument_and_final_stage_var(LIBCXX_USE_COMPILER_RT "OFF" BOOL)
set(LIBCXXABI_USE_COMPILER_RT OFF CACHE BOOL "")
set_instrument_and_final_stage_var(LIBCXXABI_USE_COMPILER_RT "OFF" BOOL)
set(LIBCXXABI_USE_LLVM_UNWINDER OFF CACHE BOOL "")
set_instrument_and_final_stage_var(LIBCXXABI_USE_LLVM_UNWINDER "OFF" BOOL)
set(LLVM_BUILD_DOCS OFF CACHE BOOL "")
set_instrument_and_final_stage_var(LLVM_BUILD_DOCS "OFF" BOOL)
set(LLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN ON CACHE BOOL "")
set_instrument_and_final_stage_var(LLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN "ON" BOOL)
set(LLVM_ENABLE_CURL OFF CACHE BOOL "")
set_instrument_and_final_stage_var(LLVM_ENABLE_CURL "OFF" BOOL)

set_instrument_and_final_stage_var(PACKAGE_VENDOR "LoZ Open Source Ecosystem (https://www.ibm.com/community/z/usergroups/opensource)" STRING)

if(LLVM_RELEASE_CLANG_SYSTEMZ_DEFAULT_ARCH)
  set(CLANG_SYSTEMZ_DEFAULT_ARCH "${LLVM_RELEASE_CLANG_SYSTEMZ_DEFAULT_ARCH}" CACHE STRING "")
  set_instrument_and_final_stage_var(CLANG_SYSTEMZ_DEFAULT_ARCH "${LLVM_RELEASE_CLANG_SYSTEMZ_DEFAULT_ARCH}" STRING)
endif()
EOF

git clone -b "llvmorg-18.1.8" --depth=1 https://github.com/llvm/llvm-project.git
cd llvm-project
curl -sSL https://github.com/llvm/llvm-project/commit/a356e6ccada87d6bfc4513fba4b1a682305e094a.patch | git apply -
curl -sSL https://github.com/llvm/llvm-project/commit/ddaa5b3bfb2980f79c6f277608ad33a6efe8d554.patch | git apply -
curl -sSL $PATCH_URL/clang-ubuntu-2510.patch | git apply - # Only for Ubuntu 25.10
2.2.3 Build Clang

The Stage 2 Clang build must use GCC 13 on SLES 16 and Ubuntu 25.10. The commands below select the appropriate host compiler and pass its GCC installation directory to the bootstrap build. For all other supported distributions, the system GCC/G++ toolchain is used.

cd "$SOURCE_ROOT/clang-build"
mkdir build

# Identify the distribution and select the System Z default architecture.
source /etc/os-release
DISTRO="$ID-$VERSION_ID"
z_default_arch="z13"

case "$DISTRO" in
  rhel-9.* | rhel-10.* | sles-16.*)
    z_default_arch="z14"
    ;;
  sles-15.*)
    z_default_arch="zEC12"
    ;;
esac

# Use GCC 13 for the Stage 2 build on SLES 16 and Ubuntu 25.10.
host_cc="gcc"
host_cxx="g++"
gcc_install_dir=""
cmake_extra_args=()

case "$DISTRO" in
  sles-16.*)
    host_cc="gcc-13"
    host_cxx="g++-13"
    gcc_install_dir="/usr/lib64/gcc/s390x-suse-linux/13"
    ;;
  ubuntu-25.10)
    host_cc="gcc-13"
    host_cxx="g++-13"
    gcc_install_dir="/usr/lib/gcc/s390x-linux-gnu/13"
    ;;
esac

if [[ -n "$gcc_install_dir" ]]; then
  cmake_extra_args=(
    -DGCC_INSTALL_PREFIX=/usr
    -DCLANG_DEFAULT_CXX_STDLIB=libstdc++
    -DCLANG_DEFAULT_RTLIB=libgcc
    -DBOOTSTRAP_CMAKE_C_FLAGS="--gcc-install-dir=${gcc_install_dir}"
    -DBOOTSTRAP_CMAKE_CXX_FLAGS="--gcc-install-dir=${gcc_install_dir}"
    -DBOOTSTRAP_CMAKE_EXE_LINKER_FLAGS="--gcc-install-dir=${gcc_install_dir}"
    -DBOOTSTRAP_CMAKE_SHARED_LINKER_FLAGS="--gcc-install-dir=${gcc_install_dir}"
    -DBOOTSTRAP_CMAKE_MODULE_LINKER_FLAGS="--gcc-install-dir=${gcc_install_dir}"
  )
fi

env CC="gcc" CXX="g++" \
  cmake -G "Ninja" -B build -S llvm-project/llvm \
    -DLLVM_RELEASE_ENABLE_LTO="OFF" \
    -DLLVM_PARALLEL_LINK_JOBS=4 \
    -DBOOTSTRAP_LLVM_PARALLEL_LINK_JOBS=4 \
    -DLLVM_RELEASE_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi" \
    -DLLVM_RELEASE_ENABLE_PROJECTS="clang;lld;clang-tools-extra" \
    -DLLVM_RELEASE_CLANG_SYSTEMZ_DEFAULT_ARCH="$z_default_arch" \
    "${cmake_extra_args[@]}" \
    -C llvm-project/clang/cmake/caches/Release.cmake \
    -C Release-s390x.cmake

ninja -C build stage2-package

cd "$SOURCE_ROOT"
tar xf "${SOURCE_ROOT}/clang-build/build/tools/clang/stage2-bins/LLVM-18.1.8-Linux.tar.gz"
rm -rf "${SOURCE_ROOT}/clang-build"

2.3. Install Bazel 7.7.1

cd $SOURCE_ROOT mkdir bazel && cd bazel wget https://github.com/bazelbuild/bazel/releases/download/7.7.1/bazel-7.7.1-dist.zip unzip -q bazel-7.7.1-dist.zip chmod -R +w . env EXTRA_BAZEL_ARGS="--tool_java_runtime_version=local_jdk" BAZEL_DEV_VERSION_OVERRIDE="7.7.1"bash ./compile.sh sudo cp output/bazel /usr/local/bin/

2.4. Install Rust

cd $SOURCE_ROOT
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh /dev/stdin -y
export PATH="$HOME/.cargo/bin:$PATH"

2.5. Install Go 1.24.6

cd "$SOURCE_ROOT"
wget -q https://golang.org/dl/go1.24.6.linux-s390x.tar.gz
chmod ugo+r go1.24.6.linux-s390x.tar.gz
sudo rm -rf /usr/local/go
sudo tar -C /usr/local -xzf go1.24.6.linux-s390x.tar.gz
export PATH=/usr/local/go/bin:$PATH
go version

2.6. Build and install cargo-bazel from rules_rust

cd "$SOURCE_ROOT"
git clone -b 0.68.1 --depth 1 https://github.com/bazelbuild/rules_rust.git
cd rules_rust/crate_universe/
cargo build --release --locked --bin cargo-bazel
cp target/release/cargo-bazel "$SOURCE_ROOT"/

3. Build and Install Envoy

  • Download Envoy and Apply Patches

    cd "$SOURCE_ROOT"/
git clone --depth 1 -b v1.37.1 https://github.com/envoyproxy/envoy.git
cd envoy

# Set the location of the s390x cargo-bazel binary required by rules_rust
echo "build --repo_env=CARGO_BAZEL_GENERATOR_URL=file:${SOURCE_ROOT}/cargo-bazel" >> "${SOURCE_ROOT}/envoy/user.bazelrc"
# Disable heap checking because it slows down the tests and causes timeouts
echo "build --test_env=HEAPCHECK=" >> "${SOURCE_ROOT}/envoy/user.bazelrc"
echo "test --test_env=HEAPCHECK=" >> "${SOURCE_ROOT}/envoy/user.bazelrc"
# Ensure LLVM toolchain is available for all operations
echo "common --repo_env=BAZEL_LLVM_PATH=${LLVM_HOME_DIR}" >> "${SOURCE_ROOT}/envoy/user.bazelrc"
echo "common --repo_env=BAZEL_LLVM_PATH=${LLVM_HOME_DIR}" >> "${SOURCE_ROOT}/envoy/user.bazelrc"

# Apply patches to allow envoy to build
curl -sSL $PATCH_URL/envoy.patch | git apply -

# Move patch files to envoy/bazel which will be applied to external packages while building envoy
curl -sSL $PATCH_URL/boringssl-s390x.patch > $SOURCE_ROOT/envoy/bazel/boringssl-s390x.patch
curl -sSL $PATCH_URL/quiche-s390x.patch > $SOURCE_ROOT/envoy/bazel/external/quiche-s390x.patch
curl -sSL $PATCH_URL/proxy_wasm_cpp_host-s390x.patch > $SOURCE_ROOT/envoy/bazel/proxy_wasm_cpp_host-s390x.patch
curl -sSL $PATCH_URL/rules_foreign_cc-s390x.patch > $SOURCE_ROOT/envoy/bazel/rules_foreign_cc-s390x.patch
curl -sSL https://github.com/iii-i/moonjit/commit/dee73f516f0da49e930dcfa1dd61720dcb69b7dd.patch > $SOURCE_ROOT/envoy/bazel/foreign_cc/luajit-s390x.patch
curl -sSL https://github.com/iii-i/moonjit/commit/035f133798adb856391928600f7cb6b4f81578ab.patch >> $SOURCE_ROOT/envoy/bazel/foreign_cc/luajit-s390x.patch
curl -sSL https://github.com/openresty/luajit2/commit/e598aeb7426dbc069f90ba70db9bce43cd573b0e.patch >> $SOURCE_ROOT/envoy/bazel/foreign_cc/luajit-s390x.patch
curl -sSL $PATCH_URL/highway-s390x.patch > $SOURCE_ROOT/envoy/bazel/highway-s390x.patch
curl -sSL $PATCH_URL/luajit-as.patch > $SOURCE_ROOT/envoy/bazel/foreign_cc/luajit-as.patch
curl -sSL $PATCH_URL/grpc-s390x.patch > $SOURCE_ROOT/envoy/bazel/grpc-s390x.patch
curl -sSL $PATCH_URL/toolchains_llvm-s390x.patch > $SOURCE_ROOT/envoy/bazel/toolchains_llvm-s390x.patch
curl -sSL $PATCH_URL/protobuf-s390x.patch > $SOURCE_ROOT/envoy/bazel/protobuf-s390x.patch

  • Build Envoy

    bazel build envoy -c opt --config=clang

The binary will be generated in $SOURCE_ROOT/envoy/bazel-bin/source/exe/envoy-static.

  • Verify the version of Envoy 
    $SOURCE_ROOT/envoy/bazel-bin/source/exe/envoy-static --version
    Output should be similar to: 
    bazel-bin/source/exe/envoy-static  version: 5ef4e4cea57f63e7e2970b9c1ad696278db927d6/1.37.1/Modified/RELEASE/BoringSSL

4. Test (optional)

cd $SOURCE_ROOT/envoy
  • Run Tests 
    bazel test //test/... -c opt --config=clang --keep_going --test_output=errors

Notes:

  • Some tests are flaky and may fail randomly. Individual tests can be re-run with a command like:

    ```bash
bazel test <bazel test target> -c opt --config=clang
```

References:

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue if you have any questions or feedback.

⚠️ **GitHub.com Fallback** ⚠️