Building Envoy - linux-on-ibm-z/docs GitHub Wiki

Building Envoy

The instructions provided below specify the steps to build Envoy version 1.34.0 on Linux on IBM Z for the following distributions:

  • RHEL (8.10, 9.4, 9.5, 9.6)
  • SLES 15 SP6
  • Ubuntu (22.04, 24.04, 24.10, 25.04)

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Build and Install Envoy

1. Build using script

If you want to build Envoy using manual steps, go to Step 2.

Use the following commands to build Envoy using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Envoy/1.34.0/build_envoy.sh

# Build Envoy
bash build_envoy.sh   [Provide -t option for executing build with tests]

In case of error, check logs for more details or go to STEP 2 to follow manual build steps.

2. Install Dependencies

export SOURCE_ROOT=/<source_root>/
export PATCH_URL="https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Envoy/1.34.0/patch/"

2.1. Install Basic Dependencies

  • RHEL (8.10, 9.4, 9.5, 9.6)

    sudo yum install -y wget curl zip unzip patch clang-14.0.6 llvm-devel-14.0.6 gcc-toolset-12-gcc gcc-toolset-12-gcc-c++ gcc-toolset-12-libstdc++-devel gcc-toolset-12-binutils-devel gcc-toolset-12-binutils-gold gcc-toolset-12-annobin-plugin-gcc gcc-toolset-12-libatomic-devel pkgconf-pkg-config openssl-devel java-21-openjdk-devel python3.11
    
    # set gcc 12 as default
    source /opt/rh/gcc-toolset-12/enable
    
    # set JAVA_HOME location
    export JAVA_HOME=/usr/lib/jvm/java-21-openjdk
    export PATH=$JAVA_HOME/bin:$PATH
    

    Note: Add the '--allowerasing' option to the yum command above if any dependency conflicts are encountered for RHEL 9.*.

  • SLES 15 SP6

    sudo zypper addrepo --priority 199 http://download.opensuse.org/distribution/leap/15.6/repo/oss/ oss
    sudo zypper --gpg-auto-import-keys refresh -r oss
    
    sudo zypper install -y awk autoconf curl libtool patch pkg-config wget git gcc12-c++ gcc-c++ clang14 llvm14-devel binutils-gold gzip make cmake python311 java-21-openjdk-devel unzip zip tar xz libopenssl-devel
    
    # set JAVA_HOME location
    export JAVA_HOME=/usr/lib64/jvm/java-21-openjdk
    export PATH=$JAVA_HOME/bin:$PATH
    
  • Ubuntu 22.04

    sudo apt-get update
    sudo apt-get install -y autoconf curl git libtool patch python3-pip unzip virtualenv pkg-config locales libssl-dev build-essential openjdk-21-jdk-headless python2 python2-dev python3 python3-dev zip
    
    # Install GCC 12 from repo
    sudo apt-get install -y gcc-12 g++-12
    
    sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-12 12
    sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12
    sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-12 12
    sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-12 12
    
    export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-s390x
    export PATH=$JAVA_HOME/bin:$PATH
    
  • Ubuntu (24.04, 24.10, 25.04)

    sudo apt-get update
    sudo apt-get install -y autoconf curl wget git libtool patch python3-pip virtualenv pkg-config locales clang-14 gcc g++ libstdc++-12-dev openssl libssl-dev build-essential openjdk-21-jdk-headless python3 zip unzip binutils-gold
    
    export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-s390x
    export PATH=$JAVA_HOME/bin:$PATH
    

2.2. Install Bazel 7.6.0

cd $SOURCE_ROOT
mkdir bazel && cd bazel
wget https://github.com/bazelbuild/bazel/releases/download/7.6.0/bazel-7.6.0-dist.zip
unzip -q bazel-7.6.0-dist.zip
chmod -R +w .
env EXTRA_BAZEL_ARGS="--tool_java_runtime_version=local_jdk" bash ./compile.sh
sudo cp output/bazel /usr/local/bin/

2.3. Install Clang (Only for Ubuntu 22.04)

    cd "$SOURCE_ROOT"
    sudo apt-get update
    sudo apt-get install -y lsb-release wget software-properties-common gnupg
    wget https://apt.llvm.org/llvm.sh
    chmod +x llvm.sh
    sudo ./llvm.sh 14

2.4. Install Rust

    cd $SOURCE_ROOT
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh /dev/stdin -y
    export PATH="$HOME/.cargo/bin:$PATH"

2.5. Install Go 1.24.2

    cd "$SOURCE_ROOT"
    wget -q https://golang.org/dl/go1.24.2.linux-s390x.tar.gz
    chmod ugo+r go1.24.2.linux-s390x.tar.gz
    sudo rm -rf /usr/local/go
    sudo tar -C /usr/local -xzf go1.24.2.linux-s390x.tar.gz
    export PATH=/usr/local/go/bin:$PATH
    go version

2.6. Setup GCC libstdc++ 12 for use with Clang (Only for Ubuntu 24.04, 24.10 and 25.04)

    sudo ln -s "/usr/include/s390x-linux-gnu/c++/12" "/usr/include/s390x-linux-gnu/c++/99"
    sudo ln -s "/usr/include/c++/12" "/usr/include/c++/99"
    sudo ln -s "/usr/lib/gcc/s390x-linux-gnu/12" "/usr/lib/gcc/s390x-linux-gnu/99"

2.7. Build and install cargo-bazel from rules_rust

    cd "$SOURCE_ROOT"
    git clone -b "$RULES_RUST_VERSION" --depth 1 https://github.com/bazelbuild/rules_rust.git
    cd rules_rust/crate_universe/
    cargo build --release --locked --bin cargo-bazel
    cp target/release/cargo-bazel "$SOURCE_ROOT"/

2.8. Build and install buf binaries for rules_buf

    cd "$SOURCE_ROOT"
    BUF_DIR="${SOURCE_ROOT}/rules_buf/local"
    mkdir -p "$BUF_DIR"
    git clone -b "v$BUF_VERSION" --depth 1 https://github.com/bufbuild/buf.git
    cd buf
    GOBIN="$BUF_DIR" go install "github.com/bufbuild/buf/cmd/buf@v${BUF_VERSION}"
    mv "$BUF_DIR/buf" "$BUF_DIR/buf-Linux-s390x"
    GOBIN="$BUF_DIR" go install "github.com/bufbuild/buf/cmd/protoc-gen-buf-breaking@v${BUF_VERSION}"
    mv "$BUF_DIR/protoc-gen-buf-breaking" "$BUF_DIR/protoc-gen-buf-breaking-Linux-s390x"
    GOBIN="$BUF_DIR" go install "github.com/bufbuild/buf/cmd/protoc-gen-buf-lint@v${BUF_VERSION}"
    mv "$BUF_DIR/protoc-gen-buf-lint" "$BUF_DIR/protoc-gen-buf-lint-Linux-s390x"
    cd "$BUF_DIR"
    sha256sum buf-Linux-s390x protoc-gen-buf-breaking-Linux-s390x protoc-gen-buf-lint-Linux-s390x > sha256.txt

3. Build and Install Envoy

  • Download Envoy and Apply Patches

    cd "$SOURCE_ROOT"/
    git clone --depth 1 -b 1.34.0 https://github.com/envoyproxy/envoy.git
    cd envoy
    
    ./bazel/setup_clang.sh "/usr" ### For RHEL and SLES only
    ./bazel/setup_clang.sh "/usr/lib/llvm-14" ### For Ubuntu only
    
    # Set the location of the s390x cargo-bazel binary required by rules_rust
    echo "build --repo_env=CARGO_BAZEL_GENERATOR_URL=file:${SOURCE_ROOT}/cargo-bazel" >> "${SOURCE_ROOT}/envoy/user.bazelrc"
    # Set the location of the s390x buf binaries required by rules_buf
    echo "build --repo_env=BUFBUILD_BUF_TOOLCHAIN_URL=file:${SOURCE_ROOT}/rules_buf" >> "${SOURCE_ROOT}/envoy/user.bazelrc"
    # Disable heap checking because it slows down the tests and causes timeouts
    echo "build --test_env=HEAPCHECK=" >> "${SOURCE_ROOT}/envoy/user.bazelrc"
    echo "test --test_env=HEAPCHECK=" >> "${SOURCE_ROOT}/envoy/user.bazelrc"
    
    # Apply patches to allow envoy to build
    curl -sSL $PATCH_URL/envoy-build.patch | git apply -
    
    # Apply patches for failing tests
    curl -sSL $PATCH_URL/envoy-test.patch | git apply -
    
    # Move patch files to envoy/bazel which will be applied to external packages while building envoy
    curl -sSL $PATCH_URL/boringssl-s390x.patch > $SOURCE_ROOT/envoy/bazel/boringssl-s390x.patch
    curl -sSL $PATCH_URL/quiche-s390x.patch > $SOURCE_ROOT/envoy/bazel/external/quiche-s390x.patch
    curl -sSL $PATCH_URL/proxy_wasm_cpp_host-s390x.patch > $SOURCE_ROOT/envoy/bazel/proxy_wasm_cpp_host-s390x.patch
    curl -sSL $PATCH_URL/rules_foreign_cc-s390x.patch > $SOURCE_ROOT/envoy/bazel/rules_foreign_cc-s390x.patch
    curl -sSL https://github.com/iii-i/moonjit/commit/dee73f516f0da49e930dcfa1dd61720dcb69b7dd.patch > $SOURCE_ROOT/envoy/bazel/foreign_cc/luajit-s390x.patch
    curl -sSL https://github.com/iii-i/moonjit/commit/035f133798adb856391928600f7cb6b4f81578ab.patch >> $SOURCE_ROOT/envoy/bazel/foreign_cc/luajit-s390x.patch
    curl -sSL $PATCH_URL/luajit-as.patch > $SOURCE_ROOT/envoy/bazel/foreign_cc/luajit-as.patch
    curl -sSL $PATCH_URL/rules_buf-s390x.patch > $SOURCE_ROOT/envoy/api/bazel/rules_buf-s390x.patch
    curl -sSL $PATCH_URL/grpc-s390x.patch > $SOURCE_ROOT/envoy/bazel/grpc-s390x.patch
    
  • Build Envoy

    bazel build envoy -c opt --config=clang 
    

The binary will be generated in $SOURCE_ROOT/envoy/bazel-bin/source/exe/envoy-static.

  • Verify the version of Envoy
    $SOURCE_ROOT/envoy/bazel-bin/source/exe/envoy-static --version
    
    Output should be similar to:
    bazel-bin/source/exe/envoy-static  version: d7809ba2b07fd869d49bfb122b27f6a7977b4d94/1.34.0/Modified/RELEASE/BoringSSL
    

4. Test (optional)

cd $SOURCE_ROOT/envoy
  • Run Tests
    bazel test //test/... -c opt --config=clang --keep_going
    

Notes:

  • Below mentioned test failures are observed on all distros on Intel and s390x:

    • //test/extensions/filters/listener/original_dst:original_dst_integration_test: To fix the test, follow official doc to set up the original_dst filter for a quick manual test. To run this test after applying this fix, the following command can be used.
      bazel test //test/extensions/filters/listener/original_dst:original_dst_integration_test -c opt --config=clang
      
  • Below mentioned test failure is observed on all RHEL and SLES distros on Intel and s390x:

    //test/config_test:example_configs_test: This test fails because the file "/etc/ssl/certs/ca-certificates.crt" only exists on debian like systems.

  • Some tests are flaky and may fail randomly. Individual tests can be re-run with a command like:

    ```bash
    bazel test <bazel test target> -c opt --config=clang
    ```
    

References: