Building Bazel - linux-on-ibm-z/docs GitHub Wiki
Building Bazel
Following instructions specify the steps to build Bazel version 7.2.1 on Linux on IBM Z:
- Ubuntu (20.04, 22.04)
General Notes:
- When following the steps below please use a standard permission user unless otherwise specified.
- A directory
/<source_root>/
will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.
Step 1 : Building Bazel
1.1) Build using script
If you'd like to build Bazel using the manual steps, please go to STEP 1.2.
Use the following commands to build Bazel using the build script. Please make sure you have wget installed.
wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/7.2.1/build_bazel.sh
# Build Bazel
bash build_bazel.sh [Provide -t option for executing build with tests]
In case of any errors, check logs
for more details or go to STEP 1.2 to follow the manual build steps.
1.2) Install dependencies
- Set environment variable
SOURCE_ROOT
:
export SOURCE_ROOT=/<source_root>/
sudo apt-get update
- Ubuntu 20.04
sudo apt-get install -y --no-install-recommends bind9-host build-essential coreutils curl dnsutils ed expect file git gnupg2 iproute2 iputils-ping mkisofs lcov less libssl-dev lsb-release netcat-openbsd openjdk-11-jdk-headless openjdk-21-jdk-headless zip zlib1g-dev unzip wget python2 python2-dev python-is-python3 python3 python3-dev python3-pip python3-requests python3-setuptools python3-six python3-wheel python3-yaml
- Ubuntu 22.04
sudo apt-get install -y --no-install-recommends bind9-host build-essential coreutils curl dnsutils ed expect file git gnupg2 iproute2 iputils-ping mkisofs lcov less libssl-dev lsb-release netcat-openbsd openjdk-11-jdk-headless openjdk-21-jdk-headless zip zlib1g-dev unzip wget python2 python2-dev python3
sudo ln -s /usr/bin/python3 /usr/bin/python
1.3) Build
Note: Netty and netty-tcnative jar are needed to fix test case failures related to remote execution. If remote execution functionality is not required then go to "Set JAVA_HOME to JDK21 required for building Bazel" step.
- Install and build netty-tcnative 2.0.61:
cd $SOURCE_ROOT
wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/netty-tcnative/2.0.61/build_netty.sh
bash build_netty.sh -y
- Set JAVA_HOME to JDK11 required for building netty:
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-s390x
export PATH=$JAVA_HOME/bin:$PATH
- Build netty 4.1.93:
cd $SOURCE_ROOT
git clone https://github.com/netty/netty.git
cd netty
git checkout netty-4.1.93.Final
mvn clean install -DskipTests
- Set JAVA_HOME to JDK21 required for building Bazel:
export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-s390x
export PATH=$JAVA_HOME/bin:$PATH
- Download and patch rules_java 7.6.1:
cd $SOURCE_ROOT
git clone -b 7.6.1 https://github.com/bazelbuild/rules_java.git
cd rules_java
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/7.2.1/patch/rules_java_7.6.1.patch | git apply
- Download and patch rules_graalvm 0.11.1:
cd $SOURCE_ROOT
git clone -b v0.11.1 https://github.com/sgammon/rules_graalvm
cd rules_graalvm
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/7.2.1/patch/rules_graalvm_0.11.1.patch | git apply
- Download Bazel 7.2.1 distribution archive:
cd $SOURCE_ROOT
wget https://github.com/bazelbuild/bazel/releases/download/7.2.1/bazel-7.2.1-dist.zip
mkdir -p dist/bazel && cd dist/bazel
unzip -q ../../bazel-7.2.1-dist.zip
chmod -R +w .
- Build the bootstrap bazel:
cd ${SOURCE_ROOT}/dist/bazel
env EXTRA_BAZEL_ARGS="--tool_java_runtime_version=local_jdk" bash ./compile.sh
- Checkout the Bazel 7.2.1 source code and apply the patch:
cd $SOURCE_ROOT
git clone --depth 1 -b 7.2.1 https://github.com/bazelbuild/bazel.git
cd bazel
curl -sSLO https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/7.2.1/patch/bazel.patch
sed -i "s#RULES_JAVA_ROOT_PATH#${SOURCE_ROOT}#g" bazel.patch
patch -p1 < bazel.patch
rm -f bazel.patch
- Copy the netty and netty-tcnative jar into the respective bazel directory:
cp $SOURCE_ROOT/netty-tcnative/openssl-classes/target/netty-tcnative-classes-2.0.61.Final.jar \
$SOURCE_ROOT/netty-tcnative/boringssl-static/target/netty-tcnative-boringssl-static-2.0.61.Final-linux-s390_64.jar \
$SOURCE_ROOT/netty/buffer/target/netty-buffer-4.1.93.Final.jar \
$SOURCE_ROOT/netty/codec/target/netty-codec-4.1.93.Final.jar \
$SOURCE_ROOT/netty/codec-http/target/netty-codec-http-4.1.93.Final.jar \
$SOURCE_ROOT/netty/codec-http2/target/netty-codec-http2-4.1.93.Final.jar \
$SOURCE_ROOT/netty/common/target/netty-common-4.1.93.Final.jar \
$SOURCE_ROOT/netty/handler/target/netty-handler-4.1.93.Final.jar \
$SOURCE_ROOT/netty/handler-proxy/target/netty-handler-proxy-4.1.93.Final.jar \
$SOURCE_ROOT/netty/resolver/target/netty-resolver-4.1.93.Final.jar \
$SOURCE_ROOT/netty/resolver-dns/target/netty-resolver-dns-4.1.93.Final.jar \
$SOURCE_ROOT/netty/transport/target/netty-transport-4.1.93.Final.jar \
$SOURCE_ROOT/netty/transport-classes-epoll/target/netty-transport-classes-epoll-4.1.93.Final.jar \
$SOURCE_ROOT/netty/transport-classes-kqueue/target/netty-transport-classes-kqueue-4.1.93.Final.jar \
$SOURCE_ROOT/netty/transport-native-unix-common/target/netty-transport-native-unix-common-4.1.93.Final-linux-s390_64.jar \
$SOURCE_ROOT/netty/transport-native-kqueue/target/netty-transport-native-kqueue-4.1.93.Final.jar \
$SOURCE_ROOT/netty/transport-native-epoll/target/netty-transport-native-epoll-4.1.93.Final-linux-s390_64.jar \
$SOURCE_ROOT/bazel/third_party
- Build Bazel:
cd $SOURCE_ROOT/bazel
${SOURCE_ROOT}/dist/bazel/output/bazel build -c opt --stamp --embed_label "7.2.1" //src:bazel //src:bazel_jdk_minimal //src:test_repos //src/main/java/...
mkdir output
cp bazel-bin/src/bazel output/bazel
- Bazel version:
cd $SOURCE_ROOT/bazel
./output/bazel --version
bazel 7.2.1
Step 2: Execute Test Suite (Optional)
- Run complete test suite:
cd $SOURCE_ROOT/bazel
./output/bazel --host_jvm_args=-Xmx2g test -c opt --build_tests_only --flaky_test_attempts=3 --test_timeout=3600 --show_progress_rate_limit=5 --terminal_columns=143 --show_timestamps --verbose_failures \
-- //scripts/... //src/java_tools/... //src/test/... //src/tools/execlog/... //src/tools/singlejar/... //src/tools/workspacelog/... //third_party/ijar/... -//tools/android/... //tools/aquery_differ/... //tools/python/... \
-//src/java_tools/import_deps_checker/... -//src/test/shell/bazel/android/... -//src/test/java/com/google/devtools/build/android/... -//src/test/shell/bazel:bazel_determinism_test -//src/test/shell/bazel:bazel_android_tools_test
- Run individual test:
cd $SOURCE_ROOT/bazel
./output/bazel test -c opt --cache_test_results=no --build_tests_only --flaky_test_attempts=3 --test_timeout=3600 --show_progress_rate_limit=5 --show_timestamps --verbose_failures -- //<module_name>:<testcase_name>
For example:
cd $SOURCE_ROOT/bazel
./output/bazel test -c opt --cache_test_results=no --build_tests_only --flaky_test_attempts=3 --test_timeout=3600 --show_progress_rate_limit=5 --show_timestamps --verbose_failures -- //src/test/cpp/util:md5_test
Notes:
1. Some tests have been excluded in the bazel test
command above:
//src/java_tools/import_deps_checker/...
fails on all platforms//src/test/shell/bazel/android/...
fails because an android sdk is required//src/test/java/com/google/devtools/build/android/...
fails because an android sdk is required//src/test/shell/bazel:bazel_determinism_test
fails because some archive files which must be rebuilt during the test on s390x result in new file sizes and md5sum values//src/test/shell/bazel:bazel_android_tools_test
fails on both s390x and x86_64//src/test/py/bazel:bazel_vendor_test
passes on master branch of bazel.//src/test/py/bazel:mod_command_test
fails as buildozer binary does not have support for s390x.//src/test/shell/bazel:command_profiler_test
fails as async-profiler does not have support for s390x.
2. The following test would be skipped, since prebuilt ijar
and singlejar
binaries haven't been released for s390x:
//src/test/shell/bazel:bazel_java_test_defaults_prebuilt
3. The following test would fail unless ipv6 is enabled on the system:
//src/test/java/com/google/devtools/build/lib/bazel/bzlmod:BzlmodTests
4. The following test may fail with an out of memory error if building on a machine with less than 4GB of memory:
//src/test/java/com/google/devtools/build/lib/graph:GraphTests
5. Below test cases fail on both s390x and x86_64 on all distros:
//src/test/java/com/google/devtools/build/lib/bazel/bzlmod:BzlmodTests
//src/test/shell/bazel:bazel_example_test
//src/test/shell/integration:sandboxing_test
//src/test/shell/bazel:external_integration_test
//src/test/java/com/google/devtools/build/lib/sandbox:SpawnRunnerTests
//src/test/java/com/google/devtools/build/lib/shell:CommandUsingLinuxSandboxTest
//src/test/shell/bazel:bazel_spawnstats_test
//src/test/java/com/google/devtools/build/lib/vfs:VfsTests
//src/test/shell/integration:test_test
//src/test/tools:daemonize_test
//src/test/shell/integration:bazel_hardened_sandboxed_worker_test
6. The following test passes with Python v3.9:
//src/test/java/com/google/devtools/build/lib/query2/cquery:ProtoOutputFormatterCallbackTest
7. More information on running Bazel in a low memory environment:
https://bazel.build/advanced/performance/memory
Step 3: Use the new module-focused external dependencies management system in Bazel (Optional)
The new module-focused MODULE.bazel
system can be enabled by the flag --enable_bzlmod
, for example:
cd $SOURCE_ROOT/bazel
./output/bazel build --enable_bzlmod //src:bazel_nojdk