Building CockroachDB - linux-on-ibm-z/docs GitHub Wiki

Building CockroachDB

The instructions provided below specify the steps to build CockroachDB version 24.1.0 on Linux on IBM Z for the following distributions:

  • RHEL (8.8, 8.9, 8.10, 9.2, 9.3, 9.4)
  • Ubuntu (20.04, 22.04, 24.04)

Important Notes:

  • You need to enable the EPEL repositories on RHEL distros. This can be done by installing the epel-release package:

    • RHEL (8.8, 8.9, 8.10)
    sudo subscription-manager repos --enable codeready-builder-for-rhel-8-$(arch)-rpms
sudo dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
    • RHEL (9.2, 9.3, 9.4)
    sudo subscription-manager repos --enable codeready-builder-for-rhel-9-$(arch)-rpms
sudo dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Build and Install CockroachDB

1.1) Build using script

If you'd like to build CockroachDB using the manual steps, please go to STEP 1.2.

Use the following commands to build CockroachDB using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/CockroachDB/24.1.0/build_crdb.sh

# Build CockroachDB
bash build_crdb.sh   [Provide -t option for executing build with tests]

If the build completes successfully, go to STEP 3. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps

1.2) Install the Dependencies

export SOURCE_ROOT=/<source_root>/
export PATCH_URL=https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/CockroachDB/24.1.0/patch

  • RHEL (8.8, 8.9, 8.10)

    sudo yum install -y gcc-c++ git ncurses-devel make cmake automake bison patch wget tar xz zip unzip java-11-openjdk-devel python3 zlib-devel diffutils libtool libarchive openssl-devel keyutils-libs-devel

  • RHEL 9.2

    sudo yum install -y gcc-c++ git ncurses-devel make cmake automake bison patch wget tar xz zip unzip java-11-openjdk-devel python3 ghc-resolv zlib-devel diffutils libtool libarchive keyutils-libs-devel

  • RHEL (9.3, 9.4)

    sudo yum install -y gcc-c++ git ncurses-devel make cmake automake bison patch wget tar xz zip unzip java-11-openjdk-devel python3 zlib-devel diffutils libtool libarchive keyutils-libs-devel

  • Ubuntu 20.04

    sudo apt-get update && sudo apt-get -y install zip unzip autoconf automake wget make openjdk-11-jdk libssl-dev libncurses5-dev bison xz-utils patch g++ curl git python3 libresolv-wrapper libkeyutils-dev

  • Ubuntu 22.04

    sudo apt-get update && sudo apt-get -y install zip unzip autoconf automake wget make openjdk-11-jdk libssl-dev libncurses5-dev bison xz-utils patch g++ curl git python3 cmake netbase libresolv-wrapper libkeyutils-dev

  • Ubuntu 24.04

    sudo apt-get update && sudo apt-get -y install zip unzip autoconf automake wget make openjdk-11-jdk libssl-dev libncurses5-dev bison xz-utils patch g++ curl git python3 cmake netbase libresolv-wrapper libkeyutils-dev bzip2

1.3) Build and install GCC 10.5.0 (Only for Ubuntu 24.04)

cd $SOURCE_ROOT
ver=10.5.0
wget https://ftp.gnu.org/gnu/gcc/gcc-${ver}/gcc-${ver}.tar.gz
tar xzf gcc-${ver}.tar.gz
cd gcc-${ver}
./contrib/download_prerequisites
mkdir build-gcc
cd build-gcc
../configure --enable-languages=c,c++ --disable-multilib
make -j$(nproc)
sudo make install

1.4) Build and install CMake (Ubuntu 20.04)

cd $SOURCE_ROOT
wget https://github.com/Kitware/CMake/releases/download/v3.29.0/cmake-3.29.0.tar.gz
tar -xzf cmake-3.29.0.tar.gz
cd cmake-3.29.0
./bootstrap
make
sudo make install
cmake --version

1.5) Build and install Go

cd $SOURCE_ROOT
GO_VERSION=1.22.2
mkdir go_bootstrap
cd go_bootstrap
wget https://go.dev/dl/go${GO_VERSION}.linux-s390x.tar.gz
tar -xzf go${GO_VERSION}.linux-s390x.tar.gz
export GOROOT_BOOTSTRAP=${SOURCE_ROOT}/go_bootstrap/go

cd $SOURCE_ROOT
git clone -b go${GO_VERSION} https://go.googlesource.com/go goroot
cd goroot
wget -O $SOURCE_ROOT/go.patch $PATCH_URL/go.patch
git apply --reject --whitespace=fix $SOURCE_ROOT/go.patch
cd src
./make.bash
export PATH="${SOURCE_ROOT}/goroot/bin:${PATH}"

go version

1.6) Build and Install Bazel

cd $SOURCE_ROOT
git clone -b 5.5.1 https://github.com/bazelbuild/rules_java.git
cd rules_java
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/6.4.0/patch/rules_java_5.5.1.patch | git apply || error "Patch rules_java v5.5.1"
mkdir -p ${SOURCE_ROOT}/bazel
cd ${SOURCE_ROOT}/bazel
wget https://github.com/bazelbuild/bazel/releases/download/6.4.0/bazel-6.4.0-dist.zip
unzip -q bazel-6.4.0-dist.zip
chmod -R +w .
curl -sSLO https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/6.4.0/patch/bazel.patch
sed -i "s#RULES_JAVA_ROOT_PATH#${SOURCE_ROOT}#g" bazel.patch
patch -p1 < bazel.patch || error "Patch bazel"
bash ./compile.sh
export PATH=$PATH:$SOURCE_ROOT/bazel/output/
bazel --version

1.7) Build and Install resolv_wrapper (RHEL only)

cd $SOURCE_ROOT
wget https://ftp.samba.org/pub/cwrap/resolv_wrapper-1.1.8.tar.gz
tar zxf resolv_wrapper-1.1.8.tar.gz
cd resolv_wrapper-1.1.8
mkdir obj
cd obj
cmake -DCMAKE_INSTALL_PREFIX=/usr ..
make
sudo make install

1.8) Build and Install Bazel-lib

cd $SOURCE_ROOT
git clone https://github.com/aspect-build/bazel-lib.git
cd bazel-lib/
git checkout v1.32.1
wget -O $SOURCE_ROOT/bazel-lib.patch $PATCH_URL/bazel-lib.patch
git apply --reject --whitespace=fix $SOURCE_ROOT/bazel-lib.patch
bazel build @aspect_bazel_lib//tools/copy_directory
bazel build @aspect_bazel_lib//tools/copy_to_directory

1.9) Build and Install CockroachDB

  • Download source code

    cd $SOURCE_ROOT
git clone https://github.com/cockroachdb/cockroach
cd cockroach
git checkout v24.1.0

  • Apply patches to code using below commands:

    cd $SOURCE_ROOT/cockroach
wget -O $SOURCE_ROOT/cockroachdb.patch $PATCH_URL/crdb.patch
sed -i "s#SOURCE_ROOT_PATH#${SOURCE_ROOT}#g" $SOURCE_ROOT/cockroachdb.patch
git apply --reject $SOURCE_ROOT/cockroachdb.patch
sudo cp $SOURCE_ROOT/bazel-lib/lib/private/copy_to_directory_toolchain.bzl .
sudo cp $SOURCE_ROOT/bazel-lib/lib/private/copy_directory_toolchain.bzl .

  • Build and install

    cd $SOURCE_ROOT/cockroach
echo 'build --remote_cache=http://127.0.0.1:9867' > ~/.bazelrc
echo 'build --config=dev
build --config nolintonbuild' > .bazelrc.user
echo "test --test_tmpdir=$SOURCE_ROOT/cockroach/tmp" >> .bazelrc.user
./dev doctor
./dev build
sudo cp cockroach /usr/local/bin
export PATH=$SOURCE_ROOT/cockroach:$PATH

Step 2: Testing (Optional)

cd $SOURCE_ROOT/cockroach
./dev test -v -- --test_timeout=3600

There are currently known test failures on s390x.

Many tests check for exact floating point values however due to optimizations on the s390x platform that cause small differences, tests in these packages will fail.

  • pkg/ccl/logictestccl (TestTenantLogic_geospatial)
  • pkg/geo/geomfn
  • pkg/sql/logictest (TestLogic_geospatial)
  • pkg/sql/opt/exec/execbuilder (TestExecBuild_geospatial)
  • pkg/sql/sem/eval
  • pkg/sql/sem/eval_test
  • pkg/util/metric

Tests in these packages fail because of platform dependent filenames in the stack traces.

  • pkg/util/log/logcrash

Tests in this package fail due to platform dependent hash function values.

  • pkg/sql/colflow

These specific tests in the pkg/sql package fail because the binary encoding of the sql plan is platform dependent but the plan is correct and is decoded correctly.

  • pkg/sql/opt/exec/execbuilder/tests/5node
    • TestExecBuild_dist_vectorize
    • TestExecBuild_explain_analyze_plans
    • TestLogic_distsql_stats
  • pkg/sql/logictest/tests/5node
    • TestLogic_distsql_stats

Test related to upgrade will fail because it downloads CockroachDB binary tar which is not available to s390x.

  • //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test
  • //pkg/ccl/logictestccl/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test

Note: Some test cases might experience time out problem. They should pass after increasing the timeout and running the test again. And remaining test cases will pass on re-run.

Step 3: Verify

cockroach version

Output should look like:

Build Tag:        v24.1.0-dev
Build Time:
Distribution:     CCL
Platform:         linux s390x
Go Version:       go1.22.2 X:nocoverageredesign
C Compiler:       gcc 11.4.1 20231218 (Red Hat 11.4.1-3)

Please visit this link to start a local cluster and CockroachDB SQL.

References: