Building Kibana - linux-on-ibm-z/docs GitHub Wiki

Building Kibana

The instructions provided below specify the steps to install Kibana 8.11.0 on Linux on IBM Z for following distributions:

  • RHEL (8.6, 8.8, 8.9, 9.0, 9.2, 9.3)
  • SLES (15 SP5)
  • Ubuntu (20.04, 22.04)

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Build and Install Kibana

1) Build using script

If you want to build Kibana using manual steps, go to STEP 2.

Use the following commands to build Kibana using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.11.0/build_kibana.sh

# Build Kibana
bash build_kibana.sh

If the build completes successfully, go to STEP 13. In case of error, check logs for more details or go to STEP 2 to follow manual build steps.

2) Install build dependencies

export SOURCE_ROOT=/<source_root>/

Please note EPEL is required on RHEL.

  • RHEL (8.6)

    sudo yum install -y curl git gcc-c++ gzip make python2 python3 java-11-openjdk-devel unzip zip tar wget patch xz pkg-config expat-devel glib2-devel meson ninja-build brotli gobject-introspection-devel
sudo yum install -y --allowerasing python3-bind9.16 bind9.16-chroot coreutils-single ed expect file gnupg2 iproute iproute-devel iputils lcov less openssl-devel redhat-lsb netcat python2-devel python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel python3-pyyaml zlib-devel
sudo ln -sf /usr/bin/python3 /usr/bin/python

  • RHEL (8.8, 8.9)

    sudo yum install -y curl git gcc-c++ gzip make python2 python3 java-11-openjdk-devel unzip zip tar wget patch xz pkg-config expat-devel glib2-devel meson ninja-build brotli gobject-introspection-devel
sudo yum install -y --allowerasing python3-bind bind-chroot coreutils ed expect file gnupg2 iproute iproute-devel iputils lcov less openssl-devel redhat-lsb netcat python2-devel python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel python3-pyyaml zlib-devel
sudo ln -sf /usr/bin/python3 /usr/bin/python

  • RHEL (9.0)

    sudo yum install -y curl git gcc-c++ gzip make python3 java-11-openjdk-devel unzip zip tar wget patch xz pkg-config expat-devel glib2-devel meson ninja-build brotli gobject-introspection-devel
sudo yum install -y --allowerasing python3-bind bind-chroot coreutils-single ed expect file gnupg2 iproute iproute-devel iputils lcov less openssl-devel netcat python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel python3-pyyaml zlib-devel
sudo ln -sf /usr/bin/python3 /usr/bin/python

  • RHEL (9.2, 9.3)

    sudo yum install -y curl git gcc-c++ gzip make python3 java-11-openjdk-devel unzip zip tar wget patch xz pkg-config expat-devel glib2-devel meson ninja-build brotli gobject-introspection-devel
sudo yum install -y --allowerasing python3-bind bind-chroot coreutils ed expect file gnupg2 iproute iproute-devel iputils lcov less openssl-devel netcat python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel python3-pyyaml zlib-devel
sudo ln -sf /usr/bin/python3 /usr/bin/python

  • SLES 15 SP5

    sudo zypper install -y curl git gcc10-c++ gzip make python python3 java-11-openjdk-devel unzip zip tar wget patch xz which gawk pkg-config glib2-devel libexpat-devel cmake meson ninja gobject-introspection-devel
sudo zypper install -y python3-bind bind-chrootenv coreutils curl ed expect file iproute2  iputils lcov less libopenssl-devel netcat python2 python2-devel python3 python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel unzip zlib-devel python3-python-gnupg python3-PyYAML

sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-10 10
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 10
sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-10 10

  • Ubuntu 20.04

    sudo apt-get update
sudo apt-get install -y curl git g++ gzip make python python3 openjdk-11-jdk unzip zip tar wget patch xz-utils build-essential pkg-config libglib2.0-dev libexpat1-dev ninja-build brotli libgirepository1.0-dev

  • Ubuntu 22.04

    sudo apt-get update
sudo apt-get install -y curl git g++ gzip make python2 python3 openjdk-11-jdk unzip zip tar wget patch xz-utils build-essential pkg-config libglib2.0-dev libexpat1-dev meson ninja-build brotli libgirepository1.0-dev

3) Set JAVA_HOME environment variable

export JAVA_HOME=<path to java>
export PATH=$JAVA_HOME/bin:$PATH

4) Install Bazel

  • RHEL and SLES
mkdir bazel && cd bazel
wget https://github.com/bazelbuild/bazel/releases/download/5.1.1/bazel-5.1.1-dist.zip
mkdir -p dist/bazel && cd dist/bazel
unzip ../../bazel-5.1.1-dist.zip
chmod -R +w .
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/5.1.1/patch/dist-md5.patch | git apply
env EXTRA_BAZEL_ARGS="--tool_java_runtime_version=local_jdk" bash ./compile.sh
cd $SOURCE_ROOT/bazel
git clone https://github.com/bazelbuild/bazel.git
cd bazel
git checkout 5.1.1
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/5.1.1/patch/bazel.patch | git apply
cd $SOURCE_ROOT/bazel/bazel
$SOURCE_ROOT/bazel/dist/bazel/output/bazel build -c opt --stamp --embed_label "5.1.1" //src:bazel //src:bazel_jdk_minimal //src:test_repos
mkdir output
cp bazel-bin/src/bazel output/bazel
./output/bazel build  -c opt --stamp --embed_label "5.1.1" //src:bazel //src:bazel_jdk_minimal //src:test_repos
export PATH=$PATH:$SOURCE_ROOT/bazel/output/
export USE_BAZEL_VERSION=$SOURCE_ROOT/bazel/output/bazel
  • Ubuntu
wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/5.1.1/build_bazel.sh
bash build_bazel.sh -y
export PATH=$PATH:$SOURCE_ROOT/bazel/output/
export USE_BAZEL_VERSION=$SOURCE_ROOT/bazel/output/bazel

5) Install Go v1.18.4

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Go/1.18.4/build_go.sh
bash build_go.sh -y

6) Build Bazelisk binary

cd $SOURCE_ROOT
git clone https://github.com/bazelbuild/bazelisk.git
cd bazelisk
git checkout v1.12.1
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.11.0/patch/bazelisk_patch.diff | git apply --ignore-whitespace
go build && ./bazelisk build --config=release //:bazelisk-linux-s390x

7) Install Node.js 18.18.2

cd $SOURCE_ROOT
sudo mkdir -p /usr/local/lib/nodejs
wget https://nodejs.org/dist/v18.18.2/node-v18.18.2-linux-s390x.tar.xz
sudo tar xf node-v18.18.2-linux-s390x.tar.xz -C /usr/local/lib/nodejs
export PATH=/usr/local/lib/nodejs/node-v18.18.2-linux-s390x/bin:$PATH

8) Install Yarn and patch Bazelisk

sudo chmod ugo+w -R /usr/local/lib/nodejs/node-v18.18.2-linux-s390x
npm install -g yarn @bazel/[email protected]
BAZELISK_DIR=/usr/local/lib/nodejs/node-v18.18.2-linux-s390x/lib/node_modules/@bazel/bazelisk
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.11.0/patch/bazelisk.js.diff | patch $BAZELISK_DIR/bazelisk.js
cp $SOURCE_ROOT/bazelisk/bazel-out/s390x-opt-*/bin/bazelisk-linux_s390x $BAZELISK_DIR

9) Install libvips

  • Install meson (Ubuntu 20.04 only)

    cd $SOURCE_ROOT
wget https://github.com/mesonbuild/meson/releases/download/0.55.3/meson-0.55.3.tar.gz
tar zxf meson-0.55.3.tar.gz
cd meson-0.55.3
ln -s meson.py meson
export PATH=$SOURCE_ROOT/meson-0.55.3:$PATH

  • Install brotli (SLES only)

    cd $SOURCE_ROOT
git clone https://github.com/google/brotli.git
cd brotli
git checkout v1.1.0
mkdir out && cd out
cmake -DCMAKE_BUILD_TYPE=Release .. #need cmake 3.15+
sudo cmake --build . --config Release --target install
export LD_LIBRARY_PATH=/usr/local/lib64:$LD_LIBRARY_PATH

  • Install libvips

    export npm_config_sharp_libvips_local_prebuilds="${SOURCE_ROOT}"

# RHEL or SLES
LIB_DIR="lib64"
# Ubuntu
LIB_DIR="lib/s390x-linux-gnu"

    2 versions of libvips are required and need to be built for Kibana dependencies. Run the steps below for LIBVIPS_VER=8.14.2 and LIBVIPS_VER=8.14.5:

    LIBVIPS_VER=8.14.2

cd "${SOURCE_ROOT}"
wget https://github.com/libvips/libvips/archive/refs/tags/v$LIBVIPS_VER.tar.gz
tar zxf v$LIBVIPS_VER.tar.gz
cd libvips-$LIBVIPS_VER
meson setup build --prefix $PWD/libvips_install
cd build
meson compile
meson install
export LD_LIBRARY_PATH="${SOURCE_ROOT}"/libvips-$LIBVIPS_VER/libvips_install/LIB_DIR:$LD_LIBRARY_PATH

mkdir "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x && cd "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x
wget https://github.com/lovell/sharp-libvips/releases/download/v$LIBVIPS_VER/libvips-$LIBVIPS_VER-linux-x64.tar.gz
tar -xzf libvips-$LIBVIPS_VER-linux-x64.tar.gz
rm -rf libvips-$LIBVIPS_VER-linux-x64.tar.gz
cp "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER/libvips_install/$LIB_DIR/libvips-cpp.so.42 "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x/lib/
sed -i 's/linux-x64/linux-s390x/g' "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x/platform.json
cd "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x
tar -cf libvips-$LIBVIPS_VER-linux-s390x.tar *
cd "${SOURCE_ROOT}"
mkdir v$LIBVIPS_VER && cd v$LIBVIPS_VER
mv "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x/libvips-$LIBVIPS_VER-linux-s390x.tar .
brotli -j -Z libvips-$LIBVIPS_VER-linux-s390x.tar

    Notes: Repeat the steps above for LIBVIPS_VER=8.14.5.

10) Build and Install Kibana

  • Download Kibana

    cd $SOURCE_ROOT
git clone https://github.com/elastic/kibana.git
cd kibana
git checkout v8.11.0

  • Build re2

    cd $SOURCE_ROOT
git clone https://github.com/uhop/node-re2.git
cd node-re2 && git checkout 1.20.1
git submodule update --init --recursive
npm install
gzip -c build/Release/re2.node > $SOURCE_ROOT/linux-s390x-93.gz
mkdir -p $SOURCE_ROOT/kibana/.native_modules/re2/
cp $SOURCE_ROOT/linux-s390x-93.gz $SOURCE_ROOT/kibana/.native_modules/re2/

  • Apply source code patch:

    cd $SOURCE_ROOT/kibana
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.11.0/patch/kibana_patch.diff | git apply
sed -i -e "s#https://packages.atlassian.com/api/npm/npm-remote/#https://registry.yarnpkg.com/#g" yarn.lock

  • Bootstrap Kibana

    yarn kbn bootstrap

  • Build Kibana

    cd $SOURCE_ROOT/kibana
export NODE_OPTIONS="--max_old_space_size=4096"
yarn build --skip-os-packages

11) Test (Optional)

  • Unit tests

    cd $SOURCE_ROOT/kibana
wget https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.11.0/patch/unittest.sh
bash unittest.sh

  • Integration tests

    cd $SOURCE_ROOT/kibana
wget https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.11.0/patch/integrationtest.sh
bash integrationtest.sh

  • Functional tests

    export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
sudo ldconfig
  	
cd $SOURCE_ROOT/kibana
wget https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.11.0/patch/functionaltest.sh
bash functionaltest.sh

Note:

  • The following test case failures can be ignored as they are seen on x86 also:

    packages/kbn-es/src/utils/docker.test.ts packages/kbn-es/src/utils/wait_until_cluster_ready.test.ts x-pack/plugins/screenshotting/server/config/create_config.test.ts (RHEL only)

  • The following test case failures are not related to functionality and can be safely ignored:

    src/cli_plugin/install/zip.test.js src/dev/src/dev/build/lib/config.test.ts src/dev/i18n/utils/utils.test.js src/dev/build/tasks/nodejs/node_shasums.test.ts src/dev/build/tasks/nodejs/verify_existing_node_builds_task.test.ts src/dev/build/tasks/nodejs/download_node_builds_task.test.ts src/dev/build/tasks/nodejs/extract_node_builds_task.test.ts src/dev/i18n/extract_default_translations.test.js src/dev/build/lib/config.test.ts

  • Some tests pass with below flags:

    TEST_BROWSER_HEADLESS=1 FORCE_COLOR=1 DISABLE_BOOTSTRAP_VALIDATION=true BROWSERSLIST_IGNORE_OLD_DATA=true

  • Some integration/functional tests fail as they download and start Intel specific ElasticSearch binary

  • Some X-Pack test cases will fail as X-Pack plugins are not supported on s390x

  • Some failing test cases pass when executed individually.

12) Install Kibana

cd $SOURCE_ROOT/kibana
sudo mkdir /usr/share/kibana/
sudo tar -xzf target/kibana-8.11.0-SNAPSHOT-linux-s390x.tar.gz -C /usr/share/kibana --strip-components 1
sudo ln -sf /usr/share/kibana/bin/* /usr/bin/

sudo groupadd [username]
sudo chown [username]:elastic -R /usr/share/kibana/

13) Start Kibana

Kibana requires an Elasticsearch instance to be running.

Set Kibana home directory export KIBANA_HOME=/usr/share/kibana/. You may start Kibana and also update the Kibana configuration file $KIBANA_HOME/config/kibana.yml.

kibana

Open your browser and go to https://<HOST_IP>:5601 to make sure the Kibana installation works.

Note:

  • When Kibana is run for the first time, it generates a unique link in your terminal which can be used to enroll your Kibana instance with Elasticsearch. See here for more information on starting Kibana.
  • In order to access Kibana server through web browser, you might need to change server.host field in $KIBANA_HOME/config/kibana.yml to 0.0.0.0. See here for more information.
  • For more information regarding configuring Kibana, please visit here.

References

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

⚠️ **GitHub.com Fallback** ⚠️