Building Kibana - linux-on-ibm-z/docs GitHub Wiki
The instructions provided below specify the steps to install Kibana 8.14.0 on Linux on IBM Z for following distributions:
- RHEL (8.8, 8.10, 9.2, 9.4)
- SLES (15 SP5, 15 SP6)
- Ubuntu (20.04, 22.04)
General Notes:
- When following the steps below please use a standard permission user unless otherwise specified.
- A directory
/<source_root>/
will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.
If you want to build Kibana using manual steps, go to STEP 2.
Use the following commands to build Kibana using the build script. Please make sure you have wget installed.
wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.14.0/build_kibana.sh
# Build Kibana
bash build_kibana.sh
If the build completes successfully, go to STEP 13. In case of error, check logs
for more details or go to STEP 2 to follow manual build steps.
export SOURCE_ROOT=/<source_root>/
Please note EPEL is required on RHEL.
-
RHEL (8.8, 8.10)
sudo yum install -y curl git gcc-c++ gzip make python2 python3 java-11-openjdk-devel unzip zip tar wget patch xz pkg-config expat-devel glib2-devel meson ninja-build brotli gobject-introspection-devel sudo yum install -y --allowerasing python3-bind bind-chroot coreutils ed expect file gnupg2 iproute iproute-devel iputils lcov less openssl-devel redhat-lsb netcat python2-devel python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel python3-pyyaml zlib-devel sudo ln -sf /usr/bin/python3 /usr/bin/python
-
RHEL (9.2, 9.4)
sudo yum install -y curl git gcc-c++ gzip make python3 java-11-openjdk-devel unzip zip tar wget patch xz pkg-config expat-devel glib2-devel meson ninja-build brotli gobject-introspection-devel sudo yum install -y --allowerasing python3-bind bind-chroot coreutils ed expect file gnupg2 iproute iproute-devel iputils lcov less openssl-devel netcat python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel python3-pyyaml zlib-devel sudo ln -sf /usr/bin/python3 /usr/bin/python
-
SLES 15 SP5
sudo zypper install -y curl git gcc10-c++ gzip make python python3 java-11-openjdk-devel unzip zip tar wget patch xz which gawk pkg-config glib2-devel libexpat-devel cmake meson ninja gobject-introspection-devel sudo zypper install -y python3-bind bind-chrootenv coreutils curl ed expect file iproute2 iputils lcov less libopenssl-devel netcat python2 python2-devel python3 python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel unzip zlib-devel python3-python-gnupg python3-PyYAML sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-10 10 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 10 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 10 sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-10 10
-
SLES 15 SP6
sudo zypper install -y curl git gcc10-c++ gzip make python python3 java-11-openjdk-devel unzip zip tar wget patch xz which gawk pkg-config glib2-devel libexpat-devel cmake meson ninja gobject-introspection-devel sudo zypper install -y python3-bind bind-chrootenv coreutils curl ed expect file iproute2 iputils lcov less libopenssl-devel netcat python2 python3 python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel unzip zlib-devel python3-python-gnupg python3-PyYAML sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-10 10 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 10 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 10 sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-10 10
-
Ubuntu 20.04
sudo apt-get update sudo apt-get install -y curl git g++ gzip make python python3 openjdk-11-jdk unzip zip tar wget patch xz-utils build-essential pkg-config libglib2.0-dev libexpat1-dev ninja-build brotli libgirepository1.0-dev
-
Ubuntu 22.04
sudo apt-get update sudo apt-get install -y curl git g++ gzip make python2 python3 openjdk-11-jdk unzip zip tar wget patch xz-utils build-essential pkg-config libglib2.0-dev libexpat1-dev meson ninja-build brotli libgirepository1.0-dev
export JAVA_HOME=<path to java>
export PATH=$JAVA_HOME/bin:$PATH
mkdir bazel && cd bazel
wget https://github.com/bazelbuild/bazel/releases/download/5.1.1/bazel-5.1.1-dist.zip
mkdir -p dist/bazel && cd dist/bazel
unzip ../../bazel-5.1.1-dist.zip
chmod -R +w .
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/5.1.1/patch/dist-md5.patch | git apply
env EXTRA_BAZEL_ARGS="--tool_java_runtime_version=local_jdk" bash ./compile.sh
cd $SOURCE_ROOT/bazel
git --depth 1 -b 5.1.1 clone https://github.com/bazelbuild/bazel.git
cd bazel
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/5.1.1/patch/bazel.patch | git apply
cd $SOURCE_ROOT/bazel/bazel
$SOURCE_ROOT/bazel/dist/bazel/output/bazel build -c opt --stamp --embed_label "5.1.1" //src:bazel //src:bazel_jdk_minimal //src:test_repos
mkdir output
cp bazel-bin/src/bazel output/bazel
./output/bazel build -c opt --stamp --embed_label "5.1.1" //src:bazel //src:bazel_jdk_minimal //src:test_repos
export PATH=$PATH:$SOURCE_ROOT/bazel/output/
export USE_BAZEL_VERSION=$SOURCE_ROOT/bazel/output/bazel
wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Go/1.18.4/build_go.sh
bash build_go.sh -y
cd $SOURCE_ROOT
git clone --depth 1 -b v1.12.1 https://github.com/bazelbuild/bazelisk.git
cd bazelisk
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.14.0/patch/bazelisk_patch.diff | git apply --ignore-whitespace
go build && ./bazelisk build --config=release //:bazelisk-linux-s390x
cd $SOURCE_ROOT
sudo mkdir -p /usr/local/lib/nodejs
wget https://nodejs.org/dist/v20.13.1/node-v20.13.1-linux-s390x.tar.xz
sudo tar xf node-v20.13.1-linux-s390x.tar.xz -C /usr/local/lib/nodejs
export PATH=/usr/local/lib/nodejs/node-v20.13.1-linux-s390x/bin:$PATH
sudo chmod ugo+w -R /usr/local/lib/nodejs/node-v20.13.1-linux-s390x
npm install -g yarn @bazel/[email protected]
BAZELISK_DIR=/usr/local/lib/nodejs/node-v20.13.1-linux-s390x/lib/node_modules/@bazel/bazelisk
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.14.0/patch/bazelisk.js.diff | patch $BAZELISK_DIR/bazelisk.js
cp $SOURCE_ROOT/bazelisk/bazel-out/s390x-opt-*/bin/bazelisk-linux_s390x $BAZELISK_DIR
-
Install
meson
(Ubuntu 20.04 only)cd $SOURCE_ROOT wget https://github.com/mesonbuild/meson/releases/download/0.55.3/meson-0.55.3.tar.gz tar zxf meson-0.55.3.tar.gz cd meson-0.55.3 ln -s meson.py meson export PATH=$SOURCE_ROOT/meson-0.55.3:$PATH
-
Install
brotli
(SLES only)cd $SOURCE_ROOT git clone --depth 1 -b v1.1.0 https://github.com/google/brotli.git cd brotli mkdir out && cd out cmake -DCMAKE_BUILD_TYPE=Release .. #need cmake 3.15+ sudo cmake --build . --config Release --target install export LD_LIBRARY_PATH=/usr/local/lib64:$LD_LIBRARY_PATH
-
Install
libvips
export npm_config_sharp_libvips_local_prebuilds="${SOURCE_ROOT}" # RHEL or SLES LIB_DIR="lib64" # Ubuntu LIB_DIR="lib/s390x-linux-gnu"
2 versions of
libvips
are required and need to be built for Kibana dependencies. Run the steps below forLIBVIPS_VER=8.14.2
andLIBVIPS_VER=8.14.5
:LIBVIPS_VER=8.14.2 cd "${SOURCE_ROOT}" wget https://github.com/libvips/libvips/archive/refs/tags/v$LIBVIPS_VER.tar.gz tar zxf v$LIBVIPS_VER.tar.gz cd libvips-$LIBVIPS_VER meson setup build --prefix $PWD/libvips_install cd build meson compile meson install export LD_LIBRARY_PATH="${SOURCE_ROOT}"/libvips-$LIBVIPS_VER/libvips_install/LIB_DIR:$LD_LIBRARY_PATH mkdir "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x && cd "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x wget https://github.com/lovell/sharp-libvips/releases/download/v$LIBVIPS_VER/libvips-$LIBVIPS_VER-linux-x64.tar.gz tar -xzf libvips-$LIBVIPS_VER-linux-x64.tar.gz rm -rf libvips-$LIBVIPS_VER-linux-x64.tar.gz cp "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER/libvips_install/$LIB_DIR/libvips-cpp.so.42 "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x/lib/ sed -i 's/linux-x64/linux-s390x/g' "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x/platform.json cd "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x tar -cf libvips-$LIBVIPS_VER-linux-s390x.tar * cd "${SOURCE_ROOT}" mkdir v$LIBVIPS_VER && cd v$LIBVIPS_VER mv "${SOURCE_ROOT}"/libvips-$LIBVIPS_VER-linux-s390x/libvips-$LIBVIPS_VER-linux-s390x.tar . brotli -j -Z libvips-$LIBVIPS_VER-linux-s390x.tar
Notes: Repeat the steps above for
LIBVIPS_VER=8.14.5
.
-
Download Kibana
cd $SOURCE_ROOT git clone --depth 1 -b v8.14.0 https://github.com/elastic/kibana.git cd kibana
-
Build
re2
cd $SOURCE_ROOT git clone --depth 1 -b 1.12.1 https://github.com/uhop/node-re2.git cd node-re2 git submodule update --init --recursive npm install gzip -c build/Release/re2.node > $SOURCE_ROOT/linux-s390x-93.gz mkdir -p $SOURCE_ROOT/kibana/.native_modules/re2/ cp $SOURCE_ROOT/linux-s390x-93.gz $SOURCE_ROOT/kibana/.native_modules/re2/
-
Apply source code patch:
cd $SOURCE_ROOT/kibana curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.14.0/patch/kibana_patch.diff | git apply sed -i -e "s#https://packages.atlassian.com/api/npm/npm-remote/#https://registry.yarnpkg.com/#g" yarn.lock
-
Bootstrap Kibana
yarn kbn bootstrap # If build fail on this then use `yarn kbn bootstrap --force-install` or yarn config set network-timeout 600000
-
Build Kibana
cd $SOURCE_ROOT/kibana export NODE_OPTIONS="--max_old_space_size=4096" yarn build --skip-os-packages
-
Unit tests
cd $SOURCE_ROOT/kibana wget https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.14.0/patch/unittest.sh bash unittest.sh
-
Integration tests
cd $SOURCE_ROOT/kibana wget https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.14.0/patch/integrationtest.sh bash integrationtest.sh
-
Functional tests
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH sudo ldconfig cd $SOURCE_ROOT/kibana wget https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Kibana/8.14.0/patch/functionaltest.sh bash functionaltest.sh
Note:
-
The following test case failures can be ignored as they are seen on x86 also:
packages/kbn-es/src/utils/docker.test.ts
packages/kbn-es/src/utils/wait_until_cluster_ready.test.ts
x-pack/plugins/screenshotting/server/config/create_config.test.ts
(RHEL only) -
The following test case failures are not related to functionality and can be safely ignored:
src/cli_plugin/install/zip.test.js
src/dev/src/dev/build/lib/config.test.ts
src/dev/i18n/utils/utils.test.js
src/dev/build/tasks/nodejs/node_shasums.test.ts
src/dev/build/tasks/nodejs/verify_existing_node_builds_task.test.ts
src/dev/build/tasks/nodejs/download_node_builds_task.test.ts
src/dev/build/tasks/nodejs/extract_node_builds_task.test.ts
src/dev/i18n/extract_default_translations.test.js
src/dev/build/lib/config.test.ts
-
Some tests pass with below flags:
TEST_BROWSER_HEADLESS=1
FORCE_COLOR=1
DISABLE_BOOTSTRAP_VALIDATION=true
BROWSERSLIST_IGNORE_OLD_DATA=true
-
Some integration/functional tests fail as they download and start Intel specific ElasticSearch binary
-
Some X-Pack test cases will fail as X-Pack plugins are not supported on s390x
-
Some failing test cases pass when executed individually.
cd $SOURCE_ROOT/kibana
sudo mkdir /usr/share/kibana/
sudo tar -xzf target/kibana-8.14.0-SNAPSHOT-linux-s390x.tar.gz -C /usr/share/kibana --strip-components 1
sudo ln -sf /usr/share/kibana/bin/* /usr/bin/
sudo groupadd [username]
sudo chown [username]:elastic -R /usr/share/kibana/
Kibana requires an Elasticsearch instance to be running.
Set Kibana home directory export KIBANA_HOME=/usr/share/kibana/
. You may start Kibana and also update the Kibana configuration file $KIBANA_HOME/config/kibana.yml
.
kibana
Open your browser and go to https://<HOST_IP>:5601
to make sure the Kibana installation works.
Note:
- When Kibana is run for the first time, it generates a unique link in your terminal which can be used to enroll your Kibana instance with Elasticsearch. See here for more information on starting Kibana.
- In order to access Kibana server through web browser, you might need to change
server.host
field in$KIBANA_HOME/config/kibana.yml
to0.0.0.0
. See here for more information. - For more information regarding configuring Kibana, please visit here.