Building GlusterFS - linux-on-ibm-z/docs GitHub Wiki

Building GlusterFS

Following versions of GlusterFS are available in respective distributions at the time of creation of these build instructions:

RHEL (8.8, 8.9, 9.2, 9.3) has 6.0
SLES 15 SP5 has 9.3
Ubuntu 20.04 has 7.2
Ubuntu 22.04 has 10.1
Ubuntu 23.10 has 10.3

The instructions provided below specify the steps to build or install GlusterFS 11.1 on Linux on IBM Z for following distributions:

RHEL (7.8, 7.9, 8.8, 8.9, 9.2, 9.3)
SLES (12 SP5, 15 SP5)

Note: For Ubuntu (20.04, 22.04, 23.10), please refer to this link to install GlusterFS from the community GlusterFS PPA (ppa:gluster/glusterfs-11).

General Notes:

When following the steps below please use a super user unless otherwise specified
A directory /<source root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it

Step 1: Build and Install GlusterFS

Step 1.1: Build using script

If you want to build GlusterFS using manual steps, go to Step 1.2.

Use the following commands to build GlusterFS using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/GlusterFS/11.1/build_glusterfs.sh
# Build GlusterFS
bash build_glusterfs.sh    [Provide -h option to check usage,  -t for executing build with tests]

If the build completes successfully, go to STEP 3. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps.

Step 1.2: Install the dependencies

Set environment variables:

export SOURCE_ROOT=/<source_root>/
export PREFIX=/usr/local
export PATCH_URL=https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/GlusterFS/11.1/patch

PATH=${PREFIX}/bin:${PREFIX}/sbin${PATH:+:${PATH}}
export PATH

PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig:${PREFIX}/lib64/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}
export PKG_CONFIG_PATH

LD_LIBRARY_PATH=${PREFIX}/lib:${PREFIX}/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export LD_LIBRARY_PATH

LD_RUN_PATH=${PREFIX}/lib:${PREFIX}/lib64${LD_RUN_PATH:+:${LD_RUN_PATH}}
export LD_RUN_PATH

RHEL (7.8, 7.9)

yum install -y curl gcc-c++ rpcgen libtirpc-devel langpacks-en glibc-langpack-en automake autoconf libtool flex bison openssl-devel libxml2-devel python-devel libaio-devel libibverbs-devel librdmacm-devel readline-devel lvm2-devel glib2-devel userspace-rcu-devel libcmocka-devel libacl-devel sqlite-devel fuse-devel libuuid-devel redhat-rpm-config git gperftools-devel gperftools-libs openssl popt-devel gperf gperftools-devel

RHEL (8.8, 8.9)

yum install -y curl autoconf automake bison dos2unix flex fuse-devel glib2-devel libacl-devel libaio-devel libattr-devel libcurl-devel libibverbs-devel librdmacm-devel libtirpc-devel libuuid-devel libtool libxml2-devel make openssl-devel pkgconfig xz-devel  python3-devel python3-netifaces  readline-devel rpm-build sqlite-devel systemtap-sdt-devel tar git lvm2-devel python3-paste-deploy python3-simplejson python3-sphinx python3-webob python3-pyxattr userspace-rcu-devel rpcgen liburing-devel gperf gperftools-devel

RHEL (9.2, 9.3)

yum install -y curl autoconf automake bison dos2unix flex fuse-devel glib2-devel libacl-devel libaio-devel libattr-devel libcurl-devel libibverbs-devel librdmacm-devel libtirpc-devel libuuid-devel libtool libxml2-devel make openssl-devel pkgconfig xz-devel  python3-devel python3-netifaces  readline-devel rpm-build sqlite-devel systemtap-sdt-devel tar git lvm2-devel python3-paste-deploy python3-simplejson python3-sphinx python3-webob python3-pyxattr userspace-rcu-devel rpcgen liburing-devel gperf gperftools-devel iproute

SLES 12 SP5

zypper install -y curl autoconf automake bison cmake flex fuse-devel gcc-c++ git-core glib2-devel libacl-devel libaio-devel librdmacm1 libtool liburcu-devel libuuid-devel libxml2-devel lvm2 make pkg-config python3 python3-xattr rdma-core-devel readline-devel openssl-devel zlib-devel which gawk dmraid popt-devel gperftools-devel gperf gperftools libtirpc-devel util-linux iproute util-linux-systemd keyutils-devel

SLES 15 SP5

zypper install -y curl autoconf automake bison cmake flex fuse-devel gcc-c++ git-core glib2-devel libacl-devel libaio-devel librdmacm1 libtool liburcu-devel libuuid-devel libxml2-devel lvm2 make pkg-config python3 python3-xattr rdma-core-devel readline-devel openssl-devel zlib-devel which gawk dmraid popt-devel gperftools-devel gperf gperftools libtirpc-devel rpcgen liburing-devel util-linux hostname iproute util-linux-systemd keyutils-devel

Step 1.3: Build and install dependencies

Install GCC 11

RHEL (7.8, 7.9)

yum install -y devtoolset-11-gcc
scl enable devtoolset-11 bash

RHEL (8.8, 8.9)

yum install -y gcc-toolset-11-gcc-c++
scl enable gcc-toolset-11 bash

SLES (12 SP5)

zypper addrepo https://download.opensuse.org/repositories/devel:gcc/SLE-12/devel:gcc.repo
zypper refresh
zypper install -y gcc11
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 50
gcc --version

Install userspace-rcu (Only on RHEL 7.x and SLES 12 SP5)

cd $SOURCE_ROOT
git clone git://git.liburcu.org/userspace-rcu.git
cd userspace-rcu/
git checkout v0.7.14
./bootstrap
./configure
make
make install
ldconfig

Install openssl 1.1.1 (Only on RHEL 7.x and SLES 12 SP5)

cd $SOURCE_ROOT
git clone https://github.com/openssl/openssl.git
cd openssl/
git checkout OpenSSL_1_1_1k
./config --prefix=/usr
make
make install
ldconfig
openssl version -a

Step 1.4: Download, Configure and Build GlusterFS

cd $SOURCE_ROOT
git clone https://github.com/gluster/glusterfs
cd glusterfs
git checkout v11.1
curl -sSL $PATCH_URL/GFS_RH9.patch | git apply ##For RHEL 9.x
curl -sSL $PATCH_URL/GFS_RH7.patch | git apply ##For RHEL 7.x
curl -sSL $PATCH_URL/GFS_SL12.patch | git apply ##For SLES 12 SP5
./autogen.sh
./configure --enable-gnfs  ##For RHEL 8.x, 9.x and SLES 15.x
./configure --enable-gnfs --enable-debug --disable-linux-io_uring --without-libtirpc  ##For RHEL 7.x
./configure --enable-gnfs --disable-linux-io_uring  ##For SLES 12 SP5
make 
make install

Note : If git clone fails with SSL certificate issue, execute git config --global http.sslVerify false command and then do git clone.

Step 2: Run the test cases (optional)

2.1) Install the testing dependencies

RHEL (7.8, 7.9)

yum install -y acl attr bc bind-utils boost-devel docbook-style-xsl expat-devel gdb net-tools nfs-utils psmisc vim xfsprogs yajl redhat-rpm-config perl-Test-Harness popt-devel procps-ng pyxattr python3-pip
pip3 install prettytable

RHEL (8.8, 8.9, 9.2, 9.3)

yum install -y dbench perl-Test-Harness yajl net-tools psmisc nfs-utils xfsprogs attr procps-ng gdb python3
pip3 install prettytable

SLES 12 SP5

zypper install -y acl attr bc bind-utils gdb libxml2-tools net-tools nfs-utils psmisc vim xfsprogs python-PrettyTable libselinux-devel selinux-tools popt-devel sysvinit-tools libexpat-devel boost-devel
pip3 install prettytable

SLES 15 SP5

zypper install -y wget acl attr bc bind-utils gdb libxml2-tools net-tools-deprecated nfs-utils psmisc thin-provisioning-tools vim xfsprogs python3-PrettyTable libselinux-devel selinux-tools popt-devel libyajl2 libyajl-devel sysvinit-tools
pip3 install prettytable
wget https://ftp.lysator.liu.se/pub/opensuse/ports/aarch64/distribution/leap/15.4/repo/oss/s390x/yajl-2.1.0-2.12.s390x.rpm
rpm -ivh yajl-2.1.0-2.12.s390x.rpm

Link pstack command (For SLES only)
```
ln -sf `which gstack` /usr/bin/pstack
```

Build DBENCH (For RHEL 7.x and SLES only)

cd $SOURCE_ROOT
git clone https://github.com/sahlberg/dbench
cd dbench
git checkout caa52d347171f96eef5f8c2d6ab04a9152eaf113
./autogen.sh
./configure --datadir=/usr/local/share/doc/loadfiles/
make
make install

Build thin-provisioning-tools (For RHEL 7.x and SLES 12 SP5 only)

cd $SOURCE_ROOT
git clone https://github.com/jthornber/thin-provisioning-tools
cd thin-provisioning-tools
git checkout v0.7.6
autoreconf
./configure
make
make install

Build yajl (For SLES 12 SP5 only)

cd $SOURCE_ROOT
git clone https://github.com/lloyd/yajl
cd yajl
git checkout 2.1.0
./configure
make install

2.2) Run tests

Apply patches

The following patch fixes the test case ./tests/00-geo-rep/00-georep-verify-non-root-setup.t (For RHEL 8.x and SLES only)

cd $SOURCE_ROOT/glusterfs
curl -sSL $PATCH_URL/test.patch | git apply 

## For RHEL 8.x
ln -s /usr/local/lib/libgfchangelog.so.0 /lib64/libgfchangelog.so
ldconfig /usr/local/lib
ldconfig /usr/local/lib64

The following patch fixes the test case ./tests/00-geo-rep/00-georep-verify-non-root-setup.t (For RHEL 9.x)

cd $SOURCE_ROOT/glusterfs
curl -sSL $PATCH_URL/test_RH9.patch | git apply 
ln -s /usr/local/lib/libgfchangelog.so.0 /lib64/libgfchangelog.so
ldconfig /usr/local/lib
ldconfig /usr/local/lib64

The following patch fixes the test cases ./tests/bugs/shard/bug-1468483.t and ./tests/bugs/shard/bug-1251824.t (For SLES only)

cd $SOURCE_ROOT/glusterfs
curl -sSL $PATCH_URL/test-sles.patch | git apply

Run the test cases:

cd $SOURCE_ROOT/glusterfs
./run-tests.sh

Note:

The following test cases fail on s390x and x86_64 both:
- For RHEL
  - ./tests/basic/afr/split-brain-favorite-child-policy-client-side-healing.t
  - ./tests/bugs/index/bug-1559004-EMLINK-handling.t
  - ./tests/00-geo-rep/00-georep-verify-non-root-setup.t
  - ./tests/00-geo-rep/00-georep-verify-setup.t
  - ./tests/00-geo-rep/01-georep-glusterd-tests.t
  - ./tests/00-geo-rep/bug-1600145.t
  - ./tests/00-geo-rep/georep-basic-dr-rsync-arbiter.t
  - ./tests/00-geo-rep/georep-basic-dr-rsync.t
  - ./tests/00-geo-rep/georep-basic-dr-tarssh-arbiter.t
  - ./tests/00-geo-rep/georep-basic-dr-tarssh.t
  - ./tests/00-geo-rep/georep-basic-rsync-ec.t
  - ./tests/00-geo-rep/georep-basic-tarssh-ec.t
  - ./tests/00-geo-rep/georep-config-upgrade.t
  - ./tests/00-geo-rep/georep-stderr-hang.t
- For SLES
  - ./tests/bugs/nfs/bug-1157223-symlink-mounting.t
  - ./tests/basic/distribute/directory-rebalance.t
The test case ./tests/bugs/glusterd/bug-1699339.t is passing on master branch

Step 3: Run the GlusterFS Daemon (Optional)

glusterd

Similar logs should be output in /var/log/glusterfs/glusterd.log:

[2024-04-09 20:21:55.345905 +0000] I [MSGID: 100030] [glusterfsd.c:2872:main] 0-glusterd: Started running version [{arg=glusterd}, {version=11.1}, {cmdlinestr=glusterd}]

Step 4: Test the GlusterFS volume (Optional)

Refer to Documentation for instructions.

Building GlusterFS - linux-on-ibm-z/docs GitHub Wiki

Building GlusterFS

General Notes:

Step 1: Build and Install GlusterFS

Step 1.1: Build using script

Step 1.2: Install the dependencies

Step 1.3: Build and install dependencies

Install GCC 11

Install userspace-rcu (Only on RHEL 7.x and SLES 12 SP5)

Install openssl 1.1.1 (Only on RHEL 7.x and SLES 12 SP5)

Step 1.4: Download, Configure and Build GlusterFS

Step 2: Run the test cases (optional)

2.1) Install the testing dependencies

2.2) Run tests

Step 3: Run the GlusterFS Daemon (Optional)

Step 4: Test the GlusterFS volume (Optional)

References