Building Apache Mesos - linux-on-ibm-z/docs GitHub Wiki

Building Apache Mesos

The instructions provided below specify the steps to build Apache Mesos version 1.11.0 on Linux on IBM Z for the following distributions:

  • RHEL (7.8, 7.9, 8.1, 8.2, 8.3)
  • SLES (12 SP5, 15 SP2)
  • Ubuntu (18.04, 20.04, 20.10)

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it

Step 1: Building and Installing Apache Mesos

1.1) Build using script

If you want to build mesos using manual steps, go to STEP 1.2.

Use the following commands to build mesos using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheMesos/1.11.0/build_mesos.sh

# Build mesos
bash build_mesos.sh #[Provide -j to specify java to be used from { openjdk, ibmsdk, adoptjdk }]

If the build completes successfully, go to STEP 3. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps.

1.2) Install dependencies

export SOURCE_ROOT=/<source_root>/

  • RHEL (7.8, 7.9)

    • With IBM SDK

      sudo yum install -y apr-devel autoconf bzip2 curl cyrus-sasl-devel cyrus-sasl-md5 gcc gcc-c++ git java-1.8.0-ibm-devel libcurl-devel libtool make maven openssl-devel patch python-devel python-six subversion-devel tar wget zlib-devel

    • With OpenJDK

      sudo yum install -y apr-devel autoconf bzip2 curl cyrus-sasl-devel cyrus-sasl-md5 gcc gcc-c++ git java-1.8.0-openjdk-devel libcurl-devel libtool make maven openssl-devel patch python-devel python-six subversion-devel tar wget zlib-devel

    • With AdoptOpenJDK (OpenJDK8 with Eclipse OpenJ9)

      sudo yum install -y apr-devel autoconf bzip2 curl cyrus-sasl-devel cyrus-sasl-md5 gcc gcc-c++ git libcurl-devel libtool make maven openssl-devel patch python-devel python-six subversion-devel tar wget zlib-devel

      Download AdoptOpenJDK 8 with OpenJ9, unzip binary and update JAVA_HOME and PATH accordingly.

  • RHEL (8.1, 8.2, 8.3)

    • With IBM SDK

      sudo yum install -y apr-devel autoconf bzip2 curl cyrus-sasl-devel cyrus-sasl-md5 gcc gcc-c++ git java-1.8.0-ibm-devel libcurl-devel libtool make maven openssl-devel patch python2-devel python2-six subversion-devel tar wget zlib-devel

    • With OpenJDK

      sudo yum install -y apr-devel autoconf bzip2 curl cyrus-sasl-devel cyrus-sasl-md5 gcc gcc-c++ git java-1.8.0-openjdk-devel libcurl-devel libtool make maven openssl-devel patch python2-devel python2-six subversion-devel tar wget zlib-devel

    • With AdoptOpenJDK (OpenJDK8 with Eclipse OpenJ9)

      sudo yum install -y apr-devel autoconf bzip2 curl cyrus-sasl-devel cyrus-sasl-md5 gcc gcc-c++ git libcurl-devel libtool make maven openssl-devel patch python2-devel python2-six subversion-devel tar wget zlib-devel

      Download AdoptOpenJDK 8 with OpenJ9, unzip binary and update JAVA_HOME and PATH accordingly.

  • SLES 12 SP5

    • With IBM SDK

      sudo zypper install --auto-agree-with-licenses -y autoconf bzip2 curl libnghttp2-devel cyrus-sasl-crammd5 cyrus-sasl-devel gcc gcc-c++ git java-1_8_0-ibm-devel libapr1-devel libcurl-devel libopenssl-devel libtool make patch python-devel python-pip python-six subversion-devel tar wget zlib-devel gawk gzip which

    • With OpenJDK

      sudo zypper install -y autoconf which bzip2 curl libnghttp2-devel cyrus-sasl-crammd5 cyrus-sasl-devel gcc gcc-c++ git java-1_8_0-openjdk-devel libapr1-devel libcurl-devel libopenssl-devel libtool make patch python-devel python-pip python-xml python-six subversion-devel tar wget zlib-devel gawk gzip

    • With AdoptOpenJDK (OpenJDK8 with Eclipse OpenJ9)

    sudo zypper install --auto-agree-with-licenses -y autoconf bzip2 curl libnghttp2-devel cyrus-sasl-crammd5 cyrus-sasl-devel gcc gcc-c++ git java-1_8_0-ibm-devel libapr1-devel libcurl-devel libopenssl-devel libtool make patch python-devel python-pip python-six subversion-devel tar wget zlib-devel gawk gzip

    Download AdoptOpenJDK 8 with OpenJ9, unzip binary and update JAVA_HOME and PATH accordingly.

  • SLES 15 SP2

    • With IBM SDK

      sudo zypper install --auto-agree-with-licenses -y autoconf bzip2 curl cyrus-sasl-crammd5 cyrus-sasl-devel gcc gcc-c++ git java-1_8_0-ibm-devel libapr1-devel libcurl-devel libopenssl-devel libtool make patch python-devel python-xml python3-pip python3-six subversion-devel tar wget zlib-devel gawk gzip

    • With OpenJDK

      sudo zypper install --auto-agree-with-licenses -y autoconf bzip2 curl cyrus-sasl-crammd5 cyrus-sasl-devel gcc gcc-c++ git java-1_8_0-openjdk-devel libapr1-devel libcurl-devel libopenssl-devel libtool make patch python-devel python-xml python3-pip python3-six subversion-devel tar wget zlib-devel gawk gzip

    • With AdoptOpenJDK (OpenJDK8 with Eclipse OpenJ9)

    sudo zypper install --auto-agree-with-licenses -y autoconf bzip2 curl cyrus-sasl-crammd5 cyrus-sasl-devel gcc gcc-c++ git libapr1-devel libcurl-devel libopenssl-devel libtool make patch python-devel python-xml python3-pip python3-six subversion-devel tar wget zlib-devel gawk gzip

    Download AdoptOpenJDK 8 with OpenJ9, unzip binary and update JAVA_HOME and PATH accordingly.

  • Ubuntu (18.04, 20.04)

    • With IBM SDK

      sudo apt-get update
sudo apt-get install -y autoconf bzip2 curl gcc g++ git libapr1-dev libcurl4-nss-dev libsasl2-dev libssl-dev libsvn-dev libtool make maven patch python-dev python-six tar wget zlib1g-dev

      Download IBM Java 8 SDK and follow the instructions as per given in the link. Update JAVA_HOME and PATH accordingly.

    • With OpenJDK

      sudo apt-get update
sudo apt-get install -y autoconf bzip2 curl gcc g++ git libapr1-dev libcurl4-nss-dev libsasl2-dev libssl-dev libsvn-dev libtool make maven openjdk-8-jdk patch python-dev python-six tar wget zlib1g-dev

    • With AdoptOpenJDK (OpenJDK8 with Eclipse OpenJ9)

    sudo apt-get update
sudo apt-get install -y autoconf bzip2 curl gcc g++ git libapr1-dev libcurl4-nss-dev libsasl2-dev libssl-dev libsvn-dev libtool make maven patch python-dev python-six tar wget zlib1g-dev

    Download AdoptOpenJDK 8 with OpenJ9, unzip binary and update JAVA_HOME and PATH accordingly.

  • Ubuntu 20.10

    • With IBM SDK

      sudo apt-get update
sudo apt-get install -y autoconf bzip2 curl gcc-9 g++-9 git libapr1-dev libcurl4-nss-dev libsasl2-dev libssl-dev libsvn-dev libtool make maven patch python-dev python-six tar wget zlib1g-dev
sudo ln -s /usr/bin/gcc-9 /usr/bin/gcc
sudo ln -s /usr/bin/g++-9 /usr/bin/g++

      Download IBM Java 8 SDK and follow the instructions as per given in the link. Update JAVA_HOME and PATH accordingly.

    • With OpenJDK

      sudo apt-get update
sudo apt-get install -y autoconf bzip2 curl gcc-9 g++-9 git libapr1-dev libcurl4-nss-dev libsasl2-dev libssl-dev libsvn-dev libtool make maven openjdk-8-jdk patch python-dev python-six tar wget zlib1g-dev
sudo ln -s /usr/bin/gcc-9 /usr/bin/gcc
sudo ln -s /usr/bin/g++-9 /usr/bin/g++

    • With AdoptOpenJDK (OpenJDK8 with Eclipse OpenJ9)

    sudo apt-get install -y autoconf bzip2 curl gcc-9 g++-9 git libapr1-dev libcurl4-nss-dev libsasl2-dev libssl-dev libsvn-dev libtool make maven patch python-dev python-six tar wget zlib1g-dev
sudo ln -s /usr/bin/gcc-9 /usr/bin/gcc
sudo ln -s /usr/bin/g++-9 /usr/bin/g++

    Download AdoptOpenJDK 8 with OpenJ9, unzip binary and update JAVA_HOME and PATH accordingly later.

NOTE: At the time of creation of these build instructions, Apache Mesos was verified with AdoptOpenJDK 8 version (openj9-0.23.0).

  • Set environment variables only for OpenJDK:

    export JAVA_HOME=/usr/lib/jvm/java-1.8.0                    # Only for RHEL
export JAVA_HOME=/usr/lib64/jvm/java-1.8.0                  # Only for SLES
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-s390x      # Only for Ubuntu
export JAVA_TOOL_OPTIONS='-Xmx2048M'
export PATH=$JAVA_HOME/bin:$PATH

  • Set environment variables only for IBM SDK:

    export JAVA_HOME=/usr/lib/jvm/java-1.8.0-ibm                      # Only for RHEL
export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-ibm                    # Only for SLES
export JVM_DIR=$JAVA_HOME/jre/lib/s390x/default
export JAVA_TEST_LDFLAGS="-L$JVM_DIR -R$JVM_DIR -Wl,-ljvm -ldl"
export JAVA_JVM_LIBRARY=$JAVA_HOME/jre/lib/s390x/default/libjvm.so
export PATH=$JAVA_HOME/bin:$PATH

  • Set environment variables only for AdoptOpenJdk 8 with OpenJ9:

    export JAVA_HOME=/<Path to JDK>/
export JAVA_TOOL_OPTIONS='-Xmx2048M'
export PATH=$JAVA_HOME/bin:$PATH

  • Install Maven 3.3.9 (For SLES only)

    cd $SOURCE_ROOT
wget https://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
sudo tar zxf apache-maven-3.3.9-bin.tar.gz -C /opt/
export M2_HOME=/opt/apache-maven-3.3.9
export PATH=$M2_HOME/bin:$PATH

  • Install curl 7.64 (For Ubuntu 18.04 only)

    cd $SOURCE_ROOT
wget https://curl.haxx.se/download/curl-7.64.0.tar.gz
tar -xzvf curl-7.64.0.tar.gz
cd curl-7.64.0
./configure --disable-shared
make
sudo make install
sudo ldconfig

1.3) Download source code

export PATCH_URL="https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheMesos/1.11.0/patch"
cd $SOURCE_ROOT
git clone https://github.com/apache/mesos.git
cd mesos
git checkout 1.11.0
  • Bundling gRPC 1.11.0
cd $SOURCE_ROOT/mesos/3rdparty/
git clone -b v1.11.0 https://github.com/grpc/grpc.git grpc-1.11.0
cd grpc-1.11.0/
git submodule update --init third_party/cares
cd ..
tar zcf grpc-1.11.0.tar.gz --exclude .git grpc-1.11.0
rm -rf grpc-1.11.0
  • Get $SOURCE_ROOT/mesos/3rdparty/grpc-1.11.0.patch (Ubuntu 20.04 and 20.10 only):
curl -o "grpc-1.11.0.patch"  $PATCH_URL/grpc-1.11.0.patch
  • Modify $SOURCE_ROOT/mesos/3rdparty/versions.am and $SOURCE_ROOT/mesos/src/python/native_common/ext_modules.py.in files with GRPC version 1.11.0:
$SOURCE_ROOT/mesos
sed -i -e 's/1.10.0/1.11.0/g' 3rdparty/versions.am
sed -i -e 's/1.10.0/1.11.0/g' src/python/native_common/ext_modules.py.in
  • Apply $SOURCE_ROOT/mesos/3rdparty/protobuf-3.5.0.patch:
cd $SOURCE_ROOT/mesos
curl -o "protobuf-3.5.0.patch"  $PATCH_URL/protobuf-3.5.0.patch
cat protobuf-3.5.0.patch >> 3rdparty/protobuf-3.5.0.patch
rm protobuf-3.5.0.patch

$SOURCE_ROOT/mesos
curl -o "boost-1.65.0.patch"  $PATCH_URL/boost-1.65.0.patch
cat boost-1.65.0.patch >> 3rdparty/boost-1.65.0.patch
rm boost-1.65.0.patch

1.4) Build and Install Apache Mesos

cd $SOURCE_ROOT/mesos
./bootstrap
mkdir build
cd build
../configure
make
sudo make install

Step 2: Testing (Optional)

Prerequisites for running the Apache Mesos test suite

  • Python 3
    • Python 3 for RHEL, SLES and Ubuntu can be found in their respective repositories.
  • Docker
    • Docker packages are provided for RHEL, SLES and Ubuntu in their respective repositories. You can also use the static binaries provided here. More information about Docker CE can be found here.

2.1) Steps to prepare images required for Apache Mesos test cases

Test cases of Apache Mesos uses the following images: alpine, chhsiao/overwrite, mesosphere/alpine-expect, mesosphere/inky, mesosphere/test-executor, tnachen/test-executor, haosdent/https-server, zhq527725/https-server and zhq527725/whiteout. Dockerfiles for each of these images is given below. Make sure all the images are present on your machine before you start testing.

Use docker build command to create all of the Docker images below:

docker build -t <image_name> .

  • Dockerfile for image mesosphere/alpine-expect:

    FROM s390x/alpine
RUN apk add --update expect

  • Dockerfile for image mesosphere/inky:

    FROM s390x/busybox
CMD ["inky"]
ENTRYPOINT ["echo"]

  • Dockerfile for both the images mesosphere/test-executor and tnachen/test-executor:

    FROM s390x/golang:1.7.5 as executor_builder
RUN mkdir -p src/github.com/mesos/ \
    && cd src/github.com/mesos/ \
    && git clone https://github.com/tnachen/go-mesos.git mesos-go \
    && cd mesos-go/examples/ \
    && sed -i 's/-race//g' Makefile \
    && go get github.com/tools/godep \
    && godep restore \
    && make

FROM s390x/busybox
COPY --from=executor_builder /go/src/github.com/mesos/mesos-go/examples/_output/executor /bin/test-executor

  • Dockerfile for image zhq527725/whiteout:

    FROM s390x/alpine
RUN mkdir -p /dir1/dir2 && touch /dir1/file1 && touch /dir1/dir2/file2
RUN rm -rf /dir1/file1 && rm -rf /dir1/dir2 && mkdir /dir1/dir2 && touch /dir1/dir2/file3

  • Dockerfile for image chhsiao/overwrite:

    FROM s390x/alpine
RUN mkdir /merged /replaced1 /replaced2 && touch /merged/m1 /replaced1/r1 /replaced2/r2 /replaced3 baz && ln -s /merged /replaced4 && ln -s baz foo && ln -s bar bar && ln -s ../../../../../../../abc xyz
RUN rm -rf /replaced1 /replaced2 /replaced3 /replaced4 foo bar baz xyz && mkdir /replaced3 /replaced4 && touch /merged/m2 /replaced1 bar xyz && ln -s /merged /replaced2 && ln -s bar foo && ln -s baz baz

  • Steps for building both haosdent/https-server and zhq527725/https-server docker images:

    • For haosdent/https-server, download files as given below

      git clone https://github.com/haosdent/https-server
cd https-server
docker build -t haosdent/https-server .

    • For zhq527725/https-server, download files as given below

      git clone https://github.com/qianzhangxa/https-server.git
cd https-server
docker build -t zhq527725/https-server .

Steps to save docker images to /opt/docker/images:

sudo mkdir -p /opt/docker/images
sudo chmod ugo+rw /opt/docker/images
cd /opt/docker/images
docker pull alpine
docker save alpine >> alpine.tar
docker pull hello-world
docker save hello-world >> hello-world.tar
mkdir mesosphere
docker save mesosphere/alpine-expect >> mesosphere/alpine-expect.tar
docker save mesosphere/inky >> mesosphere/inky.tar
mkdir haosdent
docker save haosdent/https-server >> haosdent/https-server.tar
mkdir zhq527725
docker save zhq527725/https-server >> zhq527725/https-server.tar
docker save zhq527725/whiteout >> zhq527725/whiteout.tar
mkdir chhsiao
docker save chhsiao/overwrite >> chhsiao/overwrite.tar

2.2) Modify the following files as given below

  • Modify $SOURCE_ROOT/mesos/src/slave/flags.cpp:

    diff --git a/src/slave/flags.cpp b/src/slave/flags.cpp
index e66b464..449e541 100644
--- a/src/slave/flags.cpp
+++ b/src/slave/flags.cpp
@@ -253,7 +253,7 @@ mesos::internal::slave::Flags::Flags()
       "path (e.g., `/tmp/docker/images`), or as an HDFS URI (*experimental*)\n"
       "(e.g., `hdfs://localhost:8020/archives/`). Note that this option won't\n"
       "change the default registry server for Docker containerizer.",
-      "https://registry-1.docker.io");
+      "/opt/docker/images");

   add(&Flags::docker_store_dir,
       "docker_store_dir",

  • Modify $SOURCE_ROOT/mesos/src/tests/containerizer/provisioner_docker_tests.cpp:

    diff --git a/src/tests/containerizer/provisioner_docker_tests.cpp b/src/tests/containerizer/provisioner_docker_tests.cpp
index b1f38aefa..b7b5eb46f 100644
--- a/src/tests/containerizer/provisioner_docker_tests.cpp
+++ b/src/tests/containerizer/provisioner_docker_tests.cpp
@@ -699,13 +699,13 @@ INSTANTIATE_TEST_CASE_P(
     ProvisionerDockerTest,
     ::testing::ValuesIn(vector<string>({
         "alpine", // Verifies the normalization of the Docker repository name.
-        "library/alpine",
-        "gcr.io/google-containers/busybox:1.24", // manifest.v1+prettyjws
-        "gcr.io/google-containers/busybox:1.27", // manifest.v2+json
+        //"library/alpine",
+        // "gcr.io/google-containers/busybox:1.24", // manifest.v1+prettyjws
+        // "gcr.io/google-containers/busybox:1.27", // manifest.v2+json
         // TODO(alexr): The registry below is unreliable and hence disabled.
         // Consider re-enabling shall it become more stable.
         // "registry.cn-hangzhou.aliyuncs.com/acs-sample/alpine",
-        "quay.io/coreos/alpine-sh" // manifest.v1+prettyjws
+        // "quay.io/coreos/alpine-sh" // manifest.v1+prettyjws
       })));


@@ -1405,7 +1405,7 @@ TEST_F(ProvisionerDockerTest, ROOT_INTERNET_CURL_ImageDigest)

   Image image;
   image.set_type(Image::DOCKER);
-  image.mutable_docker()->set_name("library/alpine@" + digest);
+  image.mutable_docker()->set_name("alpine" + digest);

   ContainerInfo* container = task.mutable_container();
   container->set_type(ContainerInfo::MESOS);

  • Modify $SOURCE_ROOT/mesos/src/tests/containerizer/runtime_isolator_tests.cpp:

    diff --git a/src/tests/containerizer/runtime_isolator_tests.cpp b/src/tests/containerizer/runtime_isola
index 539ae68..75366f1 100644
--- a/src/tests/containerizer/runtime_isolator_tests.cpp
+++ b/src/tests/containerizer/runtime_isolator_tests.cpp
@@ -411,7 +411,7 @@ TEST_F(DockerRuntimeIsolatorTest, ROOT_INTERNET_CURL_NestedSimpleCommand)
       v1::createCommandInfo("/bin/ls", {"ls", "-al", "/"}));

   taskInfo.mutable_container()->CopyFrom(
-      v1::createContainerInfo("library/alpine"));
+      v1::createContainerInfo("alpine"));

   Future<Event::Update> updateStarting;
   Future<Event::Update> updateRunning;

  • Modify $SOURCE_ROOT/mesos/src/tests/containerizer/cgroups_isolator_tests.cpp (For Ubuntu 20.x and RHEL 8.x only)

    diff --git a/src/tests/containerizer/cgroups_isolator_tests.cpp b/src/tests/containerizer/cgroups_isolator_tests.cpp
index aeac27406..2798f6ccf 100644
--- a/src/tests/containerizer/cgroups_isolator_tests.cpp
+++ b/src/tests/containerizer/cgroups_isolator_tests.cpp
@@ -2529,7 +2529,7 @@ TEST_F(CgroupsIsolatorTest, ROOT_CGROUPS_MemoryBackward)

 // This test verifies the cgroups blkio statistics
 // of the container can be successfully retrieved.
-TEST_F(CgroupsIsolatorTest, ROOT_CGROUPS_BlkioUsage)
+TEST_F(CgroupsIsolatorTest, DISABLED_ROOT_CGROUPS_BlkioUsage)
 {
   Try<Owned<cluster::Master>> master = StartMaster();
   ASSERT_SOME(master);

    Note: Disabling ROOT_CGROUPS_BlkioUsage as it is flaky and currently fails on newer Linux kernel version.

2.3) Run test cases

cd $SOURCE_ROOT/mesos/build
make check -k

NOTE: There are certain test cases that require root user permissions. They can be executed by: sudo ./bin/mesos-tests.sh --gtest_filter=*ROOT*

NOTE:

1. There are many test cases are fail intermittently. If this is the case, please run the test case individually as such below:

cd $SOURCE_ROOT/mesos/build
./bin/mesos-tests.sh --gtest_filter=<test_case>

2. Test cases failing with cache: Invalid format fail due to incorrect format of the file /etc/ld.so.cache. The file can be regenerated to the correct format as follows: sudo ldconfig -c compat. The test case should pass after an individual rerun.

Step 3: Run Apache Mesos

3.1) Start Master

cd $SOURCE_ROOT/mesos/build
sudo ./bin/mesos-master.sh --ip=<ip_address> --work_dir=/var/lib/mesos

3.2) Start Slave

cd $SOURCE_ROOT/mesos/build
sudo ./bin/mesos-agent.sh --master=<ip_address>:5050 --work_dir=/var/lib/mesos

3.3) Access Web UI

Open https://<ip_address>:5050 in your browser to access Mesos UI.

Reference

http://mesos.apache.org/

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

⚠️ **GitHub.com Fallback** ⚠️