Verification : ES - anujajakhade/anuja GitHub Wiki

Building Elasticsearch

Below versions of Elasticsearch are available in respective distributions at the time of creation of these build instructions.

  • Ubuntu 16.04 has 1.7.3

The instructions provided below specify the steps to build Elasticsearch 7.6.2 on Linux on IBM Z for following distributions:

  • RHEL (7.6, 7.7, 7.8)
  • SLES (12 SP4, 12 SP5)
  • Ubuntu (16.04, 18.04)

General Notes

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Building and installing Elasticsearch

1) Build using script

If you want to build Elasticsearch using manual steps, go to STEP 2.

Use the following commands to build Elasticsearch using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Elasticsearch/7.6.2/build_elasticsearch.sh

# Build Elasticsearch
bash build_elasticsearch.sh   [Provide -t option for executing build with tests]

If the build completes successfully, go to STEP 8. In case of error, check logs for more details or go to STEP 2 to follow manual build steps.

2) Install Build Dependencies

export SOURCE_ROOT=/<source_root>/

  • RHEL (7.6, 7.7, 7.8)

    sudo yum install -y curl git gzip tar wget patch

  • SLES (12 SP4, 12 SP5)

    sudo zypper install -y curl git gzip tar wget patch

  • Ubuntu (16.04, 18.04)

    sudo apt-get update
sudo apt-get install -y curl git gzip tar wget patch locales
sudo locale-gen en_US.UTF-8

3) Install AdoptOpenJDK

Download and install OpenJDK 13 with HotSpot here.
Note: At the time of creation of these build instructions, Elasticsearch was verified with AdoptOpenJDK 13 version (build 13.0.2+8).

4) Set the environment variables

unset JAVA_TOOL_OPTIONS
export LANG="en_US.UTF-8"
export JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF8"
export JAVA_HOME=/<Path to AdoptOpenJDK 13>/
export JAVA13_HOME=/<Path to AdoptOpenJDK 13>/
export PATH=$JAVA_HOME/bin:$PATH

Note: Ensure system locale is set up correctly for Elasticsearch to build without encoding errors.

5) Download Elasticsearch and apply patches

cd $SOURCE_ROOT
git clone https://github.com/elastic/elasticsearch
cd elasticsearch
git checkout v7.6.2

Apply gradle patch to generate tar with s390x suffix

wget https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Elasticsearch/7.6.2/patch/patch_gradle.diff
patch ./distribution/archives/build.gradle patch_gradle.diff

6) Build

cd $SOURCE_ROOT/elasticsearch
./gradlew -p distribution/archives/oss-linux-tar assemble --parallel

7) Testing (Optional)

cd $SOURCE_ROOT/elasticsearch
./gradlew --continue test -Dtests.haltonfailure=false -Dtests.jvm.argline="-Xss2m"

Notes:

  • To be able to run the tests, you'll need to install another version of the JDK to act as the runtime. AdoptOpenJDK 11 with Hotspot was used at the time of this writing. If another JDK is not set, then it will try to run all the test cases against the bundled JDK which is x64 based. You may encounter this error: Process 'command '~/elasticsearch/build/jdks/adoptopenjdk-13.0.2_linux/bin/java'' finished with non-zero exit value 2
  • Set RUNTIME_JAVA_HOME to the location of the JDK you'd like to use during testing.
  • If using java 11, ensure that you have JAVA11_HOME set to the location of the version 11 JDK. Likewise if using java 8, set JAVA8_HOME, etc...
  • Certain test cases may require an individual rerun to pass. There may be false negatives due to seccomp not supporting s390x properly.
  • If there is an stack overflow error, increase the -Xss arg value in the above command.
  • Some X-Pack test cases will fail as X-Pack plugins are not supported on s390x, such as Machine Learning features.
  • Illegal reflective access warnings in the logs are known issues. See https://github.com/elastic/elasticsearch/issues/40334.
  • For more information regarding Elasticsearch testing, please refer to their testing documentation.

8) Install Elasticsearch

cd $SOURCE_ROOT/elasticsearch
sudo mkdir /usr/share/elasticsearch
sudo tar -xzf distribution/archives/oss-linux-tar/build/distributions/elasticsearch-oss-7.6.2-SNAPSHOT-linux-s390x.tar.gz -C /usr/share/elasticsearch --strip-components 1
sudo ln -sf /usr/share/elasticsearch/bin/* /usr/bin/

sudo groupadd [username]
sudo chown [username]:[username] -R /usr/share/elasticsearch/

9) Verify Elasticsearch Server

> elasticsearch --version
> Version: 7.6.2-SNAPSHOT, Build: oss/tar/aa751e09be0a5072e8570670309b1f12348f023b/2020-03-17T21:30:56.294375Z, JVM: 13.0.2

Use curl -X GET http://localhost:9200/ when running Elasticsearch. The output should be similar to this:

{
  "name" : "#########",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "NNgHT8mqR8Ol7Or8-ZgBAQ",
  "version" : {
    "number" : "7.6.2-SNAPSHOT",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-04-28T22:26:04.556439Z",
    "build_snapshot" : true,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

10) Start Elasticsearch Server

elasticsearch

Note:

  • Elasticsearch needs to be run by a non-root user which has standard permission rights, hence we have changed the owner in the above step.

Installing Elasticsearch Curator client (Optional)

Elasticsearch Curator can be used to curate, or manage, Elasticsearch indices and snapshots. It can be installed using below steps:

1) Install dependencies

  • RHEL (7.6, 7.7, 7.8)

    sudo yum install -y python-setuptools
sudo easy_install pip

  • SLES (12 SP4, 12 SP5)

    sudo zypper install -y python-devel python-setuptools
sudo easy_install pip

  • Ubuntu (16.04, 18.04)

    sudo apt-get update
sudo apt-get install -y python-pip

2) Install Elasticsearch Curator

```bash  
sudo pip install elasticsearch-curator
```

References:

https://www.elastic.co/products/elasticsearch
https://www.elastic.co/guide/en/elasticsearch/client/curator/5.x/index.html