Elastic Search multi hosts setting - unix1998/technical_notes GitHub Wiki

To install and set up a multi-node Elasticsearch cluster, follow these general steps:

Prerequisites:

1. Java: Ensure Java is installed on all nodes. Elasticsearch requires Java 8 or later.

   java -version
   

2. System Requirements: Ensure each node meets the minimum system requirements for Elasticsearch.

Installation:

1. Download and Install Elasticsearch:

   • Download the latest version of Elasticsearch from the official website.
   • Install it on each node using the appropriate package manager or by extracting the tar.gz file.

     # For Debian-based systems
     sudo apt-get update
     sudo apt-get install elasticsearch
     
     # For RHEL-based systems
     sudo yum install elasticsearch
     

2. Configure Elasticsearch:

   • Edit the elasticsearch.yml configuration file on each node, typically located at /etc/elasticsearch/elasticsearch.yml.
   • Set unique node names and the same cluster name for all nodes:

     cluster.name: my_cluster
     node.name: node_1 # Change this for each node
     

   • Configure the network settings:

     network.host: 0.0.0.0 # or specific IP address
     http.port: 9200
     

   • Configure discovery settings for the multi-node cluster:

     discovery.seed_hosts: ["node1_ip", "node2_ip", "node3_ip"]
     cluster.initial_master_nodes: ["node1", "node2", "node3"] # Mention the master-eligible nodes
     

   • Ensure each node's path.data and path.logs directories are set appropriately:

     path.data: /var/lib/elasticsearch
     path.logs: /var/log/elasticsearch
     

3. Start Elasticsearch:

   • Start the Elasticsearch service on each node:

     sudo systemctl start elasticsearch
     

   • Enable it to start on boot:

     sudo systemctl enable elasticsearch
     

4. Verify the Cluster:

   • Verify that each node has joined the cluster by checking the cluster health and node information:

     curl -X GET "http://node1_ip:9200/_cluster/health?pretty"
     curl -X GET "http://node1_ip:9200/_cat/nodes?v"
     

Post-Installation:

1. Set Up Index Replicas:

   • Configure index settings to ensure data is replicated across nodes for high availability:

     {
       "settings": {
         "index": {
           "number_of_replicas": 2
         }
       }
     }
     

2. Monitor the Cluster:

   • Use Kibana or any other monitoring tools to keep track of the cluster's health and performance.
   • Regularly check logs and Elasticsearch APIs for any issues.

3. Secure the Cluster:

   • Implement security features such as TLS encryption, user authentication, and role-based access control (RBAC).

Example Configuration (elasticsearch.yml):

cluster.name: my_cluster
node.name: node_1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: ["node1_ip", "node2_ip", "node3_ip"]
cluster.initial_master_nodes: ["node1", "node2", "node3"]

By following these steps, you can set up a multi-node Elasticsearch cluster that is scalable and resilient.