ELK Setup On Centos 7

“ELK” is the acronym for Elasticsearch, Logstash, and Kibana.

lasticsearch: This is an open source, distributed, RESTful, JSON-based search engine. It is scalable, easy to use, and flexible
Logstash : the data processing component of the Elastic Stack which sends incoming data to Elasticsearch.
Kibana: lets users visualize data with charts and graphs in Elasticsearch.

Installing and Configuring Elasticsearch

Java Installation

yum -y install java-openjdk-devel java-openjdk

Adding ELK 7 repo

cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF

Importing GPG key

 rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

You can set JVM options like memory limits by editing the file: /etc/elasticsearch/jvm.options

The example below sets the min and max size of total heap space

-Xms1g
-Xmx1g

Starting and enable the Elasticsearch

systemctl start elasticsearch
systemctl enable elasticsearch

Now lets test whether your Elasticsearch service is running by sending an HTTP request:

curl http://127.0.0.1:9200

Response will be

{
  "name" : "swarmmasternode",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "DlV5QuhXQJOyDCREzr185Q",
  "version" : {
    "number" : "7.2.0",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "508c38a",
    "build_date" : "2019-06-20T15:54:18.811730Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Installing and Configuring Kibana

Install Kibana

yum -y install kibana

Now, let's configure Kibana:

you can expose Kibana dashboard directly or you can set up a nignx server and proxy it.
Here I'm exposing the dashboard directly.

server.host: "0.0.0.0"
server.name: "kibana.host.com"
elasticsearch.url: "http://localhost:9200"

Then enable and start the Kibana service:

systemctl enable kibana
systemctl start kibana

Now we can access kibana dashboard - http://HOST_MACHINE_IP:5601

Installing and Configuring Logstash

Install Logstash

yum install -y logstash

Now Create a configuration file called filebeat.conf where you will set up your Filebeat configurations:

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}

filebeat-conf

Test your Logstash configuration with this command:

sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash -t

If your configuration test is successful, now lets start and enable Logstash: `` systemctl start logstash systemctl enable logstash

#### Installing and Configuring Filebeat
The Elastic Stack uses several lightweight data shippers called Beats to collect data from various sources and transport them to Logstash or Elasticsearch. We are going to configure filebeat here. Fielbeat is the only part of the infrastructure that needs to be installed on a client server.<br>
Adding repo:

cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo [elasticsearch-7.x] name=Elasticsearch repository for 7.x packages baseurl=https://artifacts.elastic.co/packages/7.x/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md EOF

Install filebeat

yum install filebeat

Next, configure Filebeat to connect to Logstash. Here, we will modify the example configuration file that comes with Filebeat.

vi /etc/filebeat/filebeat.yml


Starting and Enable filebeat

systemctl start filebeat systemctl enable filebeat

ELK Setup On Centos 7 - saviovettoor/DevOps-wiki GitHub Wiki