[Archive 1.0] 8. ELK Dashboard - mata-elang-stable/MataElang-Platform GitHub Wiki

Back to HOME

• Prerequisite
• Install docker-compose
• Install ELK
• Configure ELK
• Start ELK
• Import Dashboard

Prerequisite

• Ubuntu 18.04.6 LTS installed and updated by sudo apt update && sudo apt -y upgrade command.
• Time Zone and NTP already set.
• Docker 20.10.11 or later installed and ready.

Install docker-compose

1. Install docker-compose 1.29 and check its version.

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

sudo chmod +x /usr/local/bin/docker-compose

docker-compose --version

• Result

docker-compose version 1.29.2, build 5becea4c

Install ELK

2. Make clone of Mata-Elang-Stable/Dashboard from GitHub.

cd ~

git clone https://github.com/Mata-Elang-Stable/Dashboard.git

Configure ELK

3. Run the docker-compose first for setting user password.

cd Dashboard

if you want to custom JVM heap size based on a node’s roles and total memory, change ES_JAVA_OPTS. The default uses 2 GB of memory as below

elasticsearch:
    .....
    environment:
      ES_JAVA_OPTS: -Xmx2g -Xms2g
    .....

sudo docker-compose up -d elasticsearch

4. Reset the password of the all user (elastic, kibana_system), and enter the new password.

sudo docker-compose exec elasticsearch bin/elasticsearch-reset-password --batch --user elastic -i

sudo docker-compose exec elasticsearch bin/elasticsearch-reset-password --batch --user kibana_system -i

5. Change password on "kibana.yml" and "logstash.conf", replace <NEW PASSWORD HERE>.

• Use elastic user password just changed.

nano logstash/pipeline/logstash.conf

• Use kibana_system password just changed.

nano kibana/config/kibana.yml

6. Change the new password inside the ".env" (replace changeme for each user password just changed).

nano .env

7. Add logstash pipeline configuration to consuming kafka topic from Kafka, and change <KAFKA_SERVER_IP_OR_NAME>.

nano logstash/pipeline/logstash.conf

• Configuration

input {
  kafka {
    bootstrap_servers => "<KAFKA_SERVER_IP_OR_NAME>:9092"
    topics => "event_1s"
  }
}

8. Edit "kibana.yaml" to specify the public IP or name of the server.

nano kibana/config/kibana.yml

• Configuration

server.name: kibana
server.host: 0.0.0.0
server.publicBaseUrl: http://<PUBLIC_SERVER_IP_OR_NAME>:5601

Start ELK

9. Restart Logstash and Kibana to reconnect to Elasticsearch with new passwords.

sudo docker-compose up -d logstash kibana

10. Check each user password (elastic, kibana_system).

curl http://localhost:9200 -u elastic:<YOUR_PASSWORD>

curl http://localhost:9200 -u kibana_system:<YOUR_PASSWORD>

• Reuslt

{
  "name" : "32becb30ead1",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "GA6fFSR6S3iCSYmtkDsKlg",
  "version" : {
    "number" : "8.0.0",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "1b6a7ece17463df5ff54a3e1302d825889aa1161",
    "build_date" : "2022-02-03T16:47:57.507843096Z",
    "build_snapshot" : false,
    "lucene_version" : "9.0.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

Import Dashboard

11. Access Kibana with the following URL and log in as elastic user.

• URL: http://<ELK_SERVER_IP_OR_NAME>:5601

12. Download MataElang Dashboard Template from GitHub and save it on your PC.

• URL: https://raw.githubusercontent.com/Mata-Elang-Stable/Dashboard/master/MataElang-ELK-Dasboard.ndjson

13. Import MataElang Dashboard Template to Kibana on the side panel.

• [Stack Management]->[Saved Object]->[import]

Next Step >>