Installation ElasticSearch Kibana security and FAQ - kwantu/platformconfiguration GitHub Wiki

back

Installation ElasticSearch Kibana: Security Confuguration and FAQ

https://www.elastic.co/guide/en/elasticsearch/reference/7.4/secure-settings.html https://www.elastic.co/guide/en/elasticsearch/reference/current/get-started-enable-security.html https://blog.pythian.com/how-to-secure-your-elastic-stack-plus-kibana-logstash-and-beats/ https://www.vultr.com/docs/how-to-install-and-configure-elastic-stack-elasticsearch-logstash-and-kibana-on-ubuntu-17-04

The elastic and kibana key stores.

/etc/elasticsearch/elasticsearch.keystore
/var/lib/kibana/kibana.keystore

# Note this should be run as kibana user
sudo -u kibana /usr/share/kibana/bin/kibana-keystore create
sudo -u kibana /usr/share/kibana/bin/kibana-keystore add elasticsearch.username
sudo -u kibana /usr/share/kibana/bin/kibana-keystore add elasticsearch.username
sudo -u kibana /usr/share/kibana/bin/kibana-keystore add elasticsearch.password

Installing X-Pack

This is not needed, the basic distribution has it by default.

X-Pack is packaged as Elasticsearch and Kibana plugins. You’ve to use the bundled plugin tool to install it:

# install x-pack for elasticsearch  (reply 'y' to permissions question)
cd /usr/share/elasticsearch
sudo bin/elasticsearch-plugin install x-pack

# install x-pack for kibana
cd /usr/share/kibana
sudo bin/kibana-plugin install x-pack

Using this process

https://www.elastic.co/guide/en/elasticsearch/reference/current/get-started-enable-security.html

  1. Enable xpack security that is disabled in basic distribution
 vim /etc/elasticsearch/elasticsearch.yml
##
# ---------------------------------- Network -----------------------------------
# Set the bind address to a specific IP (IPv4 or IPv6):
network.host: 197.242.158.124
#
# Set a custom port for HTTP:
http.port: 9200
# ...
#-- Security setup
xpack.security.enabled: true
discovery.type: single-node
  1. Restart elastic and tail the log to make sure it starts correctly
tail -f  /var/log/elasticsearch/elasticsearch.log
---
[2020-05-02T16:49:34,144][INFO ][o.e.n.Node               ] [kwantu9data.kwantu.net] started
[2020-05-02T16:49:34,500][INFO ][o.e.c.s.ClusterSettings  ] [kwantu9data.kwantu.net] updating [xpack.monitoring.collection.enabled] from [false] to [true]
[2020-05-02T16:49:35,222][INFO ][o.e.l.LicenseService     ] [kwantu9data.kwantu.net] license [0e1e207e-6f59-4c4f-ab6b-c28cadf96a2a] mode [basic] - valid
[2020-05-02T16:49:35,223][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [kwantu9data.kwantu.net] Active license is now [BASIC]; Security is enabled
[2020-05-02T16:49:35,233][INFO ][o.e.g.GatewayService     ] [kwantu9data.kwantu.net] recovered [91] indices into cluster_state
  1. Set the built-in users' passwords
cd /usr/share/elasticsearch
./bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y

Enter password for [elastic]: <redacted>
Reenter password for [elastic]:<redacted>
Enter password for [apm_system]:<redacted>
Reenter password for [apm_system]:<redacted>
Enter password for [kibana]:<redacted>
Reenter password for [kibana]:<redacted>
Enter password for [logstash_system]:<redacted>
Reenter password for [logstash_system]:<redacted>
Enter password for [beats_system]:<redacted>
Reenter password for [beats_system]:<redacted>
Enter password for [remote_monitoring_user]:<redacted>
Reenter password for [remote_monitoring_user]:<redacted>
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]
  1. Specify the elastic user and password for kibana to use This can either be set in the /etc/kibana/kibana.yml config file, or in the more secure keystore
cd /usr/share/kibana
./bin/kibana-keystore create --allow-root
Created Kibana keystore in /var/lib/kibana/kibana.keystore
./bin/kibana-keystore add elasticsearch.username --allow-root
Enter value for elasticsearch.username: *******
./bin/kibana-keystore add elasticsearch.password --allow-root
Enter value for elasticsearch.password: ************

# make sure that the keystore is readable by the kibana user. From version 7.9.1 it moved to /etc/kibana, so make sure you find it in the right place
chown -R kibana:kibana /usr/share/kibana
chown -R kibana:kibana /etc/kibana


# Restart kibana
service kibana restart
# Check the logs to make sure it started OK
cd /var/log/kibana
tail -f kibana.stderr
tail -f kibana.stderr

Frequently asked questions

Number of shards error

If there is an error with the maximum number of shards that are open, you can use this to increaase the default from 1000 to 3000

curl -u elastic:<password-here> -X PUT localhost:9200/_cluster/settings -H "Content-Type: application/json" -d '{ "persistent": { "cluster.max_shards_per_node": "10000" } }'

Circuit breaking exception error

2020-09-07 18:19:51,025 [qtp923013679-24002] WARN (LogFunction.java [writeLog]:196) - (Line: 475 /db/kwantu-resource/modules/reportDataSource.xqm) debug $elastic call finish {
"message" : "Error in calling REST PUT service: 429",
"url" : "http://154.0.175.164:9200/view_c737263b1cf60482dac3959c4b5585e1-1",
"inData" : {
"settings" : {
"index.mapping.ignore_malformed" : true
},
"mappings" : {
"properties" : {
"meta_rowId" : {
"type" : "keyword"
}
}
}
},
"errorCode" : "mod-server.xqm 3 server:doPutRestCall() http Statuscode = 429",
"result" : {
"error" : {
"bytes_wanted" : 1.0268482E9,
"reason" : "[parent] Data too large, data for [<http_request>] would be [1026848200/979.2mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1026847976/979.2mb], new bytes reserved: [224/224b], usages [request=0/0b, fielddata=153371/149.7kb, in_flight_requests=224/224b, accounting=37488160/35.7mb]",
"bytes_limit" : 1.020054732E9,
"durability" : "PERMANENT",
"root_cause" : [ {
"bytes_wanted" : 1.0268482E9,
"reason" : "[parent] Data too large, data for [<http_request>] would be [1026848200/979.2mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1026847976/979.2mb], new bytes reserved: [224/224b], usages [request=0/0b, fielddata=153371/149.7kb, in_flight_requests=224/224b, accounting=37488160/35.7mb]",
"bytes_limit" : 1.020054732E9,
"durability" : "PERMANENT",
"type" : "circuit_breaking_exception"
} ],
"type" : "circuit_breaking_exception"
},
"status" : 429
},
"isError" : true
}

https://www.elastic.co/guide/en/elasticsearch/reference/current//circuit-breaker.html

To address this, set the following in elasticsearch.yml

indices.breaker.total.use_real_memory: false
⚠️ **GitHub.com Fallback** ⚠️