ElasticSearch安装配置 - yiyixiaozhi/readingNotes GitHub Wiki
[TOC]
ElasticSearch安装配置
介绍了CentOS7下安装和配置elasticSearch
下载地址：
https://www.elastic.co/cn/downloads/elasticsearch
- CentOS7安装
下载网站：
https://www.elastic.co/cn/downloads/past-releases/elasticsearch-6-8-4
https://www.elastic.co/cn/downloads/past-releases/kibana-6-8-4
下载rpm包：
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.8.4.rpm
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.8.4-x86_64.rpm
建立用户并设定密码
useradd yyxzES
passwd yyxzES
设置密码：Qz@20200321
给新用户赋予安装权限
# vi /etc/sudoers
## Allow root to run any commands anywhere
root ALL=(ALL) ALL
yyxzES ALL=(ALL) ALL
使用yyxzES进行安装
sudo rpm --install kibana-6.8.4-x86_64.rpm
sudo rpm --install elasticsearch-6.8.4.rpm
进行配置
# vi /etc/elasticsearch/elasticsearch.yml
network.host: 0.0.0.0
path.data: /home/ftpsite/yyxz/sda1.5TB/elasticsearch/data
path.logs: /home/ftpsite/yyxz/sda1.5TB/elasticsearch/logs
# vi /etc/kibana/kibana.yml
server.host: "0.0.0.0"
手动启动可以看到出错log
/usr/share/elasticsearch/bin/elasticsearch
/usr/share/kibana/bin/kibana
如果出错log中报权限问题，进行设置，部分示例如下：
chown -R yyxzES:yyxzES /etc/elasticsearch/
chown -R yyxzES:yyxzES /usr/share/elasticsearch/
chown -R yyxzES:yyxzES /usr/lib/systemd/system/elasticsearch.service
chown -R yyxzES:yyxzES /var/log/elasticsearch/
chown -R yyxzES:yyxzES /var/lib/elasticsearch/
chown -R yyxzES:yyxzES /etc/sysconfig/elasticsearch
chown -R yyxzES:yyxzES /usr/share/kibana/
chown -R yyxzES:yyxzES /var/lib/kibana/
chown -R yyxzES:yyxzES /etc/kibana/
如果报jvm错误，则配置相关文件：
vim /usr/share/elasticsearch/bin/jvm.options
## 写入内容
-Xms512m
-Xmx512m
手动启动通过后，可以使用服务来启动：
# systemctl daemon-reload
# systemctl enable elasticsearch.service
# systemctl enable kibana.service
官方教程：https://www.elastic.co/guide/en/elasticsearch/reference/7.6/rpm.html#rpm-repo
# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.1-x86_64.rpm
# rpm --install elasticsearch-7.6.1-x86_64.rpm
配置文件路径：/etc/elasticsearch/elasticsearch.yml
配置成开机启动服务：
# systemctl daemon-reload
# systemctl enable elasticsearch.service
指定启动用户
# vi /etc/systemd/system/multi-user.target.wants/elasticsearch.service
# vi /etc/systemd/system/multi-user.target.wants/kibana.service
User=yyxzES
Group=yyxzES
# systemctl daemon-reload
删除命令：
yum remove elasticsearch.x86_64
- 普通安装
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.1-linux-x86_64.tar.gz
解压到当前目录
cd /home/yyxz/download/
tar -zxvf elasticsearch-7.6.1-linux-x86_64.tar.gz
mv elasticsearch-7.6.1 /usr/local/
cd /usr/local/
更改elasticsearch-7.6.1文件夹及内部文件的所属用户及组为yyxzES
chown -R yyxzES:yyxzES elasticsearch-7.6.1
配置文件路径：/usr/local/elasticsearch-7.6.1/config/elasticsearch.yml
切换用户并启动 elasticsearch
su yyxzES
cd elasticsearch-7.6.1/bin/
./elasticsearch # 前台启动，接 ctrl + c 停止elasticsearch服务
./elasticsearch -d # 后台启动
基础配置
chown -R elasticsearch:elasticsearch /home/ftpsite/yyxz/sda1.5TB/elasticsearch/logs
chown -R elasticsearch:elasticsearch /home/ftpsite/yyxz/sda1.5TB/elasticsearch/data
vi elasticsearch.yml
## 修改文件如下部分
#network.host: 192.168.0.1
network.host: 0.0.0.0 #让所有主机都可以访问（开启远程访问）
...
#path.data: /path/to/data
path.data: /home/ftpsite/yyxz/sda1.5TB/elasticsearch/data
...
#path.logs: /path/to/logs
path.logs: /home/ftpsite/yyxz/sda1.5TB/elasticsearch/logs
本地 curl 测试
curl 127.0.0.1:9200
放通防火墙端口(如果防火墙打开的话)
firewall-cmd --zone=public --add-port=9200/tcp --permanent
firewall-cmd --reload
报错整理：
cd elasticsearch-7.6.1/bin/
./elasticsearch
...
[2020-03-21T14:40:45,789][INFO ][o.e.n.Node ] [localhost.localdomain] starting ...
[2020-03-21T14:40:45,917][INFO ][o.e.t.TransportService ] [localhost.localdomain] publish_address {172.18.0.1:9300}, bound_addresses {[::]:9300}
[2020-03-21T14:40:46,178][INFO ][o.e.b.BootstrapChecks ] [localhost.localdomain] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [3] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[3]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
ERROR: Elasticsearch did not exit normally - check the logs at /usr/local/elasticsearch-7.6.1/logs/elasticsearch.log
解决第一个错误：系统限制用户的执行内存,修改系统安全限制配置文件
vi /etc/security/limits.conf
#@student - maxlogins 4
* soft nofile 65536
* hard nofile 65536
* soft nproc 65536
* hard nproc 65536
# End of file
解决第二个错误：
vi /etc/sysctl.conf
末尾添加一行
vm.max_map_count = 655360
sysctl -p # 查看参数
解决第三个错误：
vi /usr/local/elasticsearch-7.6.1/config/elasticsearch.yml
添加一行：
cluster.initial_master_nodes: ["node-1"]
elasticsearch.yml
其他：
调整 JVM 内存大小
vi elasticsearch
ES_JAVA_OPTS="-Xms512m -Xmx512m"
需要和elasticsearch版本一致。
下载地址：https://www.elastic.co/cn/downloads/kibana
yum remove kibana.x86_64 #删除已安装过的（如果安装过的话）
# 有时候要这样才能删除干净：rpm --install kibana-7.6.1-x86_64.rpm --force --nodeps
wget https://artifacts.elastic.co/downloads/kibana/kibana-7.6.1-x86_64.rpm
tar -zxvf kibana-7.6.1-linux-x86_64.tar.gz
rpm --install kibana-7.6.1-x86_64.rpm
systemctl daemon-reload
systemctl enable kibana.service
systemctl start kibana.service
配置文件路径：/etc/kibana/kibana.yml
yum install perl-Digest-SHA #为了查看shasum -a 512 kibana-7.6.1-x86_64.rpm
- 普通安装
# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.6.1-linux-x86_64.tar.gz
mv /home/yyxzES/download/kibana-7.6.1-linux-x86_64 /opt/
配置文件路径：/opt/kibana-7.6.1-linux-x86_64/config/kibana.yml
启动 kibana
cd /opt/kibana-7.6.1-linux-x86_64/bin/
./kibana # 前台启动，接 ctrl + c 停止
./kibana & # 后台启动
放通防火墙端口
firewall-cmd --zone=public --add-port=5601/tcp --permanent
firewall-cmd --reload
修改配置文件：
vi kibana.yml
## 修改文件如下部分
#server.host: "localhost"
server.host: "0.0.0.0" # 让所有主机都可以访问
github：https://github.com/medcl/elasticsearch-analysis-ik
下载
wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.6.1/elasticsearch-analysis-ik-7.6.1.zip
解压缩
unzip -o elasticsearch-analysis-ik-7.6.1.zip -d elasticsearch-analysis-ik-7.6.1/
mv elasticsearch-analysis-ik-7.6.1 /usr/share/elasticsearch/plugins/
如果报错：blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];
curl -XPUT -H "Content-Type: application/json" http://172.31.192.1:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'
常用命令示例：
GET _search
{
"query": {
"match": {
"serialType": "HTTP"
}
}
}
GET _search
{
"query": {
"range": {
"createTime": {
"gte": "2020-06-05T00:00:00"
}
}
}
}
POST sys-log/_delete_by_query
{
"query": {
"match": {
"serialType": "HTTP"
}
}
}
POST sys-log/_delete_by_query?wait_for_completion=true
{
"query": {
"range": {
"createTime": {
"lt": "1591113600000"
}
}
}
}
参考：
- https://www.cnblogs.com/heqiuyong/p/10324934.html
- https://blog.csdn.net/CNZYYH/article/details/93404794
- Elasticsearch删除数据之_delete_by_query
https://www.elastic.co/cn/downloads/past-releases/elasticsearch-6-8-4
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.8.4.rpm
https://www.elastic.co/cn/downloads/past-releases/kibana-6-8-4
https://artifacts.elastic.co/downloads/kibana/kibana-6.8.4-x86_64.rpm
本示例以更改sys-log索引为例进行演示。
一：查看原有字段动态映射类型
GET sys-log/_mapping
{
"sys-log" : {
"mappings" : {
"OperationLog" : {
"properties" : {
"actionDesc" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"createTime" : {
"type" : "long"
},
"ipAddress" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"macAddress" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"serialType" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"serialValue" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}
}
二：更改指定字段类型，新建索引sys-log-new
PUT sys-log-new
{
"mappings": {
"OperationLog": {
"properties": {
"actionDesc": {
"type": "text",
"analyzer": "ik_smart"
},
"createTime": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
"ipAddress": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"macAddress": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"name": {
"type": "text"
},
"serialType": {
"type": "text"
},
"serialValue": {
"type": "text"
},
"updateTime": {
"type": "date"
},
"username": {
"type": "text"
}
}
}
}
}
三：重建索引，将sys-log数据reindex到sys-log-new
POST _reindex
{
"source": {
"index": "sys-log"
},
"dest": {
"index": "sys-log-new"
}
}
四：删除索引sys-log
命令：
DELETE sys-log
五：将sys-log-new数据reindex到sys-log
POST _reindex
{
"source": {
"index": "sys-log-new"
},
"dest": {
"index": "sys-log"
}
}
六：删除索引sys-log-new
命令：
DELETE sys-log-new