ELK Stack with Email Plugin for Error logs - madhusudana30/AlternativeJPAForWebSphere GitHub Wiki

ELK Stack (Elastic search, Logstash, kibana)

Wednesday, May 15, 2019 5:56 PM

Install ELK and filebeat, ngnix

logstash.conf

input { file { path => "/home/tomcat/apache-tomcat-7.0.92/logs/.log" start_position => beginning type=> "my_log" } }

filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}" } } date { match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ] } }

output { stdout { codec => rubydebug } if [type] == "my_log" { elasticsearch { hosts => ["localhost:9200"] } } }

Validate configuration: ./logstash -f logstash.conf -t --verbose —debug

Run logstash:

/logstash -f logstash.conf

References: https://www.tutorialspoint.com/logstash/logstash_transforming_the_logs.htm

https://www.elastic.co/guide/en/logstash/current/plugins-outputs-email.html

Error Logs: Define grok format based on the actual Error message.

input { file { path => "/home/tomcat/apache-tomcat-7.0.92/logs/." start_position => beginning type=> "my_log" } }

filter { grok { match => { "message" => "%{LOGLEVEL:level} %{GREEDYDATA:messageText}" } } } output { stdout { codec => rubydebug } if [type] == "my_log" and [level]=="ERROR" { elasticsearch { hosts => ["localhost:9200"] } } }

Email Notification:

input { file { path => "/home/tomcat/apache-tomcat-7.0.92/logs/." start_position => beginning type=> "my_log" } }

filter { grok { match => { "message" => "%{LOGLEVEL:level} %{GREEDYDATA:messageText}" } } date { match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ] } } output { stdout { codec => rubydebug } if [type] == "my_log" and [level]=="ERROR" { email { address => "localhost" codec => "plain" contenttype => "text/html; charset=UTF-8" to => 'tomail' from => 'frommail' subject => 'Alert - Error' domain => 'localhost' via => "smtp" port => 25 use_tls => false } elasticsearch { hosts => ["localhost:9200"] } } }