Lab 3.2 - Configuring Filebeat

Filebeat is a lightweight shipper for forwarding and centralizing log data. Installed as an agent on your servers, Filebeat monitors the log files or locations that you specify, collects log events, and forwards them either to Elasticsearch or Logstash for indexing. As with our Metricbeat lab - we will be forwarding them directly to Elasticsearch.

Preparation

• Make sure Apache2 is installed and started, like in the previous lab.
• Also verify timedatectl gives the expected output.
• Make sure that Elasticsearch and Kibana are both running.

Configuring Filebeat

• Install with sudo apt-get install filebeat
• Get to the config file at /etc/filebeat/filebeat.yml, and similarly to the Metric beat file, adjust the Kibana and elasticsearch.output options to have your private IP.

Filebeat also uses modules. In this case, the modules will identify and send to data from logs for different services. In our lab, we will again use "system" which sends "system" logs from /var/log including auth.log and syslog. And "apache" which sends the Apache log data from /var/log/apache2

• Run sudo filebeat modules list to see all modules
• We are gonna use the system and apache ones so run sudo filebeat modules enable apache and sudo filebeat modules enable system
• Perform a sudo filebeat test config -e to make sure the config works
• Assuming it returns a "Config OK", run sudo filebeat setup -e. Again, this may take a minute.

Starting Filebeat

• Now, we can run sudo service filebeat start to get Filebeat up and running.
• After going to Kibana, you can go to the Discover page and view logs in the filebeat-* index. You should be able to see them for both "System" and "Apache".