Lab 3.1 Configuring Metricbeat - Hsanokklis/2023-2024-Tech-journal GitHub Wiki

Helpful Info

The Public IPv4 address will change every new session. Current IPv4 address in use: 3.83.14.248

to access your instance ssh -i hannelore-elk-key.pem ubuntu@public key

Private IPv4 address is : 172.31.87.23

When next login your system make sure to start everything again

• Start in this order: Elasticsearch, Logstash, Kibana

To get to the kibana dashboard type http://public-ip:5601 into your browser

Helpful Vocab!

Metricbeat is a lightweight shipper that you can install on your servers to periodically collect metrics from the operating system and from services running on the server. Metricbeat takes the metrics and statistics that it collects and ships them to the output that you specify, such as Logstash - or as we will do in this lab, directly to Elasticsearch.

Specifically, we will have Metricbeat ship metrics from System and Apache to Elasticsearch and then we can view in Kibana.

Lightweight shipper:

• ur mom

Step 1: Lab Prep

Install Apache2 on the Ubuntu server

We can use this as another service to get metrics from and view in Kiana

• sudo apt install apache2
• sudo systemctl start apache2

browse to your server on port 80 - you should see the default Apache page

Check your date/time on your Ubuntu server

• timedatectl to make sure your timezone and time is correct (use timedatectl to adjust if needed)

Step 2: Installing Beats

The various shippers belonging to the Beats family can be installed in exactly the same way as we installed the other Elastic Stack components.

Install Metricbeat

• sudo apt-get install metricbeat

Step 3: Configure Metricbeat

The metricbeat config file is metricbeat.yml. It should be in /etc/metricbeat

Edit metricbeat.yml

• In the Kibana section -->
  • uncomment hosts
  • change localhost to the IP of your ELK server

• In the Output-Elasticsearch section -->
  • change the localhost to the IP of your ELK server


Metricbeat uses modules to collect metrics. Each module defines the basic logic for collecting data from a specific service. In our lab, we will use "system" which collects basic system metrics (like uptime, cpu...) and apache which collects metrics on the apache service

• sudo metricbeat modules list|more
  • will show all the supported modules. You should see that "system" is enabled by default

• sudo metricbeat modules enable apache
  • will also enable the Apache module
• sudo metricbeat test config -e
  • shows you config info and output should end with "Config OK"

Use Metricbeat setup command to load the recommended index template for writing to Elasticsearch and deploy the sample dashboards for visualizing the data in Kibana.

• sudo metricbeat setup -e

Step 4: Starting Metricbeat

Start metricbeat

• sudo service metricbeat start

In the side navigation, click Discover. To see the Metricbeat data make sure the predefined metricbeat- index pattern is selected.* You should see logs from the Metricbeat Agent showing up.

SUBMIT Screenshot of metricbeat data

In the side navigation, click Dashboard

• A number of defined Metricbeat Dashboards are available
• Find [Metricbeat System] Overview ECS - it should provide an overview of the system status

Submit a screenshot

• Change Dashboard to [Metricbeat Apache] Overview ECS - it provides info on Apache

Submit a screenshot

Step 5: Metricbeat Analysis

Go back to Discover and view the Metricbeat logs

• Expand some of the entries and look at the metrics being provided

Find a "system" entry and make note of a metric that could be valuable to monitor and the value listed

system.cpu.nice.norm.pct metricbeat query is querying for the percentage of CPU time spent on processes with a "nice" priority, and the value is normalized, meaning it is adjusted to a common scale for easier comparison.

Find an "apache" entry and make note of a metric that could be valuable to monitor and the value listed`

The Metricbeat query apache.status.hostname:* is used to filter and retrieve metrics related to Apache HTTP Server's status, specifically targeting metrics associated with a specific hostname.

apache.status.hostname:* is looking for Apache server status metrics without restricting the search to a specific hostname. It aims to collect information about the performance and status of Apache servers across all available hostnames.

Click on the Search bar and start typing "system" to find the system metric you identified above.

• Create a search filter that displays log results for that metric

Submit Screenshot

Click on the Search bar and start typing "apache" to find the apache metric you identified above.

• Create a search filter that displays log results for that metric

Submit Screenshot