Prometheus is a monitoring system that allows us to monitor the performance of services with more detail.

Issues page for this lab here

1. Download Prometheus on the backup server

wget https://github.com/prometheus/prometheus/releases/download/v2.18.0-rc.1/prometheus-2.18.0-rc.1.linux-amd64.tar.gz

2. Extract Prometheus with this command

tar xvfz prometheus-2.18.0-rc.1.linux-amd64.tar.gz

Here is the current prometheus.yml

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']

note: under scrape_configs is where you list what services you what it to monitor, currently it is monitoring itself.

3. Run Prometheus with this command

./prometheus --config.file=prometheus.yml

Then by visiting the URL http://13.75.139.76:9090/targets you should be able to see the Prometheus giving data about itself as that's what its currently configured for.

1. On the backup server, download this: sudo wget https://github.com/prometheus/node_exporter/releases/download/v1.0.0-rc.0/node_exporter-1.0.0-rc.0.linux-amd64.tar.gz

2. Extract with this command: sudo tar xvfz node_exporter-1.0.0-rc.0.linux-amd64.tar.gz

3. cd node_exporter-1.0.0-rc.0.linux-amd64 , then run this command : ./node_exporter The exporter provides data at the url http://13.75.139.76:9100.

4. Add the following to that file in the scrape_config section inside the prometheus.yml file:

- job_name: ’node’
static_configs:
- targets: [’localhost:9100’]

This file’s contents are in YAML, so be sure the indenting is consistent.

5. Run both the node exporter and prometheus on 2 different shells (in their respective directories)

• ./node_exporter
• ./prometheus --config.file=prometheus.yml

6. Go to the prometheus target url, you should see this:

7. Query the Node Exporter’s metrics in the expression browser in the "Graph" tab.

Examples:

• node_filesystem_free_bytes shows free disk space

• node_load1 shows the CPU load average.

Note: you can also use the "insert metric at cursor" option for more options

Choosing this as an example:

• node_timex_maxerror_seconds