In this guide, we’ll learn how to run Prometheus, Node Exporter and Grafana as Docker containers on a Raspberry Pi 4 machine, with the containers managed by Docker Compose allowing us to collect detailed metrics, store them efficiently, and visualize them in a Grafana dashboard.

We’ll mount the relevant host directories into the Node Exporter and Prometheus containers, and configure Prometheus to scrape Node Exporter metrics and push them to Grafana. We’ll then install a preconfigured dashboard to visualize these system metrics.

Before we begin we should have the following available:

• A Raspberry Pi 4 machine
• Docker and Docker Compose installed on our Raspberry Pi 4 machine

In this step, we’ll create a docker-compose.yml file which will define our prometheus and node-exporter services, as well as our monitoring bridge network.

This will be the content of the docker-compose.yml file:

version: '3.7'
services:
  prometheus:
    image: prom/prometheus:latest
    networks:
      - monitorpi
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - ./prometheus_data:/prometheus
    command:
      - --config.file=/etc/prometheus/prometheus.yml
    ports:
      - 9090:9090

  node-exporter:
    image: prom/node-exporter:latest
    networks:
      - monitorpi
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    command:
      - --path.procfs=/host/proc
      - --path.sysfs=/host/sys
      - --path.rootfs=/rootfs

networks:
  monitorpi

For the node-exporter service, we mount some necessary paths from the host into the container in :ro or read-only mode:

• /proc
• /sys
• /

The prometheus service persists its data to a local directory on the host at ./prometheus_data. Docker Compose will create this directory after starting the prometheus container.

In the next step, we’ll create the Prometheus configuration file, which Compose will read from ./prometheus.yml.

In this step, we’ll configure Prometheus to scrape node-exporter metrics and ship them to Grafana. We’ll configure the following sections:

• global: Global Prometheus config defaults. In this example, we set the scrape_interval for scraping metrics from configured jobs to 5 seconds.
• scrape_configs: Defined scrape jobs.

Create a Prometheus configuration file named prometheus.yml in the same directory as docker-compose.yml with the following:

global:
  scrape_interval: 5s
  external_labels:
    monitor: 'node-exporter'

scrape_configs:
  - job_name: 'node-exporter'
    static_configs:
      - targets: ['node-exporter:9100']

Start the prometheus and node-exporter containers using the docker-compose command. Instruct Compose to run the containers in the background with the -d flag:

$ docker compose up -d

We can get container status using docker compose ps.

Verify the status of prometheus by checking its logs. It should look something like this:

$ docker compose logs -f prometheus

Verify the status of node-exporter by checking its logs. It should look something like this:

$ docker-compose logs -f node-exporter

We can now move on to querying these metrics from Grafana.

It is recommended to install Grafana in a different machine. So in case the Raspberry Pi is having performance issues, this won't be affecting the visualization of the metrics in Grafana.

So, to install Grafana in a Linux machine, we need to create a new docker compose file with this content:

version: "3.7"
services:
  grafana:
    image: grafana/grafana
    container_name: grafana
    restart: always
    ports:
      - 3000:3000
    environment:
      GF_SECURITY_ADMIN_USER: <admin>
      GF_SECURITY_ADMIN_PASSWORD: <password>
    volumes:
      - ./grafana_data:/var/lib/grafana

In the volumes section, ./grafana_data points to the grafana_data directory in the same location as our docker-compose.yml file. This directory will be mounted to /var/lib/grafana inside the Grafana Docker container, and Grafana will store its persistent data in this directory.

We have to replace and placeholders by our own values.

If the grafana_data directory does not exist, Docker will attempt to create it. However, note that this might cause permission issues depending on the Docker setup and the permissions of your user and the directory.

If we have an error like this:

You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied

This is because the Docker daemon doesn't have the required permissions to write to the bind-mounted directory on the host system.

To fix this, we'll need to ensure proper ownership of the grafana_data directory by the same user or group ID that's used inside the container. Grafana Docker image runs as a non-root user grafana (uid: 472, gid: 472 by default). However, since we're using Docker bind mounts, the ownership and permissions of the directory on the host machine must match these.

1. First, ensure you have a grafana_data directory in the same directory as your docker-compose.yml. This directory will be used as the persistent storage for your Grafana data.
2. Change the owner of the grafana_data directory to match the UID:GID of the Grafana user in the container.

sudo chown -R 472:472 grafana_data

3. Now, start the Grafana service using `docker compose'

docker compose up -d

Once Grafana is up an running, we should open it by navigating to http://localhost:3000.

In this step, you’ll query out Prometheus metrics from Grafana.

First, we need to create a Prometheus datasource.

Then, we need to configure a dashboard to visualize metrics, for this, we will import a Grafana dashboard into our Grafana instance.

Official and community-built dashboards are listed on the Grafana website Dashboards page. Dashboards on this page will include information in the Overview tab about required configurations you may need to get the dashboard to work.

In this case, we’ll use the Node Exporter Full dashboard. Note the dashboard’s ID: 1860.

In Grafana, click on Dashboards in the left-side menu to go to the Dashboards page.

Click New and select Import in the dropdown menu. Enter the dashboard’s ID and click Load. Select the appropriate Prometheus data source, and then click on Import.

Depending on our node exporter configuration, some panels may not function correctly. To learn how to configure node exporter to emit the required metrics, please see the node_exporter GitHub repo.

• Monitoring on Raspberry Pi with Node Exporter, Prometheus, and Grafana
• Monitoring a Linux host with Prometheus, Node Exporter, and Docker Compose