Monitoring a Single Kubernetes Cluster - datnguyendv/monitoring_tools GitHub Wiki
This tutorial demonstrates how to monitor a single Kubernetes cluster using Prometheus (via kube-prometheus-stack
) with custom exporters for both node metrics and MongoDB metrics.
For this example, we’ll simulate the setup in Minikube, and assume the cluster name is singapore. It will be the name of helm.
For reference architecture, see this diagram.
Prepare Namespaces
We will create two namespaces:
kubectl create namespace monitoring
kubectl create namespace mongodb
Install Prometheus with Helm
We’ll use the official kube-prometheus-stack Helm chart to deploy Prometheus and related components into the monitoring namespace.
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install singapore prometheus-community/kube-prometheus-stack -f scrape-prometheus/helm/kps-scls.yaml -n monitoring
After installation, Prometheus will automatically collect node-level metrics and Kubernetes control plane metrics.
Access Prometheus and Verify Setup
Port-forward Prometheus UI to your localhost:
kubectl port-forward -n monitoring svc/singapore-kube-prometheus-prometheus 9090
Then open http://localhost:9090/ in your browser.
To verify that node metrics are being collected, try running this query:
node_cpu_seconds_total
You should see time series data from all nodes in your Minikube cluster.
Deploy Grafana visualization
helm install singapore-graf grafana/grafana -f visualization-grafana/helm/grafana.yaml -n monitoring
Access Grafana and Verify Setup
Port-forward Prometheus UI to your localhost:
kubectl port-forward -n monitoring svc/singapore-graf-grafana 9090
Then open http://localhost:9090/ in your browser.
Add prometheus
as a datasource.
To verify that node metrics are being collected, try running this query:
node_cpu_seconds_total
Deploy MongoDB Example
Let’s now deploy MongoDB into the mongodb
namespace:
kubectl -n mongodb create deployment mongodb --image=mongodb/mongodb-community-server:latest
kubectl -n mongodb expose deployment mongodb --port=27017 --target-port=27017
Ensure that the MongoDB pod and service are running correctly:
kubectl get pods -n mongodb
kubectl get svc -n mongodb
Create mongodb user for mongdb-exporter
service:
db.createUser({
user: "exporter",
pwd: "exporterpass",
roles: [
{ role: "clusterMonitor", db: "admin" }, // required for serverStatus
{ role: "readAnyDatabase", db: "admin" }
]
})
Add MongoDB Exporter
Install a MongoDB exporter (e.g., [bitnami/mongodb-exporter](https://github.com/bitnami/bitnami-docker-mongodb-exporter)
) in the same namespace.
Or using a manifest-based setup, deploy it like this:
kubectl -n mongodb apply -f metrics-exporter/mongodb/k8s/deployment.yaml
kubectl -n mongodb apply -f metrics-exporter/mongodb/k8s/service.yaml
Make sure the MongoDB exporter is pointing to your MongoDB instance via environment variables or connection string.
Add a ServiceMonitor for MongoDB Exporter
ServiceMonitor is custom CRD of prometheus operator
. When apply ServiceMonitor, prometheus will scrape resource automaticaly. Ensure namespaceSelector.matchNames
and metadata.labels.release
includes:
...
metadata:
labels:
release: singapore <--- name of release of prometheus
...
namespaceSelector:
matchNames:
- mongodb <--- it must be namespace of exporter deployment
...
Apply ServiceMonitor:
kubectl -n monitoring apply -f scrape-prometheus/helm/service_monitor/mongo-exporter.yaml
Verify MongoDB Metrics in Prometheus
Wait a few seconds, then go back to Prometheus UI and try this query:
mongodb_up
Or explore metrics such as:
mongodb_ss_mem_resident_bytes
mongodb_mongod_op_counters_total
If data appears, your setup is working!
End of guide.