InfluxDB VS Prometheus - kdwivedi1985/system-design GitHub Wiki
InfluxDB:
- InfluxDB is general purpose time-series database handle high write loads with flexible data structure. It is used for IoT applications, data logging, and real-time analytics.
- InfluxDB 1.x and 2.x are open-source but some features like clustering etc, are not available.
- InfluxDB Cloud/Enterprise are proprietary. Designed for scale, high availability, and clustering.
Prometheus:
- Prometheus is open-source time-series database, specially used for real-time monitor systems and applications, especially in dynamic environments like microservices and Kubernetes.
- It can be configured to send alerts like- High CPU, Memory Issue, Kublet down, Container CPU Throttling, High Network Latency etc.
How does Prometheus Stores Data?
- Prometheus stores time-series in Disc identified as Metric Name + Label(Key-Value).
- e.g. http_requests_total{method="GET", handler="/api", status="200"}
- It uses WAL(write ahead log) + block storage for durability.
- First data is written in WAL for safety.
- Every 2 hours durability.
- Time-Series engine manages block, index and queries.
Clustering with Prometheus
- Prometheus is single node system by design. Thanos is an open-source project that extends Prometheus to provide:
- High-availability, Horizontal Scaling, Deduplication of HA data, Long-term storage (S3 etc.), Global view of multiple prometheus instances.
- Thanos cluster is not like traditional tightly couple cluster like DB but it is loosely coupled set of components that works together to extend prometheus.
- Prometheus instances run normally while Thanos is deployed as side car.
- Thanos act as query engine and storage gateway.
- Thanos instances can be scalled horizontally (Multiple sidecars, store gateways, query nodes, etc.) which makes it like a cluster.
- There can be multiple Prometheus instances connected to Thanos through side car.
- We may have a regional cluster like below:
- prometheus-us-east + sidecar-us-east
- prometheus-eu-west + sidecar-eu-west
- prometheus-k8s + sidecar-k8s
Compare InfluxDB and Prometheus
| Feature | Prometheus | InfluxDB |
|---|---|---|
| Data Collection | Pull-based (pull from clients in few seconds- http://my-app:8080/metrics) | Push-based |
| Data Model | Metric name + labels | Measurement, tags, fields |
| Query Language | PromQL | InfluxQL, Flux |
| Data Types | float64 (main), limited string | float, int, bool, string |
| Timestamp Precision | Millisecond | Nanosecond |
| Clustering | No native (open source) | Enterprise only |
| Alerting | Built-in (Alertmanager) | External (Kapacitor) |
| Visualization | Grafana, basic UI | Grafana, Chronograf, InfluxDB UI |
| Best For | Monitoring, alerting, cloud-native infra | Historical data, IoT, event logging |
- Many organizations use both Prometheus and InfluxDB, leveraging Prometheus for real-time monitoring and InfluxDB for historical data analysis and long-term storage.