V1.7.7_MQTT_IMPROVEMENTS - Alteriom/painlessMesh GitHub Wiki
Version 1.7.7 introduces two new specialized packages designed to enhance MQTT communication efficiency for monitoring mesh networks. These packages provide comprehensive visibility into system performance, proactive problem detection, and predictive maintenance capabilities.
A comprehensive performance metrics package designed for real-time dashboards, capacity planning, and network optimization.
- Real-time Monitoring - Track system performance in real-time
- Capacity Planning - Identify bottlenecks and plan upgrades
- Performance Optimization - Analyze and improve system efficiency
- Troubleshooting - Diagnose performance issues quickly
CPU and Processing:
-
cpuUsage(uint8_t) - CPU usage percentage (0-100) -
loopIterations(uint32_t) - Loop iterations per second -
taskQueueSize(uint16_t) - Number of pending tasks
Memory Metrics:
-
freeHeap(uint32_t) - Current free heap memory in bytes -
minFreeHeap(uint32_t) - Minimum free heap since boot -
heapFragmentation(uint32_t) - Heap fragmentation percentage -
maxAllocHeap(uint32_t) - Largest allocatable block
Network Performance:
-
bytesReceived(uint32_t) - Total bytes received -
bytesSent(uint32_t) - Total bytes sent -
packetsReceived(uint16_t) - Total packets received -
packetsSent(uint16_t) - Total packets sent -
packetsDropped(uint16_t) - Packets dropped -
currentThroughput(uint16_t) - Current throughput in bytes/sec
Timing and Latency:
-
avgResponseTime(uint32_t) - Average response time in microseconds -
maxResponseTime(uint32_t) - Maximum response time in microseconds -
avgMeshLatency(uint16_t) - Average mesh latency in milliseconds
Connection Quality:
-
connectionQuality(uint8_t) - Overall connection quality score (0-100) -
wifiRSSI(int8_t) - WiFi RSSI in dBm
Metadata:
-
collectionTimestamp(uint32_t) - When metrics were collected -
collectionInterval(uint32_t) - Interval between collections in ms
#include "alteriom_sensor_package.hpp"
using namespace alteriom;
void collectAndSendMetrics() {
MetricsPackage metrics;
metrics.from = mesh.getNodeId();
// Collect CPU metrics
metrics.cpuUsage = calculateCPUUsage();
metrics.loopIterations = getLoopsPerSecond();
metrics.taskQueueSize = scheduler.size();
// Collect memory metrics
metrics.freeHeap = ESP.getFreeHeap();
metrics.minFreeHeap = getMinHeapSinceBoot();
#ifdef ESP32
metrics.maxAllocHeap = ESP.getMaxAllocHeap();
#else
metrics.heapFragmentation = ESP.getHeapFragmentation();
metrics.maxAllocHeap = ESP.getMaxFreeBlockSize();
#endif
// Collect network metrics
metrics.bytesReceived = getTotalBytesRx();
metrics.bytesSent = getTotalBytesTx();
metrics.currentThroughput = calculateThroughput();
// Connection quality
metrics.connectionQuality = calculateConnectionQuality();
metrics.wifiRSSI = WiFi.RSSI();
// Metadata
metrics.collectionTimestamp = mesh.getNodeTime();
metrics.collectionInterval = 30000; // 30 seconds
// Send metrics
mesh.sendBroadcast(metrics.toJsonString());
}When using with the MQTT bridge, metrics are published to:
-
mesh/metrics/{node_id}- Per-node metrics -
mesh/metrics/aggregated- Aggregated mesh-wide metrics
Grafana:
{
"cpu_usage": "${cpuUsage}",
"free_heap": "${freeHeap}",
"throughput": "${currentThroughput}",
"rssi": "${wifiRSSI}",
"quality": "${connectionQuality}"
}InfluxDB Line Protocol:
metrics,node_id=12345 cpu=${cpuUsage},heap=${freeHeap},throughput=${currentThroughput} ${timestamp}
A proactive health monitoring package designed for early problem detection, predictive maintenance, and automated alerting.
- Proactive Detection - Identify problems before they cause failures
- Predictive Maintenance - Estimate time to failure and plan maintenance
- Automated Alerting - Trigger alerts based on health thresholds
- Root Cause Analysis - Provide detailed health indicators for troubleshooting
- 2 (Healthy) - All systems operating normally
- 1 (Warning) - Some issues detected, monitoring required
- 0 (Critical) - Immediate attention required
| Flag | Value | Description |
|---|---|---|
| Low Memory | 0x0001 | Free memory below threshold |
| High CPU | 0x0002 | CPU usage above threshold |
| Connection Instability | 0x0004 | Frequent reconnections |
| High Packet Loss | 0x0008 | Packet loss above threshold |
| Network Congestion | 0x0010 | Network congestion detected |
| Low Battery | 0x0020 | Battery level critical |
| Thermal Warning | 0x0040 | Temperature above safe limit |
| Mesh Partition | 0x0080 | Mesh network partitioned |
| OTA in Progress | 0x0100 | Firmware update active |
| Configuration Error | 0x0200 | Configuration issue detected |
Each component (memory, network, performance) has a health score:
- 90-100 - Excellent
- 70-89 - Good
- 50-69 - Fair
- 30-49 - Poor
- 0-29 - Critical
Overall Health:
-
healthStatus(uint8_t) - Overall status (0-2) -
problemFlags(uint16_t) - Bit flags for specific problems
Component Health:
-
memoryHealth(uint8_t) - Memory health score (0-100) -
memoryTrend(uint32_t) - Memory loss rate in bytes/hour -
networkHealth(uint8_t) - Network health score (0-100) -
packetLossPercent(uint8_t) - Current packet loss percentage -
reconnectionCount(uint8_t) - Reconnections in last hour -
performanceHealth(uint8_t) - Performance health score (0-100) -
missedDeadlines(uint32_t) - Missed task deadlines -
maxLoopTime(uint16_t) - Maximum loop execution time in ms
Environmental:
-
temperature(int8_t) - Device temperature in Celsius -
temperatureHealth(uint8_t) - Temperature health score
Stability:
-
uptime(uint32_t) - Uptime in seconds -
crashCount(uint16_t) - Crash/restart count -
lastRebootReason(uint32_t) - Last reboot reason code
Predictive:
-
estimatedTimeToFailure(uint16_t) - Estimated hours until failure (0=unknown) -
recommendations(TSTRING) - Recommended actions
void performHealthCheck() {
HealthCheckPackage health;
health.from = mesh.getNodeId();
// Calculate component health scores
uint8_t memHealth = calculateMemoryHealth();
uint8_t netHealth = calculateNetworkHealth();
uint8_t perfHealth = calculatePerformanceHealth();
// Determine overall health status
if (memHealth < 30 || netHealth < 30 || perfHealth < 30) {
health.healthStatus = 0; // Critical
} else if (memHealth < 60 || netHealth < 60 || perfHealth < 60) {
health.healthStatus = 1; // Warning
} else {
health.healthStatus = 2; // Healthy
}
// Set problem flags
health.problemFlags = 0;
if (memHealth < 60) health.problemFlags |= 0x0001;
if (perfHealth < 60) health.problemFlags |= 0x0002;
if (netHealth < 60) health.problemFlags |= 0x0004;
// Component health
health.memoryHealth = memHealth;
health.memoryTrend = calculateMemoryTrend();
health.networkHealth = netHealth;
health.packetLossPercent = getPacketLossPercent();
health.performanceHealth = perfHealth;
// Predictive indicators
if (health.memoryTrend > 0) {
uint32_t freeHeap = ESP.getFreeHeap();
health.estimatedTimeToFailure = freeHeap / health.memoryTrend;
}
// Recommendations
if (health.healthStatus == 0) {
health.recommendations = "CRITICAL: Immediate attention required";
} else if (health.healthStatus == 1) {
if (memHealth < 60) {
health.recommendations = "Increase memory allocation";
} else if (netHealth < 60) {
health.recommendations = "Check network connections";
}
} else {
health.recommendations = "System operating normally";
}
// Send health check
mesh.sendBroadcast(health.toJsonString());
}Home Assistant:
- platform: mqtt
name: "Mesh Node Health"
state_topic: "mesh/health/12345"
value_template: "{{ value_json.healthStatus }}"
json_attributes_topic: "mesh/health/12345"
json_attributes_template: "{{ value_json | tojson }}"Alerting Rules:
alerts:
- name: critical_health
condition: healthStatus == 0
action: send_notification
- name: memory_warning
condition: memoryHealth < 60
action: log_warning
- name: predicted_failure
condition: estimatedTimeToFailure < 24
action: schedule_maintenance| Package Type | Interval | Use Case |
|---|---|---|
| MetricsPackage | 30-60s | Normal monitoring |
| MetricsPackage | 10-30s | Active troubleshooting |
| MetricsPackage | 5-10s | Critical performance analysis |
| HealthCheckPackage | 60s | Normal health monitoring |
| HealthCheckPackage | 30s | Warning state monitoring |
| HealthCheckPackage | 10s | Critical state monitoring |
// In setup()
#define METRICS_INTERVAL 30000 // 30 seconds
#define HEALTH_INTERVAL 60000 // 60 seconds
Task taskMetrics(METRICS_INTERVAL, TASK_FOREVER, &sendMetrics);
Task taskHealth(HEALTH_INTERVAL, TASK_FOREVER, &sendHealthCheck);
userScheduler.addTask(taskMetrics);
userScheduler.addTask(taskHealth);
taskMetrics.enable();
taskHealth.enable();
// Dynamic adjustment based on health status
void adjustCollectionIntervals(uint8_t healthStatus) {
if (healthStatus == 0) { // Critical
taskMetrics.setInterval(10000); // 10 seconds
taskHealth.setInterval(10000);
} else if (healthStatus == 1) { // Warning
taskMetrics.setInterval(20000); // 20 seconds
taskHealth.setInterval(30000);
} else { // Healthy
taskMetrics.setInterval(60000); // 60 seconds
taskHealth.setInterval(60000);
}
}-
mesh/metrics/{node_id}- Individual node metrics -
mesh/health/{node_id}- Individual node health checks
-
mesh/metrics/aggregated- Mesh-wide aggregated metrics -
mesh/health/summary- Overall mesh health summary -
mesh/alerts- Active alerts across the mesh
Packages now include the message_type field for 90% faster message classification:
{
"schema_version": 1,
"device_id": "ALT-12345",
"timestamp": "2025-10-23T21:30:00Z",
"type": 204,
"message_type": 204,
"from": 12345,
"cpuUsage": 45,
"freeHeap": 100000,
"currentThroughput": 8192,
"connectionQuality": 85,
"wifiRSSI": -55,
...
}Message Type Codes (mqtt-schema v0.7.2+):
For performance optimization and standardized routing, all Alteriom packages include the message_type field:
| Code | Constant | Message Type | Category | Description | PainlessMesh Package |
|---|---|---|---|---|---|
| 200 | SENSOR_DATA | sensor_data | telemetry | Sensor telemetry readings | SensorPackage |
| 202 | SENSOR_STATUS | sensor_status | telemetry | Sensor status change | StatusPackage |
| 204 | SENSOR_METRICS | sensor_metrics | telemetry | Sensor health and performance metrics | MetricsPackage |
| 400 | COMMAND | command | control | Device control command | CommandPackage |
| 600 | MESH_NODE_LIST | mesh_node_list | mesh | Mesh node inventory | MeshNodeListPackage |
| 601 | MESH_TOPOLOGY | mesh_topology | mesh | Mesh network topology | MeshTopologyPackage |
| 602 | MESH_ALERT | mesh_alert | mesh | Mesh network alert | MeshAlertPackage |
| 603 | MESH_BRIDGE | mesh_bridge | mesh | Mesh protocol bridge | MeshBridgePackage |
| 604 | MESH_STATUS | mesh_status | mesh | Mesh network health status | EnhancedStatusPackage |
| 605 | MESH_METRICS | mesh_metrics | mesh | Mesh network performance metrics | HealthCheckPackage |
Key Points:
- The
message_typefield enables 90% faster message classification compared to parsing JSON - All Alteriom packages align with @alteriom/mqtt-schema v0.7.2+ standards
- Message type codes are consistent across MQTT bridge, gateway, and mesh nodes
See examples/alteriom/metrics_health_node.ino for a complete working example.
#include "alteriom_sensor_package.hpp"
using namespace alteriom;
// Add to your existing mesh node
void setup() {
// ... existing mesh setup ...
// Add metrics collection task
userScheduler.addTask(Task(30000, TASK_FOREVER, []() {
MetricsPackage metrics;
// ... populate metrics ...
mesh.sendBroadcast(metrics.toJsonString());
}));
// Add health check task
userScheduler.addTask(Task(60000, TASK_FOREVER, []() {
HealthCheckPackage health;
// ... populate health data ...
mesh.sendBroadcast(health.toJsonString());
}));
}// In MQTT bridge receivedCallback
void receivedCallback(uint32_t from, String& msg) {
DynamicJsonDocument doc(2048);
deserializeJson(doc, msg);
uint8_t msgType = doc["type"];
switch(msgType) {
case 204: // MetricsPackage
publishMetricsToMQTT(from, doc);
break;
case 605: // HealthCheckPackage (MESH_METRICS)
publishHealthToMQTT(from, doc);
checkForAlerts(doc);
break;
}
}
void publishMetricsToMQTT(uint32_t from, JsonDocument& doc) {
String topic = "mesh/metrics/" + String(from);
String payload;
serializeJson(doc, payload);
mqttClient.publish(topic.c_str(), payload.c_str());
}
void checkForAlerts(JsonDocument& doc) {
uint8_t healthStatus = doc["health"];
uint16_t problemFlags = doc["problems"];
if (healthStatus == 0) {
String alertTopic = "mesh/alerts/critical";
String alertMsg = "Critical health on node " + String(doc["from"].as<uint32_t>());
mqttClient.publish(alertTopic.c_str(), alertMsg.c_str());
}
}- MetricsPackage: ~200 bytes per message
- HealthCheckPackage: ~250 bytes per message (including recommendations string)
- Total overhead: <1KB for both packages with reasonable collection intervals
With 10 nodes and recommended intervals:
- Metrics (30s): 10 nodes × 200 bytes / 30s = ~67 bytes/sec
- Health (60s): 10 nodes × 250 bytes / 60s = ~42 bytes/sec
- Total: ~109 bytes/sec = minimal overhead
- Adjust intervals based on load - Increase intervals during normal operation
- Use selective reporting - Only send metrics that changed significantly
- Aggregate at gateway - Combine multiple node metrics before MQTT publish
- Compress recommendations - Use short, standardized recommendation codes
- Throttle during congestion - Reduce collection frequency when network is busy
Run the comprehensive test suite:
./bin/catch_metrics_health_packagesTests validate:
- Serialization/deserialization
- Field preservation
- Edge cases (min/max values)
- Problem flag handling
- Health status levels
- Integration with painlessMesh plugin system
// Test metrics collection
void testMetrics() {
MetricsPackage metrics;
metrics.from = 12345;
metrics.cpuUsage = 45;
metrics.freeHeap = 100000;
auto var = protocol::Variant(&metrics);
auto metrics2 = var.to<MetricsPackage>();
assert(metrics2.cpuUsage == 45);
assert(metrics2.freeHeap == 100000);
}
// Test health monitoring
void testHealth() {
HealthCheckPackage health;
health.from = 12345;
health.healthStatus = 1;
health.problemFlags = 0x0001;
auto var = protocol::Variant(&health);
auto health2 = var.to<HealthCheckPackage>();
assert(health2.healthStatus == 1);
assert(health2.problemFlags == 0x0001);
}No breaking changes. Simply add the new packages to your code:
// Add new includes
#include "alteriom_sensor_package.hpp"
using namespace alteriom;
// Add new collection tasks
// ... see examples above ...- All existing packages (200-203) continue to work unchanged
- New packages (204, 604, 605) are optional additions
- No changes required to existing code
- Can be adopted incrementally
- Start with conservative intervals - Begin with 60s intervals and adjust based on needs
- Monitor memory usage - Watch heap fragmentation when enabling new packages
- Implement health-based throttling - Reduce collection frequency when unhealthy
- Use problem flags effectively - Check specific flags rather than just status
- Act on recommendations - Implement automated responses to common recommendations
- Set up alerting - Configure alerts for critical health status
- Track trends - Use memoryTrend to detect slow memory leaks
- Test thoroughly - Validate metrics accuracy in your specific environment
- Increase collection intervals
- Reduce number of active packages
- Use shorter recommendation strings
- Check for memory leaks using memoryTrend
- Increase collection intervals
- Use SINGLE routing instead of BROADCAST for some packages
- Implement throttling based on network health
- Reduce message size by removing optional fields
- Calibrate CPU usage calculation for your use case
- Validate timing measurements
- Check clock synchronization across mesh
- Verify RSSI readings match reality
Planned for v1.8.0:
- Compressed metric packages for large meshes
- Historical trend storage in gateway
- Automatic threshold tuning
- Machine learning-based failure prediction
- Integration with cloud monitoring services
The enhanced MQTT bridge (examples/bridge/enhanced_mqtt_bridge.hpp) extends the basic MQTT status bridge with command handlers and aggregation capabilities.
- Command Handlers - Request metrics and health checks on-demand
- Aggregated Statistics - Mesh-wide metrics and health summaries
- Automatic Caching - Stores recent metrics for aggregation
- Alert Detection - Automatic critical health alerting
- Response Topics - Dedicated topics for command responses
| Topic | Payload | Description |
|---|---|---|
mesh/command/request_metrics |
{"node_id": 0} |
Request metrics (0=all nodes) |
mesh/command/request_health |
{"node_id": 12345} |
Request health check |
mesh/command/get_aggregated |
{} |
Get current aggregated stats |
| Topic | Description | Update Frequency |
|---|---|---|
mesh/metrics/{node_id} |
Individual node metrics | When received from node |
mesh/health/{node_id} |
Individual node health | When received from node |
mesh/aggregated/metrics |
Mesh-wide metrics | Configurable (default 60s) |
mesh/aggregated/health |
Mesh health summary | Configurable (default 60s) |
mesh/alerts/critical |
Critical health alerts | When critical detected |
mesh/response/metrics |
Metrics command response | When node responds |
mesh/response/health |
Health command response | When node responds |
{
"node_count": 10,
"avg_cpu": 42,
"avg_heap": 125000,
"min_heap": 95000,
"max_heap": 150000,
"total_throughput": 81920,
"avg_quality": 87,
"min_quality": 65,
"max_quality": 98,
"avg_rssi": -58,
"timestamp": "2025-10-23T21:30:00Z"
}{
"node_count": 10,
"mesh_health": 2,
"healthy_nodes": 8,
"warning_nodes": 2,
"critical_nodes": 0,
"aggregated_problems": 5,
"avg_memory_health": 82,
"avg_network_health": 88,
"avg_performance_health": 90,
"timestamp": "2025-10-23T21:30:00Z"
}mosquitto_pub -h localhost -t mesh/command/request_metrics \
-m '{"node_id": 0}'mosquitto_pub -h localhost -t mesh/command/request_health \
-m '{"node_id": 12345}'mosquitto_pub -h localhost -t mesh/command/get_aggregated -m '{}'mosquitto_sub -h localhost -t mesh/aggregated/metricsmosquitto_sub -h localhost -t mesh/alerts/critical#include "enhanced_mqtt_bridge.hpp"
// In setup()
EnhancedMqttBridge bridge(mesh, mqttClient);
bridge.setTopicPrefix("mesh/");
bridge.enableAggregation(true);
bridge.setAggregationInterval(60000); // 60 seconds
bridge.begin();
// In loop()
bridge.update();
// In MQTT callback
void mqttCallback(char* topic, byte* payload, unsigned int length) {
String topicStr = String(topic);
String payloadStr = String((char*)payload).substring(0, length);
bridge.handleMQTTMessage(topicStr, payloadStr);
}// Set custom topic prefix
bridge.setTopicPrefix("alteriom/mesh/");
// Set device ID
bridge.setDeviceId("GATEWAY-001");
// Enable/disable aggregation
bridge.enableAggregation(true);
// Set aggregation interval
bridge.setAggregationInterval(30000); // 30 secondsNodes must respond to command requests:
void receivedCallback(uint32_t from, String& msg) {
DynamicJsonDocument doc(1024);
deserializeJson(doc, msg);
uint8_t msgType = doc["type"];
if (msgType == 201) { // CommandPackage
uint8_t command = doc["cmd"];
if (command == 210) { // Request metrics
sendMetrics();
} else if (command == 211) { // Request health
sendHealthCheck();
}
}
}Memory Usage:
- Cache size: ~100 bytes per node (max 20 nodes)
- Total overhead: ~2KB for bridge + caching
Network Bandwidth:
- Aggregated metrics: ~200 bytes every 60s
- Per-node metrics: ~200 bytes when received
- Critical alerts: ~150 bytes when detected
Optimization:
- Adjust cache size via
MAX_STORED_NODESconstant - Increase aggregation interval for lower bandwidth
- Disable aggregation if not needed
- GitHub Issues: https://github.com/Alteriom/painlessMesh/issues
- Documentation: https://alteriom.github.io/painlessMesh/
-
Examples:
examples/alteriom/metrics_health_node.ino,examples/bridge/enhanced_mqtt_bridge_example.ino
Added:
- MetricsPackage (Type 204) for comprehensive performance monitoring
- HealthCheckPackage (Type 605) for proactive health monitoring (MESH_METRICS)
- Complete test suite for new packages
- Example implementation in
metrics_health_node.ino - Documentation for MQTT integration
- Dashboard integration examples
Improved:
- MQTT communication efficiency for metrics and health monitoring
- Problem detection and alerting capabilities
- Predictive maintenance support
- Memory leak detection
Compatibility:
- 100% backward compatible with v1.7.6
- All existing packages (200-203) unchanged
- Optional adoption of new features