Large Image containing Influxdb and Grafana

An extended version of the large image was investigated to hold influxdb and Grafana, allowing users with no or limited access to the internet to store and dashboard data.

Due to the restrictions on being 32bot, this meant that influxdb v1.0 had to be used, this has issues with index files once the database gets over 2Gb. This should not cause issues for normal day to day operation, but a large site trying to store lots of data with large history could run into issues.

The image consisted of the following components

• Venus-InfluxDB-Loader, to take data from VenusOS via MQTT and load it into the influxdb.
• InfluxDB, to store the data
• Grafana, for dashboard reporting

Build

The following branches are used to build the image.

meta-victronenergy (mbath/grafana-v3.20-12-R1)

Commits exist to add the following recipes to the build

• go, added recipes from from openembedded-core so as too have the latest available.
• rust/cargo, added recipes from openembedded-core so as too have the latest available.
• influxdb, used multiple source to create the recipes.
• grafana, used multiple source to create the recipes.
• venus-influx-loader, created a new recipe.
• venus-platform, modified the existing receipts to pull in the below branch.

meta-victronenergy-private (mbath/grafana-v3.20-8-R1)

Commits exist to add the following recipes to the build

• gui, modified the existing recipes to pull in the below branch, to add the Grafana stack to the large menu.

gui (mbath/grafana-R1)

Commit to extend the large menu to allow for starting and stopping the Grafana stack.

venus-platform (mbath/grafana)

Commit to add support for the grafana and associated services starting and stopping.

Configuration and operation

Each of the three key services, venus-influxdb-loader, influxdb and graffana can be started through the GUI. This causes /service/venus-influx-loader; /service/influxdb; and /service/grafana to be started and stopped. These can manually be started and stopped by using the svc command.

Venus-influx-loader

The configuration file for this is stored in /data/conf/venus-influx-loader. The default configuration is pulled in from the recipe and is available here. This will connect to the local MQTT server, and therefore this must be started. At this time when grafana is started from the GUI, the system does not check if it is running. A future change could be that on grafana startup, if MQTT is not running then an error should be displayed, or even MQTT started. It is configured to write to influxdb on localhost and port 8086.

The administration GUI is currently not built and deployed. There are issues with the build of this component within the openembedded build environment, so these need to be resolved. They relate to the build of the GUI element and needing to pre-build components. ADD DETAILS

If no Venus database exists, it will create one and set the retention period to 30 days. There is no configuration capability to change this, at this stage.

the service is started through /service/venus-influx-loader, which calls a startup script. This makes sure that a default config.json file exists in /data/conf/venus-influx-loader/, and if not copies one into that directory.

influxdb

The main configuration file is stored in /etc/influxdb. This is on the readonly environment, which is not an issue as there is generally no need to modify the configuration. The default configuration is populated by the build system and the file is available here.

Influxdb is started on port 8086; the RPC service has been moved from 8088 to 8087, due to some other service already using 8088 (INSERT WHICH)

Influxdb is configured to store data in /data/home/influxdb/storage/, this should be a symbolic link to some USB storage. The user will need to set this link up, some future iteration of the system should manage the external storage and define what is used for influxdb and create the symbolic link. The file system needs to be ext4, as FAT does not allow for setting file permissions, which influxdb needs to do.

Influx is started through a /service/ file and svc. The run script calls a prepare script which checks that the influxdb home directory exists.

Grafana

The main configuration area for grafana is /data/conf/grafana. The svc run script triggers two events.

• Creates the home directory if it does not already exist via the script
• Populates the standard configuration, if it does nota already exist through the script

Issues

At present the CerboGX and Ekrano images will not fit on the internal flash, this means that it is only available to run on the RPI images. The reason for this is that at present VenusOS 3.x is shipping with both QT4 and QT6 while the new interface is being built. It might be possible once QT4 is removed that there will be enough flash to handle this large image.