Cloud Integrations Expressions - SolarNetwork/solarnetwork GitHub Wiki

Cloud Datum Stream Mapping Property entities can be configured as a dynamic expression that can perform a calculation on the acquired cloud data values.

Expressions are evaluated after all datum for a given Cloud Datum Stream request have been resolved. Each expression configured on the Cloud Datum Stream Mapping is evaluated in the order defined, against all resolved datum. Thus the expression will be evaulated against each datum's resolved node (or location) and source ID values and all property values as resolved by the Cloud Datum Stream Mapping's non-expression references.

For example, imagine a Cloud Datum Stream acquires a datum with properties like this:

{
    "created": "2024-11-13 04:20:00Z",
    "nodeId": 123,
    "sourceId": "test/meter",
    "i": {
        "voltage_a": 287.652,
        "voltage_b": 287.652,
        "voltage_c": 287.318
    }
}

Here the meter has provided 3-phase voltage properties, but you would like the overall voltage as well, on a voltage property. You can use an expression property to do that, like this:

{
    "enabled": true,
    "propertyType": "i",
    "propertyName": "voltage",
    "valueType": "s",
    "valueReference": "round(rms({voltage_a, voltage_b, voltage_c}), 1)"
}

The valueType value s means Spel Expression and the valueReference value is the actual expression. In this example, a rms() function is passed the three phase voltage values to calculate the root-mean-square value, which is then passed to round() to round the result to at most one decimal place.

After the expression is evaulated, the resulting datum then looks like this:

{
    "created": "2024-11-13 04:20:00Z",
    "nodeId": 123,
    "sourceId": "test/meter",
    "i": {
        "voltage_a": 287.652,
        "voltage_b": 287.652,
        "voltage_c": 287.318,
        "voltage":   287.5
    }
}

Restricting an expression to a specific datum stream

As each expression Cloud Datum Stream Mapping Property is evaulated against all resolved datum from each Cloud Datum Stream request, for integrations that resolve multiple datum streams at once you might want to restrict the expression to apply only to one specific stream (or set of streams). You can do that in the expression by making use of the sourceId property that is available to the expression.

For example, imagine the integration produces several INV/* streams like INV/1, INV/2, and so on for a set of solar inverters, along with a GEN/1 stream for an energy meter that tracks the overall energy production of all meters. You'd like to include a sum total of the inverter power output on the meter stream as a invTotPower property to compare with the value recorded by the energy meter, which could be done with an expression like this:

sourceId == "GEN/1"
? sum(latestMatching('INV/*').![watts])
: null

Spel Syntax

"Spel" stands for Spring Expression Language. See the Spring Expression Language Syntax guide for more details.

The Cloud Datum Stream expression support is very similar to SolarNode Datum Expressions. Expressions can refer to datum properties directly, and many helper functions are provided.

ExpressionRoot object

Some functions return ExpressionRoot objects. An ExpressionRoot object supports all the properties and functions described in this guide.

Properties

The following properties are available:

Property Type Description
datum Datum A Datum object, in case you need direct access to the functions provided there.
node NodeInfo A NodeInfo object, for node datum streams only. This can be used to obtain the node's time zone, for example.
locId Number For location datum streams, the location ID of the datum.
nodeId Number For node datum streams, the node ID of the datum.
props Map<String,Object> Simple map based access to all properties in datum. As datum properties are also available directly as variables, this is rarely needed but can be helpful for accessing dynamically-calculated property names or properties with names that are not legal variable names.
sourceId String The source ID of the datum stream.
timestamp Instant The date of the datum.

NodeInfo object

The NodeInfo object has the following properties:

Property Type Description
nodeId long A node ID.
userId long The ID of an account owner.
country String The ISO 3166-1 alpha 2-character country code associated with the node, for example NZ.
zone ZoneId The time zone associated with the node. Useful with date time functions, for example now(node.zone) will return the current node-local time.

Functions

The following sections detail the functions that are available to Cloud Datum Stream expressions.

Bit functions

The same bitwise integer manipulation functions provided in SolarNode are available.

Collection functions

Function Arguments Result Description
union(map1, map2...) Map, Map... Map Combine the properties of two or more maps into a new map. Later map argument values will replace earlier map argument values.

Date time functions

The same date time functions provided in SolarNode are available.

Datum stream functions

Often Cloud Datum Streams acquire multiple datum at once, for example multiple datum for a single datum stream over time, or multiple datum streams, or both. The following functions make it possible to perform calculations that draw from values in any of the available datum streams.

For example, imagine a Cloud Datum Stream that collects two datum streams, one from a weather station and another from a PV inverter. You could configure an expression on the PV inverter stream that uses data from the weather stream, perhaps a calculation of the expected PV generation based on the weather conditions. Here's a contrived example that multiplies a "cloudiness" percentage collected by the weather station by a pre-computed expected output rate provided by a tariff schedule in the node's metadata:

latest('weather').cloudiness
* resolveTariffScheduleRate(nodeMetadata(), '/pm/modelled-output', now(node.zone))

For another example, imagine a Cloud Datum Stream that collects many PV inverter datum streams, with source IDs like inv/1, inv/2, and so on. You could use an expression to sum up the total combined output of all inverter watts properties like this:

sum(latestMatching('inv/*', timestamp).![watts])

💡 The timestamp property is a useful value to pass for the date argument in these functions; the effect is to find datum from other streams that are near (in time) to the current stream. For example latestMatching('inv/*', timestamp) will find all inv/X datum nearest in time to the datum currently being evaulated.

👇 Note that the date argument is optional in these functions. When omitted the functions use the latest available data as the time reference point from which offsets are calculated. When provided the given date becomes the time reference point.

👇 The pattern argument is a wildcard-style pattern.

Datum stream test functions

The following functions test the availability of datum matching certain criteria. Generally they are each closely associated with a datum stream query function.

Function Arguments Result Description
hasLatest(source, date) String, [Instant] boolean Returns true if latest(source, date) returns a value.
hasLatestMatching(pattern, date) String, [Instant] boolean Returns true if latestMatching(pattern, date) returns a non-empty result.
hasOffset(offset, date) int, [Instant] boolean Returns true if offset(offset) returns a value.
hasOffset(source, offset, date) String, int, [Instant] boolean Returns true if offset(source,offset) returns a value.
hasOffsetMatching(pattern, offset, date) String, int, [Instant] boolean Returns true if offsetMatching(pattern, offset) returns a non-empty result.

Datum stream query functions

Function Arguments Result Description
latest(source, date) String, [Instant] ExpressionRoot Provides access to the latest available datum matching the given source ID, or null if not available. This is a shortcut for calling offset(source,0,date).
latestMatching(pattern, date) String, [Instant] ExpressionRoot[] Return a collection of the latest available datum matching a given source ID wildcard pattern. This is a shortcut for calling offsetMatching(pattern,0,date).
offset(offset, date) int, [Instant] ExpressionRoot Provides access to a datum from the same stream as the current datum, offset by offset in time, or null if not available. Offset 1 means the next earlier datum with the same source ID as this datum, and so on.
offset(source, offset, date) String, int, [Instant] ExpressionRoot Provides access to an offset from the latest available datum matching the given source ID, or null if not available. Offset 0 represents the "latest" datum, 1 the one before that, and so on.
offsetMatching(pattern, offset, date) String, int, [Instant] ExpressionRoot[] Return a collection of datum matching a given source ID wildcard pattern, each offset by offset in time. Offset 0 represents the "latest" datum, 1 the next earlier, and so on.

Datum stream function limitations

  • The *Matching() functions only match source IDs of datum in the current request. Functions like hasOffsetMatching() and offsetMatching() fall into this category. That means they will not find datum previously stored in SolarNetwork for source IDs not also present in the current request.

  • The hasOffset(), offset(), hasLatest(), and latest() functions have access to datum previously stored in SolarNetwork, including sources not present in the current request. That means you can reference datum from past requests or even datum collected by other Cloud Integrations or SolarNode devices.

  • The offset functions, when querying datum previously stored in SolarNetwork, will search backwards in time at most 90 days for a previous datum, and will include at most 100 datum. That means the maximum offset you can query is 100 from the oldest datum in the current request.

  • All functions are restricted to the node or location ID configured on the Cloud Datum Stream configuration.

Datum stream function examples

Virtual meter

Here is an expression that derives an accumulating irradianceHours property on a datum stream that has an irradiance property, similar to what the SolarNode Virtual Meter filter does. It calculates the average irradiance value between the current and previous datum, multiplied by the fractional number of hours between the two, and adds that to the previous datum's calculated value. The result is that irradianceHours becomes an ever-increasing "irradiance energy meter" property.

hasOffset(1, timestamp) && offset(1, timestamp).props['irradianceHours'] != null
? offset(1, timestamp).irradianceHours + round(
	(secondsBetween(offset(1, timestamp).timestamp, timestamp) / 3600.0)
	  * avg({offset(1, timestamp).irradiance, irradiance})
  )
: 0

Metadata functions

Both datum stream and node metadata for the datum the expression is evaulating can be accessed with the following functions:

Function Arguments Result Description
metadata() MetadataOperations Returns a MetadataOperations object for the metadata associated with the datum's stream.
metadata(path) String Object Extract the value at a metadata key path on the metadata associated with the datum's stream.
nodeMetadata() MetadataOperations Returns a MetadataOperations object for the metadata associated with the datum's node ID. For location datum this returns null.
nodeMetadata(path) String Object Extract the value at a metadata key path on the metadata associated with the datum's node ID. For location datum this returns null.
tariffSchedule(meta, path) MetadataOperations, String TariffSchedule Extracts a TariffSchedule object from metadata at a given path.
resolveTariffScheduleRate(meta, path, date, rateName) MetadataOperations, String, LocalDateTime, String Object Resolve a tariff schedule rate from metadata at a given path, date, and tariff rate name. The date and rateName arguments are optional. If no date is given then the current time in the UTC time zone will be used. If no rateName is given then the first-available resolved rate will be returned.

Metadata operations

A MetadataOperations object provides the same properties and functions as SolarNode Datum metadata.

Tariff schedules

A TariffSchedule object represents a set of time-based criteria with associated tariffs. A tariff in this scheme is nothing more than a set of one or more number values, each with an associated name. See the SolarNode TariffSchedule object functions guide for details on the properties and functions that are avaialble on a TariffSchedule object.

HTTP functions

Function Arguments Result Description
httpBasic(user, pass) String, String String Generate an HTTP Authorization header value for the Basic authentication scheme.
httpGet(uri, params, headers) String, [Map, Map] Result<Map> Make a HTTP GET request for JSON content. Both params and headers are optional. Returns a Result object with a Map value for the data property.

Result object

A Result<T> object is the General Response as returned by the SolarNetwork API.

For example, the Result<Map> returned from httpGet() might look like this:

{
	"success": true,
	"data": {
		"answer": "thing"
	}
}

In an expression, you could refer to answer like this:

httpGet('http://example.com/thing')?.data?.answer

HTTP function example

Here is an expression example that makes use of node metadata to provide query parameters for a web service that can calculate POA irradiance from a GHI irradiance value. It includes Basic authentication as well:

has('irradiance') ? httpGet(
	'https://example.com/ghi-to-poa',
	union(nodeMetadata('/pm/pv-characteristics'), {
		date: local(timestamp.atZone(nodeMetadata('/pm/pv-characteristics/zone'))),
		irradiance: irradiance
	}), {
		'Authorization': httpBasic('foo', 'bar')
	}
)?.data?.poa_global : null
  1. First has('irradiance') checks for the existance of an irradiance property.
  2. Then httpGet() is called.
    • Query parameter are provided via a union() function that combines a node metadata object at /pm/pv-characteristics with a literal map containing the datum's timestamp (converted to the time zone specified in node metadata at /pm/pv-characteristics/zone) and the datum's irradiance property.
    • Headers are provided as a literal map with Authorization set to HTTP Basic authorization value generated by the httpBasic() function.
  3. The HTTP result value poa_global is then returned (the ?.data?.poa_global snippet).

Math functions

The same math functions provided in SolarNode are available.

Property functions

The same property functions provided in SolarNode are available.

User secret functions

The following functions provide access to user secret values. The secret topicId must be c2c/i9n, which means you must configure a key pair with that value as the key pair key (not to be confused by the key arguments below, which refer to a secret key).

Function Arguments Result Description
secret(key) String String Decrypt a user secret with key key as a string.
secretData(key) String byte[] Decrypt a user secret with key key as bytes.
⚠️ **GitHub.com Fallback** ⚠️