Understanding the to PI flow - langanjp/PI-NodeRED GitHub Wiki
Using PI Web API
The PI Web API requires the use of a WebId for most calls. The WebId can represent one or multiple tags and/or attributes. This flow takes information about the element, parent element and template. It checks to see if there is an existing WebId for the element in memory; if there is, it separates the message into individual attributes and sends it to write to PI. If there is no WebId, it checks to see if the element WebId exists. If it does, it stores the element's WebId into a global variable and goes to write to pi; if not, it finds the parent webid, and with the referenced AF element template, and automatically creates the new AF element and associated attributes. It is then able to get a WebId and write the data to PI. Ideally, this flow should be improved to include error handling and some sort of caching / buffering for communication errors
This flow take the following two messages and uses the information to send the data to the PI System
Example of a JavaScript objects key: value pairs that will be mapped to PI AF Attributes:
{Current Frequency: "60.019",
Instantaneous Time Error: "-13.126",
Actual System Demand: "32783",
Total System Capacity (not including Ancillary Services): "35099",
Total Wind Output: "4321",
DC_E (East): "86",
DC_L (Laredo VFT): "0",
DC_N (North): "0",
DC_R (Railroad): "0",
DC_S (Eagle Pass): "1"}
Example OSIsoft PI Information needed (including timestamp):
{elementname: "realtime",
elementdescription: "Real-Time System Conditions",
sourcesys: "externalservices",
template: "ercot_com_realtime",
auth: "Basic base64username:password",
url: "https://<piwebapiserver>/piwebapi",
pathelementparent: "\\PIAFSERVER\AFDB\externalservices\ercot_com",
pathelement: "\\PIAFSERVER\AFDB\externalservices\ercot_com\realtime",
timestamp: "2017-01-21T14:53:20.000Z"}
Example mapping (this maps from the source message to the AF element and attribute)
ercot_com_realtime = {
"Current Frequency": {attribute: "frequency"},
"Instantaneous Time Error": {attribute: "timeerror"},
"Actual System Demand": {attribute: "demand"},
"Total System Capacity (not including Ancillary Services)": {attribute: "capacity"},
"Total Wind Output": {attribute: "windoutput"},
"DC_E (East)": {attribute: "DC_E(East)"},
"DC_L (Laredo VFT)": {attribute: "DC_L(LaredoVFT)"},
"DC_N (North)": {attribute: "DC_N(North)"},
"DC_R (Railroad)": {attribute: "DC_R(Railroad)"},
"DC_S (Eagle Pass)": {attribute: "DC_S(EaglePass)"},
};
The corresponding AF Template:
Environment readings JSON messages from the Raspberry PI - PI Sense HAT
This is a pretty simple flow that takes the json output from a PI Sense HAT, separates the "environment" readings, slows down the scan rate, associates them with the right AF / global variables template and sends them to PI
Current weather XML messages from National Weather Service (weather.gov) weather stations
This flow goes to the National Weather Service site every 10 minutes to get weather updates by station (id). The xml results are then converted into json, re-formatted and then sent to PI.
Current grid conditions HTML messages from ERCOT (ercot.com), who manages the electrical power grid in Texas
This flow requests current grid information every 15 seconds from ercot. The resulting html are converted into json, re-formatted ad then sent to PI. HTML requires more effort than XML to convert to json (some querying and more manipulation)
As displayed in PI Coresight:
Global variables
Node-RED allows for the use of global variables or functions. The globals init tab runs a function that sets a number of global variables, objects and functions on start-up. These globals can be used in any of the flows, but do not persist once Node-RED is shut down. For the data that is written to the PI System, there is a representation of the AF template stored in the globals. This maps the data sources to the AF elements. See this link for more information on globals: https://nodered.org/docs/writing-functions#storing-data
Why PI Web API and Node-RED?
For web and IOT data sources and json manipulation, I find Node-RED a lot easier to work with than something like the PI UFL Interface or PI UFL Connector. Since a lot of IOT and web feeds that I am dealing with are in json (or something like XML that can easily be converted to json), JavaScript is a great language when working with json. The basic parsing ability of the PI UFL is fine for delimited text files and even some simple xml and json; however, when needing to any more advanced functions (e.g. looping through a file with hundreds of inputs to break into multiple values, or trying to do a look-up to map a json key value pair automatically to an AF Element), it falls short (or needs a lot of IF statements!). The PI UFL Connector does offer a REST client to poll a web api endpoint, however, it is limited to http(s) "get"; it does not appear to yet support authentication or more advanced settings (like oauth2 or additional headers) or channel/pub-sub functionality like websockets or http streaming.
For delivery into the PI System, with many of the data sources use a REST endpoint or a similar delivery, the PI Web API seemed to be an obvious choice. The same skills/mechanisms to gather the data are the same being used to write it to the PI System. However, in the future, it might be worth looking at a hybrid approach. Using Node-RED for all of the gathering and transformation of the messages from the various system, and PI UFL as the input mechanism to the PI System (either via the PI UFL Connector's REST endpoint or by using formatted text files). PI UFL has been around a long time and is very tried and true.