State Collection Interface Documentation for Multiple Devices - FREEDM-DGI/FREEDM GitHub Wiki

This documentation is mainly for documentation State Collection model in DGI. Multiple devices states could be collected by State Collection requested from other module.

Algorithm in State Collection:

The DGI State Collection module is implemented based on the Chandy-Lamport algorithm [1], which is used to collect consistent states of all participants in a distributed system. A consistent global state is one corresponding to a consistent cut. A consistent cut is left closed under the causal precedence relation. In another words, if one event is belongs to a cut, and all events happened before this event also belong to the cut, then the cut is considered to be consistent. The algorithm works as follows:

  • The initiator starts state collection by recording its own states and broadcasting a marker out to other peers. At the same time, the initiator starts recording messages from other peers until it receives the marker back from other peers.

  • Upon receiving the marker for the first time, the peer records its own state, forwards the marker to others (include the initiator) and starts recording messages from other peers until it receives the marker back from other peers.

The following diagram illustrates the Chandy-Lamport algorithm working on three nodes. The initiator is the leader node chosen by Group Management module in DGI.

State Collection Algorithm

Data Structure in State Collection:

  • Marker: std::pair<std::string, int>. Marker is tracked by the local node’s UUID in addition to a locally incremented integer.
  • Collected states: multimap <Marker, ptree>. Collected states use the ptree structure and are recorded in multimap with marker’s version. A Ptree is a data structure in Boost that stores an arbitrarily deeply nested tree of values, indexed at each level by some key [2]. Each node of the tree stores its own value, plus an ordered list of its subnodes and their keys. The tree allows easy access to any of its nodes by means of a path, which is a concatenation of multiple keys.

The state of device and channel message will have similar ptree structure. TakeSnapshot() will be called to obtain the states of various devices through function provided by physical device manager. The state of each device will be added as a sub ptree.

ptree sub_ptree;
sub_ptree.add("type", deviceType);
sub_ptree.add("signal", signalType);
sub_ptree.add("value", VALUE);
m_.m_submessages.add_child("sc.collects.collect", sub_ptree);
m_curstate = m_.GetSubMessages();

HandleAny() will be called to save any in-transit messages during the state collection cycle. Each in-transit message will be added as a sub ptree.

ptree sub_ptree;
sub_ptree.add("type", "Message");
sub_ptree.add("signal", "inchannel");
sub_ptree.add("value", intransit);
m_.m_submessages.add_child("sc.collects.collect", sub_ptree);
m_curstate = m_.GetSubMessages();

Functions in State Collection:

  • Initiate(): The initiator use this function to start a state collection by recording its own states and broadcasting the marker out to all other peers. Currently, the initiator is the leader node chosen by the Group Management module.
  • TakeSnapshot (deviceType, valueType): This function is used to save a local state. Given the type of the device (deviceType) and the type of the value (valueType), request state will be recorded.
  • SendStateBack(): This function indicates any node except the initiator has completed its local state collection and is called to send its collected states to the initiator.
  • HandleRead(): This function is called upon every incoming message.
  • StateResponse(): This function is called after the initiator has received states from all other peers and has completed its local state collection. All collected states will be separated into different kinds of strings: for example, string for gateway value, string for channel message.

State Collection Interface:

State Collection is a passive process that won’t be started until it receives a message from other modules. It communicates with other modules through message passing. The following example illustrates the Load Balance (LB) calling State Collection (SC) for the gateway value, generation value, storage value and messages passing back and forth:

LB prepares message for requesting gateway, generation and storage:

CMessage m_sc;
m_sc.m_submessages.put("sc", "request");
m_sc.m_submessages.put("sc.source", GetUUID());
m_sc.m_submessages.put("sc.module", "lb");

//for multiple devices
m_sc.m_submessages.put("sc.deviceNum", 3);

//SST device
ptree subPtree1;
subPtree1.add("deviceType", "Sst");
subPtree1.add("valueType", "gateway");
m_sc.m_submessages.add_child("sc.devices.device", subPtree1);

//DRER device
ptree subPtree2;
subPtree2.add("deviceType", "Drer");
subPtree2.add("valueType", "generation");
m_sc.m_submessages.add_child("sc.devices.device", subPtree2);

//DESD device
ptree subPtree3;
subPtree1.add("deviceType", "Desd");
subPtree1.add("valueType", "storage");
m_sc.m_submessages.add_child("sc.devices.device", subPtree3);

Upon receiving states back in HandleCollectedState():

LB can print out the states based on the type:

ptree pt = msg.GetSubMessages();
if(pt.get_child_optional("CollectedState.gateway"))
{
    BOOST_FOREACH(ptree::value_type &v, pt.get_child("CollectedState.gateway"))
    {
        Logger.Notice << "SC module returned gateway values: "
                      << v.second.data() << std::endl;
    }
}
else if(pt.get_child_optional("CollectedState.geneartion"))
{
    BOOST_FOREACH(ptree::value_type &v, pt.get_child("CollectedState.generation"))
    {
        Logger.Notice << "SC module returned generation values: "
                      << v.second.data() << std::endl;
    }
}
else if(pt.get_child_optional("CollectedState.storage"))
{
    BOOST_FOREACH(ptree::value_type &v, pt.get_child("CollectedState.storage"))
    {
        Logger.Notice << "SC module returned storage values: "
                      << v.second.data() << std::endl;
    }
}
else if(pt.get_child_optional("CollectedState.drain"))
{
    BOOST_FOREACH(ptree::value_type &v, pt.get_child("CollectedState.drain"))
    {
        Logger.Notice << "SC module returned drain values: "
                      << v.second.data() << std::endl;
    }
}
else if(pt.get_child_optional("CollectedState.intransit"))
{
    BOOST_FOREACH(ptree::value_type &v, pt.get_child("CollectedState.intransit"))
    {
        Logger.Status << "SC module returned intransit messages: "
                      << v.second.data() << std::endl;
    }
}

State Diagram for State Collection:

  • For the initiator: State Diagram for the initiator

  • For the Peer node: State Diagram for the peer

Test Environment Requirement:

Load_Timeout, State_Timeout in LoadBalance.hpp and phase for State Collection in PosixMain.cpp need increasing as the number of device is increasing.

Two devices:

  • Load_Timeout = 400ms
  • State_Timeout = 800ms
  • SC_timer = 800ms

Three devices:

  • Load_Timeout = 400ms
  • State_Timeout = 1000ms
  • SC_timer = 1000ms