The System - SimDynamX/SpaceTeamsPRO GitHub Wiki

The Fundamentals → The System

Up to date for Platform 0.54.2

Written by Connor Jakubik.

The System

Space Teams Systems are pieces of code that hold all of the behavior of the simulation. Systems are applied to Entities, and during simulation execution they affect the Parameters of the entities to evolve the simulation over time. Systems typically have zero or very few dependencies on other Systems, meaning they can be specialized to a certain environmental effect (e.g. gravity), physical process (e.g. heat transfer), analysis computation (e.g. logging telemetry), or autonomous logic (e.g. path-planning) which can be applied on any Entity.

Example System Pseudocode

System simulating a component on a vehicle:

  1. On each update,
  2. Read parameters to see current state of things such as a battery.
  3. React to those parameters by changing internally tracked state and/or modifying Entity parameters.
    1. For example, a System simulating the power system on a vehicle may update the battery state of charge based on the total power usage of the vehicle.

Dynamics Propagator System:

  1. On each update,
  2. Read in the states of each entity in the previous update's time context.
  3. Use numerical integration with a certain set of ODEs to propagate dynamics forward from the previous update to the current update's time.
  4. Set the states of each entity with the propagated values.

System set up to incorporate external telemetry into this sim:

  1. On each update, or with an on-receive async task,
  2. Parse input from a socket or other inter-process messaging system.
  3. Set Entity parameters or trigger other functionality based on the latest telemetry.

System for taking controller input:

  1. On each update, or with a loop handled by third party code,
  2. Read button and axis states.
  3. Apply those directly to Entity parameter values, or
  4. Perform operations to present the button presses and axis changes to Entity parameters in a different way, such as applying a response curve to axes, or treating button presses as events.

System Instances

In a Space Teams sim, all Systems are initialized as System Instances in the SimConfig. A System Instance has a Nametag, Type, Source, and other properties that depend on the type. All Systems additionally have Instance Parameters. System Instance Parameters differ from Entity parameters in that they are entirely internal to that System; they are only accessible on that System Instance and don't have value changes transmitted to other programs in the sim.

Multiple instances of the same System code (same Source) can be declared in a Sim Config, each with a different Nametag, Inst_Parameters, assignment to entities, etc.

System Instances are applied to Entities by adding their Nametag to the Entity's Systems list. Alternatively, Systems can be given access to Entities through EntityRef parameters, either on the System Instance Params or on the params of another Entity the System has direct access to.

C++ Systems

To write simulation behavior in C++, a user duplicates our template System subclass through a tool to establish all the boilerplate code, then writes code inside the load(), init(), update() and other System member functions that will be called by the Compute_Server in a thread specific to that System during the simulation. The Space Teams Platform is distributed publicly with certain C++ core code header files available, which allows users to integrate their System code directly with the server for maximum performance. Systems are subclasses of the core System class. Before running the simulation, Systems are compiled into dynamically linked libraries which are loaded to create System Instances according to what is specified in a Sim Config file.

Each C++ System Instance gets its own thread, where the Compute_Server attempts to call its update functions at the specified frequency, or as close to that as possible. This frequency is specified in Frequency on the definition of that System Instance.

Taskflows

Taskflow Systems are declared in the Taskflows section of the Sim Config. TaskflowTask System Instances, instead of operating in their own thread, operate as part of the Taskflow executor, which runs the tasks in a concurrent environment while satisfying dependencies on the run-order of certain tasks. This is how you can enforce strict ordering between Systems if required. Typically, it will be possible to avoid bad behavior such as race conditions even without resorting to a Taskflow by designing your System code well and using Events where appropriate.

See the Sim Config page for examples of setting up Systems and Taskflows in a Sim Config.

Python Systems

For writing simulation behavior in Python instead of C++, Space Teams includes a Python API library. This library encapsulates most of the functionality of the Compute_Server. The Python scripts that use it are 100% "normal" Python, as opposed to "embedded" python scripting where you can't install pip packages. We include a Python installer for a particular version (accessible in windows as .\sc_python.bat) alongside our application because we don't make prebuilt Python API versions for more than that one version. Python System scripts are spun off from the Compute_Server with special configuration options in order to connect to the running sim. The st.connect_to_sim(sys.argv) function is where the Python API starts a local copy of the simulation and connects.

Python Systems have their .py file specified as Source instead of their DLL, and they do not use the Frequency parameter. Python systems are free to set up their own loop, use the loop of a UI library, just run through some scripts without a loop, or whatever the programmer desires. Exit the sim without stopping it by st.leave_sim(), or stop the sim with st.stop_sim().

The Python API is as close as possible to the C++ API syntax. Instead of templates on param functions to identify the variable type of the param value, the first argument is st.VarType.something.

See Programming STP Sims with Python for more on this topic.

Order of Ops Guarantees

During a simulation, this is a timeline of events relevant to Systems in the Compute_Server:

  • SimState Loading
    • Load the sim config as a starting point, independent of any other programs
    • Python System processes are started but will hang on st.connect_to_sim() until SimState::Runtime happens.
    • System load() functions are run
  • Lobby
    • Wait for other clients to finish connecting/loading, communicate and organize with other users (if applicable).
  • Sync
    • With a final user list known, enter a process where we ensure all clients are joined and warmed up before Init/Running SimState is made available.
    • Some propagators such as SPICE start running in order to initialize the solar system environment
    • (v0.41.2+) Python Systems BeforeInit delegate functions are run
  • Init
    • System init() then initPerEntity() functions are run in sequence, on a single thread
    • (v0.41.2+) Python Systems BeforeRuntime delegate functions are run
  • Runtime
    • Timed updates begin. Physics starts moving.
    • Normal UI interaction resumes
    • Async Scheduler:
      • System update() then updatePerEntity() are called in a loop on a unique thread per System or Taskflow, with time calculations to try to maintain the given frequencies with the OS scheduler.
    • Deterministic Scheduler:
      • System update() then updatePerEntity() are called in a single update thread,
    • Python Systems continue from connect_to_sim() call
  • Stopping
    • (If Async Scheduler) Wait for an update/updatePerEntity loop to complete for each System.
    • end() for each System.
    • After all Systems end(), Compute_Server program is allowed to terminate.

If the program is terminated by other means such as a Ctrl-C in the terminal or through Task Manager, the program terminates immediately without finishing init/update stuff or calling the end() function.

Publish / Subscribe

Space Teams has an Publish-Subscribe messages feature to fill the gap of modeling individual events such as a button-push instead of continuously-updating effects such as gravity. This is similar to ROS messages. A Subscribe "delegate function" can be added in code for a certain topic, optionally filtering by an entity ID. On reception of a published message, all Subscribes are checked for a topic/ID match, and all matching Subscribe delegates are executed in their own thread*. Messages are Published with a ParamMap "payload", which is supplied to any delegates it executes.

[!NOTE] * There are options for how the delegate is ran. By default, a fresh thread is spun up to run the delegate. On Subscribe, the code can specify the delegate should run immediately, which blocks other delegates from running (among other things) but does not incur thread-creation overhead (can be >1ms delay in some scenarios). This functionality may change in a future (> v0.54.2) version.

The user must be careful when using Publish/Subscribe to implement complex behavior, as these async delegate functions violate the design constraint of each C++ and Python System Instance having its own single thread. Subscribe delegate functions are ran immediately upon reception of the Published message on a topic, and that code may cause multithreading issues if the data that is accessed by the delegate is also being accessed by the code that registered the delegate. In event listener delegates, using parameter data structures and features built on top of that like entity body-fixed frames are not going to cause exceptions or crashes, but they still might introduce issues like race conditions.

  • Publish/Subscribe New Functionality
    • An optional entityId argument is available to make your Publish/Subscribe more specific. If unspecified, the entityId is the SimEntity's entityId. This should be used to make messages on a topic specific to a certain entity in cases where multiple entities in the simulation are doing a similar thing and may otherwise receive each others messages when subscribing.
    • To avoid most issues with deadlocking, all Publishes only enqueue a Message, and all Subscriber delegates are only ran asynchronously in one of two modes:
      • If argument independentThread is true, the delegate is ran in its own thread. This is a suitable default.
      • Else, the delegate is ran in either its own thread or some program-dependent threadpool that has lower overhead. This is a better option if your delegate is expected to be called many times per second, should take less than 10 milliseconds to run, and does NOT wait for other conditions to be met before returning. Without independentThread, there is more risk of a deadlock situation due to the structure of your simulation code.

Request / Response

One level more complicated than Publish/Subscribe is Request/Response. As opposed to the multicast fire-and-forget behavior of Publish/Subscribe, Request/Response is meant for more HTTP-like requests that expect 1 response with a payload, have a timeout, and can fail.

  • Use Request / Respond in situations where you want to get data or request an action that you do not have access to in the current System or thread you are programming in. The behavior can be treated somewhat similarly to HTTP requests.
  • Only one Response is expected for a single Request. Any more responses than one are treated as an error. So, don't set up Respond() on the same topic/entity twice, or include code that takes in more data to decide whether to respond.
  • A Respond call sets up a delegate function that takes a Request's payload as input and returns a "Response" payload as output. A Request call sends a payload along with a unique request ID to the entire STP sim and waits for the first Response that it receives on the same request ID. When the response is received, the Request returns the response payload.
  • Respond is a blocking call, and has a timeout argument
    • timeout defaults to 10 (real-world) seconds
    • The timeout happens if a response isn't received in time, no matter if a responder exists somewhere in the sim.
    • This timeout will end the blocking call and return an invalid object
      • In C++, this is an invalid std::optional value
      • In Python, this is a None object
    • Always check the output of the Request
      • C++: 
        auto response = SimGlobals::Request(...);
if(!response)
{
    //Do something to handle failure, log an error, or fatal-log and stop the sim.
}
      • Python: 
        response = st.SimGlobals.Request(...)
if response is None:
    #Do something to handle failure, log an error, or fatal-log and stop the sim.
        or 
        response = st.SimGlobals.Request(...)
if response is not None:
    #Continue inside this indented scope
    • RequestAsync is an alternative function available in C++ where the std::future object is returned for the user to implement their own waiting scheme.
    • RequestAsyncWithDelegate is suitable for situations when you want to wrap some Request/Response code with a delegate function that will be ran with the received response payload to transform it into a more structured form. A good example of this is for CaptureImageAsync, where we have a recv-delegate that rearranges the parameter response into a CapturedImage object which fulfills the future the CaptureImageAsync returns.
  • Respond and RequestAsyncWithDelegate also have optional independentThread, with the same behavior as described above.
  • Request/Respond also allow setting a more specific entityId to filter which Responders will respond.

CaptureImage

TODO move to camera wiki page

  • Making use of Request/Response, we have the CaptureImageAsync and CaptureImageBlocking functions available as an alternative to CaptureImage/OnImageReceived.
  • CaptureImageBlocking allows the user to make much more simple usage of the Camera entity type Unreal rendering. Instead of registering an OnImageReceived delegate function, the C++ or Python code will blocking-wait until the image is captured and received before continuing.
    • A timeout argument is required, and should probably be set to at least 1 second if not 10 to handle lag spikes.