Model Metrics Report - openmpp/openmpp.github.io GitHub Wiki

Home > Model Development Topics > Model Metrics Report

This topic describes and illustrates the Model Metrics Report produced by the OpenM++ compiler. The report summarizes code inputs and code outputs, model inputs and model outputs, model symbols, and maintained dependencies.

Related topics

  • Model Code: How model code is organized into modules, syntactic islands, and C++
  • Model Resource Use: Activating and interpreting information about model resource use

Topic contents

Introduction

The OpenM++ compiler (omc) produces the Model Metrics Report in the file MODEL/.../src/ModelMetrics.txt. The report consists of 7 summary tables about the model's

  • source code base
  • user interface
  • symbols
  • implicit causal web

The Model Metrics Report is about the model, not about a run of the model. To produce metrics for a run of the model, see Model Resource Use.

The report begins with a title line like

Model Metrics for RiskPaths 2023-10-13

which gives the model name and the date of compilation.

The remainder of this topic describes each of the 7 tables in turn using the RiskPaths model as an example.

[back to topic contents]

Code input

The CODE INPUT table is a summary of the model code processed by omc.
Here it is for RiskPaths:

+-----------------------------------+
| CODE INPUT (model source code)    |
+-----------------+-------+---------+
| Source          | Files |   Lines |
+-----------------+-------+---------+
| Model-specific  |    14 |    1504 |
|   Islands       |       |     286 |
|   C++           |       |    1218 |
| Common          |     9 |    1707 |
|   Islands       |       |      11 |
|   C++           |       |    1696 |
| Both            |    23 |    3211 |
|   Islands       |       |     297 |
|   C++           |       |    2914 |
+-----------------+-------+---------+
| Total           |    23 |    3211 |
+-----------------+-------+---------+
Note: Does not include parameter data.

The table groups the model source code files into two categories:

  • model-specific files in the MODEL/code folder
  • common files supplied by OpenM++ and incorporated into the model by use statements

The model-specific RiskPaths source code consists of 1,504 lines in 14 .mpp and .ompp files. An example of one of those 14 files is RiskPaths/code/PersonCore.mpp, which implements core functionality for the Person entity.

The common source code in RiskPaths consists of 1,707 lines in 9 .ompp files specified by use statements. An example of one of those 9 files is OM_ROOT/use/random/random_lcg41.ompp, which implements a family of simple and fast random number generators.

Common source code files are specified in use statements, which in RiskPaths are in the model-specific source code file RiskPaths/code/ompp_framework.ompp. Common source code files can also contain use statements.

The MODEL CODE table assigns each line of a source code file to one of 2 categories:

  • Syntactic island
  • C++ code

A syntactic island is a section of declarative code written in the ompp language. Here's an example from RiskPaths - the declaration of the table T02_TotalPopulationByYear:

table Person T02_TotalPopulationByYear  //EN Life table
{
   //EN Age
   integer_age *		
   {
      unit,       //EN Population start of year
      duration()  //EN Average population in year
   }
};

A syntactic island starts with a keyword, e.g. table, and ends with a matching ;.

Source code which is not part of a syntactic island is C++ code. Model C++ code implements C++ functions associated with events, derived parameters, and derived tables. It can also implement other functions to support the model.

Here's an example of model C++ code from RiskPaths - the C++ function which implements the DeathEvent:

void Person::DeathEvent()
{
   life_status = LS_NOT_ALIVE;
   Finish();
}

For more on model source code, see Model Code.

The CODE INPUT table does not report on parameter value files read by omc to publish the starting Default scenario/set for the model. For RiskPaths those are the 2 files RiskPaths.dat and Framework.odat in the folder RiskPaths/parameters/Default.

[back to topic contents]

Code output

The CODE OUTPUT table contains summary information about C++ code generated by omc.
Here it is for RiskPaths:

+-----------------------------------------+
| CODE OUTPUT (generated C++)             |
+-----------------------+-------+---------+
| Description           | Files |   Lines |
+-----------------------+-------+---------+
| Header (.h)           |     3 |    1395 |
| Implementation (.cpp) |     1 |    3766 |
+-----------------------+-------+---------+
| Total                 |     4 |    5161 |
+-----------------------+-------+---------+
Note: Table does not include C++ initializers for burned-in parameter data.

omc uses the declarative model specification in syntactic islands to generate procedural C++ code which implements the model specification. For RiskPaths, the 297 lines in syntactic islands which specify the model are used by omc to generate the 5,161 of C++ code which implement the model. The large difference between the # of lines of input declarative code and the # of lines of output procedural code reflects the conciseness and power of the ompp declarative language.

Generally, model developers don't need to examine generated C++ code because it just works, but here's a peek at it anyway. The following 2 extracts show the code generated by omc for the internal function om_side_effects_age_status, one of the C++ functions used to implement RiskPaths.

The function is declared in the generated header file RiskPaths/.../src/om_declarations.h and looks like this:

// model entity classes
class Person : public Entity<Person>
{
public:
    ...
    /// Implement side effects of changing age_status in entity Person.
    void om_side_effects_age_status(int om_old, int om_new);
    ...
};

The function is defined in the generated implementation file RiskPaths/.../src/om_definitions.cpp and looks like this:

void Person::om_side_effects_age_status(int om_old, int om_new)
{

    // Code Injection: group=8, injector=formation_hazard
    // Maintain identity for 'formation_hazard'
    formation_hazard_update_identity();

    // Code Injection: group=8, injector=preg_hazard
    // Maintain identity for 'preg_hazard'
    preg_hazard_update_identity();

    // Code Injection: group=10, injector=FirstPregEvent
    // Recalculate time to event FirstPregEvent
    if (om_active) om_FirstPregEvent_om_event.make_dirty();

    // Code Injection: group=10, injector=Union1FormationEvent
    // Recalculate time to event Union1FormationEvent
    if (om_active) om_Union1FormationEvent_om_event.make_dirty();
}

The generated function om_side_effects_age_status handles the side-effects of a change in the age_status attribute of the Person entity. It is automatically called if the value of age_status changes during an event in a run, and when called it

  • updates the identity attributes formation_hazard and preg_hazard because their declarations in syntactic islands use age_status.
  • marks for recalculation the future times of the events FirstPregEvent and UnionFormationEvent because the time functions of those events use age_status, which could affect those future times.

These 4 actions in om_side_effects_age_status are examples of Maintained dependencies, which have a dedicated table of their own in the Model Metrics report.

This example shows that the C++ code generated by omc contains comments as well as code and is formatted for readability. This can help to understand the generated code if there's ever a need to examine or trace into it in a debug session of a model.

omc also generates, in the output C++ header file MODEL/.../src/om_declarations.h, doxygen brief descriptions for model symbols such as classifications, ranges, partitions, parameters, and attributes. For example, the generated doxygen brief description for the age_status attribute looks like this:

    /// attribute(identity) int: Current age interval
    age_status_om_type age_status;

The generated doxygen brief description which starts with /// says that age_status is an attribute, more specifically an identity attribute, that its type is int, and its label in the model's default human language is Current age interval. Doxygen brief descriptions are automatically recognized by the Visual Studio and VSCode IDEs, and those IDEs display a popup with the doxygen brief description for a symbol when the cursor is hovered over that symbol name in model C++ source code. For more, see Doxygen brief descriptions for model symbols.

omc transforms burned-in parameter values to C++ initializers in the output file MODEL/.../src/om_fixed_parms.cpp. The CODE OUTPUT table does not report on this file or its contents.

[back to topic contents]

Model input

The MODEL INPUT table provides summary information about the model's input parameters. It reports counts of parameters and counts of parameter cells.

MODEL_INPUT classifies parameters into 4 mutually exclusive categories:

  • Visible parameters, which are immediately available in the UI.
  • Hidden parameters, which are made visible in the UI when the user clicks a reveal button.
  • Burned-in parameters, which have fixed values and are absent from the model database and UI. A parameter is burned into the model executable if its Default value file is located in the Fixed folder, or if it was removed from the model database and UI by a suppress_parameters or retain_parameters statement.
  • Derived parameters, which are computed by model-specific code during run initialization.

Here's the MODEL INPUT table for RiskPaths:

+----------------------------------+
| MODEL INPUT (parameters)         |
+---------------+-------+----------+
| Kind          | Count |    Cells |
+---------------+-------+----------+
| Visible       |     2 |       18 |
| Hidden        |     7 |      133 |
| Burned-in     |     0 |        0 |
| Derived       |     0 |        0 |
+---------------+-------+----------+
| Total         |     9 |      151 |
+---------------+-------+----------+
Note: Burned-in includes fixed and suppressed parameters.

The table shows that RiskPaths presents a highly simplified UI to users by hiding all but 2 parameters. The 2 visible parameters contain 18 values (they're arrays). The remaining parameters are hidden but can be examined and modified by a user by clicking the reveal button in the UI. This design reflects that RiskPaths was used as a hands-on simple example in a demography course.

[back to topic contents]

Model output

The MODEL OUTPUT table provides summary information about the model's output tables. It reports counts of tables and counts of table cells.

MODEL_OUTPUT classifies tables in two ways:

Entity/Derived:

  • Entity tables are declared in model-specific syntactic islands. They are computed dynamically during a run.
  • Derived tables are computed by model-specific C++ code after the simulation phase completes.

Visible/Hidden:

  • Visible tables are immediately available in the UI after a run completes.
  • Hidden tables are made visible in the UI when the user clicks a reveal button.

Here's the MODEL OUTPUT table for RiskPaths:

+----------------------------------+
| MODEL OUTPUT (tables)            |
+---------------+-------+----------+
| Kind          | Count |    Cells |
+---------------+-------+----------+
| Entity        |       |          |
|   Visible     |     2 |      205 |
|   Hidden      |     5 |      148 |
| Derived       |       |          |
|   Visible     |     0 |        0 |
|   Hidden      |     0 |        0 |
| Both          |       |          |
|   Visible     |     2 |      205 |
|   Hidden      |     5 |      148 |
+---------------+-------+----------+
| Total         |     7 |      353 |
+---------------+-------+----------+
Note: Cells includes margins and expression dimension.

For didactic purposes, RiskPaths shows only 2 key tables in the UI when a run completes. However, all tables in RiskPaths can be made visible and explored in the UI. No derived tables were required in RiskPaths, so those rows are zero in the report.

[back to topic contents]

Model Symbols

The MODEL SYMBOLS table reports counts of all symbols in syntactic islands by category, together with counts of associated labels and notes in the model's default human language.

Some model symbols are given names in the model code, such as the table T02_TotalPopulationByYear in the example in Code input. Some model symbols are declared positionally in the model code with no name, such as the expression dimension of that table. Other model symbols are declared when used, such as the derived attribute duration(parity_status,PS_CHILDLESS) which automatically maintains a running sum of the time a Person has spent in the PS_CHILDLESS state.

Here's the MODEL SYMBOLS table for RiskPaths:

+-------------------------------------------------+
| MODEL SYMBOLS                                   |
+-------------------------+-------+-------+-------+
| Description             | Count | Label |  Note |
+-------------------------+-------+-------+-------+
| Language (human)        |     2 |     0 |     0 |
| Enumeration             |       |       |       |
|   Classification        |     4 |     4 |     0 |
|     Level               |    12 |    12 |     0 |
|   Range                 |     1 |     1 |     0 |
|   Partition             |     4 |     4 |     0 |
|   Aggregation           |     0 |     0 |     0 |
| Input/Output            |       |       |       |
|   Parameter             |     9 |     9 |     0 |
|     Dimension           |     7 |     7 |     0 |
|     Group               |     3 |     3 |     0 |
|   Table                 |     7 |     7 |     0 |
|     Dimension           |    13 |     6 |     0 |
|     Expression          |    13 |    13 |     0 |
|     Group               |     3 |     3 |     0 |
|   Import                |     0 |     0 |     0 |
| Entity                  |       |       |       |
|   Kind                  |     1 |     0 |     0 |
|   Event                 |     7 |     0 |     0 |
|   Attribute             |    38 |    13 |     0 |
|     Built-in            |     6 |     0 |     0 |
|     Simple              |     5 |     5 |     0 |
|     Identity            |    13 |     8 |     0 |
|     Derived             |    14 |       |       |
|     Link                |     0 |     0 |     0 |
|     Multilink aggregate |     0 |       |       |
|   Function              |     2 |     2 |     2 |
|   Multilink             |     0 |     0 |     0 |
|   Array                 |     0 |     0 |     0 |
|   Foreign               |     0 |     0 |     0 |
| Entity set              |     0 |     0 |     0 |
|   Dimension             |     0 |     0 |     0 |
+-------------------------+-------+-------+-------+
| Total                   |   126 |    84 |     2 |
+-------------------------+-------+-------+-------+
| Supplementary Info      |       |       |       |
|   Random streams        |     6 |       |       |
|   Eligible microdata    |     0 |       |       |
+-------------------------+-------+-------+-------+
Notes: Parameter includes derived parameters.
       Table > Dimension includes the expression dimension.
       Entity > Attribute > Identity includes those generated from filters.
       Entity > Function does not include event time and implement functions.

The RiskPaths model is relatively simple, and contains only 126 symbols in syntactic islands.

The Eligible microdata row has a count of 0 because RiskPaths does not enable microdata output. For more, see Microdata Output.

[back to topic contents]

Published Symbols

The PUBLISHED SYMBOLS table is similar to MODEL SYMBOLS but reports only on symbols which are published to the model database and visible to model users in the UI, exported model metadata, or run data. Model code can suppress or retain parameters and tables to create a trimmed down model for publishing, while full preserving model logic. For more on that, see the subtopic Model trim down.

Here's the PUBLISHED SYMBOLS table for RiskPaths:

+-------------------------------------------+
| PUBLISHED SYMBOLS                         |
+-------------------+-------+-------+-------+
| Description       | Count | Label |  Note |
+-------------------+-------+-------+-------+
| Language (human)  |     2 |     0 |     0 |
| Enumeration       |       |       |       |
|   Classification  |     2 |     2 |     0 |
|     Level         |     8 |     8 |     0 |
|   Range           |     1 |     1 |     0 |
|   Partition       |     4 |     4 |     0 |
| Input/Output      |       |       |       |
|   Parameter       |     9 |     9 |     0 |
|     Dimension     |     7 |     7 |     0 |
|     Group         |     3 |     3 |     0 |
|   Table           |     7 |     7 |     0 |
|     Dimension     |    13 |     6 |     0 |
|     Expression    |    13 |    13 |     0 |
|     Group         |     3 |     3 |     0 |
|   Import          |     0 |     0 |     0 |
+-------------------+-------+-------+-------+
| Total             |    72 |    63 |     0 |
+-------------------+-------+-------+-------+
Note: Table > Dimension includes the expression dimension.

The table shows that RiskPaths supplies labels for almost all published model symbols, which can help model users navigate and understand model inputs and outputs. RiskPaths eschews the use of notes, which would have been viewable with a click in the UI, providing additional information about the model's parameters and tables.

This table reveals a couple of minor issues in RiskPaths. First, human-language labels for the 2 languages declared in RiskPaths (EN and FR) are absent for the model's default human language. This causes OpenM++ to present language codes rather than language names in the UI for switching languages. Second, 7 table dimensions lack labels in the model's default language, which causes omc to generate fall-back labels mechanically.

[back to topic contents]

Maintained dependencies

As described in Code input, the ompp language is declarative. Here again is the declaration of the table T02_TotalPopulationByYear:

table Person T02_TotalPopulationByYear  //EN Life table
{
   //EN Age
   integer_age *		
   {
      unit,       //EN Population start of year
      duration()  //EN Average population in year
   }
};

The resulting table has 101 rows for integer age and 2 columns for the computed expressions. Whenever a Person has a birthday and integer_age changes, the cell in T02_TotalPopulationByYear to which they are contributing changes and the table must be updated. In other words, there is a dependency between integer_age and T02_TotalPopulationByYear.

The example in Code output illustrates other kinds of dependency, where changes in the age_status attribute requires updating the identity attributes formation_hazard and preg_hazard and and recalculating the events FirstPregEvent and Union1FormationEvent.

To implement the model specification, omc generates C++ code which maintains all such dependencies, both direct and indirect. The MAINTAINED DEPENDENCIES table groups and counts these dependencies by category.
Here's it is for RiskPaths:

+----------------------------+-------+
| MAINTAINED DEPENDENCIES            |
| (in generated C++ runtime code)    |
+----------------------------+-------+
| Dependency                 | Count |
+----------------------------+-------+
| Reciprocal link            |     0 |
| Attribute maintenance      |       |
|   Identity                 |    18 |
|   Derived                  |    23 |
|   Multilink aggregate      |     0 |
| Table dimension/filter     |     8 |
| Set dimension/filter/order |     0 |
| Event maintenance          |    17 |
+----------------------------+-------+
| Total                      |    66 |
+----------------------------+-------+

The table shows that the implicit causative web in RiskPaths contains 17 dependencies between attributes and events.

RiskPaths is a simple model with only 286 lines of model code in syntactic islands. Nevertheless, it has 66 dependencies which are automatically maintained by the C++ code generated by omc.

[back to topic contents]

⚠️ **GitHub.com Fallback** ⚠️