Model Data Get and Set Quick Start - openmpp/openmpp.github.io GitHub Wiki

Overview

There multiple examples how to set input parameters, run the model and get results:

Also openM++ support following APIs:

oms: openM++ web-service which you can use from any modern environment: Python, .NET, JavaScript, etc.
openMpp R package
Go library and tools

Quick examples below do not cover all possible options, please check links above and Model Run: How model finds input parameters for more details.

Sub-values: sub-samples, members, replicas

Following terms: "simulation member", "replica", "sub-sample" are often used in micro-simulation conversations interchangeably, depending on context. To avoid terminology discussion openM++ uses "sub-value" as equivalent of all above and some older pages of that wiki may contain "sub-sample" in that case.

Model output tables: sub-values, accumulators and expressions

There are two kind of model output tables:

expressions table: model output value calculated as accumulators aggregated across sub-values (e.g. mean or CV or SE)
accumulators table: output sub-values (similar to Modgen sub-samples)

All output accumulator tables always contain same number of sub-values, for example model run:

model.exe -OpenM.SubValues 16

will create 16 sub-values for each accumulator in each output accumulator table.

Model parameters: sub-values (optional)

OpenM++ parameters can also contain sub-values. Parameters sub-values are not required, it is a user choice to run the model and supply sub-values for some parameters.

For example, if user wants to describe statistical uncertanty of parameter SalaryByYearByProvince then .csv or .tsv file with 16 sub-values can be supplied to run the model:

model.exe -OpenM.SubValues 16 -SubFrom.SalaryByYearByProvince csv -OpenM.ParamDir C:\MyCsv\

Parameters: Re-use same parameters values as in previous model run

Most of the model parameters are not changing between simulations and only few are varying. It is convenient to select all unchanged parameters from previous model run (from "base" run):

model.exe -Parameter.Ratio 0.7 -OpenM.BaseRunId 1234
model.exe -Parameter.Ratio 0.7 -OpenM.BaseRunDigest 5dc848891ea57db19d8dc08ec7a30804
model.exe -Parameter.Ratio 0.7 -OpenM.BaseRunName "My base run of the Model"

Above command do run the model with parameter Ratio = 0.7 and the rest of parameters values are the same as it was in previous run with id = 1234.

It is also possible to use run diegst or run name to identify "base" model run:

model.exe -Parameter.Ratio 0.7 -OpenM.BaseRunDigest 5dc848891ea57db19d8dc08ec7a30804
model.exe -Parameter.Ratio 0.7 -OpenM.BaseRunName "My base run of the Model"

Please keep in mind, model run may not be unique and if database contains multiple model runs with the same name then first run will be selected.

Parameter: Value as command line argument

It is possible to specify value of any scalar parameter as command line argument, for example:

model.exe -Parameter.Ratio 0.7

There is an example of such technique at Run model from R: simple loop over model parameter page, where we using NewCaseBased model to study effect of Mortality Hazard input parameter on Duration of Life output:

for (mortalityValue from 0.014 to 0.109 by step 0.005)
{
  # run the model
  NewCaseBased.exe -Parameter.MortalityHazard mortalityValue
}

If parameter is enum-based (e.g. classification) then you can specify code or enum id:

modelOne.exe -Parameter.baseSalary Full
modelOne.exe -Parameter.baseSalary 22 -OpenM.IdParameterValue true

Parameter: Sub-values [0, N-1] as command line argument

If we want to run the model with multiple sub-values (a.k.a. sub-samples) and want "Grade" parameter sub-values to be created as [0, N-1] then:

model.exe -OpenM.SubValues 16 -SubFrom.Grade iota

as result sub-values parameter Grade would be: [0, ..., 15]

Parameter: Value inside of ini.file

Also any scalar parameter can be defined in model ini-file, i.e.:

model.exe -ini my.ini

; inside of my.ini file:
;
[Parameter]
Z_Parameter = XYZ      ; string parameter
SomeInt     = 1234     ; integer parameter
OrLogical   = true     ; boolean parameter
Anumber     = 9.876e5  ; float parameter

Parameters: CSV or TSV files

It is also possible to supply some (or even all) model parameters as CSV or TSV file(s). For example:

model.exe -OpenM.ParamDir C:\my_csv

If directory C:\my_csv\ exist and contains ParameterName.csv or ParameterName.tsv file then model will use it parameter values.

It is important to describe your parameter values to make sure model users clearly understand scenario data. In order to do that you can supply ParameterName.LANG-CODE.md file(s).

For example, C:\my_csv\Sex.csv values of "Sex" parameter:

sub_id,dim0,param_value
0,     F,   true
0,     M,   false

or it can be TSV file: C:\my_csv\Sex.tsv:

sub_id	dim0	param_value
0	F	true
0	M	false

And parameter value notes C:\my_csv\Sex.EN.md:

Sex parameter values in this scenario contain indicators of increased gender-specific hazards.

Parameters: CSV or TSV files with multiple sub-values

If user want to supply up to 32 sub-values of "Sex" parameter:

sub_id,dim0,param_value
0,     F,   true
0,     M,   false
1,     F,   true
1,     M,   true
.................
31,    F,   false
31,    M,   true

Important: Presence of multiple sub-values in csv or tsv file (or in database) does not mean model will use all parameter sub-values. Only explicitly specified parameter(s) receiving sub-values.

For example, if user run the model 3 times:

model.exe -OpenM.SubValues 16
model.exe -OpenM.SubValues 16 -OpenM.ParamDir C:\my_csv
model.exe -OpenM.SubValues 16 -OpenM.ParamDir C:\my_csv -SubFrom.Sex csv

"Sex" parameter expected to be in database and no sub-values used
"Sex" parameter value is sub-value 0 from C:\my_csv\Sex.csv or C:\my_csv\Sex.tsv
"Sex" parameter using sub-values [0, 15] from C:\my_csv\Sex.csv or C:\my_csv\Sex.tsv

Output Tables: Suppress output tables

By default model calculate all output tables and write it into database as model run results. Sometime it may be convenient to save only some output tables to reduce a time of each model run. This can be done by either suppressing model output table(s) or table group(s):

model.exe -Tables.Suppress ageSexIncome
model.exe -Tables.Suppress ageSexIncome,fullAgeSalary,A_TablesGroup

Or by suppressing output for all tables except of some:

model.exe -Tables.Retain ageSexIncome
model.exe -Tables.Retain ageSexIncome,fullAgeSalary,A_TablesGroup

Suppress and Retain options are mutually exclusive and cannot be mixed. For example, this model run would fail:

model.exe -Tables.Suppress ageSexIncome -Tables.Retain fullAgeSalary

Use dbget: Export model run results into text files

Please notice: development of dbget utility is not completed, additional functionality coming.

Dbget is a command line tool to export OpenM++ model metadata, input parameters and run results. It is reading from model database and produce CSV, TSV or JSON output.

Output all modelOne model runs input parameters and output tables into CSV or TSV files:

dbget -m modelOne -do all-runs
dbget -m modelOne -do all-runs -csv
dbget -m modelOne -do all-runs -tsv

By default dbget produces language specific output based and language is determined by matching of user OS language to model languages. Dimension labels in CSV or TSV files are language specific, for example it can be Männlich, Weiblich for Deutsche OS version. If teher is no match between user OS language and model languages in database then default model language is used. User can explicitly specify a language for dbget output:

dbget -m modelOne -do all-runs -lang FR
dbget -m modelOne -do all-runs -lang en-CA

Sometime you may want to have dimension item codes instead of language specific labels, e.g.: M, F instead of Male, Female. It can be achived by using -dbget.NoLanguage option:

dbget -m modelOne -do all-runs -dbget.NoLanguage

You can suppress zero values or NULL values in output tables:

dbget -m modelOne -do all-runs -dbget.NoZeroCsv
dbget -m modelOne -do all-runs -dbget.NoNullCsv
dbget -m modelOne -do all-runs -dbget.NoZeroCsv -dbget.NoNullCsv

To redirect CSV files into a specific directory use:

dbget -m modelOne -do all-runs -dir my/output/dir

To use dbget output in commands pipe do:

dbget -m modelOne -do all-runs -pipe

Important: by using -pipe option you suppressing all error messages. Please always check exit code to avoid errors.

If model SQLite database file located outside of current directory then you use -db option:

dbget -db /some/dir/my.sqlite -m RiskPaths -do all-runs

To get only a specific run results and not all model runs please specifiy model run digest, run name, run stamp or run ID or special -dbget.LastRun, -dbget.FirstRun options:

dbget -m modelOne -do run -dbget.FirstRun
dbget -m modelOne -do run -dbget.LastRun
dbget -m modelOne -do run -r "My special run"
dbget -m modelOne -do run -r "abcd12345efgh789eca"
dbget -m modelOne -do run -r "2024_07_16_18_34_42_038"
dbget -m modelOne -do run -dbget.RunId "1234"

To get only one parameter or one output table use -parameter or -table option:

dbget -m modelOne -dbget.LastRun -parameter ageSex
dbget -m modelOne -dbget.LastRun -table salarySex

It may be convinient for you to use Modgen compatibilty views to get output from openM++ model databse compatible with old Modgen tools. Important: We are trying to mimic old Modgen database views as close as possible, however it is never be identical.* To get Modgen comaptible model metadata:

dbget -m modelOne -do old-model
dbget -m modelOne -do old-model -csv
dbget -m modelOne -do old-model -tsv
dbget -m modelOne -do old-model -json
dbget -m modelOne -do old-model -pipe

Get model run parameters and output tables values from Modgen compatibility views:

dbget -m modelOne -do old-run
dbget -m modelOne -do old-run -csv
dbget -m modelOne -do old-run -tsv
dbget -m modelOne -do old-run -lang fr-CA
dbget -m modelOne -do old-run -dbget.NoLanguage
dbget -m modelOne -do old-run -pipe
dbget -m modelOne -do old-run -dir my/dir
dbget -m modelOne -do old-run -dbget.NoZeroCsv
dbget -m modelOne -do old-run -dbget.NoNullCsv

Get parameter run values from compatibility (Modgen) views:

dbget -m modelOne -do old-parameter -dbget.Parameter ageSex
dbget -m modelOne -do old-parameter -dbget.Parameter ageSex -csv
dbget -m modelOne -do old-parameter -dbget.Parameter ageSex -tsv
dbget -m modelOne -do old-parameter -dbget.Parameter ageSex -lang fr-CA
dbget -m modelOne -do old-parameter -dbget.Parameter ageSex -dbget.NoLanguage
dbget -m modelOne -do old-parameter -dbget.Parameter ageSex -pipe

Get output table values from compatibility (Modgen) views:

dbget -m modelOne -do old-table -dbget.Table salarySex
dbget -m modelOne -do old-table -dbget.Table salarySex -csv
dbget -m modelOne -do old-table -dbget.Table salarySex -tsv
dbget -m modelOne -do old-table -dbget.Table salarySex -lang fr-CA
dbget -m modelOne -do old-table -dbget.Table salarySex -dbget.NoLanguage
dbget -m modelOne -do old-table -dbget.Table salarySex -pipe
dbget -m modelOne -do old-table -dbget.Table salarySex -dbget.NoZeroCsv
dbget -m modelOne -do old-table -dbget.Table salarySex -dbget.NoNullCsv

There are more features avaliable in dbget, for example you can do microdata run comparison. And also additional functionality will be added because dbget is still under active development.

Use dbcopy: Export entire model into text files

dbcopy -m modelOne
dbcopy -m modelOne -dbcopy.Zip
dbcopy -m modelOne -dbcopy.NoAccumulatorsCsv
dbcopy -m modelOne -dbcopy.NoMicrodata

It will create modelOne directory and modelOne.Zip file with:

all model metadata (e.g. parameters, description, notes,...) in .json files
csv files with sets of model input parameters
csv files with model run results, input parameters and microdata

You can create .tsv output files instead of .tsv:

dbcopy -m modelOne -dbcopy.IntoTsv
dbcopy -m modelOne -dbcopy.IntoTsv -dbcopy.Zip

Model run microdata can be huge and if you are not intersted in it then use -dbcopy.NoMicrodata to suppress it:

dbcopy -m modelOne -dbcopy.NoMicrodata

For each model run output table openM++ store expression values (e.g. average, CV, SE) and also accumulators. Accumulators are sub-samples (a.k.a. sub-values or members or replicas, etc.) which used to produces output table aggregated expression value(s). By default dbcopy do output both: output table expressions and accumulators. If you are interestred only expression values then use -dbcopy.NoAccumulatorsCsv to suppress accumulators and get your results faster:

dbcopy -m modelOne -dbcopy.NoAccumulatorsCsv

Use dbcopy: Export entire model into csv files

dbcopy -m modelOne -dbcopy.To csv
dbcopy -m modelOne -dbcopy.To csv -dbcopy.Zip
dbcopy -m modelOne -dbcopy.To csv -dbcopy.NoAccumulatorsCsv

It will create modelOne directory and modelOne.Zip file with:

all model metadata (e.g. parameters, description, notes,...) in .csv files
csv files with sets of model input parameters
csv files with model run results and input parameters Each model run result and each input parameters set will be in separate sub-directory. Use -dbcopy.NoAccumulatorsCsv option to get your results faster by suppressing accumulators (a.k.a sub-sanples) output to CSV files.

You can create .tsv output files by using -dbcopy.IntoTsv option:

dbcopy -m modelOne -dbcopy.To csv -dbcopy.IntoTsv
dbcopy -m modelOne -dbcopy.To csv -dbcopy.IntoTsv -dbcopy.Zip

Other variation of csv or tsv output is:

dbcopy -m modelOne -dbcopy.To csv-all
dbcopy -m modelOne -dbcopy.To csv-all -dbcopy.IntoTsv

In that case all model runs will be in "all_model_runs" sub-directory and all input sets are in "all_input_sets".

You can suppress zero values and / or NULL (missing) values in output tables and microdata CSV files:

dbcopy -m modelOne -dbcopy.To csv -dbcopy.NoZeroCsv
dbcopy -m modelOne -dbcopy.To csv -dbcopy.NoNullCsv
dbcopy -m modelOne -dbcopy.To csv -dbcopy.NoNullCsv -dbcopy.NoZeroCsv

Use dbcopy: Export set of input parameters into text files

dbcopy -m modelOne -s modelOne_other -dbcopy.ParamDir pDir

It will create pDir directory with:

input parameters set metadata (name, description, notes,...) in .json file
csv files with sets of model input parameters

Use dbcopy: Export model run results into text files

dbcopy -m modelOne -dbcopy.LastRun
dbcopy -m modelOne -dbcopy.RunId 101
dbcopy -m modelOne -dbcopy.RunName modelOne_2016_11_22_11_38_49_0945_101
dbcopy -m modelOne -dbcopy.LastRun -dbcopy.NoAccumulatorsCsv

It will create a directory with:

model run metadata (name, description, notes,...) in .json file
csv files with input parameters used to run the model
csv files with model output tables values Use -dbcopy.NoAccumulatorsCsv option to get your results faster by suppressing accumulators (a.k.a sub-sanples) output to CSV files.

Use dbcopy: Import parameters from csv files into database

dbcopy -m myModel -s MyInput -dbcopy.ParamDir Any -dbcopy.ToSqlite myModel.sqlite

If any ParameterName.csv file(s) exist in directory Any then it will be loaded into MyInput set of input parameters.

It is recommended to run dbcopy -m modelOne -s modelOne_other -dbcopy.ParamDir Any to get familiar how csv files look like.

Use dbcopy: Import parameters, description and notes from text files into database

dbcopy -m myModel -s MyInput -dbcopy.ToSqlite myModel.sqlite

It will insert or update MyInput set of input parameters in database with:

if json metadata file exist then input set description, notes and parameter value note updated
if any ParameterName.csv files exist then it will be loaded into database

It is recommended to run dbcopy -m modelOne -s modelOne_other -dbcopy.ParamDir Any to get familiar how json and csv files look like.

Example of json metadata file for "ageSexData" input set of parameters with description, notes and ageSex parameter value notes:

{
  "ModelName" : "modelOne",
  "Name" : "ageSexData",
  "Txt" : [{
      "LangCode" : "EN",
      "Descr" : "Model One set of parameters"
    }
  ],
  "Param" : [{
      "Name" : "ageSex",
      "SubCount" : 1,
      "Txt" : [{
          "LangCode" : "EN",
          "Note" : "Age by Sex values"
        }
      ]
    }
  ]
}

It is also must exist csv file with parameter values: ageSex.csv

Example of json metadata file for "emptyData" input set of parameters with description and notes in English and French:

{
  "ModelName" : "modelOne",
  "Name" : "emptyData",
  "Txt" : [{
      "LangCode" : "EN",
      "Descr" : "Model One set of parameters",
      "Note" : "Notes for model One set of parameters"
    }, {
      "LangCode" : "FR",
      "Descr" : "Je suis désolé je ne parle pas français"
    }
  ]
}