model run options and ini file - openmpp/openmpp.github.io GitHub Wiki
There are many options which control model run, i.e.: number of cases, random generator starting seed, etc. OpenM++ model gets run options in following order:
- as command line arguments
- from ini-file (similar to Modgen .sce file)
- from database
profile_optiontables - use default values
Each option has unique key string associated with it, i.e. "Parameter.StartingSeed" is model input parameter "StartingSeed", which is most likely, random generator starting seed. You can use this key to specify model parameter on command line, in ini-file or database. For example:
modelOne.exe -Parameter.StartingSeed 123 -ini small.ini
would run "modelOne" model with starting seed = 123 and other options from small.ini file.
Note: We recommend to use normal Windows command line cmd.exe. If you are using Windows PowerShell then it may be necessary to put "quotes" around command line options, e.g:
modelOne.exe "-Parameter.StartingSeed" 123 "-ini" "small.ini"
Typically we are using SQLite database files to run the model and in that case you just specifiy a path to your sqlite file:
modelOne.exe -db C:\My-Model\m1.sqlite
It is often SQLite database file is in the same directory as model.exe file and file name is ModelName.sqlite, you can run model as:
my/model/dir/model.exe -OpenM.SqliteFromBin
If database connection string is not specified then model try to open SQLite database OM_MODEL_NAME.sqlite (i.e.: modelOne.sqlite) in current working directory. Default database connection string is:
Database=OM_MODEL_NAME.sqlite; Timeout=86400; OpenMode=ReadWrite;
Please notice, Linux file names are case sensitive and modelOne.sqlite is different from ModelOne.sqlite.
You can specify database connection string as command line argument, i.e.:
modelOne.exe -OpenM.Database "Database=C:\My Model\m1.sqlite; Timeout=86400; OpenMode=ReadWrite;"
Or, more convenient, by using ini-file
modelOne.exe -ini C:\MyModel\small.ini
Following parameters allowed for SQLite database connection:
- Database - (required) database file name or URI, file name can be empty
- Timeout - (optional) table lock "busy" timeout in seconds, default=0
- OpenMode - (optional) database file open mode: ReadOnly, ReadWrite, Create, default=ReadOnly
- DeleteExisting - (optional) if true then delete existing database file, default: false
Please notice: to run the model you need OpenMode=ReadWrite.
Model developer can pass an arbitrary run option from ini-file and use it to debug model code. In order to do that model should be started with following command line arguments:
model.exe -ini some.ini -OpenM.IniAnyKey
Or any of equivalent formats:
model.exe -ini some.ini -OpenM.IniAnyKey true
model.exe -OpenM.IniFile some.ini -OpenM.IniAnyKey true
model.exe -OpenM.IniFile some.ini -OpenM.IniAnyKey 1
model.exe -OpenM.IniFile some.ini -OpenM.IniAnyKey yes
Special boolean option -OpenM.IniAnyKey true allow to pass any key and values to model development code from ini-file.
For example, you can process following ini-file development options:
[MyTest]
ShowReport = yes ; true if: "yes", "1", "true" or empty value, false if missing
ReportStyle = readable ; string option
MinimumTime = 1234.56 ; double value, use as default: -inf
LineCount = 4321 ; integer option
EntityId = 1234567890123456789 ; long long integer
SelectedNames = e1,e2,e3 ; comma separated list of event namesby including code below into ompp_framework.ompp:
// process development model run options from model ini-file
void ProcessDevelopmentOptions(const IRunOptions * const i_options)
{
using namespace std;
bool isShowReport = i_options->boolOption("MyTest.ShowReport");
string rptStyle = i_options->strOption("MyTest.ReportStyle");
double minTime = i_options->doubleOption("MyTest.MinimumTime", -numeric_limits<double>::infinity());
int lineCount = i_options->intOption("MyTest.LineCount", 0);
long long entityId = i_options->longOption("MyTest.EntityId", 0);
// option is a list of comma separated names
list<string> evtList = openm::splitCsv(i_options->strOption("MyTest.SelectedNames"));
// if option is not specified at all
if (!i_options->isOptionExist("MyTest.ShowReport")) {
// do something
}
// get a copy of all model run options, including openM++ standard options
vector<pair<string, string>> allOpts = i_options->allOptions();
// each option is a pair of key and value
for (const auto & opt : allOpts) {
// string key = opt.first;
// string value = opt.second;
}
}Important:
Model development options should not be used as model parameters and should not affect modeling results. It is strictly for debugging and development purpose. OpenM++ does not provide any guarantee about model development options.
To specify name of ini-file you can use -s or -ini or -OpenM.IniFile command line option. Please see OpenM++ ini-file format
to find out more about ini-file structure supported by openM++.
Example of model ini-file:
; Lines started with ; semicolon are just a comments
# Lines started with # hash are just a comments
;#============================================================================
;#
;# model parameters
;# any scalar model parameter can be specified in [Parameter] section
;# or as command line argument or in profile_option table
;
[Parameter]
;# random seed value
;
; StartingSeed = 16807
;# base salary is classification parameter
;# using enum code "Full" to specify parameter value
;# if [OpenM]IdParameterValue=true (see below) then we must use baseSalary=22 instead
;
; baseSalary = Full
;#============================================================================
;#
;# openM++ run options
;#
;# OpenM++ boolean options:
;# True value is any of: "yes", "1", "true" or empty value
;# False value is any of: "no" "0", "false"
;# Boolean values are not case sensitive, e.g.: "yes" == "YES" and it is a true value
;
[OpenM]
;# number of sub-values, default: 1
;
; SubValues = 16
;# max number of modeling threads, default: 1
;#
;# if number of sub-values per process < number of modeling threads then sub-values run sequentially.
;# if more threads specified then sub-values run in parallel.
;#
;# for example:
;# model.exe -OpenM.SubValues 8
;# model.exe -OpenM.SubValues 8 -OpenM.Threads 4
;# mpiexec -n 2 model.exe -OpenM.SubValues 31 -OpenM.Threads 7
;
; Threads = 4
;# if NotOnRoot is true then do not use "root" process for modeling
;# default value: false
;# empty value: true
;#
;# this option can be used only if multiple model.exe processes are running in parallel
;# otherwise it has no effect.
;#
;# for example:
;# (a) mpiexec -n 4 model.exe -OpenM.SubValues 16
;# (b) mpiexec -n 4 model.exe -OpenM.SubValues 16 -OpenM.NotOnRoot true
;# both commands above do launch four model.exe processes
;# but in second case only three children are doing modeling
;# and root process dedicated to run controlling activity
;
; NotOnRoot = false
;# database connection string
;# default database name: ModelName.sqlite
;
; Database = "Database=ModelName.sqlite; Timeout=86400; OpenMode=ReadWrite;"
;# path to SQLite database file
;#
;# If Database option (see above) specified then this SQLite option has no effect
;# Database option has higher priority over this Sqlite option.
;
; Sqlite = /path/to/my-model.sqlite
;# if SqliteFromBin is true the use model SQLite database file located next to model.exe
;# model database file path: directory/of/model/exe/ModelName.sqlite
;#
;# If any of Database or SQLite options (see above) specified then this SqliteFromBin option has no effect
;# Database and SQLite option has higher priority over this Sqlite option.
;
; SqliteFromBin = false
;# name of model run results
;# if not specified then automatically generated
;
; RunName = my-default-scenario
;# set id is an id of input set of model parameters
;#
;# default: min(set id)
;
; SetId = 101
;# set name is name of input set to get model parameters
;# if set name specified then it used to find set of model input parameters
;# if SetId option specified then SetName is ignored
;
; SetName = Default
;# if specified then use parameters from base run instead of input set
;# find base run by run id
;
; BaseRunId = 1234
;# if specified then use parameters from base run instead of input set
;# if BaseRunId option NOT specified then find base run by run digest
;
; BaseRunDigest = 6866f742cabab735ced1577c56b23e93
;# if specified then use parameters from base run instead of input set
;# if BaseRunId and BaseRunDigest options are NOT specified then find base run by run name
;# run name is not unique and as result it will be a first model run with that name
;
; BaseRunName = My_Model_Run
;# run id to restart model run (i.e. after power failure)
;
; RestartRunId =
;# task id is an id of modeling task
;# if modeling task id specified then
;# model will run all input sets included into that modeling task
;
; TaskId = 1
;# task name is name of modeling task
;# if task name specified then it used to get task id
;# if task id specified then set name is ignored
;
; TaskName = taskOne
;# task run name is name of modeling task run
;# if not specified then automatically generated
;
; TaskRunName = run-first-task-with-16-sub-values
;# task "wait":
;# default value: false
;# empty value: true
;#
;# allow to dynamically append new input data into modeling task
;# modeling task not completed automatically
;# it is waiting until some external script signal:
;# UPDATE task_run_lst SET status = 'p' WHERE task_run_id = 1234;
;
; TaskWait = false
;# profile name to select run options from profile_option database table
;
; Profile = modelOne
;# convert to string format for float, double, long double, default: %.15g
;
; DoubleFormat = %.15g
;# path to parameters csv file(s) directory
;# if specified then for each parameter where exist param/dir/parameterName.csv
;# values from csv file are used to run the model
;
; ParamDir = ./csv
;# if true then parameter(s) csv file(s) contain enum id's, default: enum code
;# default value: false
;# empty value: true
;
; IdCsv = false
;# value of scalar parameter(s) can be specified in [Parameter] section (see above)
;# or as command line argument -Parameter.Name of model.exe
;#
;# if IdParameterValue is true
;# then scalar parameter(s) value is enum id's, default: enum code
;# default value: false
;# empty value: true
;
; IdParameterValue = false
;# if true then use sparse output to database, default: false
;# default value: false
;# empty value: true
;
; SparseOutput = false
;# if use sparse and abs(value) <= SparseNullValue then value not stored
;# default = FLT_MIN
;
; SparseNullValue = 1.0E-37
;# if positive then used to report percent completed of simulation, default: 1
;
; ProgressPercent = 1
;# if positive then used to report simulation progress, default: 0
;# for case based models it is number of cases completed and must integer value
;# for time based models it is time passed from first event and must positive value, e.g.: 0.1
;
; ProgressStep = 1000
;# language to display output messages
;# default: set in Windows Control Panel or by Linux LANG
;
; MessageLanguage = en-CA
;# process run stamp, default: log file time stamp
;# use it to find model run(s) in run_lst table
;# or model task run in task_run_lst table
;
; RunStamp = 2012_08_17_16_04_59_148
;# log settings:
;# log can be enabled/disabled for 3 independent streams:
;# console - standard output
;# "last run" file - log file with specified name, overwritten on every model run
;# "stamped" file - log file with unique name, created for every model run
;#
;# "stamped" name produced from "last run" name by adding time-stamp and/or pid-stamp, i.e.:
;# modelOne.log => modelOne.2012_08_17_16_04_59_148.987654.log
;
; LogToConsole = true ; log to console, default: true
; LogToFile = true ; log to file, default: true
; LogToStampedFile = false ; log to "stamped" file
; LogUseTimeStamp = false ; use time-stamp in log "stamped" file name
; LogUsePidStamp = false ; use pid-stamp in log "stamped" file name
; LogFilePath = model.log ; log file path, default = current/dir/modelExeName.log
; LogNoMsgTime = false ; if true then do not prefix log messages with date-time
; LogRank = false ; if true then prefix log messages with MPI process rank
; LogSql = false ; debug only: log sql statements
;# trace settings:
;# trace can be enabled/disabled for 3 independent streams:
;# console - cout stream
;# "last run" file - trace file with specified name, overwritten on every model run
;# "stamped" file - trace file with unique name, created for every model run
;#
;# "stamped" name produced from "last run" name by adding time-stamp and/or pid-stamp, i.e.:
;# trace.txt => trace.2012_08_17_16_04_59_148.987654.txt
;#
;# If trace to file is enabled
;# then existing "last run" trace file is overwritten even if model does not write anything to trace output
;
; TraceToConsole = false ; trace to console, default false
; TraceToFile = false ; trace to file
; TraceToStampedFile = false ; trace to "stamped" file
; TraceFilePath = trace.txt ; trace file path, default: current/dir/modelExeName.trace.txt
; TraceUseTimeStamp = false ; use time-stamp in trace "stamped" file name
; TraceUsePidStamp = false ; use pid-stamp in trace "stamped" file name
; TraceNoMsgTime = true ; if true then do not prefix trace messages with date-time
; TraceRank = false ; if true then prefix trace messages with MPI process rank
;#============================================================================
;#
;# language-specific options
;#
[EN]
;#
;# model run description in English
;
; RunDescription = model run with 50,000 cases
;#
;# path to file with model run notes in English
;
; RunNotesPath = run_notes-in-english.md
;#
;# run entity description in English
;
; Person--EntityDescription = base Person entities
;#
;# path to file with entity run notes in English
;
; Person--EntityNotesPath = entity-run_notes-in-english.md
[FR]
;#
;# model run description in French
;
; RunDescription = je suis désolé je ne parle pas français
;#
;# path to file with model run notes in French
;
; RunNotesPath = run_notes-fr-français.md
;#============================================================================
;#
;# Ouput tables suppression.
;#
;# It can be in one of the two forms:
;# Suppress = ATable,BTable,Group1,Group2
;# Or:
;# Retain = ATable,BTable,Group1,Group2
;#
;# Suppress and Retain options are mutually exclusive and cannot be mixed.
;# For example, this model run would fail:
;# model.exe -Suppress.A -Retain.B
[Tables]
;#
;# Suppress output table "ageSexIncome"
;# and suppress group of output tables "AdditionalTables"
;
; Suppress = ageSexIncome,AdditionalTables
;# Or suppress all output tables
;# except of "ageSexIncome" table and tables included into "AdditionalTables" group:
;
; Retain = ageSexIncome,AdditionalTables
;#============================================================================
;#
;# where to find sub-values for model parameter or group of parameters: db, csv, iota
;
[SubFrom]
;# where to find sub-values for parameter "Member"
;# "iota" means create parameter "Member" sub-values as 0,1,...[OpenM].SubValues-1
;
; Member = iota
;# where to find sub-values for "baseSalary" parameter
;# "db" means read sub-values from input set (read from model database)
;# modelOne default input set has 4 sub-values for "baseSalary" and "salaryFull"
;
; baseSalary = db
;# where to find sub-values for "salaryFull" parameter
;# "csv" means read all sub-values from parameter.csv file
;# by default only one sub-value read from csv file
;
; salaryFull = csv
;# sub-value for all members of "age_sex_parameters" group coming from .csv files:
;#
;# age_sex_parameters = csv
;#
;# it is the same as:
;# -SubFrom.ageSex csv -SubFrom.salaryAge csv
;# because this group consist of: "ageSex" and "salaryAge"
;#============================================================================
;#
;# how many sub-values to select for parameter and which sub id to select
;# it is also can be applied to the parameters group
;#
;# SubValues option can be:
;# range: SubValues.Age [1,4]
;# list of id's: SubValues.Age 2,1,4,3
;# bit mask: SubValues.Age x0F
;# single id: SubValues.Age 7
;# default id: SubValues.Age default
;#
;# if you running:
;# model.exe -OpenM.SubValues 4 -SubFrom.Age csv
;# then Age.csv file must have at least 4 sub values with sub id's 0,1,2,3
;#
;# to use only one single sub-value either specify "default" id:
;# model.exe -OpenM.SubValues 4 -SubFrom.Age db -SubValues.Age default
;# or explicit sub-value id:
;# model.exe -OpenM.SubValues 4 -SubFrom.Age csv -SubValues.Age 7
;#
;# to select 4 sub-values use [first,last] range or comma-separated list or bit mask:
;# model.exe -OpenM.SubValues 4 -SubFrom.Age csv -SubValues.Age [4,7]
;# model.exe -OpenM.SubValues 4 -SubFrom.Age csv -SubValues.Age 4,5,6,7
;# model.exe -OpenM.SubValues 4 -SubFrom.Age csv -SubValues.Age xF0
;#
[SubValues]
; baseSalary = default
; isOldAge = 4,2,1,3
;# use sub-values 2 and 3 for all members of "age_sex_parameters" group:
;
; age_sex_parameters = 2,3
;
;# it is the same as:
;# -SubValues.ageSex 2,3 -SubValues.salaryAge 2,3
;# because this group consist of: "ageSex" and "salaryAge"
;#============================================================================
;#
;# import model parameters from other model(s)
;
[Import]
;# if "All" is true then import all parameters (all parameters which has import statement).
;# default value: false
;# empty value: true
;
; All = true
;
;# for each upstream model last succesful run is used to import parameters
;#
;# if "ModelName" is true then import all parameters from upstream "ModelName".
;# default value: false
;# empty value: true
;# Example:
;# import parameters from last succesful run of upstream model "RedModel"
;
; RedModel = true
;#============================================================================
;#
;# import model parameters from run specified by run digest
;#
[ImportRunDigest]
;# Example:
;# import parameters from upstream model "RedModel" where run digest = abcdefghef12345678
;
; RedModel = abcdefghef12345678
;#============================================================================
;#
;# import model parameters from run specified by run id
;
[ImportRunId]
;# Example:
;# import parameters from upstream model "RedModel" where run id = 101
;
; RedModel = 101
;#============================================================================
;#
;# import model parameters from last sucessful run with specified run name
;
[ImportRunName]
;# Example:
;# import parameters from last successful run of upstream model "RedModel" where run name = GoodRun
;
; RedModel = GoodRun
;#============================================================================
;#
;# import model parameters from last sucessful run of model with specified digest
;
[ImportDigest]
;# Example:
;# import parameters from last successful run of upstream model "RedModel" where model digest = 87654321fedcba
;
; RedModel = 87654321fedcba
;#============================================================================
;#
;# import model parameters from last sucessful run of model with specified id
;
[ImportId]
;# Example:
;# import parameters from last successful run of upstream model "RedModel" where model id = 123
;
; RedModel = 123
;#============================================================================
;#
;# import model parameter from specified expression of output table
;
[ImportExpr]
;# If upstream output table has multiple measure values (multiple expressions)
;# the by default first expression of output table is used to import parameter value.
;# To override default measure name (expression name) can be explicitly specified.
;#
;# Example:
;# import parameter from AgeTable of upstream model "RedModel" using "expr2" value as parameter values
;
; AgeTable = expr2
;#============================================================================
;#
;# import model parameter from specified model database
;
[ImportDatabase]
;# By default upstream model imported from the same database as current (downstream) model
;# or, if not exist there then from defalut SQLite database with name ModelName.sqlite
;# Use connection string to override default database rule.
;#
;# Example:
;# import parameters from upstream model "RedModel" in database ../RedHot.sqlite
;
; RedModel = "Database=../RedHot.sqlite;OpenMode=RedaOnly;"
;#============================================================================
;#
;# model development options
;#
;# Are available for processing in model code only if model.exe started with command line options:
;#
;# model.exe -ini iniFileName.ini -OpenM.IniAnyKey
;#
;# Or:
;#
;# model.exe -ini iniFileName.ini -OpenM.IniAnyKey 1
;# model.exe -ini iniFileName.ini -OpenM.IniAnyKey yes
;# model.exe -ini iniFileName.ini -OpenM.IniAnyKey true
;#
;# OpenM++ boolean options:
;# True value is any of: "yes", "1", "true" or empty value
;# False value is any of: "no" "0", "false"
;# Boolean values are not case sensitive, e.g.: "yes" == "YES" and it is a true value
;#
;# For example of model development option processing, see function ProcessDevelopmentOptions
;# in OM_ROOT/NewCaseBased/code/ompp_framework.ompp
;
; [LargeOutput]
;
; incomeByYear = true ; 4824 * 4 expression cells
; incomeByLow = true ; 48240 * 4 expression cells
; incomeByMiddle = true ; 144720 * 4 expression cells
; incomeByPeriod = true ; 969624 * 4 expression cells
;
;#============================================================================
;#
;# event trace model development options
;#
;# Requires activation of model development options using -OpenM.IniAnyKey (see above).
;# Requires that model code contains the statement
;# options event_trace = on;
;# Requires OpenM.TraceToFile = true
;# you can enable TraceToFile in section [OpenM] above, e.g.:
;# [OpenM]
;# TraceToFile = true
;#
;# See wiki for explanation of EventTrace options
;
; [EventTrace]
;
; format
;
; ReportStyle = readable ; "modgen", "readable", or "csv", default: modgen
; MaximumLines = 100000 ; integer value, default: 20000
; NameColumnWidth = 20 ; integer value, default: 40
;
; filters
;
; SelectedEntityKinds = e1,e2,e3 ; comma separated list of entity kinds, if empty all entity kinds
; SelectedEntities = 1,2,3 ; comma separated list of integers, if empty all entities
; SelectLinkedEntities = no ; default: no
; SelectedCaseSeeds = 1,2,3 ; comma separated list of case seeds, if empty all cases
; MinimumTime = 2025 ; double value, default: -inf
; MaximumTime = 2025 ; double value, default: +inf
; MinimumAge = 65 ; double value, default: -inf
; MaximumAge = 66 ; double value, default: +inf
;
; events
;
; ShowEnterSimulation = yes ; default: yes
; ShowExitSimulation = yes ; default: yes
; ShowEvents = yes ; default: yes
; SelectedEvents = e1,e2,e3 ; comma separated list of event names, if empty all events
; ShowQueuedEvents = no ; default: no
; ShowQueuedUnchanged = no ; default: no
; ShowSelfSchedulingEvents = no ; default: no
; ShowQueuedSelfSchedulingEvents = no ; default: no
;
; attributes
;
; ShowAttributes = no ; default: no
; SelectedAttributes = year,alive ; comma separated list of attribute names, if empty no attributes
; MinimumAttribute = 1 ; double value, default: -inf
; MaximumAttribute = 1 ; double value, default: +inf
;
;#============================================================================
;#
;# Model run microdata: entity name and attributes to store at each model run
;#
;# run time list of attributes can include less attributes than entity have
;# for example, full list of Person attributes is:
;
; Person = age, ageGroup, sex, income, salary, salaryGroup, fullTime, isOldAge, pension
;
;# order of attributes is not important, it is defined by entity metadata and cannot be changed at run time
;
[Microdata]
; Person = ageGroup,sex,age,income,isOldAge,pension
; Store all non-internal attributes of Person entity
;
; Person = All
; Store all non-internal attributes of all entities
; NOT recommended for production, use for debug only
;
; All = true
; Allow to store entities internal attributes
; NOT recommended for production, use for debug only
;
; UseInternal = true
; Write microdata entity attributes into database
; Important: each microdata entity MUST have unique key
;
; ToDb = false
; Write microdata entity attributes and events (if enabled) into CSV file(s)
; each microdata entity is written in it's own file
;
; ToCsv = false
; Directory where microdata CSV file(s) should be created, must be existing directory
; default value: current directory
;
; CsvDir = path/to/some/directory
; Write microdata entity(s) attributes and events (if enabled) into model Trace output
; Trace output must be enabled to produce any results;
; see Trace options in [OpenM] section above
;
; ToTrace = false
; Write selected events into Trace or CSV file
;
; Events = Birth,Union,Death
; If true then write event name into CSV file
;
; CsvEventColumn = true