Markov Models - zward/Amua GitHub Wiki

Node Types

Markov models are comprised of 5 types of nodes:

  • Decision node Decision nodes define decision points in the model. These nodes indicate choices that are under your control.
  • Chance node Chance nodes define probability events in the model.
  • Markov Chain A Markov Chain contains a set of discrete Markov states and the transitions between them.
  • Markov State Markov States are a set of mutually exclusive, completely exhaustive discrete states in the model.
  • State Transition State Transitions define the allowable transitions between Markov States.

Markov Trace

The current model cycle is represented by the variable t. The model begins at cycle 0 and all transitions are assumed to happen at the end of each cycle.

The Markov Trace records the prevalence of each state and rewards/counters for each cycle and can be referenced by the variable trace[cycle,column] where cycle is the model cycle and column is either the column index (integer) or column name (string) in the trace.

The value of each variable is also recorded in the trace each cycle. In Monte Carlo Markov models the expected value (i.e. mean across individuals) of each variable is recorded.

Node Attributes

  • Name: Each node can be given a name. Markov State names must be unique within the Markov Chain .
  • Probability: Chance nodes must have probabilities that sum to 1.0. Markov States must have initial probabilities (i.e. starting prevalence) that sum to 1.0. The letter c or C can be entered to indicate a complementary probability (i.e. sums to 1.0).
  • Rewards: Cycle rewards for each Markov State can be defined for each dimension in the model. Rewards that vary by cycle can be defined using a table or other math expression. A half-cycle correction and/or discounting can also be applied in Markov Properties.
  • State Transition: The next Markov State must be entered for each transition node .
  • Termination Condition: A termination condition must be specified for each Markov Chain . The chain runs until the condition evaluates to true.
  • Variable Updates: Variable update expressions can be added to nodes by clicking the Add/Remove Variable Update button on the toolbar or pop-up menu. See variables for more information.
  • Costs: Costs (or transition rewards) can be added to nodes by clicking the cost button on the toolbar or pop-up menu.
  • Notes: More information (ex: notes, citations/sources, etc.) about nodes can be entered in the notes pane at the bottom of the screen.

Markov Properties

See model properties