Multistate Markov Models

  • Method

One of the goals of Ki is to explain the probability of growth faltering, as well as the degree of recovery that can be expected of children born small for gestation age (SGA). Multistate models are often used to model the progression of disease by describing the probability for an individual to transition from one disease state into another. Ki used multistate Markov models to describe the probability of transitioning between three growth states from birth to 2 years of age: stunted, at-risk for stunting, and not stunted where the outcome is length by age z-score.


Multistate models are defined by states (represented by the boxes in Figure 1) and transitions between them (represented by arrows).

States can be transient, in which individuals can enter and exit, or absorbing, in which individuals never exit once they enter (e.g. dead).

The simplest model contains two states, for example, alive and dead or healthy and ill, and is commonly called the mortality model. This type of model can be analyzed using survival analysis methods.[1, 2]

FIGURE 1. Common multistate models used in biostatistics (A) is the illness–death mode, sometimes called the disability model, (B) is the progressive, staged disease model, and (C) is the competing risk model.

Multistate models enable the analysis of longitudinal data in which individuals may experience more than one health event.

Multistate Markov models can be used to estimate the transition hazard (the instantaneous risk of transitioning from one state into another), as well as transition probabilities and the mean sojourn time in a given state.

Markov models are used most frequently due to their simplicity. A multistate model is considered Markov if it assumes that the probability of transitioning to a new state depends only on the current value of the model.[1, 3]

In general, a random process can be described as a Markov if it determines future probabilities solely based on its current values.

This means that the past, current, and future states of the system are all independent of one another. For this reason, Markov processes are sometimes described as “memoryless.”[1, 3]

Advantages of multistate Markov models

  • Allows easy understanding ‘centile crossing’ probabilities. For example, we can calculate the probability of crossing the 90th percentile for LAZ.
  • Can allow for movement between states in both directions (except for absorbing states, like death). This allows the same model to estimate the hazard rate of recovery as well as illness.

Disadvantages of multistate Markov models

  • Requires categorization of states. For example, in modeling stunting based on a continuous length-for-age z-score (LAZ) variable, it requires binning the continuous variable into discrete categories.



Model to describe how probability of transitioning between three states—defined as stunted, at-risk for stunting, and not stunted—changes from birth to two years of age.

This models LAZ outcomes, categorized into 4 bins.


  1. Meira-Machado L, de Una-Alvarez J, Cadarso-Suarez C, Andersen PK. Multi-state models for the analysis of time-to-event data. Stat Methods Med Res. 2009;18(2):195-222.
  2. Jackson CH, Sharples LD, Thompson SG, Duffy SW, Couto E. Multistate Markov models for disease progression with classification error. Journal of the Royal Statistical Society: Series D (The Statistician). 2003;52(2): 193-209.
  3. Hougaard P. Multi-state Models: A Review. An International Journal devoted to Statistical Methods and Applications for Time-to-Event Data. 1999;5(3):239-64.
  4. Ledoux Sofeu C, Tejiokem MC, Protopopescu C, Ida Penda C, Ateba Ndongo F, Tetang Ndiang S, et al. Early treated HIV-infected children remain at risk of growth retardation during the first five years of live: Results from the ANRS-Pediacam cohort in Cameroon. 2017. p. S55-S6.

Last Updated

October, 2020