Skip to main content

Table 2 Statistical performance of selected and alternative statistical impact prediction models

From: Malaria intervention scale-up in Africa: effectiveness predictions for health programme planning tools, based on dynamic transmission modelling

Statistical model

Metric

Health burden outcome, and time period from intervention start

Case incidence 0–4 years, years 1–3

Case incidence 0–4 years, years 4–6

Malaria mortality 0–4 years, years 8–10

Malaria mortality 15+ years, years 8–10

Coefficient of variation, i.e. ratio of standard deviation of the simulated distribution to the mean (does not depend on the statistical model)

 

125 %

129 %

138 %

279 %

Variance in simulated outcomes (does not depend on statistical model)

 

1.1

1.2

1.04e−4

3.1e−6

Selected (best) model: simulated 0 values imputed and remaining results re-scaled in the range 0–0.99

MSE, from out-of-sample predictiona

7.5 %

17.0 %

43.4 %

73.3 %

Adjusted R2

96.5 %

92.7 %

90.3 %

74.1 %

MSE as  % of simulated variance

12.6 %

17.9 %

52.0 %

73.9 %

Simulated 0 values dropped and remaining results re-scaled in the range 0–0.99

Adjusted R2

96.6 %

92.2 %

86.8 %

70.1 %

MSE as  % of simulated variance

19.3 %

28.4 %

72.2 %

75.0 %

Dropping EIR and model variant (the two variables with no country data)

Adjusted R2

88.4 %

80.4 %

72.7 %

42.6 %

MSE as  % of simulated variance

75.1 %

406 %

142 %

89.8 %

Log-transformation instead of logit-transformation of health outcomes

Adjusted R2

96.0 %

93.0 %

90.6 %

77.1 %

MSE as  % of simulated variance

37.9 %

324 %

405 %

75.3 %

  1. aAverage of 25 simulations in which sub-samples of 100,000 simulations were randomly drawn to train and select the statistical model, and each time the remaining 65,888 simulations were used to assess its MSE