The authors have declared that no competing interests exist.
Conceived and designed the experiments: WLMR SH. Performed the experiments: WLMR SH. Analyzed the data: MvB. Contributed reagents/materials/analysis tools: MvB. Wrote the paper: MvB WLMR JW PDO SH.
Classical approaches to estimate vaccine efficacy are based on the assumption that a person's risk of infection does not depend on the infection status of others. This assumption is untenable for infectious disease data where such dependencies abound. We present a novel approach to estimating vaccine efficacy in a Bayesian framework using disease transmission models. The methodology is applied to outbreaks of mumps in primary schools in the Netherlands. The total study population consisted of 2,493 children in ten primary schools, of which 510 (20%) were known to have been infected, and 832 (33%) had unknown infection status. The apparent vaccination coverage ranged from 12% to 93%, and the apparent infection attack rate varied from 1% to 76%. Our analyses show that vaccination reduces the probability of infection per contact substantially but not perfectly (
Less than two decades ago, it was generally believed that in developed countries infectious diseases such as measles, mumps, and pertussis were under firm control via vaccination. Nowadays, it is increasingly recognized that this picture has been overly optimistic. A central question is whether recurrent disease outbreaks are caused by vaccination coverage having dropped below safe levels, or by vaccines having become less effective. To answer this question, the authors study outbreaks of mumps in primary schools in the Netherlands. Using disease transmission models, the authors estimate vaccine efficacy and the critical vaccination coverage needed to prevent large outbreaks. The analyses show that the vaccine has been highly effective in preventing infection, but that vaccination coverage has been insufficient in some schools. The authors argue that catch-up vaccination campaigns aimed at populations with intermediate vaccination coverage will be most efficient, as these would maximize the (direct and indirect) benefits of vaccination.
Mass vaccination programs for childhood diseases have been highly successful in reducing the incidence and public health impact of the targeted diseases. Nevertheless, with the exception of smallpox, eradication has not been achieved, and outbreaks continue to occur even in highly vaccinated populations
In the Netherlands, large outbreaks of mumps genotypes D and G have occurred in recent years
To determine whether the outbreaks of mumps are the result of low vaccination coverage or insufficient protection conferred by the vaccine, we estimate vaccine efficacy using outbreak data from ten primary schools in the Netherlands
number of persons | number infected | vaccination coverage | attack rate in unvaccinated persons | attack rate in vaccinated persons | overall attack rate | |||
all schools | 2493 | 510–1342 | 0.62 |
0.68 | (485/709) | 0.03 | (25/952) | 0.31 |
school 1 | 432 | 205–369 | 0.12 | 0.86 | (204/237) | 0.03 | (1/31) | 0.76 |
school 2 | 338 | 135–289 | 0.13 | 0.82 | (131/160) | 0.17 | (4/24) | 0.73 |
school 3 | 259 | 68–159 | 0.42 | 0.72 | (68/94) | 0 | (0/74) | 0.40 |
school 4 | 184 | 40–70 | 0.54 | 0.53 | (37/70) | 0.04 | (3/84) | 0.26 |
school 5 | 130 | 13–33 | 0.75 | 0.46 | (13/28) | 0 | (0/82) | 0.12 |
school 6 | 263 | 28–171 | 0.76 | 0.70 | (19/27) | 0.10 | (9/93) | 0.23 |
school 7 | 194 | 6–43 | 0.78 | 0.19 | (6/31) | 0 | (0/126) | 0.04 |
school 8 | 227 | 3–27 | 0.79 | 0.05 | (2/41) | 0.01 | (1/162) | 0.01 |
school 9 | 258 | 6–119 | 0.93 | 0.18 | (2/11) | 0.03 | (4/134) | 0.04 |
school 10 | 208 | 6–62 | 0.93 | 0.30 | (3/10) | 0.02 | (3/142) | 0.04 |
The column ‘number infected’ shows the possible range of actual infections, ranging from the number known to be infected to the sum of this number and the number of persons with unknown infection status. Vaccination coverages and attack rates are calculated using persons with known vaccination status (vaccination coverage), and known vaccination and infection status (attack rates). See
: averaged over schools.
Classical methods to estimate vaccine efficacy from outbreak data compare the infection attack rates in the vaccinated versus unvaccinated groups (i.e. the cohort method)
To take account of the dependencies between individuals that arise naturally in infectious disease outbreaks we base the statistical analyses on a Bayesian inferential framework using infectious disease transmission models. In this framework, missing vaccination and infection information is imputed in a consistent manner, thereby making efficient use of the available information, and enabling precise estimation of vaccine efficacy and the critical vaccination coverage needed to prevent epidemic outbreaks
The analyses reveal that mumps vaccine effectively prevents infection, and that herd immunity against mumps is achieved with moderate vaccination coverages. We argue that resource-limited catch-up vaccination efforts should be focused at communities with intermediate vaccination coverages, thereby maximizing both the direct and indirect benefits of vaccination.
Our baseline scenario assumes a common transmissibility and vaccine efficacy across schools. The analysis indicates that mumps is moderately transmissible (
The median of the basic reproduction number is 2.49 (95% CrI: 2.36–2.63) and the median of vaccine efficacy is 0.933 (95CrI: 0.908–0.954). The estimated critical vaccination coverage is 0.642 (95% CrI: 0.617–0.666).
unvaccinated | vaccinated | vaccination coverage | attack rate | |||
total | infected | total | infected | |||
school 1 | 382 (372–389) | 327 (315–337) | 50 (43–59) | 3 (1–6) | 0.12 (0.10–0.14) | 0.77 (0.74–0.79) |
school 2 | 293 (286–301) | 243 (232–256) | 45 (37–52) | 6 (4–11) | 0.13 (0.11–0.15) | 0.74 (0.70–0.78) |
school 3 | 150 (141–158) | 110 (101–119) | 109 (101–117) | 3 (0–8) | 0.42 (0.39–0.45) | 0.44 (0.41–0.47) |
school 4 | 84 (79–90) | 45 (41–50) | 100 (94–104) | 4 (3–6) | 0.53 (0.51–0.56) | 0.27 (0.24–0.30) |
school 5 | 33 (30–37) | 15 (13–18) | 97 (93–100) | 0 (0–2) | 0.75 (0.72–0.77) | 0.12 (0.10–0.15) |
school 6 | 67 (59–76) | 50 (42–59) | 196 (187–204) | 20 (14–27) | 0.75 (0.71–0.78) | 0.27 (0.23–0.30) |
school 7 | 42 (38–46) | 10 (7–13) | 152 (148–156) | 0 (0–3) | 0.78 (0.76–0.80) | 0.05 (0.04–0.07) |
school 8 | 48 (45–52) | 3 (2–5) | 179 (175–182) | 1 (1–2) | 0.79 (0.77–0.80) | 0.02 (0.01–0.03) |
school 9 | 18 (13–25) | 6 (3–11) | 240 (233–245) | 11 (7–16) | 0.93 (0.90–0.95) | 0.07 (0.05–0.09) |
school 10 | 13 (10–17) | 4 (3–7) | 195 (191–198) | 5 (3–8) | 0.94 (0.92–0.95) | 0.05 (0.03–0.06) |
Estimates are represented by posterior medians with 95% credible intervals.
We use estimates of transmissibility and vaccine efficacy to obtain estimates of the critical vaccination coverage. The analyses yield an estimated critical vaccination coverage of 0.642 (95%CrI: 0.617–0.666), indicating that herd immunity in the school setting can be obtained with moderate vaccination coverages. Estimates of transmissibility and vaccine efficacy are used to obtain an estimate of the number of infections prevented per vaccination. This number is highest for vaccination coverages just below the critical vaccination coverage, as at these values the slope of attack rate versus vaccination coverage is steepest (
The figure shows the medians of the posterior vaccination coverages versus posterior attack rates in the ten schools (blue dots), the deterministic final size attack rate using the posterior medians of the basic reproduction number and vaccine efficacy (dotted line), and the results of simulations in populations of size 200 using samples from the posterior distributions of the basic reproduction number and vaccine efficacy (black line: median; grey area: 2.5%–97.5% percentiles). See text for details.
Schools in our study population span a large range of possible vaccination coverages, and it is of interest to evaluate the consistency of the estimates of vaccine efficacy and pathogen transmissibility.
To investigate the information contained in the data by school we perform analyses in which each school is equipped with its own transmissibility and vaccine efficacy. It appears that precise estimates of transmissibility and vaccine efficacy can be obtained in schools with high attack rates (schools 1–4), but not in schools with only a handful of infections (schools 7–10). In fact, in schools with less than 10 confirmed infections credible intervals of the reproduction number range from well below 1 to more than 3, while vaccine efficacy estimates can range from less than 0.20 (schools 8–10) to almost 1 (schools 7–10;
Note that estimated vaccine efficacy is consistently high in schools with high exposure (schools 1–6), but cannot be estimated with any precision in schools with low exposure (schools 8–10).
basic reproduction number |
vaccine efficacy |
critical vaccination coverage |
|
school 1 | 2.5 (2.3–2.7) | 0.97 (0.91–1.0) | 0.63 (0.59–0.67) |
school 2 | 2.3 (2.1–2.5) | 0.87 (0.67–0.96) | 0.65 (0.57–0.80) |
school 3 | 2.8 (2.4–3.3) | 0.99 (0.96–1.0) | 0.66 (0.60–0.70) |
school 4 | 2.5 (1.9–3.2) | 0.94 (0.84–0.98) | 0.64 (0.51–0.74) |
school 5 | 3.6 (2.1–6.0) | 0.98 (0.89–1.0) | 0.74 (0.54–0.85) |
school 6 | 3.0 (2.2–3.9) | 0.88 (0.78–0.94) | 0.76 (0.66–0.83) |
school 7 | 1.9 (0.73–3.8) | 0.95 (0.67–1.0) | 0.51 (0–0.77) |
school 8 | 1.1 (0.25–3.5) | 0.67 (0.06–0.96) | 0.10 (0–1) |
school 9 | 0.96 (0.31–2.9) | 0.67 (0.08–0.92) | 0 (0–0.74) |
school 10 | 1.6 (0.40–4.5) | 0.80 (0.16–0.96) | 0.46 (0–0.85) |
The table shows the estimates of the basic reproduction number, vaccine efficacy, and critical vaccination coverage. Estimates are represented by posterior medians with 95% credible intervals.
In comparison with our estimates of vaccine efficacy as the reduction in the probability of infection (
vaccine efficacy |
|
school 1 | 0.93 (0.81–0.99) |
school 2 | 0.76 (0.56–0.92) |
school 3 | 0.98 (0.93–1.0) |
school 4 | 0.91 (0.80–0.98) |
school 5 | 0.97 (0.90–1.0) |
school 6 | 0.84 (0.73–0.93) |
school 7 | 0.96 (0.83–1.0) |
school 8 | 0.75 (0.0–0.98) |
school 9 | 0.78 (0.25–0.96) |
school 10 | 0.90 (0.68–0.98) |
Estimates are represented by posterior medians with 95% credible intervals. See text for details.
Our analyses have shown that mumps is moderately transmissible in the setting of primary schools, and that the vaccine used in these populations is highly effective in preventing infection. These results are largely in line with earlier studies
Estimates of the transmissbility of mumps are most precise in schools 1–4, i.e. in schools with low vaccination coverage and large numbers of infections. In these schools, the basic reproduction number is estimated at 2.5, 2.3, 2.8, and 2.5, with credible intervals ranging from 1.9 to 3.2. Vaccine efficacy, on the other hand, is estimated most precisely in schools 1, 3, 4, and 5 (
The schools included in this study differ greatly with respect to vaccination coverages (range: 12%–93%) and infection attack rates (range: 4%–76%). Nevertheless, estimates of vaccine efficacy are remarkably consistent across schools (
Schools in our study were included based on confirmed mumps infections. It is therefore possible that large outbreaks are more likely to be detected and included than small outbreaks. In other words, it is conceivable that the inclusion process systematically favours inclusion of schools with uncharacteristically high attack rates, thereby leading to selection bias. For schools with low vaccination coverage (and high attack rates) this is arguably not a problem as variation in outbreak sizes is expected to be minor, given the sizes of the schools included (
We have assumed throughout that infections outside the school played a marginal role. Again, this assumption is probably less problematic in schools with low vaccination coverage and high infection attack rates than in schools with high vaccination coverage and lower attack rates, as variation in the expected number of infections is expected to be small in schools with low vaccination coverage. Moreover, there was no sustained community transmission during the study period, suggesting that the impact of infection outside the schools may have been small. Nevertheless, it would be interesting to extend the current analyses, e.g., along the lines of
Classical estimates of mumps transmissibility have been based on the mean age at infection in the pre-vaccination era (
Vaccine efficacy and transmissibility together determine the critical vaccination coverage needed to prevent epidemic outbreaks. In our study, estimates of the critical vaccination coverage are 64% (95%CrI: 62%–67%) in the baseline scenario, and range from 63% (95%CrI: 58%–68%; school 1) to 76% (95%CrI: 66%–83%; school 6) in schools with more than 10 confirmed infections (schools 1–6). This indicates that the critical vaccination coverage does not need to be as high as suggested by early population-based estimates, which are in the range of 86%–95%.
In none of the analyses presented here have we made a distinction between children who had been vaccinated once and those that had been vaccinated twice. This was done because preliminary analyses and previous results
Further, in our analyses we assume that the vaccine works by reducing the probability of transmission (i.e. we assume a leaky vaccine), rather than by providing all-or-nothing immunity. This was done for simplicity, and since the current data do not allow us to distinguish between the different workings of the vaccine. If additional data was available, e.g., on the pre-outbreak antibody titres, one could consider extension of the method by using pre-outbreak antibody titres as an indicator for the ‘level of immunity’, and use this indicator to estimate how the level of pre-existing immunity relates to the probability of infection. In most situations, however, such information will be hard to get, as this would necessitate a large prospective study.
Our definition of vaccine efficacy has a clear-cut biological interpretation (reduction of the probability of infection per contact). This makes it possible to meaningfully average over populations with varying vaccination coverages and exposure levels, and also to extrapolate beyond the study population. This contrasts with traditional estimates of vaccine efficacy that are based on a comparison of attack rates in vaccinated and unvaccinated individuals (the cohort method), or that simply use the vaccination status of the infected individuals together with the population vaccination coverage (the screening method)
Even though our definition of vaccine efficacy differs fundamentally from vaccine efficacy measured by the cohort method, the results are quantitatively in fair agreement with traditional estimates, especially in populations with low vaccination coverage and large number of infections (
Our results point to strategies to efficiently allocate catch-up vaccination efforts in heterogeneously vaccinated populations. No additional vaccination is needed in schools with high vaccination coverage (>75%, say) as these are already protected against epidemic outbreaks affecting a large fraction of students. Similarly, allocating vaccines to schools with low vaccination coverage (<50%, say) is inefficient as it does not markedly reduce the probability of infection for those who are not vaccinated, i.e. the indirect benefits of vaccination are small in these populations. Our analyses suggest that vaccination of populations in the range between these two extremes is most efficient, and that in these populations a single vaccination can potentially prevent almost two infections. Of course, in practice other considerations, for instance on ethical issues, communication, and cost-effectiveness would also come into play.
In the Netherlands, several large outbreaks of mumps virus (genotype D) occurred in 2007–2009. We collected data from children attending primary schools with evidence of mumps virus transmission (report of at least one laboratory confirmed mumps case or more than one clinical mumps case)
The analyses are based on the distribution of the number of persons infected in an outbreak
In SEIR models, each individual in the population can either be susceptible (i.e. healthy), exposed (infected but not able to infect others), infective (infected and able to infect others) or recovered (not infectious and now immune). We assume that infectious contacts are made at the level of the school, and not at other organizational levels (e.g., class, household, community). These assumptions seem reasonable since there was no evidence of sustained community transmission during the study period, while only limited information was available on class structure within schools.
We focus on estimation of two key epidemiological quantities, the basic reproduction number
Throughout we assume that a pair of individuals makes contacts at a rate that is inversely proportional to the school size
Final size data alone do not allow us to estimate parameters with respect to calendar time, but only relative to other model parameters. To set a time-scale, and for simplicity, we therefore assume that the infectious period (i.e. the time that an individual is in the infective state) is fixed at length 1 time unit and set the basic reproduction number
Our model contains two epidemiological parameters, namely the basic reproduction number
The parameters
In a Bayesian framework the key object of interest is the posterior density of the model parameters
The augmented posterior density is given by
The above description can be made mathematically precise
The parameter vector
The posterior density is explored using a Markov chain Monte Carlo method, whereby the missing vaccination statuses are included as latent parameters
The basic reproduction number and vaccine efficacy are updated with a random-walk Metropolis algorithm using Gaussian proposal distributions with standard deviations of 0.2–2 and 0.02–0.2, respectively. Vaccination statuses are updated by flipping the vaccination status of a randomly selected person with unknown vaccination status
Updating is performed in blocks, in the order 1) update the value of the reproduction number, 2) update the value of vaccine efficacy, 3) for each school update the vaccination status of a randomly chosen person with missing vaccination information, 4) for each school attempt to add an infectious contact, and 5) for each school attempt to delete an infectious contact. Each cycle of the chain thus contains 32 updating events. To improve mixing every 50th cycle the positions in the digraph of infected persons (both vaccinated and vaccinated) are randomly permuted so that the chain does not get stuck in topologies from which links to persons with unknown infection status cannot easily be removed. Notice that this operation leaves the topology of the graph intact (distribution of links and types), and thus does not affect the likelihood.
After running a number of exploratory analyses output is generated for a single chain of length 30,000–50,000, of which the last 20,000–25,000 cycles are used to obtain a thinned sample of size 5,000 or 10,000. Inspection of convergence of the chain is performed visually. Run times are approximately 7–10 days on a 3.2Ghz eight-core workstation.
To explore the correspondence between the parameter estimates with the data, we simulated outbreaks in schools of size 200 using the digraph construction described above. To prevent early extinction we introduced three infectious persons with random vaccination status in each simulation. For each vaccination composition, we generated 5,000 random digraphs with the values of the basic reproduction number and vaccine efficacy sampled without replacement from the posterior distribution. Subsequently, for each graph we calculated the attack rate among those that were initially susceptible, and present the median and 2.5% and 97.5% percentiles of the resulting distributions (the black line and grey area in
To compare our results with estimates of vaccine efficacy using the cohort method
Overview of the outbreaks of mumps in Dutch primary schools. See Ruijs et al. (2011) (ref
(DOC)
Summary statistics of the study population, distinguishing between one and two vaccinations (cf.
(DOC)
Classical estimates of vaccine efficacy by the cohort method, i.e. as 1 minus the relative risk of infection in vaccinated versus unvaccinated persons (Orenstein et al. 1985) (ref
(DOC)
We thank all children and parents who participated in this study. Jan van de Kassteele is acknowledged for helpful discussions.