1. Modeling gene expression in a gene set with mixed models.

Let S be a gene set of interest. We start by the case of a one group experiment, where each patient act as her/his own respective control, her/his condition changing over time. The expression of genes inside S is modeled over time according to a function f as:

for all the genes g ∈ S, (1) where y_gpi is expression of the g^th gene for the p^th patient at the i^th time, μ is the intercept in the gene set S, β_g is the fixed effect of the g^th gene, c_gp ∼ 𝓝(0, σ_c) is a random effect grouped by the g^th gene of the p^th patient, t_i is the i^th measurement time, ɛ_gpi ∼ 𝓝(0, σ) is an error term. Finally f_g(t_i) is a function of time, that can be linear, polynomials, etc. Every time coefficient of the trend f_g(t_i) is actually divided into a fixed effect η_⋅ (representing the average trend in the gene set S) and a random effect h_g,⋅ ∼ 𝓝(0, σ_h⋅) grouped on the gene g, accounting for the possible heterogeneity between the genes in the gene set S. In this paper we focus on three forms for f_g (but other forms, such as exponential, etc. could easily be envisaged):

• linear polynomials:
  
• cubic polynomials:
  f_g(t) = (η₁+h_g,1)t+(η₂+h_g,2)t²+(η₃+h_g,3)t³
• natural cubic splines:
  

where the N_k(t) form the natural cubic splines basis [43] for the time variable t (with K internal knots), η_⋅ are the fixed effects of time shared across the gene set S, and h_g,⋅ are the random effects of time accounting for possible heterogeneity between genes. (h_g,1,…, h_{g, d}) ∼ 𝓝(0,Σ_h) with d being the degree of the time function, and for k = 1,…, d h_{g, k} ∼ 𝓝(0, σ_hk). Alternatively, one can make the assumption that the patient effect is the same for all the genes. In that case, the random effect c is no longer grouped on the gene level, and the model can be written as:

for all the genes g ∈ S, (1bis) with the random effect of the patient p, and the random effect of the gene g. This alternative modeling has the advantage to be more parsimonious than the Eq (1), with less parameters to be estimated.

Let’s now consider the case of a multiple group experiment (such as treatment/vaccine groups for instance). The expression of genes inside S is modeled over time according to a function f_{g, m} that is now stratified on the groups:

for all the genes g ∈ S, (2) where m indicates which group is concerned and δ_m is the fixed intercept of the m^th group, everything else being the same as in the Eq (1).

2. Testing the significance of a gene set.

In TcGSA, a “significant” gene set is a gene set whose expression is not stable either over time (in one group experiments) or over groups (in several groups experiments), once between genes and patients variability is taken into account. In other words, we want to test the significance of the time trend while being sensitive to both homogeneous and heterogeneous changes of gene expression over time inside a gene set. Testing the significance of a given gene set S therefore means testing both fixed and random effects at once, in a single test. A likelihood ratio test is the natural way to do so, fitting models under both the null hypothesis and the alternative.

In the case of one group experiment (Eqs (1) and (1bis)) the null hypothesis (H₀) is that the genes inside S are stable over time, i.e. that their expressions are constant and homogeneous over time (all coefficients of the function of time f are not significantly different from zero). The alternative hypothesis (H₁) is that the genes inside S vary significantly over time: (1.0) (1.1)

In the case of a multiple group experiment (Eq (2)), the null hypothesis is that inside the gene set S, the evolution of gene expressions over time is the same regardless of the group. The alternative hypothesis is that time trends f are different depending on the group m: (2.0) (2.1)

In both case, one model is fitted under the null hypothesis, and one is fitted under the alternative. The likelihood ratio is then computed.

However, since both fixed and random effects are tested simultaneously in this likelihood ratio, its null distribution is not the standard chi-square distribution (because of boundary constraints due to the variance of random effects). According to Self et al. [44], it can be approximated by a mixture of chi-square distributions with the following formula: where q is the number of fixed effects and r the number of random effects to be tested simultaneously. This approximation implies that the tested random effects are independent of one another [45–47]. This seems an acceptable assumption according to our simulations under the null (see S1 and S2 Figs). This allows to compute a p-value for the significance of the variation of a given gene set over time.

When several gene sets are investigated at a time, it is necessary to take into account multiple testing. A number of procedures are available to do so [48]. As the TcGSA is mostly an exploratory analysis (even though hypothesis driven in the sense that gene set are defined a priori), we recommend using the Benjamini-Yekutieli procedure for controlling the False Discovery Rate [49], as gene sets are necessarily correlated between each others and this procedure is robust to some of these dependances. Other mutliple testing correction procedures are available in the TcGSA R package.

3. Estimation of individual gene profiles.

In the estimation of linear mixed model, it is common to use the Restricted Maximum Likelihood (REML) instead of the classic Maximum Likelihood (ML) in order to avoid biased estimates of the variance components [50]. But note that REML cannot be used to estimate the likelihood ratios presented here. Indeed, REML estimation of the likelihood ratio between two models can only be used when both models have the same fixed part [51]. Since here the compared models (under H₀ and under H₁) have different fixed components (due to the η_⋅ coefficients under H₁), the use of ML estimation is needed.

For the inference of the random effects, Best Linear Unbiased Predictor (BLUP) are used [52], giving access to estimations of a single profile for each gene among a gene set, in each patient. As a result, the estimations from the mixed model are shrunken towards the average expression inside the gene set. This shrinkage occurs when the residuals variability is relatively large compared to the the random effects estimated variances [52]. The mixed model uses the repeated pattern of the longitudinal measurements to structure the variation. Its estimations give smoother trajectories for the genes than the raw data, which makes the general evolution of the set clearer [53], as it can be seen in Figs 3 and 4.

Dynamic of a significant gene set.

Once a gene set S has been identified as significant through the previous mixed likelihood ratio statistics, a summary of its dynamic over time is needed. However, due to the possible heterogeneity of S, giving a summary representation of S dynamic is not obvious. We propose to automatically identify the number of trends in a significant gene set from the fit of the model. Predicted gene expressions from the linear mixed model are clustered, and the optimal number of trends is selected with the gap statistic [54]. It is a formalization of the elbow criterion for the within-cluster variance. In order to determine the optimal partition of each gene set here, the gap statistics is applied onto a hierarchical clustering of gene expressions inside each gene set. Then the median within each of the identified clusters can summarize each trend. Therefore, gene sets are actually split when heterogeneous, before being summarized. The predicted gene expression from the linear mixed model is used for this (and not the observed expression) because smoothness of trajectories facilitates classification [53]. Examples of such representations are given in Figs 3 and 4.

Global dynamics.

Most often, TcGSA will be used to investigate a large number of gene sets (from a few hundreds to a few thousands). This multiplicity can make visualization of the results more challenging, in addition of requiring a multiple testing correction. TcGSA is designed to identify gene sets that shows a simultaneous evolution of gene expression, but possibly of a small intensity. The method can therefore be quite sensitive, and it can be of interest to rank the significant gene sets to identify the most acute signals. The likelihood ratio provides insight on the magnitude of the variation of each gene set. The percentile of their corresponding likelihood ratio gives an idea of the importance of the variation for a significant gene set. Examples of such representations are given in Fig 5.

The DALIA-1 trial design.

All of the nineteen HIV infected patients received the therapeutic vaccine while under antiretroviral treatment. The patients received four injections at week 0, 4, 8 and 12. This vaccination period was followed by an antiretroviral treatment interruption (ATI) at week 24. The patients were followed up to week 48. Antiretroviral treatment was resumed from week 24 to week 48 at any time under the following criteria: i) if the patients or their doctors wished so; ii) if CD4+ T-cell count was < 350 cells/μL and < 25% of total lymphocytes. Fourteen time points (five in pre-ATI from week 0 to week 22, and nine in post-ATI from week 24 to week 44) were used for this analysis (see Fig 6). One patient was removed from the analysis as his/her antiretroviral treatment compliance was irregular during the vaccination phase.

In the following analysis, two distinct datasets were considered: pre-ATI and post-ATI. The two datasets were normalized separately—via a normal-exponential convolution model [55, 56], followed by the application of the ComBat method [57] to correct for batch effects. Splitting the data allows us to study separately the vaccine effect and the treatment interruption, otherwise the ATI effect would mask any noticeable vaccine effect, because of the huge modification of gene expression related to viral replication [1, 2]. We investigated the gene sets defined by Chaussabel et al. [13], which are oriented towards the immune system. The definition and annotations of those 260 gene sets (called ‘Modules’) are available online (http://www.biir.net/public_wikis/module_annotation/V2_Trial_8_Modules).

Pre-ATI: The vaccination phase.

During the vaccination period, a standard gene-by-gene mixed model analysis, with a cubic polynomial function of time, did not found any significant change of gene abundance at a 5% False Discovery Rate (see Table 1). However, during this vaccination phase, cytokines production analysis of the same blood samples (as measured by Luminex or intracellular staining) have showed that a response was induced by the vaccine at week 16 [5]. Therefore, one expected to observe a signal at the gene expression level between week 0 and week 16, the gene expression preceding molecular activation. Although the measurements were not performed in the hours or days following vaccination, the changes of gene abundance may reflect a change of the equilibrium of the overall expression in some gene sets. This kind of results has already been reported in cross-sectional studies [58]. Likewise, GSEA for time-series did not identify any significant gene set during vaccination. This can be explained by the lack of power of GSEA for time-series, as this method does not take into account the repeated structure of the data and is not suitable for longitudinal measurements. Finally, CAMERA did not identified any significant gene set either, in spite of testing a competitive null hypothesis.

We applied the Time Course Gene set Analysis (see Methods) that allows to detect any change over time of gene abundance inside a gene set by detecting either trends over time or heterogeneity between gene dynamics. Fitting the Eq (1) with a cubic polynomial function of time, 69 gene sets out of 260 turned out to vary significantly. Fig 3 displays the raw sample-normalized and batch-corrected) and estimated gene expressions of 3 of the significant gene sets identified by TcGSA: T-cell, inflammation and B-cell gene sets. The identification of gene sets such as M4.1: T-cell (that includes CD402, CCR7, BCl2) was expected with regards to the CD4 T-cell response observed at Week 16 [5]. Also, the gene sets M4.6: inflammation and M6.7: B-cell are good examples of how smoothing from the estimations can give a much clearer dynamic pattern compared to the raw expression (see Methods).

Post-ATI: After antiretroviral treatment interruption.

Eq (1) was then fitted to the data after antiretroviral treatment interruption that occurred at week 24: 216 gene sets out of 260 were found to be significant with a cubic polynomial function of time. Fig 4 displays the raw and estimated expressions of nine of those significant gene sets. It features heterogeneous gene sets, such as M4.16 and M7.1, which are both also good examples of the shrinkage that occurs with the estimations (see Methods). Meanwhile, GSEA for time series identifies 67 significant, whereas CAMERA identified only 2. These results are consistent with the lower statistical power found in our simulation study.

The large number of significant gene sets post-ATI illustrates the tremendous impact of the treatment interruption on the organism. Followed by a viral rebound, the treatment interruption is indeed a major event that triggers the expression of thousand of genes. Indeed, a gene-by-gene analysis revealed 3,389 significant probes (more than 10% of all the investigated probes—an unusually high number of differentially expressed genes). The immune system is very much in demand during the viral rebound. Therefore most of the gene sets from the Modules defined by Chaussabel et al. [13] are activated, as they are tightly linked with the immune system activity. Of particular interest are the three gene sets M1.2, M3.4 and M5.12 which are all annotated as interferon-related. These three gene sets exhibit similar dynamics (see Fig 4). Such a timely upregulation was expected, as it is associated to the viral rebound after treatment interruption and was previously reported [1, 2]. The gene set M3.4 is also linked with antiviral response.

Study design.

Healthy, young adults were randomly split in three groups of six volunteers each, receiving either a 2009–2010 seasonal influenza vaccine (Fluzone), a 23-valent pneumococcal vaccine (Pneumovax23), or a placebo (saline injections). Blood samples were collected at days -7, 0, 1, 3, 7, 10, 14, 21, and 28 to measure gene expression in whole blood. A more detailed description of the study can be found in Obermoser et al. [6].

Original analysis.

In their modular analysis, Obermoser et al. [6] focused on 62 of the 260 available gene sets defined in Chaussabel et al. [13]. They investigated the changes of gene expression in those 62 gene sets for each of the seven time points from day 1 to day 28 in regards of the baseline, that was considered as the average of the two measurements at days -7 and 0. So hierarchical structure of the data was not taken into account. The three arms (saline, flu and pneumococcal) were analyzed separately, and only significant gene sets at day 1 and day 7 (not further on) are presented in their paper. Changes in eight gene sets were common to both vaccines: M4.6 (inflammation), M6.6 and M6.13 (apoptosis/cell death) and modules M4.1 and M4.15 (T cells), M4.3 (protein synthesis), M5.11, and M6.9 (no functional annotation). Nine gene sets were uniquely changing after the influenza vaccine, three were associated with antiviral responses (M1.2, M3.4, M5.12) and included genes coding for interferon (IFN)-inducible molecules. Six gene sets were uniquely responsive to the pneumococcal vaccine. Of these, five were modules including genes associated with inflammation: M3.2, M4.2, M4.13, M5.1 and M5.7.

TcGSA results.

To compare the gene expression at the gene set level between the vaccine arm (flu or pneumococcal) and the placebo (saline) arm, we applied TcGSA on these data using Eq (2) (for each vaccine separately). In both vaccines, a large response is observed at Day 1. To avoid smoothing down the expression at t_i = 1, we used the following function of time to model the dynamic evolution of gene expression: with , and m the group (either vaccine or placebo).

Most of the 62 investigated gene sets presented a significantly different evolution in vaccine arms compared to the placebo arm. Globally, the intensity of the response was stronger with the pneumoccocal vaccine than with the flu vaccine (Fig 5). The early response induced by the pneumococcal vaccine was dominated by inflammation whereas the top signal triggered by the flu vaccine involved an interferon signature (Fig 5B and 5D). In both vaccine, a T-cell response was also visible. In the pneumoccocal vaccine, a plasma cell signal, in association with cell cycle gene sets (Figs 5A and 5C), started at Day 7 until Day 14. This plasma blast signal was much less clear in the flu vaccine (Figs 5B and 5D). This is in agreement with the results of Obermoser et al. modular analysis.

TcGSA offers an extended and appropriate hierarchical analysis of these data. It provides a truly longitudinal insight into the vaccine responses, that are intrinsically compared to the placebo response. One of the main difference from the results presented in Obermoser et al. paper [6] is that, according to our analysis, the inflammation gene sets (M3.2, M4.13, M5.1 and M5.7) were also involved with the flu vaccine and were not specific to the pneumoccocal vaccine. This result is important as it means that both vaccine involved these inflammatory pathways. This result was not obvious from the original analysis because their approach was less powerful compared to TcGSA.