Quantitative Gap Gene mRNA Expression Data

We have created a quantitative mRNA expression dataset with high spatial and temporal resolution for the trunk gap genes Krüppel (Kr), knirps (kni) and giant (gt), which spans the entire duration of the blastoderm stage in the early embryo of Drosophila melanogaster (cleavage cycles C10–C14A; C14A is further subdivided into time classes T1–8). In contrast to previously published semi-quantitative gap gene mRNA data—based on colorimetric (enzymatic) staining protocols, wide-field microscopy, and an efficient but simple data processing pipeline [66], [68] —we used fluorescent staining protocols, confocal microscopy and fully quantitative data processing methods (see Materials and Methods, and [69]). This work extends a previously published fully quantitative expression dataset for gap gene mRNAs, which only covered the early part of the blastoderm stage (C10–C13) [58]. Our data consist of quantified time-series of gap gene mRNA expression (Figure 2; Supplementary Figures S1, S2, S3), which are equivalent and comparable in quality, as well as spatio-temporal range and resolution, to the comprehensive protein expression data available from the FlyEx database (http://urchin.spbcas.ru/flyex, [57], [59], [60]). This allows us, for the first time, to rigorously and accurately compare gap gene expression during the blastoderm stage at both mRNA and protein level.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Comparison of gap gene mRNA and protein expression patterns.

(A) Time series showing integrated one-dimensional expression patterns of gap gene mRNA (left column) and protein (right column) along the A–P axis in cleavage cycle C14A (time classes T1–T8). Kr is shown in green, kni in red, and gt in blue. X-axes represent A–P position in %, where 0% corresponds to the anterior pole of the embryo. Y-axes represent mRNA and protein concentrations in relative units. Grey background indicates the region displayed in (B). (B) Space-time plots indicating domain boundary positions of the central Kr domain (top), the abdominal kni domain (middle), and the anterior and posterior domains of gt (bottom). Solid patterns indicate mRNA patterns, dashed lines protein. Time flows downwards. (C) Temporal dynamics of peak expression for the central Kr domain (top), the abdominal kni domain (middle), and the posterior gt domain (bottom). Solid lines indicate mRNA, dashed lines protein. Relative concentrations (as in (A)) are plotted against time.

https://doi.org/10.1371/journal.pcbi.1003281.g002

Such a comparison between spatio-temporal gap mRNA and protein expression reveals that, to a first approximation, the mRNA patterns appear very similar to those observed for the corresponding proteins: all mRNA transcripts are expressed in broad, overlapping domains, whose relative timing and spatial arrangement with regard to each other strongly resemble that of gap protein domains (Figure 2A). In addition, the central, abdominal, and posterior mRNA domains of Kr, kni, and gt shift towards the anterior of the embryo over time (Figure 2B). The extent of this movement is on the same order of magnitude as the analogous shifts of the corresponding protein domains (Figure 2B, Supplementary Table S1) [22], [59], [66]. Since anterior domain boundaries generally move less far than posterior ones, domain width of both mRNA and protein domains decreases over time (Figure 2B, Supplementary Table S1) [59]. Based on these observations, we can conclude that expression dynamics of gap mRNA and protein domains are largely qualitatively equivalent with regard to each other.

However, if we examine the data more closely, two significant differences between mRNA and protein become apparent. First, boundary positions of mRNA domains—with the exception of the anterior Kr border—are displaced anteriorly compared to their corresponding protein domains (Figure 2B; Supplementary Table S1). This displacement is caused by the anterior shift of gap domains over time [22], [59]. The effect is substantially more pronounced for posterior domain borders than for anterior ones. It is associated with a generally slightly smaller width of mRNA domains compared to protein (Supplementary Table S1). Second, the timing of initial and maximum peak expression is delayed for protein compared to mRNA (Figure 2C). Delayed first appearance of protein versus mRNA patterns during the early blastoderm stage has been reported and quantified previously [58], [59]. Our data reveal a similar phenomenon in late-blastoderm expression dynamics: mRNA expression of all three gap genes peaks around 30 min before gastrulation (time class T3 in Figure 2C), while protein expression shows a maximum approximately 15–20 min later (time class T6–7). This obviously agrees with the fact that it takes time to export the mRNA from the nucleus, to process and translate it into protein. In addition, we detect a post-peak decrease in mRNA abundance that was not reported in an earlier PCR- and microarray-based analysis of pre-gastrulation gene expression with a lower temporal resolution than our data [70]. Interestingly, this trend is not reflected in levels of gap proteins, which only show a marginal decrease (if any) before the onset of gastrulation (see Figure 2C and also Supplementary Text S1).

In addition to our quantification of the timing and position of averaged gene expression patterns, we have also analysed the embryo-to-embryo variability of gap domain width as well as peak and boundary positions during the blastoderm stage. This had not been possible with our earlier, semi-quantitative dataset [66]. It has been reported previously that the precision of gap protein domain boundary positions increases over time due to cross-regulatory interactions [24], [25], [59]. Such a trend—although much less obvious—is also present in our mRNA data (Supplementary Figure S4). Noise levels in the mRNA data are generally higher than in protein patterns. Both Kr and kni (but not gt) show higher levels of variability in mRNA compared to protein data at a majority of sampled data points. Moreover, we observe a high level of fluctuations between time points in our mRNA data (for all three genes) indicating increased levels of experimental noise. This may be due to the harsher treatment of embryos for in situ hybridisation compared to antibody staining (see [58] for a more detailed discussion). Nevertheless, the overall trends are similar for mRNA and protein, as levels in variability of all measured expression features is generally lower at T6 than at earlier time points during C14 (Supplementary Figure S4). This indicates canalisation of development at two levels: first, protein patterns are generally more precise than mRNA domains at comparable stages, and second, mRNA precision—as is the case for protein—increases over time.

Reverse Engineering: Structural Identifiability Analysis

The fact that mRNA and protein patterns of the gap genes Kr, kni, and gt are similar (yet not identical), and show a delay in dynamics with regard to each other, raises the non-trivial question whether protein patterns simply reflect earlier mRNA levels (with a small additional contribution by protein diffusion), or whether spatially and temporally specific post-transcriptional regulation is required to account for the observed distribution of proteins. We use a reverse-engineering approach to distinguish between these two alternative possibilities. In this approach, we test the hypothesis that no post-transcriptional regulation is required by fitting a simple dynamical model to data. This model incorporates the following assumptions (see Introduction): It includes time-delayed but linear production of protein from its mRNA, as well as protein diffusion and decay. It takes mRNA patterns as an input to predict protein expression. A good fit of the model to protein expression data would therefore favor absence of post-transcriptional regulation, while a failure to fit the data would point us to specific features of gap protein expression that require regulated nuclear export, splicing, or translation.

Our reverse-engineering approach can only give us quantitative and specific insights into the problem of post-transcriptional regulation if it is fit to data in a manner which is as rigorous and reproducible as possible. As a first step, this requires us to determine whether the model is formulated in a way such that the fitting procedure has a unique solution. Since our model is feed-forward and linear, this can be achieved using structural (or a priori) parameter identifiability analysis. This analysis is performed under an ideal scenario where noise-free time-continuous experimental data are assumed to be available, and the objective is to answer the question whether under those ideal conditions the parameters can be given unique values. There are three possible outcomes: (1) The model is structurally globally identifiable (s.g.i.) if all parameters can be uniquely identified within a biologically meaningful region of parameter space, which we will call (). (2) The model is structurally locally identifiable (s.l.i.) if one or more parameters can be uniquely identified in a given neighborhood , or (3) the model is not structurally identifiable, if neither of (1) or (2) apply.

Although several methods for the analysis of structural identifiability of linear models exist, the model described by equation (1) presents the peculiar challenge of incorporating a delay parameter within the input function (the production term). In this scenario the Laplace transform () based method may be used to assess whether the s.g.i. condition holds (see [28] and references cited therein). The underlying idea is to verify whether a canonical form of the transfer matrix of the system 1 is unique. The basic steps are the following: the model (equation 1) is rewriten in matrix form and its Laplace transform is computed; the possibility of computing the transfer matrix is demonstrated by an invertibility conditon; the analytical canonical form of the transfer matrix is then obtained; symbolic manipulation is finally used to prove uniqueness of the transfer matrix. Details of our calculations can be found in Supplementary Text S2.

In the case of our model for post-transcriptional regulation (see Introduction, equation 1) structural identifiability analysis reveals that model parameters are globally identifiable in the realm of non-negative real numbers (which is expected given that rate and delay parameters cannot be less than zero). For this result to apply, the following conditions must be met by the experimental data: (1) some concentrations of mRNA and proteins must be non-zero in the interior of the model range at sampling times, and (2) protein data must be available for time points before or after mitosis (i.e. when the production rate is not zero; see Supplementary Text S2 for details). Both of these conditions are met in our data, and we conclude that our model parameters are globally structurally identifiable, i. e. our optimisation problem has a specific and unique solution.

Reverse Engineering: Model Fitting

Next, we proceeded to fit our model to quantitative spatio-temporal protein expression data. This was done for Kr, kni, and gt separately by solving the model for each gene numerically, and minimising a weighted sum of squared differences between expression patterns predicted by the model and those measured by experiment (see equation 2). Minimisation of squared differences was achieved by two different global optimisation algorithms, based on parallel simulated annealing (pLSA), and an enhanced scatter search (eSS) method, respectively (see Materials and Methods for details and references). Both of these independent optimisation approaches resulted in equivalent model fits and parameter estimates (Figure 3; Table 1). In order to further corroborate the robust performance of our algorithms, we performed a systematic comparison of eSS with a number of standard global optimisation approaches. The results can be found in Supplementary Text S3. They indicate that different algorithms show significant differences in computational efficiency, but converge to very similar solutions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Comparison of model output and measured protein concentrations.

(A) Spatial profiles of Kr (green), kni (red), and gt (blue) for early (T1), mid (T4), and late (T8) time classes during C14A. X-axes represent A–P position (in %), Y-axis show relative concentrations (as in Figure 2A). (B) Temporal dynamics of peak concentrations for the central Kr domain (left), the abdominal kni domain (centre), and the posterior gt domain (right). X-axes represent time, Y-axes show relative concentrations (as in Figure 2C). In all panels, model output is shown as a dashed black line; measured protein concentrations are shown as dark colored lines (mean) and lightly shaded background (standard deviations).

https://doi.org/10.1371/journal.pcbi.1003281.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Comparison of residual scores and parameter estimates obtained from pLSA and eSS optimisation approaches.

https://doi.org/10.1371/journal.pcbi.1003281.t001

Fitting results differ slightly between genes. The best fit between model and data is obtained by Kr optimisation runs, with a root-mean-square (RMS) score of around 11.1–11.2 (Table 1; see Methods for a mathematical definition of RMS, which represents the average deviation of model from data for each data point). Although minor patterning defects can be observed at early stages (especially between C13 and T2) and expression levels disagree somewhat at the last time points (T7/T8, see below), model and data match to within the noise level of the data at intermediate times (Figure 3, left column).

Of all fitting solutions, Kni shows the largest overall deviation between model and data with a RMS score of 21.1–21.2 (Table 1). Nevertheless, position and shape of the Kni protein domain, as well as the temporal dynamics of expression, are reproduced correctly (Figure 3, centre column). In particular, the model shows peak expression at the correct stage, and agrees with data to high accuracy in non-expressing areas. In contrast, protein expression predicted by the model is consistently lower than measured within the area of the abdominal Kni domain. This discrepancy accounts for the large RMS value.

Expression patterns resulting from Gt optimization runs exhibit similar properties as those of Kni, with a slightly lower residual error (RMS around 17.2; see Table 1). Again, the timing, position, and shape of both Gt domains are reproduced quite accurately in the model, while predicted expression levels are generally too low (Figure 3, right column). The only noticeable positional defect is a slight anterior displacement of the posterior gt domain at early stages (up to T1).

Despite the problems with reproducing accurate levels of expression for Kni and Gt, all three models show initiation and build-up of gap proteins at the appropriate stages of development, and the qualitative shape of the temporal expression profile is reproduced correctly up to T6 (Figure 3B). In contrast, model and data disagree conspicuously at later time points (T7/8), since the model predicts rapidly diminishing concentrations of gap proteins before the onset of gastrulation. This downregulation is much weaker (Kni), or entirely absent (Kr, Gt) in the data.

In summary, our models capture the position, shape, and width of gap protein domains accurately. Minor deviations in these spatial expression features are only observed during earlier time points, when noise levels in the data are high. Temporal features of gap protein expression—such as initiation of expression, shifts in domain position, or the time point of maximum expression—are also reproduced correctly. However, our models fail to reproduce the exact levels of gap protein expression, as well as their downregulation towards the end of C14. These two specific failures of our model fits have interesting implications for our understanding of gap gene regulation (see Discussion).

Reverse Engineering: Parameter Estimation and Analysis

Model fitting resulted in reproducible and biologically plausible estimates of parameter values for Kr, kni, and gt. We performed a number of independent optimisation runs for each model using both of our two alternative fitting strategies (100 runs of pLSA, 10 runs of eSS per model). Parameter estimates from different runs varied only minimally between solutions (for pLSA see Supplementary Figure S5), and estimated parameter values provided by either of the two alternative optimisation strategies agreed to high accuracy. This indicates that our parameter estimates are robust with regard to the choice of optimisation strategy. Table 1 shows parameter values from a representative optimisation run for each fitting approach.

Predicted values for the delay parameter (see equation 1) require some more detailed attention, since such parameters are notoriously difficult to estimate. For this reason, we verified the validity of our estimates for by the following numerical approach: we performed a series of fits for the gt model, with fixed to values between 0 and 8 minutes, including a particularly high-density sampling of around the values predicted by optimisation (Supplementary Figure S6). These control fits show a minimum of the residual error which coincides precisely with the parameter values inferred from optimisation. This corroborates the reproducibility and accuracy of our approach.

Our parameter estimates are informative from a biological point of view, and yield experimentally testable predictions. First of all, we note that production and decay rates ( and , respectively) are well balanced in this system. Decay rates correspond to protein half lives of 9.1 (Kr), 9.0 (Kni), and 6.2 (Gt) minutes, indicating that gap proteins must be very unstable—to enable patterning at the extremely short time scale of gap gene expression dynamics. Gap protein diffusion is generally very low, especially in the case of Kr. Protein production delays (incorporating contributions of transcription, splicing, nuclear export, and translation; see Introduction) range between and minutes. While the upper value is within the expected range (see Discussion) the former estimate is rather low, and may need further investigation (see also next section).

Reverse Engineering: Practical Identifiability Analysis

While it is encouraging that independent optimisation runs and methods give consistent parameter estimates, it is necessary to test the reliability and accuracy of these estimates using practical (or a posteriori) parameter identifiability analysis (see Introduction). We have performed such an analysis using two complementary approaches.

One approach to the practical analysis of parameter identifiability is based on a geometrical interpretation of the ‘optimisation landscape’ given by the value of the weighted-least-squares cost function (, in equation (2)) [35], [36]. To illustrate this approach, we will assume a two-dimensional parameter space for simplicity. This results in a three-dimensional topography of the optimisation landscape, where minima lie in ‘troughs’ or ‘depressions’ of the contour determined by the cost function. The more shallow the trough in which a minimum lies, the more uncertain the parameter estimate, since changing parameter values around the optimum will lead to only a slight increase in the value of the cost function. It is possible to characterise the local surface of any optimisation landscape around a given minimum using linear approximations. This allows us to define an ellipsoidal confidence region around our minimum, resulting in estimates for the confidence intervals for each of our parameters.

If there is no correlation among parameters, the principal axes of the confidence ellipsoid will lie parallel to those of parameter space. Confidence intervals for parameters can then be calculated as the intersect of the ellipsoid with these axes. Correlations among parameters are detectable as an inclination between the ellipsoid's principal axes and the axes of parameter space. This makes it possible to calculate two distinct ranges: the dependent confidence interval is given by the intersection of the ellipsoid with a given parameter axis (as above), while the projection of the ellipsoid region onto the parameter axis specifies the independent confidence interval. Independent confidence intervals typically overestimate the uncertainty in parameters, while dependent confidence intervals underestimate it. If both confidence intervals turn out to be similar and small, a parameter can be considered well determined.

Confidence intervals, as calculated by equations (4) and (5) (see Materials and Methods) are shown in Figure 4. Compared to the entire range of search space, confidence regions for all three rate parameters (, , and ; see equation (1)) are small in models of all three gap genes. Dependent and independent confidence intervals for and deviate significantly, suggesting strong mutual correlation among model parameters. This is not the case for , where independent and dependent confidence intervals are very similar. Note that the lower limits of some intervals for and are negative, and therefore lie outside the allowed range of parameter values. This artifact results from the linear approximation of the optimisation landscape used in this method. Confidence intervals for delay parameters are larger compared to rate parameters. Nevertheless, they lie within a well confined and biologically plausible range (Figure 4). As for the case of and above, there appears to be a high degree of correlation for with other parameters in all three models.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Confidence intervals for parameter estimates.

This figure shows 95% confidence intervals for parameters (production rate), (decay rate), (diffusion rate), and (production delay; see equation 1) for Kr (green), kni (red), and gt (blue). are independent, dependent intervals obtained from linear analysis (connected solid lines), are intervals obtained from bootstrapping (dashed lines). Dots (on solid lines) represent eSS parameter estimates, diamonds (on dashed lines) those from SA. Striped grey background indicates parameter values that lie outside the search space limits used for optimisation. Note that only a subregion of the search space is shown in each panel (see Materials and Methods for values of search space limits).

https://doi.org/10.1371/journal.pcbi.1003281.g004

Correlation coefficients between parameters can be calculated from the covariance matrix (Figure 5A; see equation 6 in Materials and Methods). In all three models, correlation is high between and . This is expected since high decay rates can compensate for high production rates. Both of these parameters are also correlated to the delays given by . These correlations are highest for gt, and still very substantial for both Kr and kni. Again, this is to be expected since production delay can be mimicked to some degree by low production rates. In contrast, we found that diffusion rates are largely independent of other model parameters, except for a slight negative correlation between and , and between and . This could be due to the extremely low values of diffusion rates in all of our models, or due to the fact that diffusion affects spatial, rather than strictly local, regulatory mechanisms, which could explain the increased degree of decoupling between the two processes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 5. Parameter correlations.

This figure shows correlation matrices for parameter values derived from linear analysis (A), and bootstrapping (B), for Kr (green frame), kni (red frame), and gt (blue frame). Parameter notation: (production rate), (decay rate), (diffusion rate), and (production delay; see equation 1). Colors indicate sign and strength of correlations. Matrices in (A) are calculated from equation 6 (see Materials and Methods). Matrices in (B) are derived from the singular value decomposition of bootstrap distributions.

https://doi.org/10.1371/journal.pcbi.1003281.g005

While computationally efficient, the linear identifiability analysis described above can lead to serious artifacts or biases in the estimation of confidence intervals due to its simplifying assumptions. Therefore, we validated its results by using the computationally much more expensive approach of bootstrapping [31], [71]–[73]. The bootstrap method is based on resampling protein expression patterns from distributions defined by the mean and variance of our measurements (for an equivalent analysis of the sensitivity of parameter estimates with regard to perturbations in the mRNA data see Supplementary Text S4). The model is then fitted to a large number ( in our case) of such sampled noisy patterns. Confidence intervals and correlations for parameter estimates can be directly extracted from the resulting parameter distributions.

Distributions of parameter estimates obtained by bootstrapping are shown in Figure 6. In all cases, estimated parameter values are confined to relatively small subregions of search space. Only diffusion rates show a tendency towards saturation at their lower limit (; Figure 6G–I). Distributions are generally unimodal, with the exception of Kr which shows two distinct clusters in parameter space. The cause of this bimodal distribution remains unclear. One of the clusters (568 solutions) has implausibly small values for ( min). Therefore, we only considered the remaining 432 solutions (see dashed circle in Figure 6D) for further analysis. Another interesting feature of these parameter distributions are the two horizontal lines visible in Figure 6E, which indicate an exclusion of values around 6.25 and 6.30 minutes. This corresponds roughly to the time between data points, which seems to indicate that the structure of the data used for model fitting has a non-negligible impact on parameter estimation in this case.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 6. Parameter distributions obtained from bootstrapping.

This figure shows illustrative examples of scatter plots for parameter values derived from fits to simulated noisy data (sampled from the distributions of protein data measurements; see Figure 3 for mean and standard deviations of spatial expression profiles). Parameter values for Kr are shown in green (left column, A, D, G), for kni in red (centre column, B, E, H), and for gt in blue (right column, C, F, I). Parameter notation: (production rate), (decay rate), (diffusion rate), and (production delay; see equation 1). Black triangles indicate the original parameter estimate obtained with unperturbed data. Dashed ellipse around parameter values for Kr (in D) indicates parameters selected for further analysis. Arrow in E indicates striped interference pattern in the distribution of kni parameter values. See text for details.

https://doi.org/10.1371/journal.pcbi.1003281.g006

None of these irregularities observed in parameter distributions seriously affects our ability to compute confidence intervals. This was done by determining the 95-percentile range for each parameter separately. The resulting confidence intervals and the initial guess are shown in Figure 4. The optimal solution of the unperturbed data set in every case lies within or very close () to the limits of the corresponding confidence interval (diamonds in Figure 4). With the exception of confidence intervals around , the size of bootstrap intervals lies between those of the dependent and independent intervals calculated by linear approximation. In general, this confirms the accuracy and reliability of this method.

Some notable exceptions apply. First, most confidence intervals based on bootstrapping are clearly asymmetric around the estimated optimal values. This asymmetry reflects a non-linear dependence of the optimisation problem on parameter values, which cannot be captured by confidence intervals calculated from linear approximation. Second, size of confidence intervals for obtained by the bootstrap are frequently more similar to the size given by the dependent confidence interval, indicating that delay ranges are more accurately determinable than estimated by linear approximation.

Correlation matrices calculated by linear approximation or bootstrapping are very consistent. The anisotropic shape of parameter distributions resulting from bootstrapping reveal strong positive correlations between rate parameters for production and decay ( and ; Figure 5B, 6A–C). Somewhat weaker, but nevertheless strong correlations occur between these same rate parameters and the production delays (Figure 5B, 6D–F). In contrast, diffusion rates are much less correlated with any of the other parameters (Figure 5B, 6G–I).

Data Acquisition

Blastoderm stage embryos of Drosophila melanogaster (raised at 25°C) were collected 1–4 hrs after egg laying. Embryos were fixed and stained using FITC- (Kr, gt) or DIG-labeled (kni) riboprobes, plus polyclonal antiserum against Even-Skipped (Eve) [85], according to standard experimental protocols [58], [86]–[88]. Nuclei were counterstained using Hoechst 34580. Imaging took place on a Leica TCS SP5 confocal microscope using a 20× objective, and an additional digital zoom of 1.3x. We imaged the blastodermal nuclear layer of laterally oriented embryos at two -positions, 1.0–1.2 μm apart. Data channels were scanned sequentially at a resolution of 1024×1024 pixels. Only embryos at cleavage cycle 14A (C14A) [89] were chosen for further processing. For earlier time points, we use previously published gap mRNA expression data [58].

Data Processing

Data processing and quantification methods are described elsewhere in detail [58], [69]. In brief, we create a binary whole-embryo mask by thresholding, which is used to automatically crop and align embryo images such that anterior is left, dorsal up. We identify nuclei and their surrounding territories of cytoplasm using watershed-based image segmentation algorithms [58], [57]. From these watershed masks, we determine the position of nuclei, as well as the concentration of mRNA (in nuclei plus surrounding cytoplasm), and Eve protein (in nuclei only). Embryos are classified into eight time classes (T1–8; each min long) during C14A, based on Eve expression patterns and morphological markers [69]. Non-specific background staining is removed as previously described [90]. Expression data are registered using a spline-based approach [91]. Background removal and data registration are implemented in an integrated tool [92]. Finally, registered mRNA data within a lateral strip (covering 10% of the embryo) are placed into 100 bins along the A–P axis, and concentration values in each bin are averaged per gene and time class. Individual expression profiles, integrated patterns, and the number of embryos used for each time class, are shown in Supplementary Figures S1, S2, S3. Gap mRNA expression patterns for C10–C13 were taken from [58], and scaled to provide a smooth transition between the two datasets. Integrated protein expression data used in our analysis are from the FlyEx database: http://urchin.spbcas.ru/flyex [57], [60]. We normalise our mRNA data (using the same scaling factor for all time classes) by adjusting peak concentrations to the maximum expression level observed for protein. Expression peaks, domain boundary positions (points of 50% maximum expression), and domain widths were calculated from spline approximations to the expression data as described in [63].

Model Structure and Numerical Solver

The basic objects of our model represent nuclei plus their associated surrounding cytoplasm (energids). The state variables of the model describe the concentration of intra-nuclear gap protein within each energid. Change in gap protein concentration across time and space is described by a system of ordinary differential equations (ODEs; see equation 1 in the Introduction), and depends on protein production from mRNA (concentration averaged across both nuclear and cytoplasmic portions of the energid), protein diffusion between energids, and protein decay. The model spans the entire blastoderm stage, from min after the onset of cleavage cycle 10 (C10; min) to the end of C14A at the onset of gastrulation ( min) [89]. During this time, four mitotic divisions occur (division 10, min; division 11, min; division 12, min; division 13, min; [89]). During mitosis, transcription of mRNA is interrupted and unfinished transcripts are actively degraded [93]. This process has never been quantified. Here, we assume fast mitotic mRNA degradation: therefore, the protein production term in our model (see equation 1) is set to zero, whenever the time point (current time minus the production delay) comes to lie within the time of mitosis. At the end of each mitotic phase, nuclei divide instantaneously (and thus the number of ODEs in the model increases approximately two-fold), and the distance between them is halved. Due to the presence of diffusion, our ODEs are coupled across space. The spatial range of our model is defined for each gap gene independently. In general, models cover most of the segmented trunk region of the embryo. Ranges were defined to include the posterior boundary of the anterior Gt domain, the central Kr domain, the abdominal Kni domain, and the posterior domain of Gt (Kr: 25.5–88.5%, kni: 32.5–88.5%, gt: 32.5–95.5% A–P position, where 0% is the anterior pole).

Although our models are feed-forward and linear, they depend on a non-linear external input (the mRNA expression profiles). Therefore, we solve these systems of ODEs numerically using an implementation of the MATLAB dde23 solver in C [94]. The solver was modified to satisfy the following requirements: (i) it must be able to provide dense output at time points for which there is no external input data; (ii) it must be able to handle discontinuities propagating through the system due to the delay history; and (iii) it has to handle implicit formulas if the stepsize becomes bigger than the delay. We use linear interpolation between data points to provide mRNA concentrations as external inputs at arbitrary points.

Structural Identifiability Analysis

Structural parameter identifiability analysis was performed using the Laplace transform based approach [28]. The idea is to obtain the transfer matrix of the system in rational canonical form and to assess whether the transfer matrix is unique. Supplementary Text S2 provides a detailed description of this approach, and the calculations we performed.

Parameter Estimation

Numerical solutions are produced for time points C10–C13, and T1–T8 within cleavage cycle 14A. We then calculate a weighted sum of squared differences (according to equation 2), which is minimised using two alternative optimisation strategies. The first of these consists of global optimisation using (parallel) Lam Simulated Annealing (pLSA; [95]–[98]). This method is reliable and robust, and has been successfully used in previous reverse-engineering studies of the gap gene system [22], [58], [64]. pLSA is computationally intensive and was implemented in C. The second approach consists of a scatter search approach (eSS), which systematically explores parameter space, triggering a local gradient-based search [99] whenever a promising parameter set has been found [100]–[104]. eSS is implemented in the AMIGO toolbox (based on MATLAB; [105]). Both strategies resulted in virtually identical model fits and parameter estimates (Figure 3; Table 1).

The following search space limits were used for optimisation. : 0.005–5.0; : 0.0347–0.6931; : 0.0–0.3; : 0.0–10.0.

The quality of the fit between data and model output is measured by the root mean square () score, which represents the average difference between modeled and measured protein concentrations across all data points:(3)where denotes the concentrations of protein in nucleus at timepoint , while corresponds to the according simulated value of protein concentration dependent on the chosen parameter set . The RMS—unlike the weighted least square sum (see equation (2))—is independent of not only the noise in measurement but also the number of data points used for fitting. It therefore makes model fits of different gap genes comparable to each other on a quantitative basis.

Practical Identifiability Analysis

We used two alternative strategies for practical parameter identifiability analysis. The first one is based on a local linear approximation of the ‘energy’-landscape given by the objective function (2) around a given optimum as described in [35], [36], [66]. Estimates of the local contour of this landscape are used to determine an ellipsoidal confidence region around the optimal parameter set obtained by eSS. Dependent confidence intervals () are then given by the intersection of the ellipsoid with the parameter axes(4)while independent confidence intervals are specified by the projection of the ellipsoid onto the parameter axes(5)

Correlations among model parameters can be calculated based on the covariance matrix as(6)In all equations, diagonal entries in correspond to with beeing the standard deviation on measurement. is the inferred parameter set obtained via eSS and denotes the sensitivity matrix of the model given by the first order derivative of the observables with respect to the parameters. is calculated as the upper part of Fishers distribution with and degrees of freedom(7)with(8)

The second strategy is based on a bootstrapping approach, where we sample a normal distribution (based on measured means and variances) for each of our protein expression data points. Data points for which no variance estimates were available were not randomised. Resulting sampled expression profiles were corrected by setting negative concentration values to zero. bootstrapping samples were generated for each gap gene in this way. These samples were then fitted by pLSA as described above. Bootstrapping runs were performed in parallel on a cluster provided by the Spanish Supercomputing Network (RES—Red Española de Supercomputación). From the resulting distribution of parameter values, we directly calculate confidence intervals, plus correlation coefficients indicating mutual dependence of model parameters. In the case of Kr, which shows a bimodal parameter distribution (Figure 6), we only considered a subset of the sampled estimates (see Results). For an analysis of the sensitivity of parameter values to changes in mRNA expression patterns, analogous to the protein bootstrap, please refer to Supplementary Text S4.