The authors have declared that no competing interests exist.
Platt's 1964 essay on strong inference
A half century ago Platt proposed a formal schema for scientific inquiry based, in part, on his assessment of the rapid progress made in molecular biology and theoretical physics in the middle part of the twentieth century. Building on the “simple old-fashioned method of inductive inference that goes back to Francis Bacon”
“1. Devising alternative hypotheses;
2. Devising a crucial experiment (or several of them), with alternative possible outcomes, each of which will, as nearly as possible, exclude one or more of the hypotheses;
3. Carrying out the experiment so as to get a clean result;
1′. Recycling the procedure, making subhypotheses or sequential hypotheses to refine the possibilities that remain; and so on.”
Current use of the scientific method ideally applies this logic, but all too often work stagnates at a particular step without completing the cycle. In adopting and effectively applying Platt's steps, hypotheses are systematically disproved and refined, successively moving toward a more complete understanding of nature. Hypotheses are not precious “personal property” to be protected and defended. Instead, successful disproof is the key to progress. Likewise, the goal of publication of an experimental result is to alert the community of progress and invite criticism, alternative explanations, and—what often hurts an investigator's ego—a more rigorous experiment.
Platt warns that when alternative hypotheses are not sought, scientific inquiry becomes a “conflict between men, each with his single Ruling Theory.” In that case, “the scientist has no choice but to be either soft-headed or disputatious.” (Where unattributed, all quotations are from Platt
What are the symptoms of the sickness that Platt identified? Platt enumerates several: The Frozen Method, The Eternal Surveyor, The Never Finished, The Great Man with a Single Hypothesis, The Little Club of Dependents, The Vendetta, The All-Encompassing Theory Which Can Never Be Falsified. It may be that a major emphasis on technology development has alleviated the problem of frozen methodology. It is probably fair to say that biology has become even more method-oriented than in Platt's day. Regarding the other symptoms, we suspect that most readers can quickly identify current examples within their own fields of specialization.
While many of these symptoms remain, it is also clear that several revolutionary developments have shaped the scientific landscape since the 1964 publication of Platt's essay. One of the most obvious and critical was the development and application of genome sequencing technology, highlighted by the Human Genome Project. Now in the era of post-genomics and systems biology, it is widely appreciated that biology is largely discovery-driven and perhaps less hypothesis-driven than in the past. As has been convincingly argued, discovery-driven and hypothesis-driven research are really not fundamentally distinct, so long as careful logic is applied in cycling between generating ideas and data to test the ideas
However, while discovery-driven approaches to research should offer no philosophical resistance to strong inference, in practice they have. The problem was foreshadowed by Platt who cautioned that we not “become method-oriented rather than problem-oriented.” In the twenty-first century biology has become unduly method-oriented; new tools to generate data appear to be the most highly prized. Perhaps this trend originated in the Human Genome Project. A 2000 White House press release announced the completion of the first draft sequence of the human genome, promising “a new era of molecular medicine” in which scientists will (among other things) “discover which DNA sequence changes in a gene can cause disease” and “develop new treatments at the molecular level.” Surely the project was an enormous success, providing not just the sequence itself, but also new technologies for sequencing and associated tasks that have become indispensible parts of the biomedical research toolbox. Yet nearly a decade and a half later, progress on many of the health concerns described in the White House press release proceeds much as it did before, during, and immediately after completion of the Human Genome Project. Thanks in part to information revealed by the Human Genome Project, it is now appreciated that most diseases (including diabetes, cancer, and Parkinson disease, which are specifically mentioned in that particular press release) involve a complex interplay between many genes and environmental factors. Sequences and sequencing technologies are helping us to see where there are gaps in knowledge and helping to fill in those gaps. The sequence itself neither identifies the gaps nor fills them in.
The point is that large-scale sequencing has introduced important new tools—perhaps even revolutionary tools—for biomedical research, while the underlying logical framework for “exploring the unknown” has remained unchanged. Some hype and oversimplification might be expected in promoting a grand technological achievement. But the Human Genome Project truly set a new standard for expectations. With this achievement hailed as an epoch-changing event, what's next? If the epoch has changed, what does that mean for the old-fashioned scientist who is still engaged in careful, cautious hypothesis testing? Why would anyone want to make deliberate progress on a particular biological question when the lab down the hall is getting the attention (and funding) for fully embracing The Next Really Big Thing?
Systems biology is one of the names we are calling the next big thing these days. More than just a name, systems biology represents a potential vehicle for systematic application of Platt's principles to biomedical research. The opportunity exists because the need is especially obvious at this particular time. Former NIH director Elias Zerhouni points out that while “discovery in the life sciences is accelerating at an unprecedented rate,” we are now faced with the critical “need to understand complex biological systems”
Apparently Platt did not hold much stock in mathematical modeling and simulation: “Equations and measurements are useful when and only when they are related to proof: but proof or disproof comes first and is in fact strongest when it is absolutely convincing without any quantitative measurement.” This is perhaps where Platt's vision needs to be recalibrated for the times. The mechanisms underlying the operation of biological systems—e.g., gene regulatory networks, metabolic networks, signaling networks, and the interoperation of all of the above—cannot be cast in a meaningful way into the simple qualitative framework of an earlier era of molecular biology. Increasingly, describing the operation of the biological systems under current investigation requires invoking computational models to simulate the system. For example, efforts to capture the physiome and construct a virtual human simulation represent the height of ambition of computational biology
The fundamental key is to recognize that a computational model, however simple or complex, is a hypothesis, or perhaps a series of hypotheses bundled together into a set of equations or a computer algorithm that best represents how we think a complex systems works. Formulating hypotheses as computational models has at least two advantages when using strong inference to systematically uncover the important mechanisms governing a complex system:
Computational/mathematical models are precise. Cartoons and thought models constructed out of words serve useful purposes in textbooks and in discussion sections of articles. However, when qualitative methods (such as drawing boxes and arrows to indicate interactions in a molecular network) are used to describe a hypothesis, that hypothesis can be interpreted many ways, and is therefore difficult to disprove without ambiguity.
Computational/mathematical models, as necessarily simplified representations of reality, are sometimes useful but always incomplete. By virtue of their incompleteness, computational models fit naturally into Platt's scheme. Models can remain the precious “personal property” of their inventors and still be subject to disproof and refinement. It is the natural process of mathematical and computational modeling to close the loop between Platt's Step 3 and Step 1.
Closing the loop, of course, requires data. After all, “without data, there is nothing to model; and without models, there is no source of deep predictive understanding”
In the remainder of this essay we apply strong inference to analyze two long-standing questions in the control and regulation of cardiac energy metabolism: the metabolic stability hypothesis—the hypothesis that the rate of oxidative ATP synthesis in the heart is not influenced by any significant changes in substrate concentrations with changes in rate of ATP hydrolysis—and the phosphocreatine shuttle theory—a loosely organized set of ideas regarding possible roles of creatine and phosphocreatine in transporting ATP hydrolysis potential in the cardiomyocyte. These examples show how significant progress can be made on largely stalled areas of investigation by applying strong inference.
The landmark metabolic stability hypothesis was formulated based on the observation that the heart maintains relatively stable concentrations of phosphate metabolites over the observed range of cardiac oxygen consumption. This phenomenon was first reported by Neely et al.
(A) [ADP], (B) [Pi], (C) [CrP]/[ATP]. Data were obtained from 31P-MRS of the canine myocardium, with MVO2 varied by pacing or infusion of epinephrine or phenylephrine. Data are adapted from Figures 2, 4, and 6 of Katz et al.
Although the experiments summarized in
The hypothesis that the substrates for oxidative phosphorylation (ADP and Pi) are not the primary regulators of oxidative phosphorylation has been stated by Balaban:
“…recent data from intact tissues with high oxidative phosphorylation capacities (i.e., heart, brain, and kidney) indicate that the cytosolic concentration of ADP and Pi do not change significantly with work. These data imply that [a] simple feedback model is not adequate to explain the regulation of energy metabolism in these tissues…”
The insufficiency of feedback regulation has inspired a number of theories to explain the control of oxidative phosphorylation in the heart in vivo. Indeed, several putative explanations of metabolic stability in the heart have been formulated as computational models simulating cardiac energy metabolism. Examples include: feed-forward activation by Ca2+, in which Ca2+ ion stimulates oxidative phosphorylation mainly through activating tricarboxylic acid (TCA) cycle enzymes
Yet, however important it has been in inspiring theory and experiments, the above quotation laying out the hypothesis of metabolic stability was not authored under the philosophy of strong inference. In fact, while we have recast the message as a hypothesis, this and similar statements are presented as conclusions in the original sources. The metabolic stability hypothesis is widely accepted and serious effort has been put into trying to explain it. Yet if the hypothesis is disprovable, then the longer it goes un-disproved, the less it inspires and the more it impedes progress.
(A) CrP/ATP is plotted versus cardiac oxygen consumption rate (MVO2). (B) ΔPi/CrP is plotted as a function of MVO2. Solid lines indicate model predictions from
Also note that in
It is possible that the baseline signal identified as Pi by Katz et al.
Contrasting that evidence for constant Pi with, for example, the six studies summarized in
Synthesizing the available data on phosphate metabolites in the normal heart in vivo shows: CrP/ATP decreases either slightly (perhaps 15%) or not at all with increases in MVO2, while Pi increases from somewhere below the minimum detectable concentration at baseline to more than two times the minimum detectable concentration at maximal or near maximal MVO2. Even in the detectable range the Pi measurements suffer from substantial noise, resulting in large uncertainty in ΔPi/CrP. Thus the nature of the relationship between Pi and MVO2 is not clearly revealed due to noise and individual variability in the measurements. Moreover, due to both the low signal-to-noise ratio and the fact that the Pi signal originates from both the cytoplasm and the mitochondrial matrix, absolute Pi concentrations (in mass per unit volume) are not easily estimated from these relative measurements.
These data inspire the formulation of an alternative to the metabolic stability hypothesis. Formally, we state the metabolic stability hypothesis as “biochemical feedback is not adequate to explain the regulation of energy metabolism in the heart,” and its alternative, “biochemical feedback is adequate to explain the regulation of energy metabolism in the heart.”
These alternative hypotheses are clearly mutually exclusive, satisfying the requirement of Platt's Step 2. In this case, since these hypotheses relate to potential mechanisms underlying data that have already been collected, Platt's Step 3 takes the form of a simulation experiment. Specifically, if an independently validated simulation of energy metabolism in the heart, in which the rate of oxidative phosphorylation is controlled by biochemical feedback, is able to match the available data, then the stability hypothesis will be disproved and the alternative proved. (Formal proof of the stability hypothesis would be difficult since it would require exhaustively ruling out all theoretical possibilities. Luckily, we shall disprove the stability hypothesis, demonstrating that such a proof is impossible.)
In a recent series of papers we have developed a computer model of cardiac energy metabolism that provides the means to distinguish the hypotheses. Briefly, the model to simulate mitochondrial metabolism and electrophysiology is based on a large number of kinetic and steady-state data from purified mitochondria
Integrating this energy metabolism model with a model to simulate oxygen transport in the myocardium, we are able to simulate the experiments summarized in
Furthermore, these simulations help resolve an old scientific debate that was never adequately concluded. One of us recalls that twenty years ago the findings of Balaban and colleagues generated extensive and compelling discussions both in the literature and at scientific meetings: “You did not consider alternative hypotheses!”, “The methods are not precise enough to justify your conclusion!”, “How did you calibrate the NMR [nuclear magnetic resonance] spectral signals into concentrations?”, “How do you know that the CK is maintained at equilibrium in the cell so that the ADP concentration can be calculated?”, “What experiment can distinguish feed-back from feed-forward control?” While such questions were debated extensively at poster sessions for years, the critical disproofs that came later were presented in terms that decidedly avoided challenging the status quo.
Recall that the metabolic stability hypothesis is formulated on the foundation that both biochemical substrates for oxidative phosphorylation, ADP and Pi, remain essentially constant in the heart. Thus the observations of significant increases in Pi with MVO2 are appropriately interpreted as a simple direct disproof. Yet, those observations were presented as consistent with the Balaban data. Specifically, much was made of the fact that the later experiments achieved rate-pressure products (RPP) significantly higher than the earlier studies and that only at the higher values of RPP does the Pi signal become observable. Making this point, one of us is guilty of this sort of equivocation intended to avoid conflict. In discussing our model predictions in comparison to the data on inorganic phosphate, Wu et al.
Putting equivocating statements aside, the experiments of Zhang and colleagues directly contradict the assertion that concentrations of substrates for ATP synthesis do not significantly change in response to changes in ATP hydrolysis rate. By comparing computer simulations to these data, we are able to firmly disprove the associated metabolic stability hypothesis. This outcome is an affirmation of strong inference. The stability hypothesis of Balaban and coworkers has stood for twenty years and inspired deeper investigations into the control of energy metabolism in the heart. Its disproof is an opportunity for further refinement and progress. Furthermore, this disproof does not represent a proof of our computational model. However, the integrated model has been validated on the basis of a comparison of the data in
Histological inspection of a skeletal and cardiac myocyte shows that they do not resemble a reaction chamber akin to a well-stirred chemistry flask. Bessman and Geiger
Phosphocreatine (CrP) is synthesized from ATP near sites of ATP synthesis and diffuses to sites of ATP hydrolysis, where ATP is synthesized from CrP.
Hypotheses about the nature of the cellular interior and the properties of water as solvent and metabolites as solutes remain fruitful avenues of investigation. Macroscopic features can hide microscopic properties. For example, measured diffusivities of water and of low molecular weight solutes in gels that are macroscopically a solid, such as a gelatin cube, are slightly reduced compared to diffusivity in pure aqueous solution. From these studies Wang developed a view of proteins as macromolecular strands in an aqueous environment that obstruct random three-dimensional diffusion and so decrease the measured diffusivity by a calculable extent
We now consider how strong inference can advance these questions. Two issues are intertwined and need to be separated. First, the descriptions of “the creatine phosphate shuttle” and “intracellular compartments” admit various definitions and interpretations; compartments bounded by membranes are clearly defined. The concepts of the shuttle and compartmentalization have been described qualitatively from the original diagrams in
Meyer et al.
In experiments made specifically to falsify the “obligatory shuttle” or the “facilitated shuttle” hypotheses, NMR polarization transfer methods were used to measure unidirectional fluxes of the CK reaction: CrP→ATP and the reverse
(Walliman
The phosphocreatine shuttle hypothesis assigns a transport role to the CK system that is more significant and complex than the facilitated diffusion process described by Meyer et al.
The facilitated diffusion theory predicts that there are “no significant diffusion gradients” of ATP, ADP, or Pi over the distance from mitochondrion to local sites of ATP hydrolysis in the normal cardiomyocyte
The hypothesis that there is no significant diffusion barrier between sites of mitochondrial ATP synthesis and cellular ATP hydrolysis in the cardiomyocyte is supported by a set of experiments reported by Kaasik et al.
Evidence for a more crucial role of a CK-mediated transport system includes observations that the ADP concentration necessary to achieve the half-maximal rate of oxidative phosphorylation (
However, the picture painted by the permeabilized cell data is not entirely clear, because the permeabilized cell experiments lack an ideal control. Specifically, the apparent
In addition, more direct observations on effective diffusion coefficients in striated muscle cells directly challenge subhypothesis 1. Kushmerick and Podolsky
Saks et al.
Saks et al.
All of these observations may be explained by the direct-transfer hypothesis
There is an obvious unresolved conflict when two labs report opposite results for the same experiment. Either one or both of the results is incorrect. In this case, neither result is entirely clear because it is unknown how oxidative phosphorylation could be viable with the “removal of the mitochondrial outer membrane” since cytochrome c—a necessary redox carrier in the electron transport system—diffuses freely in the intermembrane space. It is possible that the digitonin treatment in these experiments yielded a heterogeneous distribution of partial permeabilization and functional characteristics of mitochondria. Regardless of any possible explanation, from the conflicting results it can only be concluded that the hypothesis that the outer membrane presents a permeability barrier that is responsible for the apparent functional coupling remains without a rigorous attempt at disproof.
Finally, kinetic experiments on the purified mtCK enzyme
This is a simple, clear and unambiguous statement that is phenomenologically consistent with all of the independently reproduced data on functional coupling of mtCK and ANT. What remains is to determine if this statement can be successfully cast in terms of a quantitative hypothesis and associated theoretical/computational model. Attempts so far have not been successful. For example, Vendelin et al.
Saks and colleagues promote a hypothesis that “ATP is directly channeled by ANT from matrix into microcompartment (‘gap’) between” ANT and mtCK
The subhypothesis that the CK reaction mass-action ratio can be far from equilibrium in the cytoplasm of a cardiomyocyte is commonly proposed as a component of the phosphocreatine shuttle hypothesis
Simulation studies of Saks and Aliev
Here we have demonstrated the utility of applying Platt's strong inference to critical questions in cardiac energy metabolism. Regarding the hypothesis of metabolic stability, Platt's rigorous logic provides a framework for disproof and for establishing and testing alternatives. In doing so, we have shown seemingly contradictory data (on metabolic kinetics in purified mitochondria
Hence further progress in this field, and in biological systems research in general, will rely on further application of strong inference. Likewise, application of strong inference to complex biological systems requires computational simulation to formulate the hypotheses, to compare hypotheses to data, and to design the experiments to distinguish between alternatives. Biological systems research, strong inference, and computational modeling are constructively and inseparably coupled.
We are grateful for critical feedback on this manuscript from James Bassingthwaigthe, Kevin Conley, Allen Cowley, Peter Hunter, Ronald Meyer, Kalyan Vinnakota, and Robert Wiseman.