Reader Comments

Post a new comment on this article

Reviewer 2: Dennis Prangle

Posted by PLOS_CompBiol on 11 Jan 2013 at 12:26 GMT

[This is a review of the original version. See Text S1 for the version history. The authors’ responses are included in line and are reflected in the published version.]

This is a well written and accessible introductory article. I particularly like the balance struck between describing the simplicity of implementing ABC and the potential drawbacks.

Major comments

(nb I've included full references only for papers not in the original article.)

Much of the material in the "recent methodological developments" section is well established and no longer recent relative to the age of the field (e.g. the Marjoram et al paper was published in 2003). I'd suggest at least renaming this section. Alternatively, much of this material could be incorporated into the "approximation of the posterior" section, as regression correction ideas and MCMC / SMC algorithms are tools commonly used to improve the approximation.

‘’Response: The section has been removed and most of the material has been incorporated into the “approximation of the posterior” section.’’

A little more coverage of applications would be nice. One way to do this without increasing the length of the article would be to explicitly reference recent review papers (Beaumont 2010, Bertorelle et al 2010, Csillery et al 2010, Marin et al 2011[4]) for further details.

‘’Response: We have added a sentence about applications of ABC, with references to these review papers, at the end of the “Example” section.’’

The model comparison section should explain how the ABC rejection sampling algorithm can be adapted to perform inference between models (or give a reference). A reference to more advanced algorithms (e.g. Didelot et al, Toni and Stumpf 2009[5]) would also be helpful.

‘’Response: We have added a reference to the Toni & Stumpf SMC-ABC method for model selection.’’

I agree with Christian Robert's comments that the discussion of a hypothesis H in the motivation section is somewhat confusing, and that links to code could be helpful. Some additional suggestions are the "abc" R package and ABC-SysBio.

‘’Response: See our response to Christian Robert’s comment above.’’

Minor comments

The acceptance criterion should be \rho (\hat{D},D) \le \epsilon not \rho (\hat{D},D)<\epsilon if ε = 0 is to correspond to acceptance of exact matches only.

‘’Response: This has been changed.’’

"Sufficient summary statistics": As Christian writes, it would seem more natural to discuss general summary statistics first, then the special and less practically useful case of sufficient statistics.

‘’Response: This has been changed.’’

"Example": I'd point out that this is an example application only, and more accurate inference is possible here by particle filtering methods. If there were some missing data this would be a more natural ABC application e.g. if only the summary statistic was observed.

‘’Response: We have also added a sentence to point out that it is only an example application, and that the posterior can be computed exactly.’’

"Approximation of the posterior": "...has been justified theoretically under some limiting conditions". The word "limiting" doesn't seem (to me) to describe the measurement error case.

‘’Response: We agree and have reformulated this sentence.’’

"Choice and sufficiency of summary statistics": "Sufficient statistics are optimal..." I'd change to "Low dimensional sufficient statistics". For some models (e.g. iid Cauchy) the only sufficient statistics are the full data set, which would be a poor choice.

‘’Response: This has been changed.’’

"Choice and sufficiency of summary statistics": "...which is approximated with a pilot run of simulations". Something like "...which is approximated by linear regression based on simulated data" would be more accurate.

‘’Response: This has been changed.’’

"Choice and sufficiency of summary statistics": It might be useful to reference a recent comparison[6] (disclaimer: which I contributed to) between methods of choosing summary statistics.

‘’Response: A sentence was added with a reference to the paper.’’

"Bayes factor with ABC and summary statistics": "...can also be used to..." it might be more accurate to say "...is sufficient to..."

‘’Response: This has been changed.’’

"Bayes factor with ABC and summary statistics": "meaningless" seems too strong as the next sentence suggests a potentially useful alternative way of doing inference.

‘’Response: The formulation was changed to “may therefore be misinformative”.’’

"Prior distribution and parameter ranges": "...based on the principle of maximum entropy". A link to the general topic of objective priors might be helpful here.

‘’Response: A link has been added.’’

"Large data sets": "which may be a tractable approach for ABC based methods". Note it is already easy to parallelise many of the steps in ABC algorithms based on rejection sampling and SMC.

‘’Response: This has been changed.’’

"Curse of dimensionality": Some theoretical results have been proved here[7][2].

‘’Response: We have added references to these papers.’’

"Conclusion": "With faster evaluation of the likelihood function..." I'm not sure what this is getting at; in ABC applications the likelihood function typically cannot be evaluated!

‘’Response: This formulation has been changed.’’

No competing interests declared.

RE: Reviewer 2: Dennis Prangle

PLOS_CompBiol replied to PLOS_CompBiol on 11 Jan 2013 at 12:30 GMT

[This is a review of the first revision.]

I have read the revised article and discussion of the amendments, and am happy to accept it for publication.

No competing interests declared.