Topological approximate Bayesian computation for parameter inference of an angiogenesis model.
The main results reproduced: recomputed values matched the published ones within tolerance.
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Any deviation was negligible
- 🟡Could not use the authors’ exact input data
- 🟡A deviation arose in the data or preprocessing
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
REPRODUCED (Table 1, primary target) via TWO independent strategies, both run on «our HPC» compute nodes. Self-contained snakemake simulation+ABC paper, authors' own code (commit 6648c58). STRATEGY B (deterministic recompute of the ABC rejection + FNN-entropy/RSSE benchmark from the deposited Zenodo 5562670 summary statistics, «job») reproduces all four Table-1 cells BIT-IDENTICALLY to the deposited benchmarkMean.csv (Image RSSE 4.2998/2SE 0.2532, TABC RSSE 3.6142/2SE 0.2730, Image entropy -2.8635/2SE 0.1179, TABC entropy -3.3146/2SE 0.1156) = paper Table 1 to printed precision -> grade exact. STRATEGY A (fully independent fresh end-to-end: 10000 fresh prior + 100 fresh test ABM simulations -> fresh persistence/image stats -> ABC -> benchmark; no RNG seed -> Monte-Carlo, «job» COMPLETED exit 0, n050, 4:32) gives Image RSSE 4.389/2SE 0.252, TABC RSSE 3.773/2SE 0.284, Image entropy -2.739/2SE 0.116, TABC entropy -3.133/2SE 0.123 -> all four 2SE confidence intervals OVERLAP the paper's (RSSE well within 2SE; entropy CIs overlap), effect sizes match (RSSE gap 0.62 vs 0.69, entropy gap 0.39 vs 0.45) -> grade within-tol. BOTH qualitative claims confirmed in BOTH strategies: TABC lower RSSE AND lower posterior entropy than image stats (more accurate + tighter posteriors). INFRA NOTE: «our HPC» /«infra» hit 100% quota during earlier attempts, killing 4 startup jobs; space was freed and the runs proceeded. Not attempted: cosmetic figure PDFs (graded only on the numeric CSV benchmark = Table 1). Entire paper computational; nothing wet-lab.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-19 ⛓ d1ec4507d3f8
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-25
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetCan topological data analysis (TDA)-derived summary statistics (via extended persistence and persistence images) be combined with Approximate Bayesian Computation (ABC) to infer the haptotaxis (ρ) and chemotaxis (χ) parameters of the spatial Anderson–Chaplain angiogenesis model, and does this topological approach outperform ABC using simpler non-topological spatial statistics?
- ★ TDA summary statistics can be combined with ABC to infer parameters (ρ, χ) of the Anderson–Chaplain angiogenesis model method
- ★ The topological (TDA-based) ABC approach outperforms ABC using simpler statistics based on spatial features of the data finding
- ★ Extended persistence (combining ordinary, relative and extended persistence computations) forces all topological features to have finite length, avoiding information loss from features with infinite persistence method
- ★ Persistence diagrams can be vectorized into persistence images to generate topological statistics usable for statistical inference method
- This work is a first step toward a general framework of spatial parameter inference for biological systems using varied filtrations, vectorizations and summary statistics finding
- A Snakemake workflow implementing the method is publicly available on GitHub (tt104/tabc_angio) resource
- Changes in vessel structure/connectivity in the angiogenesis model make topology (connected components, loops) a useful descriptor of simulated vasculature mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| computational simulation of the Anderson–Chaplain PDE/discrete hybrid model | 2D square lattice (201x201 resolution), in silico angiogenesis model with tumour boundary and endothelial tip cells | varying haptotaxis (ρ) and chemotaxis (χ) parameter values | binary images of vasculature (tip cell/vessel presence per lattice cell) at final timepoint | implementation from Nardini et al. (2021); Snakemake workflow |
| persistent homology / extended persistence (topological data analysis) | simplicial complexes built from simulated binary vasculature images (Moore neighbourhood connectivity) | none (data processing/analysis of simulated output) | birth-death persistence pairs / persistence diagrams, vectorized into persistence images | — |
| Approximate Bayesian Computation (ABC) rejection sampling | simulated data (summary statistics) from the Anderson–Chaplain model | sampling parameters θ (ρ, χ) from prior distribution | approximate posterior distribution over model parameters ρ and χ | — |
- ▲ The topological (extended persistence/persistence image-based) ABC approach outperforms ABC using simpler spatial summary statistics for inferring ρ and χ
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper proposes a simulation-based Bayesian parameter inference framework that combines Approximate Bayesian Computation (ABC) with Topological Data Analysis (TDA) for the Anderson–Chaplain spatial model of angiogenesis. An ABC rejection sampler uses persistence images derived from extended persistent homology (on a sweeping plane filtration of simulated 2D binary images) as summary statistics to approximate posterior distributions over two model parameters (haptotaxis coefficient ρ and chemotaxis coefficient χ). The approach is conducted entirely on in-silico data and its inference quality is compared against that obtained using simpler non-topological spatial summary statistics, with the topological approach reported to outperform the simpler alternatives.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| ABC rejection sampling | Approximate posterior inference over haptotaxis (ρ) and chemotaxis (χ) parameters of the Anderson–Chaplain angiogenesis model | — | not stated |
| Comparison of approximate posteriors under topological vs. non-topological summary statistics | Evaluation of inference quality across different summary statistic choices within the ABC framework | — | not stated |
-
The paper uses an ABC rejection sampler, the simplest ABC algorithm, which draws proposals from the prior↳ Could also: Sequential Monte Carlo ABC (SMC-ABC) or ABC-MCMC could also be used for the same posterior approximation task — SMC-ABC and ABC-MCMC iteratively refine the proposal distribution, making them substantially more sample-efficient than rejection sampling—particularly relevant here because forward simulations of the Anderson–Chaplain model on a 201×201 lattice are computationally expensive
-
Persistence diagrams are vectorized into persistence images (using a kernel with a chosen bandwidth) and compared via Euclidean distance within the ABC kernel↳ Could also: Alternative vectorizations such as persistence landscapes or Betti curves, or the Wasserstein or bottleneck distance applied directly between persistence diagrams, could also serve as summary statistics or distances in ABC — Different vectorizations emphasize different aspects of the persistence diagram; Wasserstein distances are theoretically well-grounded (stable with respect to perturbations) and avoid the need to tune a kernel bandwidth, which may be non-trivial to choose optimally
-
The filtration used is a left-to-right sweeping plane (sublevel set height-function) filtration on a simplicial complex built from the binary vasculature image↳ Could also: Vietoris–Rips, alpha-complex, or Čech filtrations built on the point cloud of vascular tip-cell centroids could also be applied — Radial or proximity-based filtrations capture different geometric information (e.g., clustering density and local loop structure) that a single-direction sweeping filtration may not represent; exploring multiple filtrations could reveal which topological features are most informative about ρ and χ
-
Posterior quality is assessed by comparing the shapes of approximate posteriors visually (as described in the abstract) rather than with a formal numerical metric↳ Could also: Simulation-based calibration (SBC), expected coverage plots, or posterior predictive checks could also be used to quantitatively assess the calibration and accuracy of the ABC posteriors — Formal calibration diagnostics provide a reproducible, quantitative basis for comparing approximate posteriors obtained under different summary statistics, complementing visual inspection
-
Inference is performed using ABC, which requires specifying the tolerance ε and produces an approximation whose quality depends on that choice↳ Could also: Neural likelihood-free inference methods such as neural posterior estimation (NPE) or neural ratio estimation (NRE), as implemented in packages such as sbi, could also be applied with the same topological summary statistics — Neural posterior estimators can amortize the cost of simulation and approximate the posterior with fewer forward model evaluations than ABC rejection sampling; they also avoid the need to tune ε explicitly
-
The paper fixes all Anderson–Chaplain model parameters except ρ and χ, focusing inference on two dimensions of parameter space↳ Could also: A profile-likelihood or marginal posterior approach that treats additional parameters as nuisance variables could also be considered if those parameters are uncertain — Fixing nuisance parameters at point values ignores uncertainty in those parameters and can lead to overconfident posteriors on the parameters of interest; a joint inference or sensitivity analysis over a broader parameter set would characterize this uncertainty
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-35191485
Paper: Thorne, Wattis et al. (Taylor / Nardini group), "Topological approximate Bayesian computation for parameter inference of an angiogenesis model", Bioinformatics 38(9):2529–2535, 2022. PMID 35191485 / PMC9048691 / DOI 10.1093/bioinformatics/btac118. Preprint arXiv:2108.11640.
Code: https://github.com/tt104/tabc_angio (commit 6648c58, 2021-10-11) — the authors' OWN Snakemake workflow. Self-contained: it generates all data via simulation; no external input dataset is required to run.
Data deposit: Zenodo 10.5281/zenodo.5562670 — holds the precomputed results/outputs of the workflow (profiled separately; not an input).
What the pipeline does (all in scope — purely computational)
The workflow (workflow/Snakefile) implements a self-contained simulation +
ABC-inference benchmark:
- Prior sampling (
prior.R): draw model params from the prior. ρ (haptotaxis) ~ U(0, 0.5), χ (chemotaxis) ~ U(0, 0.5).nsampreference draws +ntest"observed/test" draws (no RNG seed set → Monte-Carlo reproduction, aggregates only). - ABM simulation (
simulation.py→angio.py): Anderson–Chaplain (1998) angiogenesis model on a 201×201 lattice (implementation from Nardini et al. 2021), run per parameter set until a sprout tip reaches x=0.95 or t=20. Emits (a) extended persistent homology via sweeping-plane filtration (gudhi), (b) the binary vessel image. - Two summary statistics:
- TABC (topological): persistence images, 4 EPD components × 3 homology
dims × 50×50 resolution, bandwidth 1.0 (
persistenceImage.py). - Image-based (baseline): mass, mean-x, mean-y, max-x (
imageStats.py).
- TABC (topological): persistence images, 4 EPD components × 3 homology
dims × 50×50 resolution, bandwidth 1.0 (
- ABC rejection (
samplePosterior.R): for each ofntestobserved datasets, keep thenpost=500 nearest reference draws by Euclidean distance in summary-stat space (α = npost/nsamp = 500/10000 = 0.05). - Benchmark (
calcBench.R): per test dataset compute- RSSE = sqrt(Σ_{posterior draws} [(ρ−ρ_true)² + (χ−χ_true)²])
- Entropy = FNN kNN entropy estimate (k=4) of the 2-D posterior.
Output
results/benchmark.csv(per-test) andresults/benchmarkMean.csv(mean ± 2·SE acrossntestdatasets) for Image vs Topological.
In-scope reproduction targets
| Target | Source | Metric |
|---|---|---|
| Table 1 (PRIMARY) | benchmarkMean.csv |
Mean RSSE, 2·SE RSSE, Mean Entropy, 2·SE Entropy, for Image vs Topological over 100 test datasets |
| Core qualitative claim | Table 1 | TABC mean RSSE < Image; TABC entropy < Image (more accurate, tighter posteriors) |
| Figure 2 (SECONDARY) | posterior + predictive plots | true params lie in posterior bulk; ridge-shaped ρ–χ correlation |
Paper config (to match): nsamp=10000, ntest=100, npost=500, pi_dim=50, pi_bw=1.0. (Repo default config.yaml ships ntest=10 for a quick demo — we set ntest=100 to match the paper.)
Reported values (Table 1, target)
| Statistic | Mean RSSE | 2σ RSSE | Mean Entropy | 2σ Entropy |
|---|---|---|---|---|
| Image-based | 4.30 | 0.25 | −2.86 | 0.12 |
| Topological | 3.61 | 0.27 | −3.31 | 0.12 |
(Note: the "2σ" columns are computed in calcBench.R as 2·standard-error of the
mean, i.e. 2*sqrt(var/n), not 2 population SD.)
Out of scope
- None wet-lab; entire paper is computational. Figure-level cosmetic plots (PDF rendering) are not graded numerically, only the CSV benchmark values.
- Posterior-predictive
predictive.Rhas a quirk (sample(1:npred)indexes only the first 5 posterior rows) but it affects only Fig-2 predictive plots, not Table 1.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.