DFAST and DAGA: web-based integrated genome annotation tools and resources.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Reported values were directly comparable
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡Could not use the authors’ exact input data
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce 1:1. The paper's reproducible computational core is its average-nucleotide-identity (ANI) analysis (Fig.3), and the Methods name the EXACT tool used (pyani, github.com/widdowquinn/pyani, Goris-2007 method). Per brief rule P16 this is an ideal third-party-tool reproduction: we ran pyani ANIb (BLAST+ blastn, 1020-nt fragments, >=30% id / >=70% coverage) on the EXACT NCBI Assembly genomes whose accessions are printed on the Fig.3 dendrogram leaves (read directly off the figure), on «our HPC» (SLURM «job», conda env built in-job). RESULTS: (1) L. gasseri (11/12 Fig-3B genomes) splits into exactly 2 subgroups with between-subgroup ANI 93.7% and within-subgroup ANI mean 99.3% -> reproduces the paper's '~93%' and '>98%'. (2) L. jensenii (14 Fig-3C genomes) splits into exactly 2 subgroups with between-subgroup ANI 88.1% and within-subgroup ANI mean 99.7% -> reproduces the paper's '~88%' and '>98%' essentially exactly. These two two-subgroup separations are the paper's HEADLINE NOVEL FINDINGS and they reproduce cleanly. (3) L. delbrueckii: 14 of the 15 subspecies-type-strain pairs fall in 97.26-98.38%, matching the paper's reported 97.2-98.4% almost exactly; the single exception is the jakobsenii(KACC13439, GCA_001263315.1) <-> delbrueckii(DSM20074, GCA_001908495.1) pair at 99.97% (near-identical). The subsp. delbrueckii assembly we used (GCA_001908495.1) was deposited ~Nov 2016 (likely AFTER the Jul-2016 paper), so the authors used a different/earlier delbrueckii genome; the near-identity may also flag a mislabeled public type-strain assembly - exactly the phenomenon DAGA was built to detect. No fabrication signal against the paper. DEVIATIONS: modern bioconda pyani 0.2.13.1 vs 2016 checkout (algorithm stable); ANIb (BLAST+) vs Goris-era blastall (sub-percent); 1 of 12 gasseri genomes (GCA_000814885.1) had no FASTA in its datasets package and was omitted (subgroup structure robust). NOT ATTEMPTED (hard 20%, see scope.md): the 704-genome all-vs-all (247,456 comparisons; exact genome set not enumerated in the paper), Prokka/DFAST annotation, Platanus de-novo assembly, CheckM grades, the web service, and the pheS/rpoA gene-identity numbers (different pipeline). All grades are PROVISIONAL - a human auditor re-derives from the checksummed matrices in reproduction/outputs/.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 89assessed: 2026-06-16 ⛓ 66ce35e23043
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-16
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a web-based integrated annotation pipeline (DFAST) coupled with a curated genome repository (DAGA) provide consistently annotated, quality- and taxonomy-assessed lactic acid bacteria genomes, and can average nucleotide identity (ANI) reliably verify taxonomic affiliation and reveal intraspecific structure within Lactobacillus and Pediococcus?
- ★ DFAST is a web-based bacterial genome annotation and DDBJ submission pipeline with integrated CheckM quality assessment and ANI taxonomic assessment. resource
- ★ DAGA is a genome repository containing 1,421 consistently annotated genomes covering 179 species and 18 subspecies of Lactobacillus and Pediococcus from DDBJ/ENA/GenBank and SRA. resource
- ★ ANI has high discriminative power to determine whether two genomes belong to the same species, with ~95% as the species threshold. method
- ★ ANI analysis detected and corrected mislabeled or misidentified genomes in public databases (77 mislabeled and 55 unidentified genomes reassigned). finding
- ★ Lactobacillus gasseri and L. jensenii each split into two previously unknown intraspecific subgroups whose between-subgroup divergence exceeds the 95% species ANI threshold. finding
- ★ A curated reference protein database tailored for Lactobacillus and Pediococcus was constructed to enable accurate, rapid, consistent annotation. resource
- 28 of 32 genomes labeled 'L. casei' were in fact L. paracasei based on ANI. finding
- Six representative strains showed anomalously high ANI indicating incongruent taxonomic positions. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Genome annotation (Prokka-based customized pipeline) | Lactobacillus and Pediococcus genomes | none | tRNA/rRNA/CRISPR/protein-coding gene predictions and functional annotation | Prokka ver. 1.11 with customized LAB reference database |
| Ortholog clustering for reference database construction | 81 genomes (69 Lactobacillus/Pediococcus + 12 others) | none | orthologous clusters (28,002) and protein/gene name assignments | GET_HOMOLOGUES v1.3 (BLASTP, OrthoMCL); LaCOGs, MBGD, NCBI CDD |
| Average nucleotide identity (ANI) calculation | 191 representative genomes; all-against-all 704 genomes | none | pairwise mean nucleotide sequence identity | pyani (BLASTN, Goris et al. method) |
| Genome quality assessment | all 1,421 DAGA genomes of Lactobacillus and Pediococcus | none | completeness and contamination via single-copy marker genes (409 for Lactobacillus, 664 for Pediococcus) | CheckM v1.0.5 |
| De novo genome assembly | raw Illumina paired-end reads from SRA | none | draft genome sequences | Platanus assembler v1.2.4 with Platanus_trim v1.0.7 |
| Hierarchical clustering / phylogenetic analysis | L. gasseri, L. jensenii, L. delbrueckii subspecies genomes | none | genome distance trees using (1-ANI) | UPGMA clustering |
| Marker gene sequence comparison | 185 representative genomes; L. gasseri and L. jensenii subgroups | none | 16S rRNA, pheS, rpoA gene identity | — |
- – DAGA contains 1,421 genomes (1,389 Lactobacillus, 32 Pediococcus); 743 from DDBJ/ENA/GenBank and 678 assembled de novo from SRA. 1,421 genomes
- – All interspecific ANI values were below 95%; exception L. zeae vs L. casei at 94.4%. <95%; 94.4%
- – In all-against-all comparison of 704 genomes (247,456 pairs), all 239,840 interspecific ANI values <95%, while 198 of 7,616 intraspecific values were <95%. 198/7,616 intraspecific <95%
- – L. gasseri subgroups differed at ANI 93% and L. jensenii subgroups at 88%, while within-subgroup ANI was >98%. 93%; 88%; >98%
- – 77 mislabeled genomes were renamed and 55 unidentified 'Lactobacillus sp.' genomes were assigned names based on ANI; all marked Rating 1. 77 mislabeled; 55 unidentified
- – 28 of 32 'L. casei' genomes were L. paracasei (ANI >98% vs L. paracasei ATCC 25302T, <85% vs L. casei ATCC 393T). 28/32; >98%; <85%
- ▲ Six strains showed anomalously high cross-species ANI (e.g. L. homohiochii 99.9% vs L. fructivorans; L. parakefiri 99.9% vs L. kefiri). 97.1–99.9%
- – ANI among six L. delbrueckii subspecies type strains ranged 97.2–98.4% yet hierarchical clustering separated them. 97.2–98.4%
- count 1,421 genomes covering 179 species and 18 subspecies (total genomes in DAGA across Lactobacillus and Pediococcus)
- count 247,456 pairwise comparisons (704 × 703/2) (all-against-all ANI comparison)
- correlation ANI 93% between L. gasseri subgroups; 88% between L. jensenii subgroups; >98% within subgroups (intraspecific subgroup divergence)
- fold_change ANI 94.4% (L. zeae vs L. casei, only interspecific exception below 95%)
- count 239,840 interspecific ANI values <95%; 198 of 7,616 intraspecific <95% (diversity analysis of 704 genomes)
- correlation pheS identity 96% (gasseri) and 93% (jensenii); rpoA 99% and 98% between subgroups (marker gene support for subgroup separation)
- count 183,469 protein sequences grouped into 28,002 orthologous clusters (reference protein database construction)
- other completeness ≥95% and contamination ≤5% (Rating 4/5); 409 and 664 markers for Lactobacillus and Pediococcus (CheckM quality rating thresholds and marker counts)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a bioinformatics resource paper describing the DFAST annotation pipeline and the DAGA genome repository, and its quantitative analyses are descriptive and comparative rather than inferential. Taxonomic relationships were assessed mainly through pairwise average nucleotide identity (ANI), with a fixed 95% threshold used to distinguish species, and genome groupings were summarized with UPGMA hierarchical clustering on (1 − ANI) distances. Genome quality was characterized with CheckM completeness and contamination metrics and reported via a 5-grade rating and counts in tables. No formal hypothesis tests, p-values, or confidence intervals are reported.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Average nucleotide identity (ANI) comparison against a fixed 95% species threshold (pyani/BLASTN, Goris et al. method) | Pairwise comparisons among 191 representative genomes (Fig. 3A), all-against-all comparison of 704 genomes (N = 247,456 pairs), and detection of mislabeled/misidentified genomes (Tables 3, 4) | 704 genomes giving 704×703/2 = 247,456 pairwise values; 239,840 interspecific and 7,616 intraspecific values; representative set of 191/185 genomes | not stated |
| UPGMA hierarchical clustering on (1 − ANI) genomic distance | Trees for L. gasseri (Fig. 3B), L. jensenii (Fig. 3C), and L. delbrueckii subspecies (Fig. 3D) | — | not stated |
| CheckM completeness and contamination estimation from single-copy marker genes | Quality rating of all genomes (Tables 1, 2); 409 markers for Lactobacillus and 664 for Pediococcus | 1,421 genomes assessed | na |
| Sequence identity comparison of marker/housekeeping genes (16S rRNA, pheS, rpoA) | Corroboration of subgroup separation in L. gasseri and L. jensenii and validation of representative genomes | — | na |
-
Species boundaries were assessed using ANI against a single fixed 95% threshold.↳ Could also: One could also report a digital DNA-DNA hybridization (dDDH/GGDC) value or an ANI value accompanied by an explicit uncertainty band around the threshold. — A second genome-distance metric or an uncertainty range would provide an additional, independent line of evidence and convey how close borderline cases (e.g., the 94.4% L. zeae/L. casei pair) sit to the cutoff.
-
Genome groupings were summarized with UPGMA clustering on (1 − ANI) distances.↳ Could also: Neighbor-joining or a model-based phylogeny (e.g., maximum likelihood on concatenated core genes) could also be used, optionally with bootstrap support values. — These approaches relax the ultrametric (constant-rate) assumption of UPGMA and can attach branch-support statistics, which helps quantify confidence in the inferred subgroups of L. gasseri and L. jensenii.
-
Intra- versus interspecific ANI distributions were described by counts and representative values relative to the 95% line.↳ Could also: The two distributions could also be summarized with medians/IQRs or visualized and compared (e.g., a histogram overlap or a gap statistic). — Distributional summaries would convey the spread and degree of separation between intra- and interspecific values, complementing the threshold-based counts.
-
The subgroup separations in L. gasseri and L. jensenii were supported descriptively by pheS and rpoA identities.↳ Could also: A clustering-validity index (e.g., silhouette width) or a formal multilocus/MLSA framework could also quantify the proposed subgroups. — An internal validity measure would put a number on how well-separated the subgroups are, supporting the suggestion that they 'might deserve subspecies-level differentiation.'
-
Reported identity figures (e.g., the 97.2–98.4% subspecies range) are given as point values or ranges.↳ Could also: Summaries such as mean ± SD or a 95% confidence interval for within- and between-group ANI could also be reported. — Dispersion statistics would communicate the variability behind the summary values, which is often preferred when comparing closely spaced groups.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
Interspecific ANI values fall below the 95% species boundary (lowest exception L. zeae vs L. casei at 94.4%).WGS lactobacillus pediococcus down 2016×1papers★ This paper is the founder (earliest)
-
In all-against-all comparison interspecific ANI is <95% while most intraspecific ANI is >=95% (198/7,616 intraspecific below 95%).WGS lactobacillus pediococcus mixed 2016×1papers★ This paper is the founder (earliest)
-
ANI-based curation renamed 77 mislabeled genomes and assigned names to 55 unidentified Lactobacillus sp. genomes.WGS lactobacillus pediococcus 2016×1papers★ This paper is the founder (earliest)
-
L. gasseri (93%) and L. jensenii (88%) subgroups separate at intermediate ANI while within-subgroup ANI exceeds 98%.WGS lactobacillus 2016×1papers★ This paper is the founder (earliest)
-
Six strains show anomalously high cross-species ANI (up to 99.9%), indicating misidentified or near-identical genomes.WGS lactobacillus up 2016×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — PMID 27867804
Paper: Tanizawa Y, Fujisawa T, Kaminuma E, Nakamura Y, Arita M. "DFAST and DAGA: web-based integrated genome annotation tools and resources." Biosci Microbiota Food Health 2016;35(4):173–184. DOI 10.12938/bmfh.16-003 · PMCID PMC5107635.
Nature of paper: A resource/tools paper. It describes (a) the DFAST genome annotation pipeline (web service) and (b) the DAGA genome repository (1,421 Lactobacillus/Pediococcus genomes). The quantitative, reproducible computational core is the average nucleotide identity (ANI) analysis used to assess taxonomy and reveal genomic diversity.
Code artifact (per brief, P16 — third-party tool is fully valid): The paper itself names the tool: "The pyani script (https://github.com/widdowquinn/pyani) was used to calculate the ANI between two genomes, based on the method by Goris et al." So we reproduce by running pyani (the exact tool the authors used) on the exact genomes they used (read off Fig. 3 panels B/C/D, which label each leaf with its NCBI Assembly accession).
ANI method as described (verbatim, Methods → "Calculation of average nucleotide identity")
"one genome was cut into 1,020 nt fragments, which were searched against the other genome by using the BLASTN algorithm. ANI was calculated as the mean identity of top-hit BLASTN matches for all fragments with a sequence identity of ≥30% and an overall aligned region of ≥70% of the fragment length."
This is exactly pyani's ANIb method (BLAST+ blastn, 1020-nt fragments, Goris
2007 thresholds). Reproduction uses average_nucleotide_identity.py -m ANIb.
In scope (pipeline-derived, attempted) — all from Fig. 3
RESULT G — Lactobacillus gasseri two-subgroup separation (Fig. 3B).
- Claim: two previously-unknown subgroups; ANI between subgroups ≈ 93%, within each subgroup > 98%.
- Genomes (12, exact accessions from Fig. 3B): GCA_000175055.1, GCA_000283135.1, GCA_000177035.2, GCA_000177415.1, GCA_000155935.2, GCA_000439915.1, GCA_000014425.1 (ATCC 33323ᵀ), GCA_000176995.2, GCA_000814885.1, GCA_000406345.2, GCA_000143645.1, GCA_001063505.1.
RESULT J — Lactobacillus jensenii two-subgroup separation (Fig. 3C).
- Claim: two subgroups; ANI between ≈ 88%, within > 98%.
- Genomes (14, exact accessions from Fig. 3C): GCA_000159335.1, GCA_001012735.1, GCA_001012745.1, GCA_001012675.1, GCA_001012655.1, GCA_000161895.2, GCA_000162435.1, GCA_001012665.1, GCA_001012685.1, GCA_000162335.1, GCA_001436455.1 (DSM 20557ᵀ), GCA_000466805.1, GCA_000155915.2, GCA_000175035.1.
RESULT D — L. delbrueckii six-subspecies type-strain ANI range (Fig. 3D + text).
- Claim: ANI among the six subspecies type strains distributed in 97.2–98.4%.
- Type-strain genomes (6):
- subsp. delbrueckii: GCA_001908495.1 (DSM 20074ᵀ, assembly from type material)
- subsp. bulgaricus: GCA_000056065.1 (ATCC 11842ᵀ)
- subsp. lactis: GCA_000192165.1 (DSM 20072ᵀ)
- subsp. indicus: GCA_001189855.1 (JCM 15610ᵀ)
- subsp. sunkii: GCA_001190005.1 (JCM 17838ᵀ)
- subsp. jakobsenii: GCA_001263315.1 (KACC 13439ᵀ)
Out of scope (not attempted) — and why
- All-against-all ANI of 704 genomes (N = 247,456 comparisons; "all interspecific ANI < 95%; 198 of 7,616 intraspecific < 95%"). This is the hard ~20%: the exact 704-genome set is not enumerated in the paper (only summary counts), and 247k BLAST-based pairwise comparisons is a large compute. Skipped; the species-level subgroup analyses (G/J/D) test the same ANI machinery on enumerated genomes.
- DFAST/Prokka annotation pipeline, CheckM quality grades, Platanus de-novo assembly of SRA reads. These are upstream/orthogonal; the headline ANI findings use the public NCBI Assembly genomes directly (Methods: genomes downloaded from NCBI Assembly DB), so we use those assemblies rather than re-assembling.
- **Web service (dfast.nig.ac.jp), mislabe
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The paper's reproducible computational core — the Fig.3 ANI analysis — reproduces cleanly: both novel findings (L. gasseri and L. jensenii each splitting into 2 subgroups, with ~93% and ~88% between-clade ANI and >98% within-clade ANI) match essentially 1:1, and 14/15 delbrueckii type-strain pairs land on the reported 97.2-98.4% range. The only real deviation — a 99.97% jakobsenii↔delbrueckii outlier — is an our-side assembly-provenance artifact (a likely post-paper near-duplicate assembly we selected), not an authors' defect. Remaining caveats (pyani version/ANIb-vs-blastall, one omitted gasseri genome, sample read off the figure) are technical/method-side and do not touch the conclusions. Overall: a strong reproduction with minor, fully explainable our-side deviations.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.