Bayesian prediction of microbial oxygen requirement.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- ✓Overall, the reproduction was clean
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Successfully reproduced all three Bayesian oxygen-requirement classification methods described in the paper (one-step, two-step, Aerobe/Anaerobe-only) via full leave-one-out cross-validation on the shipped 363-genome x 13,669-Pfam-domain matrix, run on the «our HPC» HPC cluster via SLURM. One-step and Aerobe/Anaerobe-only per-class MCCs match the paper's reported values to 2 decimal places (grade: exact). Two-step MCCs are directionally and qualitatively consistent -- correctly reproducing the paper's key claim that two-step classification improves Facultative-class MCC over one-step -- but are not numerically exact (grade: within-tol). A genuine numerical-underflow edge case (floating-point underflow to exactly 0.0 during Bayesian probability normalization) was discovered, documented, and fixed with a principled fallback without altering the core algorithm. A real discrepancy within the paper itself was found and documented: Table 5 states 378 total genomes while the shipped data (and the paper's own Tables 3/4) contain 363. Two supplementary per-genome comparison analyses (two-step and Aerobe/Anaerobe-only vs shipped predictions) were blocked by a shared-account SLURM home-directory quota exhaustion confirmed fleet-wide by the orchestrator as an external infra issue, not a reproduction failure; the one-step per-genome comparison that did complete showed 82.68% agreement with the authors' shipped predictions.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-07-31
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-07-31no human curator yet
- Last updated
- 2026-07-31
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan bacterial oxygen requirement (aerobe, anaerobe, facultative anaerobe) be predicted from genome sequence alone using naive Bayesian inference over the presence/absence of class-associated Pfam-A protein domains, and does a two-step Bayesian network outperform a single-step three-class prediction?
- ★ A naive Bayesian classifier based on presence/absence of class-associated Pfam-A domains can distinguish three oxygen requirement classes (aerobe, anaerobe, facultative anaerobe) from genome sequence, unlike prior studies that only made pairwise distinctions. method
- ★ A two-step Bayesian network (first respiring vs non-respiring, then aerobe vs facultative among respirers) performs best, improving facultative anaerobe prediction over the one-step approach. finding
- ★ The method's performance is as good as or better than previously published methods while being simpler; it is nearly twice as accurate as Wu & Moore for aerobe/anaerobe distinction. finding
- ★ Aerobes and facultative anaerobes share an enzymatic/domain profile associated with oxygen respiration that separates them from strict anaerobes, explaining why facultative genomes are most often mis-predicted as aerobes. mechanism
- Mere presence or absence of Pfam-A domains is indicative of oxygen requirement regardless of domain copy number, since performance is comparable to Lingner et al. who used domain counts. finding
- Lists of class-associated Pfam-A domains and their class-conditional likelihoods are provided as a resource, enabling classification of an arbitrary bacterial genome with a Pfam-A profile. resource
- Scripts for model construction, prediction and evaluation are publicly available (GitHub and Zenodo DOI 10.5281/zenodo.7099). resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Genome selection / curation from public database | Bacterial genomes from NCBI genome database (NCBI lproks table oxygen requirement annotation); one randomly selected member per genus per class | none | Number of genomes per oxygen requirement classification (aerobe, anaerobe, facultative) | NCBI list of sequenced genomes (http://www.ncbi.nlm.nih.gov/genomes/lproks.cgi) |
| Gene/protein prediction (in silico proteome translation) | Selected bacterial genomes (378 genomes) | none | Predicted proteomes from genome sequence | Prodigal, default settings |
| Protein domain search (HMM profile scan) | Predicted bacterial proteomes | none | Presence/absence matrix of Pfam-A domains per genome | hmmscan3 (HMMER3 package), default settings; Pfam-A database |
| Statistical enrichment analysis (two-tailed independent t-test) | Pfam-A presence/absence matrix across the three oxygen requirement classes | none | Class-associated (over-represented) Pfam-A domains, defined as present in >=65% of a class and p<0.05 vs other classes; class-conditional likelihoods p(family|class) | — |
| One-step naive Bayesian classification with N-fold (leave-one-out) cross-validation | All 378 bacterial genomes (aerobe, anaerobe, facultative) | none (flat prior 1/3, pseudo-count 0.1) | Posterior probability and assigned class per genome; MCC per class; confusion counts | Custom Python scripts (Additional files 7-9) |
| Two-step Bayesian network classification with N-fold cross-validation | All 378 bacterial genomes; step 1 respiring (aerobe+facultative) vs non-respiring (anaerobe), step 2 aerobe vs facultative among predicted respirers | none (flat prior, pseudo-count 0.1) | Assigned class per genome; MCC per class; confusion counts | Custom Python scripts (Additional files 7-9) |
| Two-class benchmarking prediction (aerobe vs anaerobe only) for literature comparison | Aerobe and anaerobe genomes only (facultative anaerobes excluded) | none | Average misclassification rate and MCC | Custom Python scripts; two-step method |
| Predictive performance evaluation (Matthew's Correlation Coefficient, one-vs-rest binarization) | Prediction outputs for all included genomes | none | MCC per classification | Custom Python script (Additional file 9) |
- – One-step prediction achieves MCC of 0.63 for aerobes, 0.76 for anaerobes and only 0.31 for facultative anaerobes MCC 0.63 / 0.76 / 0.31
- ▲ Two-step Bayesian network improves facultative anaerobe prediction while aerobe and anaerobe MCC remain unchanged facultative MCC 0.31 -> 0.39; aerobe 0.63, anaerobe 0.76 unchanged
- ▼ With the two-step method the fraction of aerobe and anaerobe genomes erroneously predicted as facultative decreased, so the facultative MCC gain comes from better aerobe/anaerobe accuracy rather than better facultative recall aerobes mis-called facultative 11% -> 5%; anaerobes 6% -> 4%; facultative correct 35% -> 33%
- – In one-step prediction, facultative anaerobes are most often mis-predicted as aerobes, consistent with shared respiration-related domain content 44% of facultative genomes predicted aerobe (48% with two-step)
- ▼ For aerobe vs anaerobe distinction alone, the two-step method has a lower misclassification rate than Wu & Moore's COG- and KEGG-based predictions slightly less than 8% (MCC = 0.84) vs 15% (COG) and 13% (KEGG)
- – Performance for aerobe/anaerobe distinction is comparable to Lingner et al. despite using only presence/absence rather than domain counts 0.84 (this study) vs 0.88 (sensitivity x specificity, Lingner et al.)
- ▲ 252 Pfam-A domains were consistently over-represented in anaerobe genomes compared with respiration-capable genomes 252 domains
- – 524 domains distinguish aerobes from facultative anaerobes (402 over-represented in aerobes, 122 in facultative anaerobes) 402 + 122 = 524 domains
- correlation MCC = 0.63 (aerobe), 0.76 (anaerobe), 0.31 (facultative) (One-step naive Bayesian prediction, N-fold cross-validation (Table 1))
- correlation MCC = 0.63 (aerobe), 0.76 (anaerobe), 0.39 (facultative) (Two-step Bayesian network prediction (Table 3))
- correlation MCC = 0.84 (Two-step method restricted to aerobe vs anaerobe genomes only; misclassification rate slightly less than 8%)
- count Aerobe 175, Anaerobe 112, Facultative 91 (Genomes included per oxygen requirement class (Table 5))
- count Aerobe: 137 (87%) aerobe, 3 (2%) anaerobe, 17 (11%) facultative; Anaerobe: 6 (6%), 95 (88%), 7 (6%); Facultative: 43 (44%), 21 (21%), 34 (35%) (One-step prediction confusion table (Table 2))
- count Aerobe: 141 (90%) aerobe, 8 (5%) anaerobe, 8 (5%) facultative; Anaerobe: 8 (7%), 96 (89%), 4 (4%); Facultative: 47 (48%), 18 (19%), 32 (33%) (Two-step prediction confusion table (Table 4))
- other 15% (COG) and 13% (KEGG) average misclassification rate (Wu & Moore published aerobe/anaerobe results used for comparison)
- other 0.88 sensitivity x specificity (Lingner et al. published aerobe/anaerobe prediction performance)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a computational/bioinformatics study rather than a wet-lab experiment: bacterial genomes were classified into aerobe, anaerobe, and facultative anaerobe groups based on Pfam-A protein domain presence/absence profiles. Domains over-represented in a class were identified using a two-tailed independent t-test (p<0.05) on domain frequency, and a naive Bayesian classifier (one-step and two-step versions) was built from these domains and evaluated by N-fold (leave-one-out) cross-validation. Predictive performance was reported using Matthews Correlation Coefficient (MCC) and misclassification/prediction-distribution percentages, with numeric comparison to two previously published classification methods.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| two-tailed independent Student's t-test | identifying Pfam-A domains with significantly different frequency (p<0.05) in one oxygen-requirement class vs. the others, used to build the 'likelihood file' for the Bayesian classifier | per-class genome counts (Table 5): Aerobe=175, Anaerobe=112, Facultative=91 | not stated |
| Matthews Correlation Coefficient (MCC), one-vs-rest per class | evaluating one-step and two-step Bayesian network prediction performance for Aerobe, Anaerobe, and Facultative classifications (Tables 1 and 3) | same per-class genome counts as above, under N-fold cross-validation | na |
-
Pfam-A domain over-representation across classes was assessed using many independent two-tailed t-tests (one per domain), evaluated at p<0.05.↳ Could also: Applying a multiple-testing correction such as Benjamini-Hochberg FDR or Bonferroni across the full set of domain-wise tests would also be a standard approach here. — When many hypothesis tests are run in parallel (one per protein domain), a correction step controls the expected rate of false positives among the domains flagged as class-associated.
-
Differences in domain presence/absence frequency between classes were assessed with a t-test, which is designed for continuous data.↳ Could also: A test built for proportions/counts, such as Fisher's exact test or a chi-square test of independence, could also be used on presence/absence data. — These tests are specifically suited to binary/categorical presence-absence data and can be more directly interpretable than a t-test applied to frequencies derived from counts.
-
Multi-class predictive performance was summarized using MCC computed separately per class (one-vs-rest).↳ Could also: A full multi-class confusion matrix combined with a macro-averaged F1 score or Cohen's kappa could also be reported alongside the per-class MCC values. — These metrics summarize overall multi-class agreement in a single number, complementing the per-class one-vs-rest MCC values already shown in Tables 1-4.
-
Predictive performance (MCC, misclassification rate) is reported as point estimates from N-fold cross-validation without a measure of variability.↳ Could also: Bootstrap resampling or repeated cross-validation could also be used to generate confidence intervals around the MCC and misclassification-rate estimates. — An interval estimate would convey how much the reported performance metrics might vary with different genome samples, complementing the single point estimates given.
-
Performance was compared to two previously published methods (Wu & Moore; Lingner et al.) by directly comparing reported misclassification rates and MCC/sensitivity-specificity numbers.↳ Could also: Where raw prediction outputs are available, a paired comparison test such as McNemar's test could also be used to compare classifiers on the same genome set. — A formal paired test would let differences between methods be expressed with an associated significance level rather than as a direct numeric comparison of summary statistics.
-
A single classifier type (naive Bayesian, in one-step and two-step forms) was used for all predictions.↳ Could also: Benchmarking additional standard classifiers, such as logistic regression, random forest, or support vector machines, on the same domain profiles could also be performed. — Comparing several classifier types on the same data provides a broader picture of how prediction performance depends on the modeling approach chosen, alongside the naive Bayesian results already presented.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Strong reproduction. Running the authors' own LOOCV protocol on the shipped 363-genome Pfam matrix reproduced the one-step MCCs (0.629/0.765/0.311 vs reported 0.63/0.76/0.31) and the Aerobe/Anaerobe-only MCC (0.836 vs 0.84) to two decimals, and the two-step MCCs came within 0.02–0.04 (Aerobe 0.623 vs 0.63, Anaerobe 0.719 vs 0.76, Facultative 0.367 vs 0.39) while preserving the paper's key qualitative claim that two-step improves Facultative prediction (0.367 > 0.311). The residual two-step gap and the 82.7% per-genome agreement with the shipped predictions sit on our side, caused by an underspecified tie-breaking/relabeling rule in the hierarchical procedure rather than by any defect in the data or in the core Bayesian math. Two blemishes are on the authors' side but are administrative, not substantive: Table 5's 378 (175/112/91) genome counts contradict Tables 3/4 and the shipped matrix's 363 (157/108/98), and the cited data DOI (zenodo 10.5281/zenodo.7099) is a code-only archive with no data. One minor per-genome concordance sub-check was left unrun due to a shared-account cluster disk-quota failure, which does not affect any MCC claim.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.