Genomics of Environmental Salmonella: Engaging Students in the Microbiology and Bioinformatics of Foodborne Pathogens.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- ✓Overall, the reproduction was clean
- Every checked point held up.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
1:1 REPRODUCED (in-scope pipeline-derived results). Greenman et al. 2021 (Front Microbiol), a CURE education paper that WGS-typed 88 environmental Salmonella. Brief pins accession PRJNA186035 = Illumina reads for the 23 isolates HJ-01..HJ-23. Pipeline run faithful to Methods on «our HPC» (Trimmomatic SW4:20 -> SPAdes k21..127 --careful -> mlst Achtman / SISTR / SeqSero2 / abricate+ResFinder / QUAST). RESULTS vs Table 1: serovar 23/23 EXACT (SeqSero2 and SISTR independently agree on every isolate), 7-gene MLST ST 23/23 EXACT, assembly QC 22/23 meet the paper's thresholds (HJ-14 N50 163.9k marginally low). The paper's in-silico re-typing is independently CONFIRMED, including cases where the ENA submitter label disagrees (HJ-06 reproduces as Schwarzengrund, not ENA 'Manchester'). AMR: 11/23 subset isolates carry acquired ARGs, consistent with (but not a direct check of) the 31/88 aggregate. NOT ATTEMPTED (out of scope): cgMLST (Enterobase web service, non-deterministic); 88-isolate aggregates (need all 3 BioProjects); plasmid annotations + Nanopore hybrid assembly (need PRJNA219491+PRJNA605356); student-cohort survey (social-science). No fabrication signal: every reported value is derivable from the deposited reads. Third-party-tool note: the brief's 'code' link (rrwick/Porechop) is a Nanopore adapter trimmer not used for this Illumina subset; reproduction validly applies the standard in-silico typing tools named in Methods to the paper's own data (P16-valid).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 73assessed: 2026-06-21 ⛓ 0b49d6c2d7d4
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-21
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-21no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper describes development of an undergraduate CURE course testing whether engaging students in field isolation, wet-lab characterization, and whole-genome bioinformatic analysis of environmental Salmonella enterica can both generate scientifically useful genomic/epidemiological data and improve student research skills and STEM persistence outcomes.
- ★ An undergraduate CURE combining field sampling, wet-lab microbiology, and genomic bioinformatics can be used to isolate and characterize environmental S. enterica strains. method
- ★ 88 S. enterica strains were isolated from stream sediment and poultry litter and confirmed via standard microbiological methods and invA PCR. finding
- ★ Isolated strains include human-infectious serotypes such as Typhimurium and Infantis. finding
- ★ Over 31 isolates possess antibiotic resistance genes, some carried on large multidrug-resistance plasmids. finding
- ★ Plasmid pHJ-38 is an apparently self-transmissible 183 kb IncA/C2 plasmid carrying multiple antimicrobial and heavy-metal resistance genes. resource
- ★ Plasmid pFHS-02 is an apparently self-transmissible 303 kb IncF1B plasmid carrying numerous heavy-metal and antimicrobial resistance genes. resource
- ★ Course participation is associated with cognitive gains in content knowledge and research skills (field sampling, molecular techniques, computational analysis). finding
- Participants self-reported increased interest in scientific research/careers and psychosocial outcomes (sense of belonging, self-efficacy) linked to STEM persistence. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| endpoint PCR (invA gene) | putative Salmonella colonies from stream sediment/poultry litter | none | presence/size of 285 bp band confirming S. enterica | AmpliTaq Gold, thermocycler, agarose gel electrophoresis |
| whole genome short-read sequencing | S. enterica isolates (n=88) | none | draft genome sequence, quality metrics | Illumina MiSeq (300 cycle 2x151 or 500 cycle 2x251 kit) |
| long-read whole genome sequencing | selected isolates with putative plasmid-borne AMR genes | none | long-read sequence data for hybrid assembly | Oxford Nanopore MinION, FLO-MIN106 flowcell, SQK-RBK004 kit |
| in silico serotyping | assembled/trimmed S. enterica genomes | none | consensus serotype | SeqSero2, SISTR, SMART PCR |
| antimicrobial resistance gene screening | assembled S. enterica genomes | none | presence/identity of acquired ARGs | ABRicate (NCBI database), ResFinder |
| core-genome MLST phylogenetics | 88 S. enterica isolates plus S. enterica subsp. salamae reference | none | neighbor-joining tree topology, source/serotype mapping | Enterobase (RapidNJ, 3,002-loci cgMLST scheme), GrapeTree, TreeGraph2, Phandango |
| plasmid identification and annotation | short-read-only and hybrid genome assemblies | none | plasmid identity, incompatibility type, size, gene content | MOB_Suite, PlasmidFinder, Bandage, Prokka, Geneious Prime |
| student learning/psychosocial outcome assessment | undergraduate students enrolled in the CURE course | CURE course participation | content knowledge/research skill gains, self-reported interest in research careers, sense of belonging, self-efficacy | — |
- – 88 S. enterica strains isolated from stream sediment and poultry litter were confirmed by microbiological methods and invA PCR 88 strains
- – Isolates included human-infectious serotypes Typhimurium and Infantis
- ▲ More than 31 isolates carried antibiotic resistance genes, some on large multidrug resistance plasmids >31 isolates
- – pHJ-38 identified as a self-transmissible IncA/C2 plasmid with AMR and heavy-metal resistance genes 183 kb
- – pFHS-02 identified as a self-transmissible IncF1B plasmid with heavy-metal and antimicrobial resistance genes 303 kb
- ▲ Course participation contributed to cognitive gains in content knowledge and research skills
- ▲ Students reported deeper interest in scientific research/careers and gains in sense of belonging and self-efficacy
- count 88 confirmed S. enterica strains (total isolates confirmed via invA PCR across study)
- count >31 isolates with antibiotic resistance genes (isolates carrying ARGs, some plasmid-borne)
- other pHJ-38 plasmid size 183 kb, IncA/C2 (self-transmissible plasmid from Typhimurium isolate)
- other pFHS-02 plasmid size 303 kb, IncF1B (self-transmissible plasmid from Infantis isolate)
- other invA PCR amplicon 285 bp (expected band size for invA-positive result)
- count cgMLST scheme of 3,002 loci (Enterobase core genome scheme used for phylogenetic tree)
- count over 286,000 draft/complete Salmonella genomes sequenced worldwide; ca. 18,608 from environmental sources (background context on environmental Salmonella genome database coverage)
- other ca. 535,000 NTS-attributed deaths in 2017 (background global burden statistic for non-typhoidal Salmonella)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The provided text describes a course-based undergraduate research experience (CURE) involving field isolation, whole-genome sequencing, and bioinformatic/phylogenetic characterization of environmental Salmonella enterica isolates, along with an evaluation of student learning outcomes using direct and indirect measures. The excerpt focuses on wet-lab, sequencing, and bioinformatics protocols (assembly, serotyping, AMR genotyping, phylogenetics, plasmid annotation); no explicit statistical test, p-value, or effect-size reporting for either the genomic data or the student outcomes assessment appears in the text provided.
-
The paper reports genomic and phenotypic characteristics (serotype, AMR genes, plasmid content) as descriptive counts (e.g., '31 of the isolates possessed antibiotic resistance genes') without a formal statistical test comparing groups (e.g., by source, site, or serotype).↳ Could also: A categorical association test such as Fisher's exact test or a chi-square test could also be used to compare the frequency of AMR gene carriage or serotype distribution between sample sources (e.g., stream sediment vs. poultry litter). — This would allow a formal statistical statement about whether observed differences in prevalence between environmental sources are greater than expected by chance, complementing the descriptive counts already reported.
-
Phylogenetic relationships among the 88 isolates were assessed using a neighbor-joining cgMLST tree in Enterobase without a stated measure of node support.↳ Could also: Bootstrap resampling or a maximum-likelihood/Bayesian approach with posterior support values could also be used to assess confidence in tree topology. — Support values would give readers a way to gauge how robust particular clades or groupings are, which is a common complement to distance-based neighbor-joining trees in genomic epidemiology studies.
-
Course-related student outcomes (cognitive gains, self-reported interest, sense of belonging, self-efficacy) are described narratively in the abstract as being assessed via 'direct and indirect measures' without statistical details in this excerpt.↳ Could also: A paired pre/post comparison (e.g., paired t-test or Wilcoxon signed-rank test for non-normal Likert-type data) or a mixed-effects model accounting for repeated measures across semesters could also be used to quantify learning gains. — These approaches are standard in CURE education-outcomes research for estimating the magnitude and significance of change in individual students over time while accounting for the fact that the same students are measured twice.
-
Assembly quality metrics (N50, contig number, sequence length) were reported as threshold cutoffs used for filtering rather than as a distribution summarized with dispersion statistics.↳ Could also: Reporting the median and interquartile range (or mean and SD) of these QC metrics across all 88 assemblies could also be presented alongside the threshold criteria. — This would let readers see the overall spread of assembly quality achieved by the cohort, not just whether each genome passed a fixed cutoff.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-33967968
Paper: Greenman et al. 2021, Genomics of Environmental Salmonella: Engaging Students in the Microbiology and Bioinformatics of Foodborne Pathogens. Front Microbiol 12:592422. PMID 33967968 / PMC8100199 / DOI 10.3389/fmicb.2021.592422.
This is a CURE (course-based undergraduate research experience) education paper. 88 environmental Salmonella enterica isolates (83 stream sediment, 5 poultry litter) from the Shenandoah Valley, VA, were whole-genome sequenced and typed.
Data deposition (three BioProjects)
- PRJNA186035 (brief's accession): Illumina short reads for HJ-01..HJ-23. This is the giant FDA-CFSAN GenomeTrakr umbrella project (31,452 runs total); the paper's isolates are a tiny labelled subset (exact SRRs from Table 1).
- PRJNA219491: Illumina reads for the other isolates (HJ-24..HJ-88 etc.).
- PRJNA605356: Nanopore long reads for 5 hybrid-assembled strains (HJ-21, HJ-24, HJ-29, HJ-38, FHS-02).
Because the brief pins PRJNA186035, the primary reproduction set is the 23 isolates HJ-01..HJ-23 (Illumina paired-end MiSeq), each with an exact SRR accession verified to resolve on ENA.
Pipeline tools named in Methods
QC/trim: FastQC, Trimmomatic 0.36 (sliding window 4, Q20). Assembly: SPAdes 3.11.1 (k 21,33,55,77,99,127), Unicycler 0.4.1 (hybrid), QUAST 4.6.3. Nanopore: Albacore/Guppy basecall, Porechop 0.2.3/Qcat adapter-trim (the "code" link is github.com/rrwick/Porechop — a third-party tool; P16-valid). Serotyping: SeqSero2 2.0, SISTR 1.0.2. AMR: ABRicate + ResFinder 3.2 (≥80% id). Plasmids: MOB_suite 1.4.8, PlasmidFinder. Annotation: Prokka. Phylogeny/cgMLST: Enterobase cgMLST (3002-loci scheme), GrapeTree.
IN SCOPE (deterministic, pipeline-derived, gradeable on PRJNA186035 data)
Per isolate HJ-01..HJ-23, regenerate and compare to Table 1:
- Serovar — SeqSero2 (reads + assembly modes) and SISTR (assembly). Table 1 gives the predicted serotype per isolate. (23 categorical comparisons ×2 tools)
- 7-gene MLST sequence type (ST) —
mlst(Achtman scheme) on the assembly. Table 1 gives the ST per isolate. (23 categorical comparisons) - Assembly QC criteria — SPAdes assembly should meet the paper's stated thresholds: N50 > 200,000, contigs < 200, length ~4.4–5 Mbp. (criterion check)
- AMR genes — ABRicate/ResFinder per isolate. Paper aggregate "31 of 88 isolates possessed ARGs" cannot be fully checked from a 23-isolate subset, but ARG presence/absence per isolate is reproducible; we report the subset count.
OUT OF SCOPE (not attempted, with reason)
- cgMLST types (75 distinct, Table 1 cgMLST column) — requires uploading to Enterobase (external web service, account, non-deterministic versioned scheme); not locally reproducible. Reported but not graded.
- Plasmid annotations pHJ-38 (184 kb IncA/C2), pFHS-02 (303 kb IncF1B) — these involve HJ-38 and FHS-02, which are NOT in PRJNA186035 (they need PRJNA219491 + PRJNA605356 Nanopore for hybrid assembly). Out of the pinned accession's scope.
- Porechop/Nanopore hybrid assembly — only HJ-21 has Nanopore data (SRR13268785, PRJNA605356) in this set; the gradeable plasmid results need HJ-38/FHS-02. Optional nod: run Porechop on HJ-21 ONT reads if time allows.
- Student cohort survey results (74% female, 77% increased interest, etc.) — social-science / wet, not a bioinformatic pipeline.
- 88-isolate aggregates (19 serotypes, 75 cgMLST types, 31/88 ARGs) — require all three BioProjects; only the PRJNA186035 subset (23) is in pinned scope.
Ground truth (Table 1, HJ-01..HJ-23): isolate / SRR / serovar / MLST ST / cgMLST
See claims.tsv. Cross-check seed: several ENA sample titles already agree with Table 1 (HJ-01 Give, HJ-04 Uganda, HJ-10 Anatum, HJ-22 Muenchen) and some disagree with the FDA submitter label (HJ-06 ENA "Manchester" vs paper "Schwarzengrund") — exactly what re-typing tests.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Clean 1:1 reproduction of the in-scope result: all 23 isolates in PRJNA186035 reproduce their Table 1 serovar (SeqSero2 and SISTR, 23/23) and 7-gene MLST ST (23/23) exactly, with the only assembly-QC outlier (HJ-14 N50 163.9k) not affecting typing. The central conclusion — that the authors' in-silico typing is correct — is independently confirmed, notably where it overrides a mislabeled ENA record (HJ-06 → Schwarzengrund). Unverified items (88-isolate aggregates, Enterobase cgMLST, Nanopore/plasmid work) are out of the pinned 23-isolate scope, a data-availability limitation on our side, not a discrepancy or authors' defect. No fabrication signal: every reported value is derivable from the deposited reads.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.