rMAP: the Rapid Microbial Analysis Pipeline for ESKAPE bacterial group whole-genome sequence data.
The main results reproduced, with only marginal, non-material deviations.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH: yes. rMAP is a TOOL/methods paper that reports NO per-accession numeric result (no N50/contig/ST/AMR table for SRR8948878), so a 1:1 numeric match is impossible by construction. The faithful reproduction is an executability + delivers-what-promised run: all 9 in-scope rMAP pipeline steps were executed at repo-pinned tool versions on the paper's own demo accession SRR8948878 (E. faecalis IW11) on «our HPC» «infra», and every promised output category was produced -- FastQC; Trimmomatic (99.95% survival); MEGAHIT assembly (3501 contigs, 5.02 Mb, N50 67150, GC 37.93%); MLST efaecalis ST16; abricate AMR across 5 DBs; AMRFinderPlus (5 genes, concordant with abricate: erm(B)/lsa(A)/dfrG/tet(M)); abricate virulence (26 VFs) + plasmidfinder (4 replicons); Prokka annotation (4216 CDS). Cross-tool concordance (AMRFinderPlus vs abricate) and biologically coherent E. faecalis output indicate the shipped demo data genuinely works through the pipeline. NOT ATTEMPTED (out of scope, stated): variant calling (needs user reference), IQ-TREE phylogeny (needs >=4 genomes), Roary pangenome (needs multiple GFFs), ISMapper, Table 4 cohort runtimes. No fabrication concern: the paper makes no falsifiable per-accession numeric claim. Per-claim grade='partial' is the closest controlled value meaning 'step delivered, but not a numeric match because none exists'; overall outcome = reproduced (executability).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-24
- Rubric version
- not recorded
- Assessed by
- —
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThere is no existing all-in-one, user-friendly bioinformatics pipeline that can comprehensively analyze whole-genome sequencing data of ESKAPE pathogens (assembly, variant calling, phylogenetics, AMR/plasmid/virulence profiling, MLST, pangenome, and insertion sequence analysis) for users without bioinformatics expertise or high-performance computing infrastructure.
- ★ rMAP is a pipeline capable of profiling the resistomes of ESKAPE pathogens using Illumina WGS data resource
- ★ rMAP automates the full WGS analysis workflow from raw reads (trimming, assembly, annotation, variant calling, phylogeny, AMR/plasmid/virulence/MLST/pangenome/IS analysis) to an interactive html report method
- ★ No previously published all-in-one tool provides this combined feature set (assembly stats, SNP calling, phylogenetics, AMR/plasmid/virulence profiling, MLST, pangenome, IS characterization) for ESKAPE pathogens finding
- ★ rMAP is designed for users with little bioinformatics expertise and can run comprehensive analyses on a personal laptop in low-resource settings resource
- ★ Megahit assembly is much faster (almost half the time) than Shovill but produces slightly lower quality assembly metrics finding
- ★ Shovill produces higher-quality assembly metrics (N50, L50, genome length) via reference-guided assembly and Pilon polishing but takes more time than Megahit finding
- ★ rMAP was benchmarked on ESKAPE pathogen WGS datasets from SRA/ENA spanning clinical, faecal, animal and sewage origins method
- rMAP uses SNP-based phylogenetic analysis because it is faster than using full sequencing data to infer phylogeny mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| de novo genome assembly | ESKAPE isolates (Enterococcus spp., Acinetobacter baumannii, Klebsiella spp., Staphylococcus aureus) from SRA/ENA | none | assembly quality metrics (N50, L50, genome length, GC content, sequencing depth) | Shovill or Megahit (Illumina reads) |
| SNP variant calling | same ESKAPE isolates | none | single-nucleotide polymorphisms (VCF files) | BWA, Freebayes, Samtools, bcftools, Vt, SnpEff |
| phylogenetic inference (SNP-based maximum likelihood) | same ESKAPE isolates | none | phylogenetic trees (rectangular/circular phylogram, cladogram) | Mafft, BMGE, IQtree (1000 bootstraps) |
| antimicrobial resistance gene profiling | same ESKAPE isolates | none | presence/absence and coverage/identity of AMR genes | Abricate/AMRfinder against CARD, ResFinder, ARG-ANNOT, NCBI, MEGARES |
| plasmid profiling | same ESKAPE isolates | none | plasmid replicon types | PlasmidFinder via Abricate |
| virulence factor profiling | same ESKAPE isolates | none | virulence gene presence/absence | VFDB via Abricate |
| multi-locus sequence typing (MLST) | same ESKAPE isolates | none | sequence types / clonal characterization | mlst tool |
| pangenome analysis | same ESKAPE isolates | none | core/accessory gene presence-absence matrix | Roary, Fasttree |
- ▼ Megahit assembly is much faster than Shovill, exhibiting almost half the computation time of its counterpart ~2-fold faster (about half the time)
- ▲ Shovill produces better quality assembly metrics (N50, L50, genome length) than Megahit but takes longer due to reference-guided mapping and Pilon polishing
- – IQtree infers phylogeny by testing substitution models and using maximum likelihood with 1000 bootstraps 1000 bootstraps
- – Antibiotic resistance gene presence/absence in the report is computed using a coverage and identity threshold >90%
- – No prior published all-in-one tool combines assembly stats, SNP calling, phylogenetics, AMR/plasmid/virulence profiling, MLST, pangenome analysis and IS characterization for ESKAPE pathogens
- other Trimmomatic default phred quality score of 27 (default quality trimming parameter)
- other minimum sequence length of 80 base pairs (default Trimmomatic trimming parameter)
- count 1000 bootstraps (IQtree maximum-likelihood phylogenetic inference)
- other coverage and identity >90% (threshold for antibiotic resistance gene presence/absence in report)
- count 14 Enterococcus isolates (benchmarking dataset (SRA accessions SRR8948878-SRR8948891))
- count 9 Acinetobacter baumannii isolates (benchmarking dataset (SRA/ENA accessions))
- count 18 Klebsiella species isolates (benchmarking dataset (SRA accessions))
- count 15 Staphylococcus aureus isolates (benchmarking dataset (ERR1794900-ERR1794914))
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This paper describes rMAP, a bioinformatics software pipeline for whole-genome sequence analysis of ESKAPE bacterial pathogens, and reports its performance by running it on a benchmarking set of publicly available Illumina sequencing datasets from SRA/ENA. Results are reported as descriptive bioinformatics outputs (genome assembly metrics such as N50, L50, genome length, and GC content; presence/absence calls for antimicrobial resistance, plasmid, virulence, and insertion-sequence genes; MLST typing; pangenome composition) rather than as outcomes of formal inferential hypothesis tests. Phylogenetic relationships among isolates are inferred using maximum-likelihood methods with bootstrap resampling for branch support. No between-group statistical comparisons (e.g., t-tests, ANOVA) or multiplicity-adjusted significance testing are described.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Maximum-likelihood phylogenetic inference (IQtree/Fasttree) with 1000 bootstrap replicates | SNP-based phylogenetic tree construction from aligned isolate genomes | all input samples analysed in a given pipeline run (specific benchmarking sample counts given per species, e.g. 14 Enterococcus, 9 A. baumannii, 18 Klebsiella, 15 S. aureus isolates) | not stated |
| Bayesian haplotype-based variant calling (Freebayes) | SNP/variant calling from aligned sequencing reads to produce VCF files | — | not stated |
-
Phylogenetic trees are built using maximum-likelihood inference with 1000 classical bootstrap replicates for branch support.↳ Could also: Bayesian phylogenetic inference (e.g., BEAST, MrBayes) or fast approximate likelihood-ratio support measures (e.g., SH-aLRT, UFBoot) — These approaches can provide posterior probability support or computationally faster approximations to bootstrap support, which some researchers use alongside or instead of classical bootstrapping to characterize confidence in tree topology.
-
Variant calling relies on Freebayes' Bayesian haplotype-based model.↳ Could also: Cross-checking with an alternative caller such as GATK HaplotypeCaller or bcftools call — Different variant callers use different statistical models and assumptions; comparing concordance across callers can characterize how sensitive variant calls are to modeling choices.
-
Genome assembly quality is summarized with single point-value metrics per sample (N50, L50, genome length, GC content) without reported variability.↳ Could also: Reporting a range or distribution (e.g., across assemblies with varied parameters or subsampled reads) — Showing spread in addition to point estimates can convey how sensitive assembly metrics are to input data or parameter choices.
-
AMR, plasmid, and virulence gene detection is reported as presence/absence based on a coverage/identity threshold (>90%).↳ Could also: Reporting concordance statistics (e.g., sensitivity/specificity, Cohen's kappa) against a curated reference standard — Quantitative agreement statistics can complement presence/absence calls by summarizing detection performance when a gold-standard comparison set is available.
-
Pipeline benchmarking is demonstrated on a defined set of publicly available isolates without a quantitative comparison to outputs from other pipelines (e.g., Tormes).↳ Could also: A systematic side-by-side benchmark reporting agreement metrics (e.g., correlation or concordance coefficients) between rMAP and comparable pipelines — Numerical agreement measures can provide an additional, quantitative way to describe how outputs from different pipelines relate to one another on the same input data.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.