Metavisitor, a Suite of Galaxy Tools for Simple and Rapid Detection and Discovery of Viruses in Deep Sequence Data.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Reported values were directly comparable
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡Could not use the authors’ exact input data
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough -> reproduced 1:1 for the in-scope claim. Metavisitor is a Galaxy tool/software paper; the brief's accession PRJNA254017 = Matranga et al. 2014 (Lassa+Ebola RNA-seq) = the paper's Use Case 3-3, whose headline result (Table 4) is a per-dataset virus presence/absence call. We ran the decisive Metavisitor step directly (P16): bowtie2 (default sensitive end-to-end) of the 150 nt paired reads against the paper's exact references (Ebola NC_002549.1; Lassa L NC_004297.1 + S NC_004296.1) on «our HPC», calling detection from mapped reads + genome coverage breadth. Result: Ebola detected in all 8/8 datasets with ~99.9% genome breadth (EXACT match to Table 4, and effectively reconstructing the genome by guided mapping); Lassa detected in 8/8 of a sampled 8 of 55 datasets (consistent with the reported 96% / 53-of-55 rate). No fabrication concern: Table 4's claims are directly derivable from the deposited reads against the stated references. NOT attempted (the optional ~20%): full Galaxy server / Docker deploy, the complete Trinity de-novo assembly + blastn contig-ID path, re-deriving the exact 53/55 count across all 55 Lassa runs, and the paper's other use cases (UC1 Drosophila Nora-virus reconstruction, UC2 Anopheles C virus discovery, UC3-1 HIV, UC3-2 febrile patients) which use different datasets than the briefed accession. Op notes: heavy compute on «our HPC» SLURM; data staged on front1 with IP-pinned downloads to work around a broken conda-wget and intermittent compute-node DNS for ENA hosts.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 78assessed: 2026-06-16 ⛓ eab7d075571b
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-16
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a modular, Galaxy-based software suite enable biologists and clinicians without specialized bioinformatics expertise to detect, identify, and assemble viral genomes from diverse deep sequencing datasets using combined de novo and reference-guided approaches?
- ★ Metavisitor is an open-source suite of modular Galaxy tools and preset workflows enabling non-experts to detect and assemble viral genomes from deep sequence data. resource
- ★ Metavisitor works with DNA, RNA, and small RNA sequencing data across a range of read lengths and combines de novo and reference-guided approaches to assemble viral genomes. method
- ★ The software has potential for both rapid diagnosis of known viruses and discovery of novel/unexpected viruses from a wide array of organisms. finding
- ★ Workflows can be adapted via the graphical Galaxy editor by adding, replacing, or modifying analysis modules (e.g., swapping assemblers). method
- Small RNA (viRNA) deep sequencing is a potent, genome-type-agnostic approach to detect viruses because antiviral RNAi generates viral small RNAs from RNA and DNA viruses alike. mechanism
- Executable use-case histories and workflows are provided publicly to increase accessibility, transparency, and reproducibility. resource
- ★ Metavisitor reconstructed Nora virus genomes from small RNA datasets with high similarity to the reference guide genome. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| small RNA sequencing (viRNA) reconstruction/assembly | Drosophila melanogaster (Nora virus infection), SRP013822 libraries | none | reconstructed viral genome consensus sequence, coverage, nucleotide identity | Illumina; Oases/Velvet assembler, blastn/blastx, CAP3, blast_to_scaffold |
| de novo assembly + BLAST detection workflow | deep sequence datasets (fruit fly, mosquito, plant, clinical/various organisms) | none | viral contigs identified via blastn/blastx against vir1 nucleotide/protein databases | Galaxy; Oases, Trinity, SPAdes assemblers; NCBI BLAST+ |
| long-read de novo assembly (Use Case 3-3) | deep sequence dataset | none | assembled viral contigs/genome | Trinity assembler (Galaxy) |
| read normalization workflow (Use Case 1-3) | Drosophila melanogaster SRP013822 small RNA reads | none | Nora_Median-Norm-reads reconstructed genome | Galaxy tool 'Normalize by median' |
| read remapping for assembly quality evaluation | SRP013822 reads vs reconstructed and guide Nora genomes | none | read mapping/coverage across reconstructed genomes | Galaxy remapping workflow |
- – All three Metavisitor-reconstructed Nora genomes and the Paparazzi JX220408 genome shared high nucleotide identity with the NC_007919.3 guide genome >96.6% nucleotide identity
- – Nora_raw_reads and Nora_Median-Norm-reads de novo assemblies entirely covered the JX220408 and NC_007919.3 genomes 12333 nt (full length)
- ▼ Nora_MV de novo assembled portion was marginally shorter than the guide genome (first 31 5' nt recovered from guide) 12298 nt vs 12333 nt
- – Previously, Paparazzi-reconstructed rNora genome (JX220408) differed from the Nora reference NC_007919.3 3.2% nucleotide difference
- ▲ Paparazzi rNora reconstruction improved viral siRNA alignment rate relative to prior reference ~121%
- ▲ In prior work, Paparazzi improved consensus and coverage of the Nora virus genome versus previous reference ~20%
- other >96.6% nucleotide identity (similarity of reconstructed Nora genomes to NC_007919.3 guide)
- other 3.2% (nucleotide difference between rNora (JX220408) and NC_007919.3)
- fold_change ~121% (improvement in viral siRNA alignment rate by rNora genome)
- fold_change ~20% (Paparazzi improvement of Nora consensus/coverage vs prior reference)
- count 12333 nt (length of JX220408 and NC_007919.3 Nora genomes fully covered by assemblies)
- count 12298 nt (de novo assembled length of Nora_MV genome)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a bioinformatics software methods paper describing Metavisitor, a Galaxy-based tool suite for virus detection and genome assembly from deep sequencing data. Validation is demonstrated through use cases applied to publicly available datasets, with assembly quality assessed by remapping reads to reconstructed genomes and reporting nucleotide identity percentages and coverage statistics. No inferential statistical tests are employed; performance is described through bioinformatics metrics (BLAST E-values, bit scores, percent nucleotide identity, subject coverage fractions, and read alignment rates).
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| BLAST similarity search (blastn / blastx) with E-value thresholding | All use cases: de novo contig alignment to viral nucleotide and protein databases (vir1) | — | not stated |
| Read remapping / alignment rate comparison | Use Cases 1-1, 1-2, 1-3: SRP013822 reads remapped to three reconstructed Nora virus genomes and JX220408 guide genome | — | not stated |
| Pairwise nucleotide identity comparison | Reconstructed Nora virus genomes versus NC_007919.3 reference (>96.6% identity reported) | — | not stated |
-
Assembly quality is evaluated by remapping reads to reconstructed genomes and reporting percent coverage and nucleotide identity to a reference↳ Could also: Formal assembly benchmarking tools such as QUAST or GAGE could also be used, reporting N50, NGA50, misassembly counts, and indel rates against the reference genome — Standardized assembly metrics provide a more comprehensive and reproducible basis for comparing multiple assembler outputs and would facilitate direct comparison with other tools in the field
-
Multiple de novo assemblers (Oases, Trinity, SPAdes) are compared with the qualitative statement that they give 'similar outputs'↳ Could also: A quantitative head-to-head benchmark reporting assembly completeness (e.g., BUSCO scores or reference-based coverage fractions) and contiguity (N50) for each assembler on the same dataset could also be used — Quantitative comparisons make it easier for users to choose among assembler options for their own data and allow the claim of similarity to be evaluated objectively
-
The improvement in viral siRNA alignment rate is reported as a single percentage point estimate (~121%) with no measure of uncertainty↳ Could also: A bootstrap confidence interval around the alignment rate difference, or reporting the rate across independently processed sub-samples, could also accompany the point estimate — A confidence interval would convey the precision of the improvement estimate and allow readers to gauge how much of the gain is attributable to the assembly versus sampling variation in the sequencing data
-
BLAST E-value is used as the sole significance threshold for determining whether a contig is of viral origin↳ Could also: Reporting the distribution of alignment identity and query/subject coverage alongside E-value, or applying a minimum bit-score cutoff in addition to E-value, could also characterize hit quality — E-value is sensitive to database size and sequence length; supplementing it with percent identity or coverage thresholds provides additional context about biological relevance and reduces spurious hits from short, low-complexity alignments
-
Reconstructed genome sequences are compared to references by reporting a single overall nucleotide identity percentage↳ Could also: A sliding-window identity plot or pairwise distance matrix across the full genome length could also illustrate where reconstruction accuracy varies regionally — Viral genomes often have heterogeneous coverage and conservation; position-level identity visualization would reveal whether any genomic regions are systematically less well reconstructed by the workflow
-
The three workflow variants (collapsed, raw, and median-normalized reads) are compared descriptively across a single dataset↳ Could also: Applying all three workflows to multiple independent datasets and summarizing reconstruction completeness with summary statistics (e.g., median and IQR of coverage across datasets) could also characterize workflow robustness — A single-dataset demonstration cannot distinguish workflow-specific effects from dataset-specific idiosyncrasies; multi-dataset evaluation would strengthen generalizability claims
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
Assessed papers, coloured by verdict. Click a node to open it.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-28045932 (Metavisitor)
Paper: Carissimo, van den Beek, Vernick, Antoniewski (2017) Metavisitor, a Suite of Galaxy Tools for Simple and Rapid Detection and Discovery of Viruses in Deep Sequence Data. PLoS One 12(1):e0168397.
Metavisitor is a tool/software paper: a suite of Galaxy wrappers + workflows (repo github.com/ARTbio/tools-artbio; Docker artbio/metavisitor:1.2) for detecting and assembling viral genomes from deep-sequencing data. The paper validates the suite on a series of use cases, each a bioinformatic pipeline applied to public sequencing data → all pipeline-derived (in principle in scope).
Brief accession PRJNA254017 → Use Case 3-3
The brief pins sra:PRJNA254017. That BioProject = Matranga et al. 2014
("Enhanced methods for unbiased deep sequencing of Lassa and Ebola RNA viruses",
Genome Biol), which supplies the Lassa & Ebola clinical RNA-seq used in the paper's
Use Case 3-3. So PRJNA254017 maps to UC 3-3, not UC 2. UC 3-3 also uses
PRJNA257197 (Gire et al. Ebola) per the manual, but the Ebola runs SRR1613377–84
are part of the Matranga deposit.
In scope (attempted) — Use Case 3-3 detection (Table 4)
- Result reproduced: per-dataset detection of Ebola virus and Lassa virus by the Metavisitor approach. Paper Table 4: Ebola detected in all 8 Ebola datasets; Lassa detected in 53 of 55 Lassa datasets.
- Pipeline (per UC 3-3 manual/methods): direct bowtie2 alignment of the 150 nt paired RNA-seq reads to the viral reference (no host depletion), then Trinity assembly + blastn against viral DB. Detection = viral reads map / viral contig recovered.
- Reference genomes used by the paper: Ebola NC_002549.1; Lassa segment L NC_004297.1, segment S NC_004296.1.
- Our faithful 1:1 core: run the detection step (bowtie2 to the paper's exact reference genomes) on the 8 Ebola datasets (SRR1613377–SRR1613384) + a sample of 8 Lassa datasets. Report mapped-read counts + genome coverage breadth → binary detection call, compared to Table 4. This is the clearly specified, low-hanging pipeline output (80%).
Out of scope / not attempted (the hard ~20%)
- Full Galaxy server deploy (Docker artbio/metavisitor:1.2) — not needed; the UC 3-3 detection signal is bowtie2 alignment, which we run directly (P16: applying the described tool to the paper's data is equally valid).
- All 55 Lassa datasets (we sample 8) — sampling, not exhaustive; the detection rate claim (53/55) is checked on the sample, not re-counted across all 55.
- Use Case 1 (Drosophila viRNA Nora-virus genome reconstruction; >96.6% identity, 12,333 nt), Use Case 2 (Anopheles C virus discovery, KU169878), UC 3-1 (HIV), UC 3-2 (febrile patients) — different datasets, not the briefed accession.
- De-novo Trinity assembly + blast contig identification — the alignment-based detection already settles the per-dataset positive/negative call; full assembly is the optional 20%.
Why detection (not assembly) is the right 1:1 target
Table 4 is a presence/absence table per dataset — the headline reproducible claim. bowtie2 to the paper's exact references is the first and decisive Metavisitor step; a sample mapping thousands of reads spanning the viral genome is unambiguous detection. We report the raw numbers so a human can re-grade.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
For the in-scope claim (Use Case 3-3, Table 4) the reproduction is strong and the reported presence/absence calls are directly derivable from the deposited reads: Ebola was an exact 1:1 match (8/8, ~99.9% breadth, depth 80-1425x) and the sampled Lassa runs were all positive, consistent with the reported 53/55 (~96.4%). Deviations are entirely on our methodology side — a self-chosen sample of 8 of 55 Lassa runs (so the exact 53/55 count was not re-derived) and a coverage-breadth proxy in place of the Trinity de-novo assembly. No fabrication or authors'-side defect is indicated. Overall a solid reproduction with explainable, self-imposed scope limits, hence yellow rather than a clean green.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.