MasterOfPores: A Workflow for the Analysis of Oxford Nanopore Direct RNA Sequencing Datasets.
The main results reproduced, with only marginal, non-material deviations.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
MasterOfPores is a Nextflow WORKFLOW paper; its only printed numbers are Table 1 (per-sample read counts + hardware timings) for 4 S. cerevisiae direct-RNA runs. Decisive constraint: GSE126213/PRJNA521324 deposited ONLY basecalled FASTQ -- NO FAST5 raw signal (ENA submitted_* empty for all 6 runs). So basecalling (hence the exact Table 1 read totals), NanoTail polyA (nanopolish/tailfindr), and NanoMod-Tombo are NOT reproducible from public data; Table 1 timings are hardware-dependent. Reproduced from the deposited FASTQ (P16, running the tools MoP wraps with its documented params): (C1) deposited read counts match ENA exactly and are a consistent 77-84% of Table 1 -> Table 1 = total basecalled (pre-filter), SRA = pass-filter subset; gap is the filter step, NOT fabrication, but exact totals are unverifiable without the un-deposited FAST5. (C2) NanoPreprocess minimap2 mapping reproduces on all 6 runs (45.9-58.2%; R64 used, paper used SK1; no printed target). (C3) NanoMod EpiNano-Error qualitatively recovers the m6A signature: WT base-calling error > ime4KO at DRACH sites (52.2% vs 51.3% background; clear high-coverage candidates e.g. chrIX:390180 WT 0.44/KO 0.05), small globally because m6A is sparse + R64!=SK1 + single-feature proxy vs EpiNano's SVM. NOT attempted: basecalling, polyA, Tombo (all need FAST5), and Table 1 timings (hardware). No fabrication indicated.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-16 ⛓ 2921f74cceea
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-16
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-18no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusThe complexity of raw Oxford Nanopore direct RNA sequencing data and the lack of systematic, reproducible pipelines have hindered its use; the paper addresses this by providing a fully benchmarked, containerized workflow (MasterOfPores) for the analysis of direct RNA sequencing reads.
- ★ MasterOfPores is a NextFlow-based, containerized workflow that processes raw FAST5 direct RNA sequencing data through base-calling, demultiplexing, filtering, QC, mapping, and quantification, plus downstream RNA modification and polyA tail length analyses. resource
- ★ The pipeline can fully process and analyze four direct RNA MinION sequencing runs in 10 h on 100 CPUs. finding
- ★ Running on GPU locally or in the cloud decreases run time fourfold compared to CPU. finding
- ★ The NanoMod module predicts RNA modifications by comparing two conditions (e.g., wild-type vs knock-out) using Tombo and EpiNano. method
- ★ The NanoTail module estimates polyA tail lengths using two algorithms, Nanopolish and TailfindR, and reports their correlation. method
- ★ The use of Linux containers (Docker/Singularity) allows execution on any Unix-compatible OS without installing additional dependencies, achieving reproducibility and portability. method
- No previously available nanopore pipelines could be used for the analysis of direct RNA sequencing datasets. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| direct RNA nanopore sequencing (polyA+) | Saccharomyces cerevisiae (yeast WT and ime△ knock-out) | KO (ime△ gene knock-out vs WT) | number of reads, base-called sequences, mapped reads, RNA modifications, polyA tail length | Oxford Nanopore MinION; SQK-RNA002 kit, FLO-MIN106 flowcell |
| base-calling (NanoPreprocess module) | S. cerevisiae direct RNA reads | none | base-called sequences (FAST5/FASTQ) | Albacore or Guppy |
| read mapping (NanoPreprocess module) | S. cerevisiae direct RNA reads | none | alignments (BAM), mapped reads, mean sequence identity | minimap2 or graphmap2 |
| gene/transcript quantification (NanoPreprocess module) | S. cerevisiae direct RNA reads | none | per-gene or per-transcript counts | HTSeq (genome) or NanoCount (transcriptome, EM algorithm) |
| RNA modification prediction (NanoMod module, paired mode) | S. cerevisiae direct RNA reads (WT vs KO) | KO vs WT comparison | predicted RNA modification sites | Tombo and EpiNano |
| polyA tail length estimation (NanoTail module) | S. cerevisiae direct RNA reads | none | polyA tail length per read | Nanopolish and TailfindR |
| demultiplexing (NanoPreprocess module) | barcoded direct RNA reads | none | demultiplexed reads | DeePlexiCon |
| quality control (NanoPreprocess module) | S. cerevisiae direct RNA reads | none | QC metrics (read length, Q-score, identity) | MinIONQC, FastQC, NanoPlot, bam2stats |
- – Four S. cerevisiae direct RNA sequencing runs processed and analyzed in 10 h on 100 CPUs 10 h
- ▼ GPU execution decreased run time relative to CPU 4-fold
- – A typical direct RNA run (1-2M reads) processes in ~2 h on a CPU cluster of 100 nodes ~2 h
- – A typical direct RNA run processes in ~1 h or less on a single GPU ~1 h or less
- – NanoPreprocess CPU time for Yeast WT rep1 (1,197,462 reads) 2 h 13 min (7 s per 1000 reads)
- – NanoPreprocess GPU (RTX 2080) time for Yeast WT rep1 1 h 8 min (3 s per 1000 reads)
- – NanoTail module CPU total time 3 h 26 min (4 s per 1000 reads)
- – NanoMod module CPU total time 5 h 40 min (7 s per 1000 reads)
- count 1,197,462 (number of raw reads, Yeast WT rep1)
- count 694,907 (number of raw reads, Yeast ime△ KO rep1)
- count 629,270 (number of raw reads, Yeast WT rep2)
- count 573,404 (number of raw reads, Yeast ime△ KO rep2)
- other 1-2M reads (typical reads per direct RNA MinION/GridION run)
- count 95% (fraction of currently known RNA modifications lacking genome-wide maps)
- other 100 nodes, 8 CPU per node (CPU configuration used for timing benchmarks)
- other 10 h on 100 CPUs (total processing of four direct RNA MinION runs)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software/workflow methods paper describing MasterOfPores, a NextFlow-based pipeline for Oxford Nanopore direct RNA sequencing analysis; it does not present a formal experimental statistical design or hypothesis testing. Quantitative reporting is largely descriptive, consisting of computing-performance benchmarks (run time and RAM across four S. cerevisiae datasets) and pipeline outputs such as per-gene/transcript quantification, polyA tail length estimates, and RNA modification predictions. Where multiple algorithms produce overlapping outputs (e.g., two polyA estimation tools), agreement is summarized via a reported correlation plot.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Correlation between two polyA tail-length algorithms (reported as a plot; correlation type not stated) | NanoTail module output comparing Nanopolish and tailfindR estimates | — | not stated |
-
Performance benchmarks (run time, RAM) are reported as single values per dataset across four runs.↳ Could also: Reporting central tendency with a measure of spread (e.g., mean with SD/range, or a 95% CI) across replicate runs would also be possible. — Summarizing across runs would convey run-to-run variability in performance and is often informative when n is small.
-
Agreement between the two polyA tail-length algorithms is summarized with a reported correlation plot.↳ Could also: A Bland–Altman (limits-of-agreement) analysis, or reporting a specific correlation coefficient (Pearson or Spearman) with a confidence interval, could also be used. — These approaches additionally quantify systematic bias and the magnitude of disagreement between methods, complementing a visual correlation.
-
The WT vs ime△ KO design uses two biological replicates per condition for downstream modification detection.↳ Could also: Pre-specifying a target number of replicates or reporting the rationale for n could also be included. — Describing how n was chosen helps readers gauge the sensitivity of comparison-based detection at the chosen replication level.
-
RNA-modification and polyA outputs are produced by wrapped tools (EpiNano, Tombo, Nanopolish, tailfindR) and written to flat files.↳ Could also: Reporting the statistical thresholds and any multiple-testing control used by each tool, or harmonizing them across tools, could also be documented. — Making the per-tool decision rules explicit would help users interpret and compare predictions consistently across algorithms.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
MasterOfPores processes a typical direct RNA nanopore run (~1-2M reads) in ~1 h on a single GPUlong-read-rna saccharomyces-cerevisiae 2020×1papers★ This paper is the founder (earliest)
-
NanoMod RNA modification prediction module processes yeast direct RNA sequencing data in ~5 h 40 min on CPU (~7 s per 1000 reads)long-read-rna saccharomyces-cerevisiae 2020×1papers★ This paper is the founder (earliest)
-
NanoTail polyA tail length estimation module processes yeast direct RNA sequencing data in ~3 h 26 min on CPU (~4 s per 1000 reads)long-read-rna saccharomyces-cerevisiae 2020×1papers★ This paper is the founder (earliest)
-
MasterOfPores processes a typical direct RNA nanopore run (~1-2M reads) in ~2 h on a 100-node CPU clusterlong-read-rna saccharomyces-cerevisiae 2020×1papers★ This paper is the founder (earliest)
-
GPU execution reduces MasterOfPores direct RNA nanopore sequencing pipeline runtime ~4-fold compared to CPUother saccharomyces-cerevisiae down 2020×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-32256520 (MasterOfPores, Cozzuto et al. 2020, Front Genet)
Paper: a workflow/tool paper. The contribution is the Nextflow pipeline
MasterOfPores (MoP1) for Oxford Nanopore direct RNA sequencing. Demonstrated on
S. cerevisiae SK1 polyA(+) direct-RNA data (WT vs ime4Δ m6A-writer knockout),
GEO GSE126213 / SRA SRP184486 / BioProject PRJNA521324.
Repo: https://github.com/biocorecrg/master_of_pores — the 2020 paper = MoP1
(tag v1.0 / v1.1), with the three modules named in the paper:
NanoPreprocess (basecall → demux → filter → QC → map → quantify),
NanoTail (polyA tail length: nanopolish + tailfindr),
NanoMod (RNA modifications: Tombo + EpiNano).
(The current repo HEAD is the rewritten DSL2 MoP3/v4.1 — different layout.)
Decisive data-availability finding
GSE126213 / PRJNA521324 deposited only basecalled FASTQ (6 GridION runs).
ENA submitted_ftp/submitted_format columns are empty for all runs → NO FAST5
raw signal deposited. GEO supplementary = 160 KB of processed TXT (per-site
tables from the linked EpiNano paper PMID 31501426). This single fact drives scope:
every step that needs the raw squiggle is blocked by missing data, not by us.
What the paper actually prints (the only quantitative claims)
- Table 1: per-sample "Number of reads" (raw, total basecalled, pre-filter) + computing time + peak RAM, for 4 of the 6 samples (WT rep1/2, ime4Δ rep1/2).
- No mapping rate, no modification-site counts, no polyA numeric values, no correlation/accuracy numbers are printed (Figs are schematics / multiQC snapshots). So there is essentially one numeric table to compare against, plus hardware timings.
IN SCOPE (reproducible from the deposited FASTQ)
| # | Result | Pipeline / module mirrored | How |
|---|---|---|---|
| C1 | Per-sample deposited read count | NanoPreprocess (post-filter reads) | count reads in each ENA FASTQ; compare to Table 1 (expect ~0.77–0.84× because Table 1 = pre-filter total) |
| C2 | Per-sample mapping rate | NanoPreprocess → minimap2 -uf -k14, RNA |
map FASTQ to yeast genome, samtools flagstat (no printed target → supporting evidence) |
| C3 | m6A WT>KO base-calling error at DRACH sites | NanoMod → EpiNano-Error | EpiNano/mpileup per-site error from BAM; show WT mismatch > ime4Δ KO (qualitative; no printed target) |
OUT OF SCOPE (cannot reproduce — reason recorded, not attempted)
| Result | Why out of scope |
|---|---|
| Basecalling (Albacore/Guppy), exact Table 1 read totals | needs raw FAST5 (not deposited) + proprietary ONT basecaller; only post-filter FASTQ is public |
| NanoTail polyA tail lengths | nanopolish-polya and tailfindr both require FAST5 raw signal — not deposited |
| NanoMod / Tombo modification calls | Tombo resquiggle needs FAST5 — not deposited (EpiNano-Error is the FASTQ-feasible alternative, see C3) |
| Table 1 computing time / RAM / throughput | hardware-dependent (100×8-core cluster, RTX 2080 GPU); not a reproducible value |
Net
Strict "match the printed number": the one numeric claim (Table 1 read counts) is pre-filter totals requiring un-deposited FAST5 → not exactly regenerable; the rest of Table 1 is hardware timing. So the faithful reproduction is to (a) confirm the deposited post-filter read counts and quantify the gap to Table 1, and (b) demonstrate the NanoPreprocess(map) and NanoMod(EpiNano) modules run on the paper's own data and produce sensible outputs. Expected verdict: partial (pipeline real & runnable; data partially deposited; only printed numbers are pre-filter totals + hardware timings).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.