miRge3.0: a comprehensive microRNA and tRF sequencing analysis pipeline.
Part of the results reproduced; minor but material deviations remained.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
▸Reproduction agent’s raw note
IN PROGRESS. Harvested code_url (XuanrZhang/miREC) is a mismatch (unrelated error-correction tool); paper's actual pipeline is miRge3.0 (mhalushka/miRge3.0). Reproducing Table 1 Islet alpha cell row (SRR1028924) with miRge3.0 on «our HPC»: total miRNA reads, unique miRNAs, miRNAs>=10RPM. Setup (data+library+conda env) running on front1.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessmentassessed: 2026-06-18 ⛓ 5690f1a4b3f5
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-18
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-07-31
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusThe paper presents miRge3.0, a third-generation small RNA-seq analysis pipeline, hypothesizing it can provide improved speed, versatility and functionality (UMI handling, error correction, isomiR reporting, GUI) over miRge2.0 and other tools for annotating miRNAs and tRFs.
- ★ miRge3.0 with 12 CPUs consistently has the best execution speed compared to miRge2.0, Chimira and sRNAbench finding
- ★ miRge3.0 is a comprehensive Python 3 small RNA-seq pipeline adding UMI handling, miREC error correction, mirGFF3 isomiR reporting, differential expression and a cross-platform GUI resource
- ★ UMI deduplication has little effect on Qiagen-type longer UMIs but a more significant effect on 4N-style UMIs, where it likely overcorrects and undercounts abundant miRNAs finding
- ★ Iterative alignment to transcribed RNA libraries with progressively looser parameters yields more accurate small RNA assignment than a single genome-wide alignment method
- Novel miRNA prediction uses an SVM model on pre-miRNA hairpin features, rebuilt in sklearn for Python 3 method
- Chimira detects many more miRNAs than other tools partly due to lacking false-positive controls like minimal read counts or percent canonical read controls finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| small RNA-seq alignment/annotation (speed benchmarking) | public SRA small RNA-seq samples (79 MB to 2.9 GB) | none | run time from start to completion across tools, with 4 or 12 CPUs, with/without GFF | miRge3.0, miRge2.0, Chimira, sRNAbench |
| small RNA-seq miRNA abundance estimation | 7 human tissue/cell SRA datasets (T cell CD8+, platelets, RPE, cortical neuron, cardiac fibroblast, renal proximal epithelium, islet alpha cell) | none | miRNA reads, unique miRNAs, miRNAs >10 RPM | miRge3.0, miRge2.0, Chimira, sRNAbench |
| UMI deduplication analysis (4N ligation adaptor) | synthetic construct (SRR6379839) and NEXTflex human brain (SRR9115360) | UMI deduplication (--umiDedup) on vs off | correlation of deduplicated vs non-deduplicated miRNA RPM counts; UMI duplication counts | miRge3.0; -a TGGAATTCTCGGGTGCCAAGG, -umi 4,4 |
| UMI deduplication analysis (Qiagen UMI) | Qiagen processed sample SRR8557389 | UMI deduplication (--umiDedup) on vs off | correlation of deduplicated vs non-deduplicated miRNA RPM counts; UMI duplication counts | miRge3.0; -a AACTGTAGGCACCATCAAT, --umiqiagen, -umi 0,12 |
| differential expression analysis | small RNA-seq miRNA read counts with control/condition metadata | group comparison (control vs condition) | log2 fold change, P-value, adjusted P-value, volcano plot, PCA | DESeq2 (release 3.12), R v4.0 |
- ▲ miRge3.0 with 12 CPUs had the best execution speed among the four tools
- – Correlation between deduplicated and non-deduplicated miRNA RPM counts for Qiagen UMIs was very high between repeated runs r2 = 0.9996
- ▼ Correlation between deduplicated and non-deduplicated counts was lower for 4N UMI samples r2 = 0.95 and 0.81
- ▲ Chimira reported much higher numbers of unique miRNAs than other tools (e.g., 1830 vs ~1178 for renal proximal epithelium) 1830 vs 1178
- – sRNAbench, miRge2.0 and miRge3.0 show similar trends in miRNA count reporting
- correlation r2 = 0.9996 (deduplicated vs non-deduplicated RPM, Qiagen UMIs)
- correlation r2 = 0.95 (deduplicated vs non-deduplicated RPM, 4N UMI sample)
- correlation r2 = 0.81 (deduplicated vs non-deduplicated RPM, 4N UMI sample)
- count 1004438 miRNA reads, 318 unique miRNAs, 217 miRNAs >10 RPM (miRge3.0 on T Cell CD8+ neonatal SRR1853808)
- count 43770112 miRNA reads, 913 unique miRNAs, 279 miRNAs >10 RPM (miRge3.0 on islet alpha cell SRR1028924)
- count 2656 miRNAs in miRBase v22 vs 1111 in miRGeneDB 2.0 (controversial number of human miRNAs)
- count 8 (nucleotides in 4N-style ligation UMI (N=8))
- other 1.6 GB (Chimira max upload limit, precluding SRR1028924)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software application note describing miRge3.0, a small RNA-seq analysis pipeline. The statistical content is primarily descriptive: run-time benchmarking across four alignment tools on seven publicly available SRA datasets, UMI deduplication impact assessed via Pearson r² correlation between deduplicated and non-deduplicated RPM counts, and miRNA annotation counts tabulated across tools. No formal hypothesis tests are reported; differential expression (DESeq2) and novel miRNA prediction (SVM) are described as integrated pipeline features rather than applied to generate the paper's own results.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Pearson r² correlation | Comparison of deduplicated vs. non-deduplicated miRNA RPM counts for Qiagen UMI (Figure 2D) and 4N UMI (Figures 2E–F) | not stated (single SRA sample per condition) | not stated |
| Support vector machine (SVM) classifier | Novel miRNA prediction from unaligned reads | null | not stated |
| DESeq2 Wald test (negative binomial GLM) | Differential expression of miRNAs — described as a pipeline feature; not applied to generate results in this paper | null | not stated |
| Descriptive comparison of run times | Speed benchmarking of miRge3.0 vs. miRge2.0, Chimira, sRNAbench across seven SRA datasets (Figure 2G) | 7 SRA samples | na |
| Descriptive count tabulation | miRNA annotation counts (total reads, unique miRNAs, miRNAs ≥10 RPM) across four tools and seven samples (Table 1) | 7 SRA samples | na |
-
Speed benchmarking was conducted as a single-pass run per tool per sample, with results reported as point estimates of wall-clock time↳ Could also: Repeated timing runs (e.g., 3–5 replicates per sample per tool) with mean and standard deviation, followed by a non-parametric test such as the Wilcoxon signed-rank test on paired run times — Repeated measurements would quantify run-time variability due to system load, I/O fluctuation, and caching effects, making speed comparisons more reproducible and statistically interpretable
-
UMI deduplication impact was assessed with Pearson r² between deduplicated and non-deduplicated RPM counts↳ Could also: Bland–Altman (limits of agreement) analysis, or Spearman's ρ for robustness to non-normality and outliers — Bland–Altman analysis directly quantifies systematic bias and spread of differences between two measurement methods, which is the primary question when evaluating whether deduplication changes counts; r² alone does not reveal directional bias or proportional error
-
miRNA annotation counts across tools were compared purely descriptively in a table (Table 1)↳ Could also: Intraclass correlation coefficient (ICC) or Jaccard similarity index for presence/absence of detected miRNAs across tools — Formal agreement metrics would provide a single quantitative summary of inter-tool concordance, making it easier to compare tool agreement across samples and across count thresholds
-
No confidence intervals or uncertainty measures were reported for any quantitative comparison↳ Could also: Bootstrap confidence intervals around r² values and around percentage differences in miRNA counts — Confidence intervals communicate the precision of each estimate and allow readers to judge whether observed differences are likely to be consistent across other datasets of similar size
-
SVM novel miRNA prediction model performance was described by reference to the prior publication (miRge2.0) rather than re-evaluated here↳ Could also: Cross-validation metrics (e.g., area under the ROC curve, precision–recall curve, or F1 score on a held-out test set) reported directly in this paper — Explicitly reporting validation metrics in the current paper would allow readers to assess whether model performance was maintained or changed after the Python 3 rebuild of the SVM feature
-
Tool comparison for miRNA detection thresholds (miRNAs ≥10 RPM) used a single fixed cutoff↳ Could also: Sensitivity analysis across a range of RPM thresholds (e.g., ≥1, ≥5, ≥10, ≥50 RPM), or receiver-operating-characteristic-style curves if a ground-truth reference set is available — A single threshold can obscure whether tool differences are consistent or threshold-dependent; a multi-threshold analysis would show how concordance changes across the expression spectrum
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-34308351 (miRge3.0)
Paper: Patil AH, Halushka MK. miRge3.0: a comprehensive microRNA and tRF sequencing analysis pipeline. NAR Genom Bioinform 2021. PMID 34308351 · PMCID PMC8294687 · DOI 10.1093/nargab/lqab068.
Code artifact — correction of a harvest mismatch
The registry code_url was harvested as https://github.com/XuanrZhang/miREC.
This is WRONG: miREC is an unrelated C++ miRNA-sequence error-correction
tool (Xuan Zhang et al., NAR 2021), with no relationship to miRge3.0 or to
Patil & Halushka. The paper's actual pipeline is miRge3.0, an authors'
Python package:
- Repo: https://github.com/mhalushka/miRge3.0
- Docs: https://mirge3.readthedocs.io
- Install:
pip install mirge3/ biocondamirge3We reproduce by running miRge3.0 (the paper's own tool) on the paper's own data (SRR1028924), per the documented parameters — this is the in-scope, faithful reproduction. (Per BRIEF P16, even a third-party tool on the paper's data would be valid; here we use the authors' own tool.)
Data
sra:SRR1028924— "Islet alpha cell" small-RNA-seq sample, Homo sapiens, Illumina, single-end 50 bp, 82,497,527 reads (ENA), 2.7 GB fastq.gz. This is one of seven SRA samples benchmarked in Table 1 / Figure 2G.
In-scope reproducible results (pipeline-derived)
From Table 1 (the row for SRR1028924 = Islet alpha cell), miRge3.0 column:
| id | result | reported |
|---|---|---|
| C1 | total miRNA reads | 43,770,112 |
| C2 | unique miRNAs detected | 913 |
| C3 | miRNAs with ≥10 RPM | 279 |
These are deterministic outputs of running miRge3.0 (-on human -db mirbase)
on SRR1028924 → directly comparable 1:1.
Out of scope / not attempted
- Figure 2G speed/runtime benchmark across 7 samples & 12 CPUs — hardware- and load-dependent; not 1:1 reproducible across a different cluster. (We do run on a comparable 12-CPU node and will note wall time as context only, not as a graded claim.)
- sRNAbench / Chimira / miRge2.0 comparison columns — require running other third-party tools / web servers (Chimira is a web server with a 1.6 GB upload cap that the paper notes could not even process this file). Not the paper's own pipeline; out of scope for this RU.
- A-to-I editing, novel miRNA, isomiR, tRF feature correctness claims — the paper demonstrates capability rather than reporting a single pinnable number for SRR1028924; not graded here (the pipeline does produce these outputs).
- Wet-lab / manual content — none relevant.
Pipeline named per result
All three in-scope claims (C1–C3) come from the miRge3.0 annotation step (bowtie alignment to the mirBase human library + RPM normalisation).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The reproduction of pmid-34308351 (miRge3.0) is incomplete: the harvested code_url was a mismatch (miREC instead of miRge3.0), and the «our HPC» job to regenerate Table 1's Islet alpha cell row (SRR1028924) was still in setup, leaving C1/C2/C3 with reproduced=null. Input data (public SRA SRR1028924) and tool (open-source miRge3.0) make the reported values derivable in principle, so this is not a fabrication or authors'-side concern. The open issues are on our side (process not finished) plus a likely benign reference-DB version drift; no deviation magnitude can yet be judged, so overall quality is provisionally yellow pending completion.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.