miRge 2.0 for comprehensive analysis of microRNA sequencing data.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce. Result is WITHIN-TOLERANCE 1:1. The paper's per-sample miRNA annotation numbers (Table 2, miRge 2.0-mb column) for human adipose sample SRR772563 reproduce to <1%: miRNA reads 2,039,835 vs 2,022,292 (-0.86%), unique miRNAs 477 vs 475, miRNAs>10RPM 238 vs 237; total input reads match exactly, confirming identical deposited data. Reproduced with the named tool's SUCCESSOR miRge 3.0 (bioconda mirge3 0.1.4, same algorithm family + miRBase v22) because miRge 2.0's own reference libraries are no longer downloadable (SharePoint human.tar.gz HTTP 401; jh.box.com folder HTTP 404, verified 2026-06-16) -- a valid P16 reproduction. The brief's accession SRR944031 is a Table-1 training accession with no reported per-sample metric, so it is ungradeable; we reproduced the Table-2 sample which has clearly-specified expected values. No fabrication signal. NOT attempted (hard ~20%): building miRge 2.0 libraries from scratch, novel-miRNA prediction, isomiR/A-to-I editing benchmarks, runtime/memory comparisons, and the miRDeep2/miRAnalyzer/MirGeneDB columns.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 89assessed: 2026-06-16 ⛓ 36838ba58223
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-16
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan miRNA sequencing analysis be made more comprehensive, specific, and standardized by redesigning the miRge aligner with a novel machine-learning-based miRNA detector, A-to-I editing analysis, and standardized isomiR reporting?
- ★ miRge 2.0 introduces a novel SVM-based miRNA detection method using both hairpin structure and isomiR composition, yielding higher specificity for miRNA identification method
- ★ The SVM model predicted miRNAs with an average MCC of 0.939 across 32 human cell datasets and outperformed miRDeep2 and miRAnalyzer on phylogenetic conservation finding
- ★ A-to-I editing detection in miRge 2.0 strongly correlates with an established reference dataset (adjusted R2 = 0.96) finding
- ★ miRge 2.0 provides custom alignment libraries for both miRBase v22 and MirGeneDB v2.0 across 6 species and a tool to create custom libraries resource
- ★ miRge 2.0 offers standardized GFF3 isomiR reporting following miRTop guidelines using CIGAR values method
- miRge 2.0 is integrated into bcbio-nextgen and installable via Bioconda for user-friendliness resource
- miRNAs are characterized by a consistent 5' start with variable 3' ends and frequent U/A non-templated additions, a pattern exploitable to distinguish them from non-miRNAs mechanism
- miRge 2.0 alignment speed is comparable to the original miRge and miRAnalyzer and significantly faster than miRDeep2 finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| small RNA-seq (miRNA-seq) novel miRNA detection / SVM model training | 17 human and mouse tissues (adrenal, bladder, blood, brain prefrontal cortex, colon, epididymis, heart, kidney, liver, lung, pancreas, placenta, retina, skeletal muscle, skin, testes, thyroid) from SRA | none | MCC of miRNA classification; structural and compositional read-cluster features | scikit-learn SVM (RBF kernel); Bowtie v1.1.1; RNAfold v2.3.5 |
| small RNA-seq model validation | 12 rat samples | none | MCC / model robustness | — |
| A-to-I editing analysis (accuracy evaluation) | pooled human brain sample (SRR095854) | none | number of significant A-to-I editing sites; A-to-I proportion correlation with reference | — |
| A-to-I editing analysis (comparative) | colon tissue, colon epithelial cells, colon cancer tissue, and colon cancer cell lines DKO1, DLD1, DKS8 (SRA samples) | none (tumor vs normal comparison) | miRNA A-to-I editing percentage per site | — |
| small RNA-seq speed and annotation benchmarking | 6 datasets: human adipose, human alpha cell, human beta cell, mouse stomach, mouse epididymal epithelial cell, mouse B3 cell | none | processing time, miRNA reads, unique miRNAs, miRNAs >10 RPM | miRge / miRge 2.0 / miRDeep2 / miRAnalyzer |
| novel miRNA prediction tool comparison | human cell datasets (unmapped reads from miRge runs) | none | PhyloP conservation score and quality score | PHAST package; PhyloP 20-way hg38 |
- – SVM model predicted miRNAs with high accuracy across 32 human cell datasets average MCC = 0.939
- ▲ A-to-I editing proportions strongly correlated with reference dataset across 16 shared sites adjusted R2 = 0.96, slope = 0.99
- – miRge 2.0 identified more significant A-to-I modification sites than the reference paper in the human brain sample 19 vs 16 sites
- – Human predictive model performance peaked and stabilized at 21 features 21 features
- ▲ miRge 2.0 (miRBase) aligns to more miRNAs than the original method after library expansion 2817 vs 2656 miRNAs
- – Novel miRNA detection is more time/memory intensive than standard annotation 17 min and 6.7 GB RAM for 25.7M reads (SRR553572)
- – miRge 2.0 processing time comparable to original miRge and faster than miRDeep2 (e.g. human adipose ~36 s vs 9.3 min) ~36 s vs 9.3 min
- correlation MCC = 0.939 (average Matthews correlation coefficient of SVM miRNA prediction over human cell datasets)
- correlation adjusted R2 = 0.96 (A-to-I editing proportion vs reference dataset (shared sites))
- other slope = 0.99 (linear regression of A-to-I proportions vs reference)
- count 19 vs 16 (A-to-I sites detected by miRge 2.0 vs reference in brain sample SRR095854)
- count 2817 (miRNAs in expanded miRBase v22 search library (vs 2656 originally))
- count 586 human miRNA genes (1171 5p/3p strands) (MirGeneDB strict-criteria human miRNA count)
- count 2656 (human miRNAs listed in miRBase v22)
- other 17 min, 6.7 GB RAM (time and peak memory for novel miRNA detection on 25.7M-read dataset SRR553572)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a bioinformatics software (miRge 2.0) methods paper that primarily reports tool development and benchmarking rather than hypothesis-driven group comparisons. Its core quantitative approach is a supervised support vector machine (SVM) classifier for novel miRNA detection, evaluated with the Matthews correlation coefficient (MCC) under repeated train/validation splits and cross-validated grid-search hyperparameter tuning; performance is also benchmarked against other tools using PhyloP conservation and quality scores. Accuracy of the A-to-I editing module is assessed by linear regression against a reference dataset (adjusted R² and slope), and other comparisons (speed, read/miRNA counts) are reported descriptively in tables without inferential testing.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Support vector machine (radial basis function kernel) classification, evaluated by Matthews correlation coefficient (MCC) | Novel miRNA detection model; mean MCC of 0.939 across 32 human cell datasets; training/validation MCC vs. number of features (Fig. 4) | 17 human and 17 mouse tissue datasets for model construction; additionally tested on 12 rat samples; SRA accessions listed in Table 1 | not stated |
| Linear regression (adjusted R² and slope) on log-log scale | A-to-I editing proportions for 16 shared sites vs. a reference dataset (Fig. 3a); adjusted R²=0.96, slope=0.99 | 16 shared A-to-I sites from pooled human brain sample SRR095854 | not stated |
| Minimum Redundancy Maximum Relevance (mRMR) feature ranking with forward stepwise feature selection | Selection of informative features for the predictive model (top 21 human features, Table 3) | — | not stated |
| PhyloP basewise conservation scoring (mean PhyloP across each miRNA) and a defined quality score [1 - ranking percentile] | Comparison of novel miRNA predictions across miRge 2.0, miRDeep2 and miRAnalyzer | — | na |
-
Model performance was reported as the mean MCC across 10 random 4:1 train/validation splits without a reported measure of spread.↳ Could also: Reporting the standard deviation or a 95% confidence interval of MCC across the splits, or showing the per-split distribution. — Adding a dispersion measure would convey the stability of the classifier across resamplings alongside the central estimate, which is commonly preferred when summarizing cross-validation results.
-
Novel miRNA detection used a single classifier (SVM with RBF kernel) selected and tuned via cross-validated grid search.↳ Could also: Benchmarking additional learners (e.g., random forest, gradient boosting, or logistic regression) under the same nested cross-validation scheme. — Comparing multiple model families would help characterize how much performance depends on the chosen algorithm versus the engineered features, a common practice in ML method papers.
-
Classifier accuracy was summarized primarily with MCC.↳ Could also: Also reporting ROC-AUC / precision-recall AUC, sensitivity and specificity, or F1 alongside MCC. — Multiple complementary metrics describe different aspects of classification performance (e.g., behavior across thresholds and on class imbalance), giving readers a fuller view of the model.
-
Agreement of the A-to-I editing module with a reference dataset was assessed using adjusted R² and the regression slope on a log-log plot.↳ Could also: Adding a Bland–Altman (agreement) analysis or a concordance correlation coefficient, and reporting a confidence interval on the slope. — Agreement-focused statistics quantify systematic bias and limits of agreement directly, which complements correlation/regression when the goal is to show two methods give equivalent measurements.
-
Tool comparisons (processing time, read counts, miRNA counts) and the tumor-vs-normal A-to-I heat map were presented descriptively in tables and figures.↳ Could also: Pairing the descriptive tables with an inferential or distributional summary across more datasets (e.g., paired comparisons of counts or editing levels across samples). — Replicate-level summaries with a significance or interval estimate would quantify how consistent the observed differences are beyond the individual examples shown.
-
Feature selection used mRMR ranking with forward stepwise selection on the assembled dataset.↳ Could also: Embedding feature selection entirely within each cross-validation fold (nested selection) or using regularization-based selection (e.g., L1/elastic net). — Performing selection inside resampling folds helps ensure the reported performance reflects fully held-out generalization, a standard safeguard against selection-related optimism.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
miRge 2.0 A-to-I editing proportions at miRNA sites in human brain correlate with reference values across 16 shared sites (adjusted R²=0.96, slope=0.99)other human brain 2018×1papers★ This paper is the founder (earliest)
-
miRge 2.0 detects more significant miRNA A-to-I editing sites in human brain than the reference method (19 vs 16 sites)other human brain up 2018×1papers★ This paper is the founder (earliest)
-
miRge 2.0 SVM classifier distinguishes miRNA reads from other small RNA reads with mean MCC of 0.939 across 32 human cell datasetsother human tissue 2018×1papers★ This paper is the founder (earliest)
-
miRge 2.0 with expanded miRBase library annotates more human miRNAs than original miRge (2817 vs 2656 miRNAs)other human up 2018×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
Assessed papers, coloured by verdict. Click a node to open it.
- No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-30153801 (miRge 2.0)
Paper: Lu Y, Baras AS, Halushka MK. miRge 2.0 for comprehensive analysis of microRNA sequencing data. BMC Bioinformatics 2018. PMID 30153801 / PMC6112139 / DOI 10.1186/s12859-018-2287-y. Tool repo: https://github.com/mhalushka/miRge (miRge 2.0).
What the paper reports (candidate reproducible results)
| Table | Content | In scope? |
|---|---|---|
| Table 1 | List of SRA accessions used to train the novel-miRNA model (per tissue). Contains the brief's accession SRR944031 (human adrenal). No per-sample numeric value — just accession identifiers. | No — nothing to grade (no reported number for SRR944031). |
| Table 2 | Per-sample annotation comparison (miRge, miRge 2.0-mb, miRge 2.0-MDB, miRDeep2, miRAnalyzer) over 6 datasets. For each: processing time, miRNA reads, unique miRNAs detected, miRNAs > 10 RPM. | YES — concrete, pinnable, pipeline-derived per-sample numbers. |
| Table 3/4, A-to-I, novel-miRNA | isomiR / editing / novel-miRNA discovery and benchmarking. | Partly in scope but heavy (the hard ~20%); not attempted first. |
In-scope target (80/20)
Reproduce the miRge 2.0 (miRBase, "mb") column of Table 2 for the smallest, fastest, cleanly-specified human sample:
- SRR772563 — Human Adipose Tissue, 2,373,604 reads (ENA, 67 MB). Reported miRge 2.0-mb: 2,039,835 miRNA reads · 477 unique miRNAs · 238 miRNAs > 10 RPM (Table 2).
(Secondary, if cheap: SRR873410 Human Beta Cell — miRge 2.0-mb 26,197,845 reads · 889 unique · 291 > 10 RPM.)
The brief's data accession SRR944031 is a Table-1 training accession with no reported per-sample metric, so it is not gradeable. We therefore reproduce the Table-2 human sample(s), which is "the paper's own data" with clearly-specified expected values — the honest auditable choice.
Tool choice — P16 (third-party / successor tool is valid)
The exact miRge 2.0 reference libraries (human.tar.gz, the
index.Libs/fasta.Libs/annotation.Libs bundle) are no longer obtainable:
- README SharePoint link → resolves to
.../BoxMigration/human.tar.gzbut returns HTTP 401 Unauthorized for anonymous download (verified 2026-06-16). - Alternate
jh.box.com/s/...folder link → HTTP 404 (gone).
The miRge3.0 libraries (same lab, same algorithm family, anchored to the same
miRBase v22) download reliably from SourceForge (human.tar.gz, ~4 GB, HTTP 200).
Per brief rule P16 ("applying an existing tool to the paper's own data is equally
valid"), we reproduce by running miRge3.0 on the paper's data with the paper's
parameters (-d miRBase -sp human -a illumina). This is the successor of the named
tool, not the exact v2.0 binary; we therefore expect within-tolerance, not
byte-exact, agreement, and label every claim provisional.
Out of scope / not attempted
- Building miRge 2.0 libraries from scratch via
mhalushka/miRge_build(needs full genome bowtie indices; the hard ~20%). - Novel-miRNA prediction, isomiR/A-to-I editing benchmarks, runtime comparisons (hardware-dependent), miRDeep2/miRAnalyzer cross-tool columns.
- SRR944031 itself (ungradeable; see above).
Pipeline
small-RNA fastq → adapter trim (cutadapt, illumina) → bowtie align to miRBase v22
human library → miRNA quantification → miR.Counts.csv / miR.RPM.csv + run report.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a clean within-tolerance reproduction: total input reads match exactly (2,373,604), confirming identical deposited data, and all three Table 2 annotation metrics reproduce to <1% (2,039,835→2,022,292; 477→475; 238→237). No fabrication signal — the reported values are derivable from the shipped pipeline on public data. The only caveat is on our side, not the authors': miRge 2.0's reference libraries are dead (401/404), so the lab's successor miRge 3.0 was run instead, and the tiny deltas are explainable version drift — hence q8 yellow rather than green.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.