A comprehensive framework for analysis of microRNA sequencing data in metastatic colorectal cancer.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- ✓Overall, the reproduction was clean
- Every checked point held up.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce 1:1. The paper's analysis code (github.com/eirikhoye/mirna_pipeline @ 09bae30) ships the miRge3.0 count matrices (miR.Counts1..7.csv), the sample sheet (sample_info_v9.csv, 268 QC-passed) and the DESeq2 analysis code + rendered report. I reproduced the counts->DESeq2->differential-expression segment on «our HPC» (R 4.5.3 / DESeq2 1.50.2): merged the 7 shipped batches by miRNA, dropped passenger(*)/per-locus(chr) rows, ran DESeq2 with the documented thresholds (|log2FC|>0.5849625, >=100 RPM in one group, FDR<0.05). RESULT = EXACT 1:1: matrix dimension (389 miRNAs), all 7 per-tissue sample counts, and up/down DE counts for ALL 8 tissue contrasts match the shipped report; the headline pCRC-vs-nCR 32-up/35-down also matches the paper Abstract verbatim. 24/24 pinnable integer claims exact. NOT attempted (out of scope): FASTQ->counts alignment (miRTrace+miRge3.0) because the COMET sequencing data is EGA-controlled (EGAS00001001127, access-on-request) -> data_restricted for that segment only; wet-lab qPCR validation (non-pipeline); UMAP/cell-composition figures and exact Table-1 per-miRNA RPM/LFC values (the hard ~20%, skipped). No fabrication concern: every reproduced number is derivable from the shipped data+code. Caveat: LFC-shrinkage used type='normal' (apeglm absent from reused env); the integer DE counts are identical either way.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 100assessed: 2026-06-15 ⛓ 0c7bb3d5f9d4
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a rigorous, miRNA-tailored bioinformatics pipeline reliably identify miRNAs associated with metastatic progression of colorectal cancer across multiple metastatic sites (liver, lung, peritoneum)?
- ★ Five miRNAs (Mir-210_3p, Mir-191_5p, Mir-8-P1b_3p [miR-141-3p], Mir-1307_5p, Mir-155_5p) are up-regulated at multiple metastatic sites in colorectal cancer. finding
- ★ Mir-210_3p and Mir-191_5p are up-regulated at all three metastatic sites (liver, lung, peritoneum) compared to primary CRC. finding
- ★ A novel miRNA-tailored bioinformatics pipeline using MirGeneDB as reference, miRTrace QC, miRge3.0 processing, a 100 RPM physiological cut-off, and correction for normal tissue background expression enables reliable differential miRNA expression analysis. method
- ★ Some identified miRNAs were previously implicated in metastasis via epithelial-to-mesenchymal transition and hypoxia, while others represent novel findings in this context. finding
- The publicly available pipeline facilitates reproducibility and allows new datasets to be added as they become available. resource
- Global miRNA expression profiles cluster by tissue of origin rather than by study of origin. finding
- Cell-type specific miRNA expression differs between tissues, reflecting differences in cellular composition that confound bulk tissue analysis. mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| miRNA small RNA next-generation sequencing (miRNA-seq) | human pCRC, nCR, liver metastases (mLi), normal liver (nLi), lung metastases (mLu), normal lung (nLu), peritoneal metastases (PM) tissue samples | none (observational tumor vs adjacent normal comparison) | miRNA read counts / expression (RPM, LFC) | Illumina HiSeq 2500 High Throughput Sequencer; TruSeq Small RNA Library protocol; Qiagen Allprep DNA/RNA/miRNA universal kit |
| qPCR validation | human mLi (n=11) and pCRC (n=11) tissue samples | none | Mir-210_3p expression (dCq normalized to Mir-103 reference) | Qiagen miRCURY LNA RT Kit, miRCURY SYBR Green Kit, miRCURY LNA miRNA PCR Assays |
| Differential expression analysis (DESeq2) | 268 miRNA-seq datasets (CRC tissues) | none | Log2 fold change, FDR | DESeq2 v1.26; miRge3.0; MirGeneDB2.0 |
| Quality control of sequencing data | 350 miRNA NGS datasets | none | read quality, length distribution, miRNA content | miRTrace |
| Global expression clustering (UMAP) | CRC tissue miRNA-seq datasets | none | two-dimensional clustering of global miRNA expression | UMAP R package; VST/DESeq2 normalization |
| Cell-type specific miRNA analysis (heatmap, PCA, t-test) | CRC tissue datasets (45 validated cell-type specific miRNAs) | none | relative expression (z-score RPM, VST values) | FactoMineR PCA |
| Gene set enrichment analysis (GSEA) | predicted miRNA-mRNA interactions from mCRC vs pCRC DE data | none | likelihood of gene set suppression (GO MF/CC/BP, KEGG) | RBiomirGS |
- ▲ Mir-210_3p up-regulated in mLi vs pCRC LFC 1.26 (SE 0.18), FDR 4.73E-11
- ▲ Mir-191_5p up-regulated at all three metastatic sites (highest expression >1000 RPM) LFC 0.74 (mLi), 0.74 (mLu), 0.61 (PM)
- ▲ qPCR validated up-regulation of Mir-210_3p in mLi vs pCRC t = -2.25, P = 0.036
- ▲ Mir-8-P1b_3p (miR-141-3p) up-regulated in mLi and mLu LFC 0.64 (mLi), 0.75 (mLu)
- ▲ Mir-155_5p up-regulated in mLu and PM LFC 0.76 (mLu), 0.70 (PM)
- ▲ Mir-1307_5p up-regulated in mLi and PM LFC 0.85 (mLi), 0.62 (PM)
- – 26 miRNAs differentially expressed in one or more metastatic tissues vs pCRC after background correction
- – 32 miRNAs up-regulated and 35 down-regulated in pCRC vs nCR, including oncomiRs Mir-21_5p, MIR-17 family, Mir-31, Mir-221
- pvalue P = 0.036 (t = -2.25, df = 19.95) (qPCR Welch t-test of Mir-210_3p dCq, mLi (n=11) vs pCRC (n=11))
- fold_change LFC 1.26 (SE 0.18), FDR 4.73E-11 (Mir-210_3p mLi vs pCRC)
- count 268 NGS datasets after QC (pCRC=120, nCR=25, mLi=35, nLi=20, mLu=28, nLu=10, PM=30) (datasets remaining after miRTrace QC of 350)
- count 537 human miRNA annotations reduced to 389 unique annotations (MirGeneDB2.0 merging by miRge3.0)
- mean 97123 RPM (pCRC) vs 164925 RPM (mLi) (Mir-10-P1a_5p exceptionally high expression)
- pvalue P = 2.20E-16 and 2.24E-16 (Mir-8-P2a_3p and Mir-8-P2b_3p higher in intestinal epithelial tissues vs nLi/nLu)
- fold_change LFC 2.69 (FDR 6.35E-09) and 2.72 (FDR 5.75E-11) (Mir-506-P3_3p and Mir-506-P4a1/P4a2/P4b_3p up-regulated in PM)
- other LFC >0.58 or < -0.58 and mean expression >100 RPM (filtering thresholds for differentially expressed miRNAs)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The study applied a custom bioinformatics pipeline (miRTrace + miRge3.0, MirGeneDB2.0 reference) to 268 miRNA-seq samples spanning primary CRC, three metastatic sites, and tumor-adjacent normal tissues. Differential expression was the primary analysis, performed independently per metastatic site versus pCRC using DESeq2 Wald tests with Benjamini-Hochberg FDR correction, LFC shrinkage, a 100 RPM expression floor, and a normal-tissue background correction step. qPCR validation of a lead finding and cell-type-specific miRNA comparisons used Welch two-sided t-tests, and pathway inference used logistic regression-based GSEA via RBiomirGS.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| DESeq2 Wald test (negative binomial GLM) with Benjamini-Hochberg FDR correction | Primary differential expression analysis: mLi vs pCRC, mLu vs pCRC, PM vs pCRC, pCRC vs nCR, and normal-tissue background comparisons (nCR vs nLi, nCR vs nLu, pCRC vs nLi, pCRC vs nLu) | pCRC n=120, nCR n=25, mLi n=35, nLi n=20, mLu n=28, nLu n=10, PM n=30 (per-tissue sample sizes used in relevant pairwise models) | not stated |
| Welch two-sided t-test | qPCR validation: Mir-210_3p dCq values compared between mLi and pCRC (Figure 5; t=-2.25, df=19.95, P=0.036) | mLi n=11, pCRC n=11 (independent validation samples not used in NGS analysis) | not stated |
| Welch two-sided t-test | Cell-type specific miRNA analysis: mean VST values compared between tissue groups (e.g., nCR vs nLi/nLu, intestinal vs non-intestinal tissues; Supplementary File S4) | based on tissue-level sample sizes as above; not stated per individual comparison | not stated |
| Logistic regression (via RBiomirGS) | Gene set enrichment analysis on GO (Molecular Function, Cellular Component, Biological Process) and KEGG pathways, using S_miRNA scores derived from DESeq2 LFC and FDR values | all miRNAs passing filters per metastatic site comparison; exact n not stated | not stated |
-
Three separate pairwise DESeq2 models were run (mLi vs pCRC, mLu vs pCRC, PM vs pCRC), with BH FDR correction applied independently within each model↳ Could also: A single multi-group DESeq2 model with tissue as a multi-level factor, or a mixed-effects model, could also have been used to jointly estimate effects across all sites in one analysis — A joint model directly estimates cross-site contrasts within the same family of tests, enabling formal tests of whether a miRNA's fold-change differs between metastatic sites, and a single FDR correction would cover the entire family of comparisons rather than each sub-family separately
-
Differential expression was performed using DESeq2 (negative binomial GLM with Wald test)↳ Could also: edgeR (exact test or quasi-likelihood F-test) or limma-voom could also have been applied to miRNA-seq count data — Both are widely validated alternatives for overdispersed RNA-seq count data; comparing results across two or more tools is a common sensitivity-check practice, as concordant findings across methods tend to be viewed as more robust
-
A Welch two-sided t-test was used to compare dCq values in the qPCR validation with n=11 per group↳ Could also: A Wilcoxon rank-sum (Mann-Whitney U) test could also have been used as a nonparametric alternative — With n=11 per group, verifying normality of dCq values is difficult; a nonparametric test requires no distributional assumption and is often applied as an alternative or sensitivity check for small-n qPCR comparisons
-
Multiple Welch t-tests were used to assess differences in cell-type specific miRNA levels across tissue groups, with exact p-values reported per comparison↳ Could also: A Benjamini-Hochberg FDR or Bonferroni correction across the family of cell-type miRNA t-tests could also have been applied — With 25 miRNAs each potentially compared across multiple tissue pairs, applying a multiplicity correction would explicitly control the false discovery rate across this family; whether such correction is warranted depends on whether the analysis is treated as confirmatory or exploratory
-
GSEA pathway enrichment used predicted miRNA-to-mRNA interactions as input to logistic regression (RBiomirGS)↳ Could also: Permutation-based GSEA (e.g., fgsea) or competitive gene-set tests (e.g., camera from limma) applied to predicted target-gene scores could also have been used — These alternatives use different null models (permutation vs. rotation-based) and handle inter-gene correlation differently, providing complementary statistical frameworks for assessing whether observed miRNA changes associate with coordinated regulation of biological pathways
-
UMAP was used as the sole dimensionality-reduction method for visualizing global miRNA expression patterns across all 268 samples↳ Could also: PCA or t-SNE could also have been used for global expression visualization, and PCA was applied separately for the cell-type specific miRNA subset — PCA is linear and directly interpretable in terms of variance explained by each component; presenting UMAP alongside PCA is common practice to show that observed clustering is not an artifact of the nonlinear embedding algorithm
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
MIR210 upregulation in colorectal cancer liver metastases versus primary tumor is independently validated by qPCRqPCR human colorectal-cancer up 2022×1papers★ This paper is the founder (earliest)
-
MIR1307 is upregulated in colorectal cancer liver and peritoneal metastases compared to primary tumorRNA-seq human colorectal-cancer up 2022×1papers★ This paper is the founder (earliest)
-
MIR141 is upregulated in colorectal cancer liver and lung metastases compared to primary tumorRNA-seq human colorectal-cancer up 2022×1papers★ This paper is the founder (earliest)
-
MIR155 is upregulated in colorectal cancer lung and peritoneal metastases compared to primary tumorRNA-seq human colorectal-cancer up 2022×1papers★ This paper is the founder (earliest)
-
MIR191 is upregulated at all three colorectal cancer metastatic sites (liver, lung, peritoneum) compared to primary tumorRNA-seq human colorectal-cancer up 2022×1papers★ This paper is the founder (earliest)
-
MIR21 is upregulated in primary colorectal cancer compared to normal colorectal tissue, part of a broader 32-miRNA oncogenic signatureRNA-seq human colorectal-cancer up 2022×1papers★ This paper is the founder (earliest)
-
MIR210 is upregulated in colorectal cancer liver metastases compared to primary colorectal cancer (LFC 1.26, FDR 4.73e-11)RNA-seq human colorectal-cancer up 2022×1papers★ This paper is the founder (earliest)
-
26 miRNAs are differentially expressed in one or more colorectal cancer metastatic sites versus primary tumor after background-expression correctionRNA-seq human colorectal-cancer mixed 2022×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-35047825
Paper: Høye et al. (2022) "A comprehensive framework for analysis of microRNA sequencing data in metastatic colorectal cancer." NAR Cancer 4(1):zcab051. DOI 10.1093/narcan/zcab051 · PMCID PMC8759566.
Code: https://github.com/eirikhoye/mirna_pipeline (cloned @ commit
09bae3087c109d38e29639e4318fb5800a78795b, «infra»).
Pipeline (as described in Methods + repo README)
Raw small-RNA-seq FASTQ → miRTrace (QC/contamination) → miRge3.0 (adapter trim + align to MirGeneDB2.0 reference) → count/RPM matrices → DESeq2 (VST exploration; differential expression with LFC-shrinkage). Downstream: UMAP, volcano plots, cell-composition inference.
Thresholds (README + Rmd, fixed): |log2FC| > 0.5849625 (=1.5×), min 100 RPM in at least one group, FDR (BH) < 0.05.
In scope (pipeline-derived, reproduced here)
The repo ships, as Supplementary_Files, the miRge3.0 count matrices
(miR.Counts1..7.csv, the 7 sequencing batches = seqdata_1,3,4,5,6,7,8 in the
authors' report), the sample sheet (sample_info_v9.csv, 268 QC-passed
samples) and the DESeq2 analysis code (scripts/deseq_functions.R,
r-markdown/diffexp_template*.Rmd) plus the rendered analysis report
(comet_analysis_report_..._mirge3_7.0.md). This makes the
count-matrix → DESeq2 → differential-expression segment fully reproducible
from shipped artifacts. Targets:
- Matrix dimension after merge+filter = 389 miRNAs (inner-join the 7 batches
on
miRNA, drop*passenger rows andchrper-locus rows). - Per-tissue sample counts (pCRC 120, mLi 35, mLu 28, nCR 25, nLi 20, nLu 10, PM 30; 268 total) — matches paper Section 3.
- Differential-expression counts (up/down) for 8 contrasts, incl. the paper headline pCRC vs nCR = 32 up / 35 down.
Pipeline named per result: DESeq2 (v from conda) on the shipped miRge3.0
counts, replicating the authors' diffexp Rmd / deseq_functions.R::DeseqResult.
Out of scope (not attempted) + why
- FASTQ → counts (miRTrace + miRge3.0 alignment): the new COMET sequencing
data is EGA-controlled (EGAS00001001127, access on request) →
data_restrictedfor that segment. The public GEO/SRA subsets (GSE57381/46622/63119, PRJNA397121) are only part of the 268 datasets, so re-aligning them would not reproduce the paper's matrix. We instead reproduce from the authors' shipped count matrices — equally valid for the downstream pipeline result (P16: applying the documented tool/params to the paper's own shipped data). - Wet-lab qPCR validation (Supplementary_file_5): non-pipeline → out of scope.
- UMAP / cell-composition / exact figure panels: the hard last ~20%; the DE count claims above are the clearly-specified, low-hanging outputs (80/20).
- Exact Table-1 per-miRNA RPM/LFC values (e.g. Mir-210_3p, Mir-191_5p): depend on figure-notebook aggregation; not attempted in this pass.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a clean 1:1 reproduction: starting from the authors' own shipped miRge3.0 count matrices and the documented DESeq2 thresholds, all 24 pinnable integer claims reproduce exactly — the matrix dimension (389 miRNAs), all 7 per-tissue sample counts, and up/down DE counts for all 8 tissue contrasts, including the Abstract headline of 32-up/35-down for pCRC vs nCR. Every reproduced value is derivable from the shared data+code, with no fabrication concern. The only unreproduced segment (FASTQ→counts) was skipped because the raw sequencing data is EGA-controlled, a data-access limitation rather than a discrepancy. The minor caveat that LFC-shrinkage used type='normal' instead of apeglm does not affect the integer DE counts.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.