Automatic discovery of 100-miRNA signature for cancer classification using ensemble feature selection.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH: yes. Authors' own repo (steppenwolf0/miRNAs100, commit 3a630d0, MIT) is self-contained and the data SHIPS IN-REPO (featureReduction/data/data_0.csv = TCGA pan-cancer miRNA matrix). Pipeline is plain scikit-learn: 8 classifiers x 10-fold stratified CV, ensemble feature selection to a 100-miRNA signature, accuracy compared full(1046) vs reduced(100) = paper Table 1. We rebuilt the EXACT paper-era env (scikit-learn 0.21.3 / pandas 0.25.3 / python 3.7) on «our HPC» and ran the exact code path. 1:1 vs different: WHERE TESTED, 1:1 to within run-to-run noise. Dataset shape reproduced exactly (8023x1046, 28 classes). 3 full-feature classifiers numerically reproduced: RidgeClassifier 0.8975 vs 0.8971 (exact), SGDClassifier 0.9099 vs 0.9100 (exact), PassiveAggressive 0.9026 vs 0.9117 (within-tol, diff 0.009). NOT ATTEMPTED/uncollected: the 5 compute-heavy classifiers (GradientBoosting300, Bagging300, RandomForest300, SVC-linear, and LogisticRegression-lbfgs) and the headline 100-miRNA-signature comparison were NOT collected within the room window - the full ensemble run (bBioInf.py) is ~3-4h on one std node and was launched («job», ran 35min) then cancelled at finalization. KEY CAVEAT: the original code sets NO random seed anywhere, so exact bit-reproduction is impossible by design; the honest target is within-tolerance, which the 3 collected points satisfy (2 exact, 1 within 0.9 pts). No fabrication signal: all attempted values derive directly from shipped data+code and match. Recipe + commit + data SHA256 pinned in agreement.json so a fuller run can complete Table 1 directly.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 69assessed: 2026-06-15 ⛓ adf3d7fa7540
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan an ensemble feature selection strategy, based on consensus feature relevance across multiple high-accuracy classifier types, identify a minimal and robust miRNA signature that classifies cancer tumor types and distinguishes tumor from normal tissue while reducing features by an order of magnitude?
- ★ An ensemble feature selection method based on classifier consensus identifies a robust 100-miRNA signature from TCGA data. method
- ★ The 100-miRNA signature yields classification accuracy differing from the full 1046-feature set by only 1.4%. finding
- ★ The 100-miRNA signature is sufficient to distinguish tumor tissue from normal tissue. finding
- ★ The proposed ensemble approach outperforms established feature selection methods (UFS, RFE, EN, LASSO, GALGO, EFS-CLA, Random) in classification accuracy. finding
- ★ The signature is cross-platform applicable, validated on 14 GEO datasets across different platforms and cancer types with some classifiers exceeding 90% accuracy. finding
- ★ A bibliographic meta-analysis confirms 77 of the 100 signature miRNAs appear in circulating-miRNA cancer studies, leaving 23 as candidates for future research. resource
- Evaluating consensus across classifiers of different typologies is more robust than relying on a single algorithm for feature selection. mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| miRNA expression sequencing (stem-loop miRNA expression) | TCGA human cancer samples (8023 cases, 28 cancer types) | none | cancer tumor type classification accuracy (10-fold CV) across 8 classifiers | miRBase V16 / TCGA (sequencing); scikit-learn classifiers |
| miRNA expression classification (tumor vs normal) | TCGA human samples | none | stratified 10-fold CV accuracy for Tumor Tissue vs Normal Tissue | scikit-learn |
| Feature selection method comparison | TCGA dataset (1046 features) | none | 10-fold CV classification accuracy for k=100 features across methods (Random, GALGO, EFS-CLA, UFS, EN, LASSO, RFE, EFS) | scikit-learn; GALGO (R) |
| miRNA microarray expression (Affymetrix Multispecies miRNA arrays) | GEO datasets HNSC/PRAD/LIHC (GPL8786) | none | cross-platform classification accuracy (10-fold CV, 10 repeats) | Affymetrix Multispecies Array miRNA-1/2/3 (GPL8786, GPL14613, GPL16384) |
| miRNA microarray expression (Agilent Human miRNA Microarray V3) | GEO datasets LIHC/BRCA (GPL10850) | none | cross-platform classification accuracy | Agilent-021827 Human miRNA Microarray V3 (GPL10850) |
| miRNA sequencing expression | GEO dataset GSE62182 LUAD (GPL11154) | none | cross-platform classification accuracy | Illumina 2000 (GPL11154) |
| miRNA expression classification of breast cancer molecular subtypes | TCGA BRCA and GEO BRCA datasets (GSE86277/86278/86281) | none | molecular subtype classification accuracy | GPL14613/GPL16384 |
| Bibliographic/literature meta-analysis | 100-miRNA signature | none | count of signature miRNAs reported as circulating biomarkers in cancer studies | — |
- – 100-miRNA signature gives nearly the same accuracy as full 1046 features across classifiers 1.4% difference
- ▲ Ensemble (EFS) achieves highest global average accuracy among feature selection methods 0.9044 global avg vs 0.8892 RFE, 0.7777 Random
- ▲ Classifiers differentiate tumor vs normal tissue well (except Ridge) 100-Global up to 0.9846 (Gradient Boosting)
- ▲ Cross-platform GEO validation: Logistic and SGD exceed 93% average accuracy across all GEO datasets >93% average accuracy
- – 77 of 100 signature miRNAs found in circulating-miRNA cancer literature 77/100
- ▼ Ridge shows decreased performance with 100-feature signature 100-feat 0.8305 vs 1046-feat 0.8971
- ▲ Approach outperforms Telonis et al. on shared GEO datasets GSE36802 baseline 76%, GSE67138 91%, GSE67139 96% all surpassed
- – Overlap of EFS-selected features with other methods varies 54 RFE, 48 EN, 44 LASSO, 38 UFS, 29 EFS-CLA, 11 GALGO, 8 Random
- other difference of 1.4% (T-test, p<0.05) (accuracy difference between 1046 features and 100-miRNA signature on TCGA)
- count 8023 cases, 28 cancer types, 1046 stem-loop miRNAs (TCGA dataset composition)
- count 77 out of 100 (signature miRNAs appearing in circulating miRNA cancer studies)
- mean 0.9044 global average (EFS ensemble accuracy across classifiers, k=100)
- mean 0.918463 (1046 feats) vs 0.9044 (100 feats) (average classifier accuracy on TCGA)
- count RFE calls classifier N-k=1046-100=946 times (computational cost of RFE)
- pvalue p<0.05 (Kolmogorov-Smirnov test) (SGD significantly better than other classifiers on GEO datasets)
- count 98 of 100 features available for most GEO datasets; 75 for GSE62182 (signature feature availability across platforms)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a machine-learning bioinformatics study whose statistical reporting centers on classification accuracy estimated by 10-fold (and stratified 10-fold) cross-validation across eight scikit-learn classifiers, comparing the full 1046-feature set to a 100-miRNA signature and to several alternative feature-selection methods. Accuracies are reported as averages with standard deviations, and a small number of inferential tests are used to compare results: a t-test to assess the 1.4% accuracy difference between full and reduced feature sets, and a Kolmogorov-Smirnov test to compare a classifier's accuracy distribution against the others. Validation is extended to 14 external GEO datasets and supplemented by a literature-based meta-analysis of the selected miRNAs.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| t-test (type not specified) | difference in classification accuracy between the 1046-feature set and the 100-miRNA signature on TCGA (reported 1.4% difference, p<0.05) | — | not stated |
| Kolmogorov-Smirnov test | comparison of SGD classifier average accuracy/standard deviation versus the other classifiers on the GEO datasets (Fig. 4, p<0.05) | — | not stated |
| ANOVA F-value (univariate score) | described as the scoring statistic underlying the Univariate Feature Selection (UFS) comparison method | — | not stated |
-
The 1.4% accuracy difference between the full feature set and the 100-miRNA signature was assessed with a t-test reported as p<0.05.↳ Could also: A paired test across the matched cross-validation folds (e.g., paired t-test or Wilcoxon signed-rank), or a corrected resampled t-test designed for cross-validation, could also be used. — Cross-validation fold accuracies for the same data are paired and overlapping; a paired or CV-aware test accounts for that structure and the alternative also reports the effect magnitude alongside the p-value.
-
The KS test was used to compare one classifier's accuracy results against the others on the GEO datasets.↳ Could also: A rank-based test for paired samples across datasets (e.g., Wilcoxon signed-rank) or a Friedman test with post-hoc comparisons across all classifiers could also be applied. — When the same datasets are evaluated by every classifier, paired/blocked tests use the dataset pairing directly, and a Friedman-style omnibus also handles the comparison of many classifiers at once.
-
Many pairwise comparisons among classifiers and feature-selection methods are reported without an adjustment for multiple comparisons.↳ Could also: A family-wise (e.g., Bonferroni, Holm) or FDR (Benjamini-Hochberg) correction, or an omnibus test followed by post-hoc adjustment, could also be applied. — Across many simultaneous comparisons a correction controls the overall error rate and makes the family of reported significances easier to interpret jointly.
-
Dispersion of cross-validation accuracy is reported as standard deviation.↳ Could also: A 95% confidence interval (or standard error) for the mean accuracy could also be reported. — A confidence interval conveys the precision of the estimated mean accuracy directly, which complements the spread shown by the SD, especially with a fixed number of folds.
-
Differences between methods and feature sets are summarized primarily through accuracy point estimates and significance thresholds.↳ Could also: Effect-size measures (e.g., mean accuracy difference with its CI) and additional class-aware metrics such as balanced accuracy, F1, or AUC could also be reported. — With class imbalance noted (e.g., normal vs tumor), class-aware metrics and explicit effect sizes give a fuller picture of performance beyond global accuracy.
-
Significance is reported as p<0.05 thresholds.↳ Could also: Reporting exact p-values could also be done. — Exact p-values let readers gauge the strength of evidence and apply their own thresholds or corrections.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
100-miRNA signature achieves >93% average accuracy in cross-platform validation on independent GEO microarray datasets (HNSC, PRAD, LIHC, BRCA, LUAD).microarray human cancer up 2019×1papers★ This paper is the founder (earliest)
-
77 of 100 ensemble-selected signature miRNAs have been independently reported as circulating cancer biomarkers in published literature.other human cancer 2019×1papers★ This paper is the founder (earliest)
-
100-miRNA ensemble-selected signature achieves pan-cancer classification accuracy within 1.4% of the full 1046-miRNA feature set across multiple classifiers on TCGA data.other human cancer none 2019×1papers★ This paper is the founder (earliest)
-
Ensemble feature selection (EFS) achieves highest global average accuracy (0.9044) among all tested feature selection methods for miRNA-based pan-cancer classification on TCGA.other human cancer up 2019×1papers★ This paper is the founder (earliest)
-
EFS-selected 100-miRNA signature shows variable overlap with other feature selection methods: 54 features shared with RFE, 48 with EN, 44 with LASSO, 38 with UFS on TCGA data.other human cancer 2019×1papers★ This paper is the founder (earliest)
-
Ridge classifier accuracy decreases from 0.8971 to 0.8305 when restricted to the 100-miRNA signature vs the full 1046-feature set on TCGA data.other human cancer down 2019×1papers★ This paper is the founder (earliest)
-
100-miRNA signature enables high-accuracy tumor vs normal tissue classification (up to 0.9846 with Gradient Boosting) across 28 TCGA cancer types.RNA-seq human cancer up 2019×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-31533612
Paper: Lopez-Rincon et al. (2019), Automatic discovery of 100-miRNA signature
for cancer classification using ensemble feature selection. BMC Bioinformatics.
Repo: https://github.com/steppenwolf0/miRNAs100 (commit 3a630d0, MIT) — authors' own code.
Data: ships IN the repo (featureReduction/data/data_0.csv = 8023 samples ×
1046 miRNAs, TCGA pan-cancer; labels.csv = 28-class labels). GEO accession
GSE62182 (and other GEO sets) are used in the paper for cross-platform validation.
Pipeline (self-contained Python / scikit-learn)
featureReduction/bBioInf.py orchestrates:
featureSelection(0)— ensemble feature selection on the FULL 1046-miRNA matrix: 8 classifiers (GradientBoosting300, RandomForest300, LogisticRegression, PassiveAggressive, SGD, SVC-linear, Ridge, Bagging300), each evaluated with 10-fold stratified CV; mean accuracy per classifier written todata/results.txt; per-classifier top-100 important features aggregated by frequency intoFS/global_0.csv(the 100-miRNA signature).reduceDataset(0)— subsets the matrix to those top-100 miRNAs →data_1.csv.featureSelection(1)— re-evaluates the same 8 classifiers (10-fold CV) on the 100-feature matrix; mean accuracy appended todata/results.txt.
In scope (attempting — pipeline-derived, clearly specified)
- Table 1: per-classifier 10-fold CV accuracy, full 1046 features (8 values).
- Table 1: per-classifier 10-fold CV accuracy, reduced 100-miRNA signature (8 values).
- Aggregate: average accuracy across the 8 classifiers (full vs reduced) and the reported "global accuracy differs only by 1.4%" claim.
- The 100-miRNA signature itself (
FS/global_0.csv) as an auditable artifact.
Out of scope / not attempted (80/20)
- Cross-platform GEO validation (GSE62182, GSE105134, … 14 GEO sets in
GEO Datasets/): separatecalA.pyruns per dataset — secondary, not the headline result. - BRCA subtype / TNBC classification (
subtype/): secondary analyses. - Figures (PCA plots, heatmaps).
- The exact identity of all 100 signature miRNAs is not bit-reproducible: the code sets no random seed anywhere (StratifiedKFold shuffle=True, RF/SGD/PAC/ Bagging all unseeded), so fold splits and several classifiers vary run-to-run. We therefore expect accuracies to match within tolerance (~±1–2%), not exactly, and the selected feature set to overlap-but-not-equal the published list.
Reproduction environment
scikit-learn 0.21.3 / pandas 0.25.3 / numpy 1.17 / python 3.7 (paper-era 2019), conda prefix env on «infra». «our HPC» SLURM (partition std, 16 cpus, «job»).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Where tested, this reproduces cleanly: data ships in-repo and the shape (8023×1046, 28 classes) plus 3 full-feature accuracies match within run-to-run noise (2 exact <0.0005, PassiveAggressive within 0.0091). The only deviations are stochastic — the original code sets no random seed, so bit-exactness is impossible by design; deviations are on the technical/expected side, not authors' or data. However, the paper's headline contribution — the 100-miRNA signature ≈ full-feature accuracy (1.4% gap) — and 5 compute-heavy classifiers were never collected (job cancelled at ~35min), so the core claim is supported in spirit but not numerically confirmed: a fuller run is needed.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.