Automatic discovery of 100-miRNA signature for cancer classification using ensemble feature selection.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH: yes. Authors' own repo (steppenwolf0/miRNAs100, commit 3a630d0, MIT) is self-contained and the data SHIPS IN-REPO (featureReduction/data/data_0.csv = TCGA pan-cancer miRNA matrix). Pipeline is plain scikit-learn: 8 classifiers x 10-fold stratified CV, ensemble feature selection to a 100-miRNA signature, accuracy compared full(1046) vs reduced(100) = paper Table 1. We rebuilt the EXACT paper-era env (scikit-learn 0.21.3 / pandas 0.25.3 / python 3.7) on «our HPC» and ran the exact code path. 1:1 vs different: WHERE TESTED, 1:1 to within run-to-run noise. Dataset shape reproduced exactly (8023x1046, 28 classes). 3 full-feature classifiers numerically reproduced: RidgeClassifier 0.8975 vs 0.8971 (exact), SGDClassifier 0.9099 vs 0.9100 (exact), PassiveAggressive 0.9026 vs 0.9117 (within-tol, diff 0.009). NOT ATTEMPTED/uncollected: the 5 compute-heavy classifiers (GradientBoosting300, Bagging300, RandomForest300, SVC-linear, and LogisticRegression-lbfgs) and the headline 100-miRNA-signature comparison were NOT collected within the room window - the full ensemble run (bBioInf.py) is ~3-4h on one std node and was launched («job», ran 35min) then cancelled at finalization. KEY CAVEAT: the original code sets NO random seed anywhere, so exact bit-reproduction is impossible by design; the honest target is within-tolerance, which the 3 collected points satisfy (2 exact, 1 within 0.9 pts). No fabrication signal: all attempted values derive directly from shipped data+code and match. Recipe + commit + data SHA256 pinned in agreement.json so a fuller run can complete Table 1 directly.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 69assessed: 2026-06-15 ⛓ adf3d7fa7540
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-09-19
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper tests whether an ensemble feature selection strategy, combining consensus from multiple classifier types, can identify a small (100-miRNA) signature from a large TCGA miRNA expression dataset that retains classification accuracy comparable to the full feature set and generalizes across independent platforms and datasets.
- ★ An ensemble feature selection strategy using consensus of feature relevance across 8 classifier types identifies a 100-miRNA signature from a 1046-feature TCGA dataset method
- ★ The 100-miRNA signature achieves classification accuracy comparable to the full 1046-feature set on TCGA data, differing by only 1.4% on average finding
- ★ The 100-miRNA signature is sufficient to distinguish tumor tissue from normal tissue finding
- ★ The proposed ensemble feature selection (EFS) method outperforms established feature selection methods (Random, GALGO, EFS-CLA, UFS, EN, LASSO, RFE) at k=100 on TCGA classification accuracy finding
- ★ The 100-miRNA signature generalizes across 14 GEO datasets spanning different platforms and cancer types, with some classifiers exceeding 90% accuracy finding
- ★ 77 of the 100 miRNAs in the signature appear in prior literature lists of circulating miRNAs used in cancer studies, per bibliographic meta-analysis finding
- Ridge classifier shows notably decreased performance with the 100-miRNA signature, likely because it relies on features outside the top 100 finding
- SGD classifier performs significantly better than other classifiers on the GEO datasets finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| miRNA expression profiling (sequencing, stem-loop miRBase v16) | TCGA pan-cancer samples (8023 cases, 28 cancer types) | none | cancer type classification accuracy (10-fold CV) with 1046 vs 100 features | scikit-learn (8 classifiers: Gradient Boosting, Random Forest, Logistic Regression, Passive Aggressive, SGD, SVC, Ridge, Bagging) |
| miRNA expression profiling | TCGA samples | none | Tumor Tissue (TT) vs Normal Tissue (NT) classification accuracy (stratified 10-fold CV) | scikit-learn classifiers |
| feature selection benchmarking on miRNA expression data | TCGA samples | none | classification accuracy comparison across feature selection methods (Random, GALGO, EFS-CLA, UFS, EN, LASSO, RFE, EFS) at k=100 | scikit-learn; GALGO (R); EFS-CLA |
| miRNA microarray/expression profiling | 14 GEO datasets (tumor types: HNSC, PRAD, LIHC, BRCA, ESCA, LUAD) | none | cross-platform classification accuracy using 100-miRNA signature (10-fold CV, 10 repeats) | Affymetrix (GPL8786, GPL14613, GPL16384), Illumina 2000, Agilent-021827 (GPL10850, GPL11154) |
| miRNA expression profiling | TCGA and GEO breast cancer (BRCA) samples | none | molecular subtype classification using 100-miRNA signature | — |
| bibliographic/literature meta-analysis | 100 miRNAs in signature | none | presence in prior circulating miRNA cancer study lists | — |
- ▼ Classification accuracy using 100-miRNA signature vs full 1046 features on TCGA differs by only 1.4% on average (global average 0.9044 vs 0.9185) 1.4%
- ▲ 100-miRNA signature yields high accuracy distinguishing tumor vs normal tissue across classifiers (e.g., Gradient Boosting 0.9846 global)
- ▲ EFS achieves highest global average accuracy (0.9044) among compared feature selection methods at k=100, ahead of RFE (0.8892), LASSO (0.8820), EN (0.8771), UFS (0.8534), EFS-CLA (0.8472), GALGO (0.8010), Random (0.7777)
- – Overlap of EFS-selected 100 features with other methods: 54 shared with RFE, 48 with EN, 44 with LASSO, 38 with UFS, 29 with EFS-CLA, 11 with GALGO, 8 with Random
- ▲ On 14 GEO datasets, Logistic Regression and SGD classifiers achieve over 93% average accuracy using the 100-miRNA signature >93%
- ▲ SGD performs significantly better than other classifiers across GEO datasets (Kolmogorov-Smirnov test) p<0.05
- – 77 of 100 signature miRNAs appear in prior circulating miRNA literature lists 77/100
- ▼ Ridge classifier is the only algorithm showing decreased performance with the 100-miRNA signature relative to full feature set
- pvalue p<0.05 (T-test for 1.4% accuracy difference between 1046 and 100 features on TCGA)
- pvalue p<0.05 (Kolmogorov-Smirnov test showing SGD significantly better than other classifiers on GEO datasets)
- mean 0.9044 (Global average accuracy across 8 classifiers using 100-miRNA signature on TCGA (10-fold CV))
- mean 0.918463 (Global average accuracy across 8 classifiers using full 1046 features on TCGA (10-fold CV))
- count 8023 samples, 28 cancer types, 1046 miRNA features (TCGA dataset used for feature selection and initial validation)
- count 77/100 (Number of signature miRNAs found in prior circulating miRNA literature lists)
- count 14 GEO datasets (Cross-platform validation datasets)
- fold_change 1046 to 100 features (~10-fold reduction) (Reduction of original feature set to signature size)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a machine-learning bioinformatics study whose statistical reporting centers on classification accuracy estimated by 10-fold (and stratified 10-fold) cross-validation across eight scikit-learn classifiers, comparing the full 1046-feature set to a 100-miRNA signature and to several alternative feature-selection methods. Accuracies are reported as averages with standard deviations, and a small number of inferential tests are used to compare results: a t-test to assess the 1.4% accuracy difference between full and reduced feature sets, and a Kolmogorov-Smirnov test to compare a classifier's accuracy distribution against the others. Validation is extended to 14 external GEO datasets and supplemented by a literature-based meta-analysis of the selected miRNAs.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| t-test (type not specified) | difference in classification accuracy between the 1046-feature set and the 100-miRNA signature on TCGA (reported 1.4% difference, p<0.05) | — | not stated |
| Kolmogorov-Smirnov test | comparison of SGD classifier average accuracy/standard deviation versus the other classifiers on the GEO datasets (Fig. 4, p<0.05) | — | not stated |
| ANOVA F-value (univariate score) | described as the scoring statistic underlying the Univariate Feature Selection (UFS) comparison method | — | not stated |
-
The 1.4% accuracy difference between the full feature set and the 100-miRNA signature was assessed with a t-test reported as p<0.05.↳ Could also: A paired test across the matched cross-validation folds (e.g., paired t-test or Wilcoxon signed-rank), or a corrected resampled t-test designed for cross-validation, could also be used. — Cross-validation fold accuracies for the same data are paired and overlapping; a paired or CV-aware test accounts for that structure and the alternative also reports the effect magnitude alongside the p-value.
-
The KS test was used to compare one classifier's accuracy results against the others on the GEO datasets.↳ Could also: A rank-based test for paired samples across datasets (e.g., Wilcoxon signed-rank) or a Friedman test with post-hoc comparisons across all classifiers could also be applied. — When the same datasets are evaluated by every classifier, paired/blocked tests use the dataset pairing directly, and a Friedman-style omnibus also handles the comparison of many classifiers at once.
-
Many pairwise comparisons among classifiers and feature-selection methods are reported without an adjustment for multiple comparisons.↳ Could also: A family-wise (e.g., Bonferroni, Holm) or FDR (Benjamini-Hochberg) correction, or an omnibus test followed by post-hoc adjustment, could also be applied. — Across many simultaneous comparisons a correction controls the overall error rate and makes the family of reported significances easier to interpret jointly.
-
Dispersion of cross-validation accuracy is reported as standard deviation.↳ Could also: A 95% confidence interval (or standard error) for the mean accuracy could also be reported. — A confidence interval conveys the precision of the estimated mean accuracy directly, which complements the spread shown by the SD, especially with a fixed number of folds.
-
Differences between methods and feature sets are summarized primarily through accuracy point estimates and significance thresholds.↳ Could also: Effect-size measures (e.g., mean accuracy difference with its CI) and additional class-aware metrics such as balanced accuracy, F1, or AUC could also be reported. — With class imbalance noted (e.g., normal vs tumor), class-aware metrics and explicit effect sizes give a fuller picture of performance beyond global accuracy.
-
Significance is reported as p<0.05 thresholds.↳ Could also: Reporting exact p-values could also be done. — Exact p-values let readers gauge the strength of evidence and apply their own thresholds or corrections.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
100-miRNA signature achieves >93% average accuracy in cross-platform validation on independent GEO microarray datasets (HNSC, PRAD, LIHC, BRCA, LUAD).microarray human cancer up 2019×1papers★ This paper is the founder (earliest)
-
77 of 100 ensemble-selected signature miRNAs have been independently reported as circulating cancer biomarkers in published literature.other human cancer 2019×1papers★ This paper is the founder (earliest)
-
100-miRNA ensemble-selected signature achieves pan-cancer classification accuracy within 1.4% of the full 1046-miRNA feature set across multiple classifiers on TCGA data.other human cancer none 2019×1papers★ This paper is the founder (earliest)
-
Ensemble feature selection (EFS) achieves highest global average accuracy (0.9044) among all tested feature selection methods for miRNA-based pan-cancer classification on TCGA.other human cancer up 2019×1papers★ This paper is the founder (earliest)
-
EFS-selected 100-miRNA signature shows variable overlap with other feature selection methods: 54 features shared with RFE, 48 with EN, 44 with LASSO, 38 with UFS on TCGA data.other human cancer 2019×1papers★ This paper is the founder (earliest)
-
Ridge classifier accuracy decreases from 0.8971 to 0.8305 when restricted to the 100-miRNA signature vs the full 1046-feature set on TCGA data.other human cancer down 2019×1papers★ This paper is the founder (earliest)
-
100-miRNA signature enables high-accuracy tumor vs normal tissue classification (up to 0.9846 with Gradient Boosting) across 28 TCGA cancer types.RNA-seq human cancer up 2019×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-31533612
Paper: Lopez-Rincon et al. (2019), Automatic discovery of 100-miRNA signature
for cancer classification using ensemble feature selection. BMC Bioinformatics.
Repo: https://github.com/steppenwolf0/miRNAs100 (commit 3a630d0, MIT) — authors' own code.
Data: ships IN the repo (featureReduction/data/data_0.csv = 8023 samples ×
1046 miRNAs, TCGA pan-cancer; labels.csv = 28-class labels). GEO accession
GSE62182 (and other GEO sets) are used in the paper for cross-platform validation.
Pipeline (self-contained Python / scikit-learn)
featureReduction/bBioInf.py orchestrates:
featureSelection(0)— ensemble feature selection on the FULL 1046-miRNA matrix: 8 classifiers (GradientBoosting300, RandomForest300, LogisticRegression, PassiveAggressive, SGD, SVC-linear, Ridge, Bagging300), each evaluated with 10-fold stratified CV; mean accuracy per classifier written todata/results.txt; per-classifier top-100 important features aggregated by frequency intoFS/global_0.csv(the 100-miRNA signature).reduceDataset(0)— subsets the matrix to those top-100 miRNAs →data_1.csv.featureSelection(1)— re-evaluates the same 8 classifiers (10-fold CV) on the 100-feature matrix; mean accuracy appended todata/results.txt.
In scope (attempting — pipeline-derived, clearly specified)
- Table 1: per-classifier 10-fold CV accuracy, full 1046 features (8 values).
- Table 1: per-classifier 10-fold CV accuracy, reduced 100-miRNA signature (8 values).
- Aggregate: average accuracy across the 8 classifiers (full vs reduced) and the reported "global accuracy differs only by 1.4%" claim.
- The 100-miRNA signature itself (
FS/global_0.csv) as an auditable artifact.
Out of scope / not attempted (80/20)
- Cross-platform GEO validation (GSE62182, GSE105134, … 14 GEO sets in
GEO Datasets/): separatecalA.pyruns per dataset — secondary, not the headline result. - BRCA subtype / TNBC classification (
subtype/): secondary analyses. - Figures (PCA plots, heatmaps).
- The exact identity of all 100 signature miRNAs is not bit-reproducible: the code sets no random seed anywhere (StratifiedKFold shuffle=True, RF/SGD/PAC/ Bagging all unseeded), so fold splits and several classifiers vary run-to-run. We therefore expect accuracies to match within tolerance (~±1–2%), not exactly, and the selected feature set to overlap-but-not-equal the published list.
Reproduction environment
scikit-learn 0.21.3 / pandas 0.25.3 / numpy 1.17 / python 3.7 (paper-era 2019), conda prefix env on «infra». «our HPC» SLURM (partition std, 16 cpus, «job»).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Where tested, this reproduces cleanly: data ships in-repo and the shape (8023×1046, 28 classes) plus 3 full-feature accuracies match within run-to-run noise (2 exact <0.0005, PassiveAggressive within 0.0091). The only deviations are stochastic — the original code sets no random seed, so bit-exactness is impossible by design; deviations are on the technical/expected side, not authors' or data. However, the paper's headline contribution — the 100-miRNA signature ≈ full-feature accuracy (1.4% gap) — and 5 compute-heavy classifiers were never collected (job cancelled at ~35min), so the core claim is supported in spirit but not numerically confirmed: a fuller run is needed.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at [email protected].
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.