Wx: a neural network-based feature selection algorithm for transcriptomic data.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓The central claim held under reproduction
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce ONE headline number 1:1 on the paper's own public data: GSE72056 melanoma external validation (Table 4). We obtained GSE72056 from GEO (the repo ships NO paper data) and classified malignant vs non-malignant cells with XGBoost over the fixed reported Wx-14 gene panel. The COHORT reproduces the paper EXACTLY (1257 malignant + 3256 non-malignant = 4513 cells) and all 14 genes are present, but our accuracy is 96.68% vs the reported 90.71% (~6 pts higher, same direction, both far above the 72.1% majority baseline); the gap is most likely the CV protocol (paper: single 3611/902 split + 5-fold; ours: StratifiedKFold-5). Grade=partial. NOT attempted (the hard ~20%): the Wx-14 panel itself (C1), TCGA 96.72% overall (C2) and Fig-2 AUCs (C5) all need the full 12-cancer TCGA matrix via the deprecated TCGA-Assembler tool, and C1 is additionally stochastic (10k softmax iterations); GSE40419/GSE103322 (C4) are a feasible same-shape follow-up deprioritised under 80/20. Caveat for reviewer: the repo's shipped data (input_data_1.csv toy; GSE_DATA = GSE105127, not in this paper) reproduces NO reported value, so headline numbers are only reproducible via independent data downloads. Not flagged as fabrication: the independent reproduction supports the claim in spirit.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-14 ⛓ af451bdaa53d
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-09-19
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetA neural network-based feature selection algorithm (Wx), which ranks genes by a discriminative index (DI) score, can identify an optimal, minimal set of gene-expression biomarkers that distinguish cancer from normal samples as well as or better than existing feature-selection/DEG-based methods.
- ★ The Wx algorithm ranks genes using a DI score reflecting each gene's classification power for distinguishing two groups. method
- ★ The top 14 genes identified by Wx (Wx-14-UGCB) were comparable to or outperformed previously reported universal gene-expression cancer biomarkers (Peng et al., Martinez-Ledesma et al.) in classification accuracy. finding
- ★ Approximately the top 100 genes ranked by DI score are sufficient to maximize average classification accuracy; adding more genes gives no further improvement. finding
- ★ Gene sets identified by Wx overlap less than 35% with genes identified by the DEG method edgeR on the same data. finding
- Common housekeeping genes GAPDH and ACTB rank among the highest DI-score genes, suggesting they are unsuitable as reference/control genes in cancer expression studies. finding
- ★ Wx-14-UGCB outperformed Peng-14-UGCB when validated on three independent RNA-seq datasets (melanoma, lung adenocarcinoma, head and neck squamous cell carcinoma single cells) not used in training. finding
- Wx showed relatively lower classification accuracy when applied to a non-cancer transcriptomic dataset (human liver zonation, GSE105127). finding
- Stand-alone and web versions of the Wx algorithm are made publicly available. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| bulk RNA-seq (mRNASeq, RSEM normalized) | TCGA pan-cancer cohort, 12 cancer types (tumor vs normal tissue) | none | DI score / classification accuracy distinguishing cancer vs normal | Illumina HiSeq (TCGA-Assembler 2) |
| single-cell RNA-seq | GSE72056 melanoma (malignant vs benign single cells) | none | classification accuracy of Wx-14-UGCB vs Peng-14-UGCB | — |
| RNA-seq | GSE40419 lung adenocarcinoma (cancer vs adjacent normal tissue) | none | classification accuracy of Wx-14-UGCB vs Peng-14-UGCB | — |
| single-cell RNA-seq | GSE103322 head and neck squamous cell carcinoma (cancer vs non-cancer single cells) | none | classification accuracy of Wx-14-UGCB vs Peng-14-UGCB | — |
| RNA-seq (laser-capture microdissection) | GSE105127 human liver, pericentral/intermediate/periportal zones (n=19 each) | none | classification accuracy of Wx algorithm on non-cancer data | — |
| gene ontology / network enrichment analysis | top 50 UGCB genes from TCGA pan-cancer analysis | none | enriched pathways (Fc gamma receptor phagocytosis, antigen processing/presentation, apoptotic signaling regulation) | Metascape |
| differential expression gene (DEG) analysis | TCGA pan-cancer RNA-seq data | none | DEG gene list (adjusted p<0.05) compared to Wx gene ranking | edgeR |
| machine learning classification (SVM vs XGBoost) | TCGA pan-cancer UGCB gene sets | none | classification accuracy comparison between algorithms | SVM; XGBoost |
- ▲ Total classification accuracy across cancer types: Wx-14-UGCB 96.72% vs Peng-14-UGCB 94.59% vs edgeR-14 (DEG) 94.81% 96.72% vs 94.59%/94.81%
- ▲ Total classification accuracy with 7-gene sets: Wx-7-UGCB 95.74% vs Martinez-Ledesma-7-UGCB 92.20% 95.74% vs 92.20%
- – Classification accuracy plateaus using approximately the top 100 genes ranked by DI score
- – Overlap between Wx and edgeR gene rankings is low: 9.0% (45/500 genes) at top 500 and 19.0% (379/2000 genes) at top 2,000 9.0%–19.0% overlap
- ▲ GSE72056 melanoma test set: Wx-14-UGCB correctly classified 818/902 samples vs 633/902 for Peng-14-UGCB (90.71% vs 70.22% accuracy) 90.71% vs 70.22%
- ▲ GSE40419 lung adenocarcinoma: Wx-14-UGCB 80.00% accuracy vs Peng-14-UGCB 56.87% 80.00% vs 56.87%
- ▲ GSE103322 HNSC single cells: Wx-14-UGCB 81.10% accuracy vs Peng-14-UGCB 68.28% 81.10% vs 68.28%
- ▲ AUC values for Wx-14-UGCB classification of BRCA, LUAD, and LUSC were 0.9944, 0.9943, and 0.9936 respectively AUC 0.9936–0.9944
- other 96.72% (Wx-14-UGCB) vs 94.59% (Peng-14-UGCB) vs 94.81% (edgeR-14) (total classification accuracy across 12 cancer types, LOOCV)
- other AUC = 0.9944 (BRCA), 0.9943 (LUAD), 0.9936 (LUSC) (Wx-14-UGCB classification performance)
- count 45 genes (9.0%) overlap (overlap between top 500 Wx and edgeR genes)
- count 379 genes (19.0%) overlap (overlap between top 2,000 Wx and edgeR genes)
- count 818/902 (Wx-14-UGCB) vs 633/902 (Peng-14-UGCB) correctly classified (GSE72056 melanoma test set classification)
- pvalue adjusted p value ≤ 0.05 (standard cutoff for DEG identification (edgeR))
- count 6,226 total samples (5,609 tumor, 617 control) (TCGA pan-cancer cohort used for UGCB identification)
- other 10,000 iterations (Wx algorithm iterated to average DI scores due to neural network weight initialization variability)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper introduces Wx, a neural network-based gene feature selection algorithm that assigns a Discriminative Index (DI) score to each gene and ranks them by classification power for cancer vs. normal discrimination. Performance was evaluated primarily by leave-one-out cross-validation (LOOCV) classification accuracy across 12 TCGA cancer types, with AUC reported for three cancer types. Benchmarking against edgeR-derived DEG sets and two previously published biomarker sets was conducted by directly comparing LOOCV accuracy percentages, without formal statistical testing of observed differences. External generalization was assessed using a single fixed 64/16/20% train/validation/test split on three independent GEO cohorts.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Leave-one-out cross-validation (LOOCV) classification accuracy with XGBoost | Primary performance evaluation of Wx-14-UGCB, Peng-14-UGCB, DEG-14-UGCB, and Wx-7-UGCB vs. MartinezL-7-UGCB across 12 TCGA cancer types (Table 3) | 6,226 total TCGA samples (5,609 cancer, 617 normal); per-type n ranges from 90 (KICH) to 1,214 (BRCA) as listed in Table 1 | not stated |
| edgeR differential expression analysis (negative binomial GLM, FDR-adjusted p-value threshold 0.05) | Identification of top 14 DEGs (DEG-14-UGCB) used as a comparison benchmark (Table 3) | Same 6,226 TCGA pan-cancer samples | not stated |
| AUC / ROC analysis | Classification performance of Wx-14-UGCB on BRCA, LUAD, and LUSC (Figure 2) | Implied from Table 1: BRCA n=1,214; LUAD n=576; LUSC n=553 | not stated |
| Fixed-split holdout accuracy (64% train / 16% validation / 20% test) with neural network classifier | External validation of Wx-14-UGCB vs. Peng-14-UGCB on three independent GEO cohorts (Table 4) | GSE72056: 4,513 total (1,257 malignant, 3,256 benign); GSE40419: 164 samples; GSE103322: 5,578 single cells | not stated |
| SVM classifier comparison | Additional benchmarking of UGCB sets as an alternative to XGBoost (Table S2) | Same TCGA cohort samples as primary analysis | not stated |
-
Performance differences between classifiers across cancer types were compared as raw accuracy percentage points with no formal statistical test↳ Could also: A paired comparison such as McNemar's test (per cancer type, on sample-level predictions) or a Wilcoxon signed-rank test across cancer types could also be used to compare classifiers — Formal testing would quantify whether observed differences (e.g., 97.20% vs. 95.79% for BLCA) are likely attributable to sampling variation; this is especially useful when margins are narrow and sample sizes vary across subtypes
-
LOOCV accuracy and AUC were reported as point estimates with no associated uncertainty↳ Could also: Bootstrap confidence intervals or variance-stabilized estimates across repeated sub-samples could also accompany each point estimate — Uncertainty quantification is particularly informative when n varies substantially across cancer types (n=90 for KICH vs. n=1,214 for BRCA), as point-estimate precision differs greatly across subtypes
-
External validation used a single fixed 64/16/20% train/validation/test split on each GEO cohort↳ Could also: Repeated random sub-sampling (Monte Carlo cross-validation) or stratified repeated k-fold validation could also be applied for external holdout evaluation — A single split's accuracy depends on which samples happen to fall in each partition; averaging over multiple random splits yields a more stable estimate and allows reporting of variance across splits
-
Gene-set overlap between Wx and edgeR was reported as percentage overlap at varying top-N cutoffs (Figure 3)↳ Could also: A hypergeometric test or Fisher's exact test could also quantify whether observed overlap levels are greater or less than expected by chance given the sizes of the gene lists and the total gene universe — Formal overlap testing places the reported percentages in a probabilistic context, which helps distinguish biologically meaningful convergence from chance overlap at different list-size thresholds
-
edgeR was the sole DEG-analysis method used as a benchmark comparator↳ Could also: DESeq2 or limma-voom are also widely used RNA-seq differential expression tools that could serve as additional comparators — Different DEG tools can produce divergent gene rankings from identical data; including multiple DEG comparators would characterize how Wx performance relates to a broader reference class rather than a single algorithm choice
-
Classification accuracy was the primary evaluation metric across all 12 cancer types, including several with high class imbalance (e.g., BLCA: 95% cancer)↳ Could also: Balanced accuracy, Matthews correlation coefficient (MCC), or F1-score could also be reported alongside raw accuracy for imbalanced cancer-type datasets — When cancer samples outnumber normal samples by 9:1 or more, raw accuracy can be dominated by the majority class; imbalance-aware metrics give a more complete picture of per-class performance
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
Wx-14-UGCB classifies cancer vs adjacent normal in human lung adenocarcinoma RNA-seq with 80.00% accuracy, outperforming Peng-14-UGCB (56.87%)RNA-seq human lung-adenocarcinoma up 2019×1papers★ This paper is the founder (earliest)
-
Wx-14-UGCB achieves near-perfect AUC for cancer subtype classification in TCGA RNA-seq (BRCA 0.9944, LUAD 0.9943, LUSC 0.9936)RNA-seq tcga pan-cancer up 2019×1papers★ This paper is the founder (earliest)
-
Wx DI-score gene ranking saturates cancer-vs-normal classification accuracy at ~100 top-ranked genes in TCGA pan-cancer RNA-seq; adding more genes yields no further improvementRNA-seq tcga pan-cancer none 2019×1papers★ This paper is the founder (earliest)
-
Wx-14-UGCB gene set achieves higher cancer-vs-normal classification accuracy (96.72%) than Peng-14-UGCB (94.59%) and edgeR DEG-based (94.81%) gene sets across 12 TCGA cancer typesRNA-seq tcga pan-cancer up 2019×1papers★ This paper is the founder (earliest)
-
Wx DI-score top-ranked genes show poor overlap with edgeR DEGs in TCGA pan-cancer RNA-seq (9% in top 500, <35% overall), indicating complementary feature selectionRNA-seq tcga pan-cancer down 2019×1papers★ This paper is the founder (earliest)
-
Wx-14-UGCB classifies cancer vs non-cancer cells in HNSCC scRNA-seq with 81.10% accuracy, outperforming Peng-14-UGCB (68.28%)scRNA-seq human hnscc up 2019×1papers★ This paper is the founder (earliest)
-
Wx-14-UGCB classifies malignant vs non-malignant cells in human melanoma scRNA-seq with 90.71% accuracy, outperforming Peng-14-UGCB (70.22%)scRNA-seq human melanoma up 2019×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
Assessed papers, coloured by verdict. Click a node to open it.
- No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
scope.md — pmid-31324856 (Wx feature selection)
Paper: Park et al. 2019, "Wx: a neural network-based feature selection algorithm for transcriptomic data." Sci Rep. PMCID PMC6642261. Code: github.com/deargen/DearWXpub (commit 85655c6). Data: GEO GSE72056 (+ TCGA, GSE40419, GSE103322).
Reported pipeline-derived results (candidate claims)
- C1 Table 2: Wx-14 pan-cancer biomarker panel = EEF1A1,FN1,GAPDH,SFTPC,AHNAK, KLK3,UMOD,CTSB,COL1A1,GPX3,GNAS,ATP1A1,SFTPB,ACTB.
- C2 Table 3: TCGA 12-cancer overall accuracy with Wx-14 = 96.72%.
- C3 Table 4: GSE72056 melanoma external validation, Wx-14 = 90.71%.
- C4 Table 4: GSE40419 lung = 80.00%; GSE103322 head/neck = 81.10%.
- C5 Fig 2 AUCs (BRCA 0.9944, LUAD 0.9943, LUSC 0.9936).
In scope vs out of scope
IN (attempted): C3 (GSE72056 90.71%). Public GEO data + the FIXED reported Wx-14 panel + XGBoost classifier (the repo's own classifier method). This is a clean "third-party-tool-on-paper-data" reproduction (brief P16) needing no deprecated tooling. OUT (not attempted — the hard 20%, documented):
- C1/C2 require the FULL TCGA RNA-seq for 12 cancer types via TCGA-Assembler
(
./TCGA_DATAS/, Module_A.R / tcga_download.R). TCGA-Assembler is deprecated and its download endpoints are unreliable in 2026; rebuilding the exact TCGA-Assembler matrix for 12 cohorts is the >20% tail. The 14-gene panel is also stochastic (10k iterations of softmax training) so an EXACT panel match is not expected even with the data. NOT attempted. - C4 (GSE40419, GSE103322) — same external-validation shape as C3 but extra data wrangling; deprioritised after C3 (80/20).
- C5 AUCs are a by-product of the TCGA LOOCV (depends on TCGA data) — out.
Repo data reality check (important caveat)
The repo SHIPS: input_data_1.csv (toy 20 samples x 20502 genes -> wx_example.py), GSE_DATA/Total_hg38.txt (= GSE105127 CV/IZ/PP, a dataset NOT in this paper), GENE_LIST_TCGA_ASSEM.txt. NONE of the shipped data corresponds to a reported number in the paper, so a "just run the repo" reproduction cannot regenerate any Table value. Hence C3 via independent GEO download is the faithful route.
Environment
Modern standalone env (conda-forge: python 3.10, pandas, scikit-learn, xgboost) on a «our HPC» compute node. The authors' legacy stack (Py3.4/TF1.4/Keras2.1.2/old sklearn.cross_validation) is NOT required for C3 — only an XGBoost classifier on the fixed gene panel, which the brief endorses as equally valid.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Cohort reproduces exactly (1257 malignant / 3256 non-malignant / 4513 total; all 14 genes present), so data identity and preprocessing match the paper 1:1 on public GEO data. The accuracy deviates +5.97pp (90.71% → 96.68%), on our methodology side — StratifiedKFold(5) vs the paper's single 3611/902 split + 5-fold — not an authors' defect. The core claim holds fully (both values far above the 72.1% baseline, same direction). Overall a solid partial reproduction with explainable deviation; the remaining 4 claims (TCGA panel/accuracy/AUCs) are data-unavailable via the deprecated TCGA-Assembler, not attempted. No fabrication signal.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at [email protected].
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.