Wx: a neural network-based feature selection algorithm for transcriptomic data.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓The central claim held under reproduction
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce ONE headline number 1:1 on the paper's own public data: GSE72056 melanoma external validation (Table 4). We obtained GSE72056 from GEO (the repo ships NO paper data) and classified malignant vs non-malignant cells with XGBoost over the fixed reported Wx-14 gene panel. The COHORT reproduces the paper EXACTLY (1257 malignant + 3256 non-malignant = 4513 cells) and all 14 genes are present, but our accuracy is 96.68% vs the reported 90.71% (~6 pts higher, same direction, both far above the 72.1% majority baseline); the gap is most likely the CV protocol (paper: single 3611/902 split + 5-fold; ours: StratifiedKFold-5). Grade=partial. NOT attempted (the hard ~20%): the Wx-14 panel itself (C1), TCGA 96.72% overall (C2) and Fig-2 AUCs (C5) all need the full 12-cancer TCGA matrix via the deprecated TCGA-Assembler tool, and C1 is additionally stochastic (10k softmax iterations); GSE40419/GSE103322 (C4) are a feasible same-shape follow-up deprioritised under 80/20. Caveat for reviewer: the repo's shipped data (input_data_1.csv toy; GSE_DATA = GSE105127, not in this paper) reproduces NO reported value, so headline numbers are only reproducible via independent data downloads. Not flagged as fabrication: the independent reproduction supports the claim in spirit.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-14 ⛓ af451bdaa53d
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a neural network-based feature (gene) selection algorithm (Wx), which ranks genes by a discriminative index (DI) score, identify an optimal minimal set of universal gene-expression biomarkers that distinguishes cancer from normal samples across multiple cancer types?
- ★ The Wx algorithm ranks genes by a discriminative index (DI) score representing classification power for distinguishing given groups, enabling intuitive selection of optimal biomarker genes. method
- ★ Wx identified 14 universal gene-expression cancer biomarkers (Wx-14-UGCB) that accurately distinguish 12 types of cancer from normal tissue samples. finding
- ★ The Wx-14-UGCB signature was comparable to or outperformed previously reported universal biomarkers (Peng et al., Martinez-Ledesma et al.) and edgeR DEGs in classification accuracy. finding
- ★ Wx-14-UGCB outperformed Peng-14-UGCB on three independent external validation cohorts (melanoma, lung adenocarcinoma, head and neck squamous cell carcinoma). finding
- Genes selected by Wx overlapped less than 35% with DEGs identified by edgeR, indicating substantial algorithmic discrepancy. finding
- Housekeeping genes GAPDH and ACTB had high DI scores, suggesting they may be unsuitable as control genes in cancer gene-expression experiments. finding
- Stand-alone and web versions of the Wx algorithm are publicly available as a resource. resource
- Top 50 UGCBs were enriched in Fc gamma receptor dependent phagocytosis, antigen processing and presentation, and regulation of apoptotic signaling pathways. mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| bulk RNA-seq (mRNASeq, RSEM normalized) | TCGA pan-cancer cohort, 12 cancer types and normal tissue (6,226 samples; 5,609 tumor, 617 control) | none | normalized expression of 20,502 genes; DI-score gene ranking and cancer vs normal classification accuracy | Illumina HiSeq (TCGA-Assembler 2) |
| single-cell RNA-seq | Melanoma (GSE72056; 1,257 malignant and 3,256 benign samples) | none | classification accuracy of malignant vs non-malignant cells (23,686 genes) | — |
| bulk RNA-seq | Lung adenocarcinoma (GSE40419; 164 samples, 87 lung cancer and 77 adjacent normal) | none | classification accuracy of cancer vs adjacent normal (36,741 genes) | — |
| single-cell RNA-seq | Head and neck squamous cell carcinoma (GSE103322; 5,578 cells, 2,215 cancer and 3,363 non-cancer) | none | classification accuracy of cancer vs non-cancer cells (23,686 genes) | — |
| RNA-seq (laser-captured microdissection) | Non-cancer human liver regions (GSE105127; pericentral n=19, intermediate n=19, periportal n=19) | none | classification accuracy across liver regions (65,671 transcripts) | — |
| differential expression analysis (edgeR) | TCGA pan-cancer cohort | none | DEG ranking by adjusted p value; overlap with Wx genes and classification accuracy | edgeR |
| GO and network enrichment analysis | Top 50 Wx UGCBs | none | enriched pathways/functions | Metascape |
| classification benchmarking (XGBoost and SVM) | TCGA cancer subtypes using UGCB gene sets | none | LOOCV classification accuracy comparison | — |
- – Approximately the top 100 genes ranked by DI score reached the highest average classification accuracy with no further increase when more genes were added. top ~100 genes
- ▲ Wx-14-UGCB achieved higher total classification accuracy than Peng-14-UGCB and edgeR DEG-14-UGCB across cancer types. 96.72% (Wx-14) vs 94.59% (Peng) vs 94.81% (edgeR)
- ▲ Wx-7-UGCB showed higher total accuracy than Martinez-Ledesma's 7-gene set. 95.74% vs 92.20%
- ▲ Wx-14-UGCB showed excellent ROC performance on BRCA, LUAD, and LUSC RNA-seq data. AUC 0.9944 (BRCA), 0.9943 (LUAD), 0.9936 (LUSC)
- ▲ Wx-14-UGCB classified melanoma cells better than Peng-14-UGCB on the test set. 90.71% vs 70.22% (818/902 vs 633/902 correct)
- ▲ Wx-14-UGCB outperformed Peng-14-UGCB on lung adenocarcinoma classification. 80.00% vs 56.87%
- ▲ Wx-14-UGCB outperformed Peng-14-UGCB on head and neck squamous cell carcinoma single cells. 81.10% vs 68.28%
- ▼ Top genes identified by Wx overlapped poorly with edgeR DEGs. top 500: 45/500 (9.0%); top 2,000: 379 (19.0%); overall <35%
- count 6,226 total samples (5,609 tumor, 617 control) (TCGA pan-cancer cohort used for Wx feature selection)
- other 10,000 iterations (Wx algorithm iterated to average DI scores and reduce random-initialization variability)
- fold_change AUC 0.9944, 0.9943, 0.9936 (Wx-14-UGCB AUC for BRCA, LUAD, LUSC)
- other 96.72% (Wx-14-UGCB total classification accuracy (LOOCV))
- count 818 out of 902 (melanoma test samples correctly classified by Wx-14-UGCB)
- other 45 genes (9.0%) (overlap between top 500 Wx and edgeR genes)
- count 2,888 training / 723 validation / 902 test (64%/16%/20%) (data split for independent validation neural network model)
- count ACTB ranked 14 out of 20,501 genes (housekeeping gene ACTB DI-score rank)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper introduces Wx, a neural network-based gene feature selection algorithm that assigns a Discriminative Index (DI) score to each gene and ranks them by classification power for cancer vs. normal discrimination. Performance was evaluated primarily by leave-one-out cross-validation (LOOCV) classification accuracy across 12 TCGA cancer types, with AUC reported for three cancer types. Benchmarking against edgeR-derived DEG sets and two previously published biomarker sets was conducted by directly comparing LOOCV accuracy percentages, without formal statistical testing of observed differences. External generalization was assessed using a single fixed 64/16/20% train/validation/test split on three independent GEO cohorts.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Leave-one-out cross-validation (LOOCV) classification accuracy with XGBoost | Primary performance evaluation of Wx-14-UGCB, Peng-14-UGCB, DEG-14-UGCB, and Wx-7-UGCB vs. MartinezL-7-UGCB across 12 TCGA cancer types (Table 3) | 6,226 total TCGA samples (5,609 cancer, 617 normal); per-type n ranges from 90 (KICH) to 1,214 (BRCA) as listed in Table 1 | not stated |
| edgeR differential expression analysis (negative binomial GLM, FDR-adjusted p-value threshold 0.05) | Identification of top 14 DEGs (DEG-14-UGCB) used as a comparison benchmark (Table 3) | Same 6,226 TCGA pan-cancer samples | not stated |
| AUC / ROC analysis | Classification performance of Wx-14-UGCB on BRCA, LUAD, and LUSC (Figure 2) | Implied from Table 1: BRCA n=1,214; LUAD n=576; LUSC n=553 | not stated |
| Fixed-split holdout accuracy (64% train / 16% validation / 20% test) with neural network classifier | External validation of Wx-14-UGCB vs. Peng-14-UGCB on three independent GEO cohorts (Table 4) | GSE72056: 4,513 total (1,257 malignant, 3,256 benign); GSE40419: 164 samples; GSE103322: 5,578 single cells | not stated |
| SVM classifier comparison | Additional benchmarking of UGCB sets as an alternative to XGBoost (Table S2) | Same TCGA cohort samples as primary analysis | not stated |
-
Performance differences between classifiers across cancer types were compared as raw accuracy percentage points with no formal statistical test↳ Could also: A paired comparison such as McNemar's test (per cancer type, on sample-level predictions) or a Wilcoxon signed-rank test across cancer types could also be used to compare classifiers — Formal testing would quantify whether observed differences (e.g., 97.20% vs. 95.79% for BLCA) are likely attributable to sampling variation; this is especially useful when margins are narrow and sample sizes vary across subtypes
-
LOOCV accuracy and AUC were reported as point estimates with no associated uncertainty↳ Could also: Bootstrap confidence intervals or variance-stabilized estimates across repeated sub-samples could also accompany each point estimate — Uncertainty quantification is particularly informative when n varies substantially across cancer types (n=90 for KICH vs. n=1,214 for BRCA), as point-estimate precision differs greatly across subtypes
-
External validation used a single fixed 64/16/20% train/validation/test split on each GEO cohort↳ Could also: Repeated random sub-sampling (Monte Carlo cross-validation) or stratified repeated k-fold validation could also be applied for external holdout evaluation — A single split's accuracy depends on which samples happen to fall in each partition; averaging over multiple random splits yields a more stable estimate and allows reporting of variance across splits
-
Gene-set overlap between Wx and edgeR was reported as percentage overlap at varying top-N cutoffs (Figure 3)↳ Could also: A hypergeometric test or Fisher's exact test could also quantify whether observed overlap levels are greater or less than expected by chance given the sizes of the gene lists and the total gene universe — Formal overlap testing places the reported percentages in a probabilistic context, which helps distinguish biologically meaningful convergence from chance overlap at different list-size thresholds
-
edgeR was the sole DEG-analysis method used as a benchmark comparator↳ Could also: DESeq2 or limma-voom are also widely used RNA-seq differential expression tools that could serve as additional comparators — Different DEG tools can produce divergent gene rankings from identical data; including multiple DEG comparators would characterize how Wx performance relates to a broader reference class rather than a single algorithm choice
-
Classification accuracy was the primary evaluation metric across all 12 cancer types, including several with high class imbalance (e.g., BLCA: 95% cancer)↳ Could also: Balanced accuracy, Matthews correlation coefficient (MCC), or F1-score could also be reported alongside raw accuracy for imbalanced cancer-type datasets — When cancer samples outnumber normal samples by 9:1 or more, raw accuracy can be dominated by the majority class; imbalance-aware metrics give a more complete picture of per-class performance
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
Wx-14-UGCB classifies cancer vs adjacent normal in human lung adenocarcinoma RNA-seq with 80.00% accuracy, outperforming Peng-14-UGCB (56.87%)RNA-seq human lung-adenocarcinoma up 2019×1papers★ This paper is the founder (earliest)
-
Wx-14-UGCB achieves near-perfect AUC for cancer subtype classification in TCGA RNA-seq (BRCA 0.9944, LUAD 0.9943, LUSC 0.9936)RNA-seq tcga pan-cancer up 2019×1papers★ This paper is the founder (earliest)
-
Wx DI-score gene ranking saturates cancer-vs-normal classification accuracy at ~100 top-ranked genes in TCGA pan-cancer RNA-seq; adding more genes yields no further improvementRNA-seq tcga pan-cancer none 2019×1papers★ This paper is the founder (earliest)
-
Wx-14-UGCB gene set achieves higher cancer-vs-normal classification accuracy (96.72%) than Peng-14-UGCB (94.59%) and edgeR DEG-based (94.81%) gene sets across 12 TCGA cancer typesRNA-seq tcga pan-cancer up 2019×1papers★ This paper is the founder (earliest)
-
Wx DI-score top-ranked genes show poor overlap with edgeR DEGs in TCGA pan-cancer RNA-seq (9% in top 500, <35% overall), indicating complementary feature selectionRNA-seq tcga pan-cancer down 2019×1papers★ This paper is the founder (earliest)
-
Wx-14-UGCB classifies cancer vs non-cancer cells in HNSCC scRNA-seq with 81.10% accuracy, outperforming Peng-14-UGCB (68.28%)scRNA-seq human hnscc up 2019×1papers★ This paper is the founder (earliest)
-
Wx-14-UGCB classifies malignant vs non-malignant cells in human melanoma scRNA-seq with 90.71% accuracy, outperforming Peng-14-UGCB (70.22%)scRNA-seq human melanoma up 2019×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
Assessed papers, coloured by verdict. Click a node to open it.
- No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
scope.md — pmid-31324856 (Wx feature selection)
Paper: Park et al. 2019, "Wx: a neural network-based feature selection algorithm for transcriptomic data." Sci Rep. PMCID PMC6642261. Code: github.com/deargen/DearWXpub (commit 85655c6). Data: GEO GSE72056 (+ TCGA, GSE40419, GSE103322).
Reported pipeline-derived results (candidate claims)
- C1 Table 2: Wx-14 pan-cancer biomarker panel = EEF1A1,FN1,GAPDH,SFTPC,AHNAK, KLK3,UMOD,CTSB,COL1A1,GPX3,GNAS,ATP1A1,SFTPB,ACTB.
- C2 Table 3: TCGA 12-cancer overall accuracy with Wx-14 = 96.72%.
- C3 Table 4: GSE72056 melanoma external validation, Wx-14 = 90.71%.
- C4 Table 4: GSE40419 lung = 80.00%; GSE103322 head/neck = 81.10%.
- C5 Fig 2 AUCs (BRCA 0.9944, LUAD 0.9943, LUSC 0.9936).
In scope vs out of scope
IN (attempted): C3 (GSE72056 90.71%). Public GEO data + the FIXED reported Wx-14 panel + XGBoost classifier (the repo's own classifier method). This is a clean "third-party-tool-on-paper-data" reproduction (brief P16) needing no deprecated tooling. OUT (not attempted — the hard 20%, documented):
- C1/C2 require the FULL TCGA RNA-seq for 12 cancer types via TCGA-Assembler
(
./TCGA_DATAS/, Module_A.R / tcga_download.R). TCGA-Assembler is deprecated and its download endpoints are unreliable in 2026; rebuilding the exact TCGA-Assembler matrix for 12 cohorts is the >20% tail. The 14-gene panel is also stochastic (10k iterations of softmax training) so an EXACT panel match is not expected even with the data. NOT attempted. - C4 (GSE40419, GSE103322) — same external-validation shape as C3 but extra data wrangling; deprioritised after C3 (80/20).
- C5 AUCs are a by-product of the TCGA LOOCV (depends on TCGA data) — out.
Repo data reality check (important caveat)
The repo SHIPS: input_data_1.csv (toy 20 samples x 20502 genes -> wx_example.py), GSE_DATA/Total_hg38.txt (= GSE105127 CV/IZ/PP, a dataset NOT in this paper), GENE_LIST_TCGA_ASSEM.txt. NONE of the shipped data corresponds to a reported number in the paper, so a "just run the repo" reproduction cannot regenerate any Table value. Hence C3 via independent GEO download is the faithful route.
Environment
Modern standalone env (conda-forge: python 3.10, pandas, scikit-learn, xgboost) on a «our HPC» compute node. The authors' legacy stack (Py3.4/TF1.4/Keras2.1.2/old sklearn.cross_validation) is NOT required for C3 — only an XGBoost classifier on the fixed gene panel, which the brief endorses as equally valid.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Cohort reproduces exactly (1257 malignant / 3256 non-malignant / 4513 total; all 14 genes present), so data identity and preprocessing match the paper 1:1 on public GEO data. The accuracy deviates +5.97pp (90.71% → 96.68%), on our methodology side — StratifiedKFold(5) vs the paper's single 3611/902 split + 5-fold — not an authors' defect. The core claim holds fully (both values far above the 72.1% baseline, same direction). Overall a solid partial reproduction with explainable deviation; the remaining 4 claims (TCGA panel/accuracy/AUCs) are data-unavailable via the deprecated TCGA-Assembler, not attempted. No fabrication signal.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.