Corpus 1,286 assessed · 1,187 scored · 648 reproduced ≥75 · 174 flagged ·∅ 73.9/100
← New search

Colorectal Cancer Prediction Based on Weighted Gene Co-Expression Network Analysis and Variational Auto-Encoder.

Biomolecules · 2020
PubMed 32825264 ↗ DOI 10.3390/biom10091207 ↗ pmid-32825264
L1 80/100 ★ ☆ ★ ★ 3/4
⚑ Flagged for review — a reproduced result did not match the reported value

Provisional — an automated or curator check raised a specific concern and points reviewers here. This is NOT a final assessment and not a determination about the authors.

Why this verdict

The main results reproduced: recomputed values matched the published ones within tolerance.

Reproduced on the brainbox compute brainarbeit.com
Scoring basis — itemised

Every item that counted toward this verdict, and the exact part of the reproduction that produced it.

Supporting (toward a concern)
Content-critical question only partially held
+2 pts
From: Q5 · Derivability / plausibility 🟡
Content-critical question only partially held
+2 pts
From: Q8 · Severity of the miss (overall human judgment) 🟡
Minor / cosmetic deviation
+1 pts
From: Q3 · Location of the main deviation 🟡
Minor / cosmetic deviation
+1 pts
From: Q6 · Severity of the deviation 🟡
Concordant (toward reproduced)
Code + data deposited & functional
-2 pts
From: Data & code availability —Available & functional
Total score +4
✓ What held up
  • ✓Same input data as the authors
  • ✓Reported values were directly comparable
  • ✓The central claim held under reproduction
What did not (or only partly)
  • 🟡A deviation arose in the data or preprocessing
  • 🔴A deviation was attributed to the published material
  • 🟡Reported values were not (fully) derivable from the shared data
  • 🟡The deviation was non-trivial in magnitude
  • 🟡Overall, the reproduction showed a material discrepancy
How its reproducibility compares
80/100
Reproducibility score
0.3 SD above mean
vs. all fields · 1187 studies
🎯 Scores higher than 57% of all assessed papers rank 485 of 1187 scored

A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.

▸Reproduction agent’s raw note

Described well enough to reproduce: YES. The repo (gutmicrobes/WGCNA-VAE @9e697d2) ships every pipeline stage's script AND its intermediate output, enabling stage-isolated checks. Headline result reproduced 1:1: linear SVC on the shipped combine.csv gives accuracy 0.9692 / AUC 0.9981 EXACTLY (deterministic). ComBat byte-exact (cor 1.0). WGCNA module count (12), hub genes (191) and DEG-intersection (165) EXACT. DEG counts within-tol (rerun limma 1166/457/709 vs reported 1159/419/740; shipped diff.csv = 1159 = paper; ~82% gene overlap; limma-version boundary effect). TWO paper-text values do not match the shipped pipeline and are flagged for the auditor: (a) turquoise module size reported 6865 but repo's own output AND our rerun = 6377; (b) soft power reported beta=12 but the shipped code auto-selects 2. NOT attempted (hard 20%): end-to-end VAE retraining — the Keras script sets no seed and the 2020 TF stack is unpinned, so latent features are not byte-reproducible; we verified S5 on the shipped VAE features instead.

💻 Code ↗ 🗄 Data: GSE8671

These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.

Assessment versions

Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.

  1. v1 current initial assessment Score 80
    assessed: 2026-06-15 ⛓ 06126848748d
✎ I am an author of this paper

Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.

Reason for the rerun

We email you a confirmation link first. The rerun is an objective re-measurement — it cannot change the verdict in your favour, only ask us to look again.

Provenance — full disclosure

When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.

Reproduced
2026-06-15
Rubric version
v1.0
Assessed by
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15
no human curator yet
Last updated
2026-09-19

Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.

Deep full-text extraction

Model: sonnet
Founding hypothesis

A two-step feature extraction approach combining WGCNA-derived hub genes with VAE-derived low-dimensional gene expression representations can improve the accuracy of an SVM classifier for predicting colorectal cancer from microarray gene expression data.

Core claims
  • ★ Combining WGCNA hub genes and VAE 10-dimensional representation as features for an SVM classifier achieves high accuracy in predicting CRC finding
  • ★ The MEturquoise gene module is the co-expression module most highly correlated with colorectal cancer finding
  • ★ Ten hub genes (including GUCA2B, CDK4, TRIP13, EIF3B, CDH3, GUCA2A) were identified from the key module based on gene significance, module membership, and intramodular connectivity finding
  • ★ A VAE can reduce 1159 differentially expressed genes to a 10-dimensional representation useful as a classifier feature method
  • The empirical Bayesian (ComBat/sva) method effectively removes batch effects across combined GEO microarray datasets method
  • A pipeline for WGCNA hub gene identification and VAE dimensionality reduction is publicly available on GitHub resource
Experimental setups
Assay System Perturbation Readout Platform
Microarray gene expression profiling / WGCNA co-expression network analysis Human colorectal cancer and normal tissue samples (combined GEO datasets GSE8671, GSE9348, GSE23878, GSE37364) none (observational, cancer vs normal) Gene co-expression modules, module eigengene (ME), Pearson correlation with cancer phenotype Affymetrix Human Genome U133 Plus 2.0 Array (GPL570)
Differential gene expression analysis Same combined CRC/normal microarray dataset (173/238 samples) none (cancer vs normal comparison) log2 fold change and FDR-adjusted p-value to define significantly altered genes limma R package
Batch effect correction Combined four-dataset gene expression matrix (13,487 genes, 238 samples) none PCA separation before/after correction sva R package (empirical Bayes/ComBat framework)
Hub gene mining (module membership, gene significance, intramodular connectivity analysis) MEturquoise module genes from WGCNA network none GS, MM, K.in values used to rank and select top 10 hub genes WGCNA R package (signedKME function)
Variational autoencoder dimensionality reduction 1159 significantly differentially expressed genes across CRC/normal samples none 10-dimensional latent representation of gene expression custom VAE neural network (5 hidden layers: 584,100,10,100,584 nodes)
SVM classification Training set (GSE8671, GSE9348, GSE23878) and test set (GSE37364) CRC/normal samples none Classification accuracy and AUC using hub genes + VAE features SVM classifier
Gene Ontology and KEGG pathway enrichment analysis 165 candidate hub genes meeting GS/MM thresholds with significant differential expression none Enriched biological functions/pathways —
Key results
  • ▲ SVM classifier using combined hub gene and VAE features achieved test accuracy 0.9692 and AUC 0.9981 accuracy 0.9692, AUC 0.9981
  • – 1159 significantly differentially expressed genes identified between cancer and normal samples 419 upregulated, 740 downregulated
  • – WGCNA clustering formed 17 initial modules, merged into 12 modules after pruning 17 to 12 modules; smallest module 36 genes, largest 6377 genes
  • ▲ MEturquoise module showed highest PCC and lowest p-value with cancer among all modules; midnight blue module was not significantly correlated
  • – 191 genes met GS>0.5 and MM>0.8 criteria in MEturquoise module; 165 of these also showed significant differential expression 191 genes, 165 significant
  • – Top 10 genes ranked by intramodular connectivity (K.in) selected as hub genes
  • – Batch effect correction eliminated separation among the four GEO datasets in PCA analysis
  • – Scale-free network requirement met at soft threshold beta = 12 beta=12, SET.R.sq>0.85
Key statistics
  • other accuracy = 0.9692 (SVM test set classification accuracy using hub genes + VAE features)
  • other AUC = 0.9981 (SVM classifier performance on test set)
  • count 1159 differentially expressed genes (419 up, 740 down) (Differential expression analysis between CRC and normal samples)
  • count 238 samples (106 normal, 132 CRC) (Combined microarray dataset from four GEO series)
  • count 17 modules merged into 12 modules (WGCNA co-expression module construction)
  • count 191 genes met GS>0.5 and MM>0.8; 165 significantly different (Hub gene candidate screening in MEturquoise module)
  • count top 10 hub genes selected by K.in (Final hub gene set for classifier)
  • other training set: 68 healthy, 105 cancer; test set: 38 healthy, 27 cancer (SVM training/test split across GSE8671, GSE9348, GSE23878 (train) and GSE37364 (test))

Statistical methods review

Model: opus

A neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.

This is a methods-development study that integrates four public microarray datasets (238 samples) to build a colorectal cancer classifier. The statistical pipeline combines batch-effect correction (empirical Bayes / ComBat via the sva package), differential expression analysis with the limma package (FDR-adjusted p < 0.05 and |log2FC| > 1), weighted gene co-expression network analysis (WGCNA) to identify gene modules, and Pearson correlation between module eigengenes and disease status to select a key module and hub genes. The selected features were then used in a variational autoencoder for dimensionality reduction and a support vector machine classifier, with performance reported as accuracy (0.9692) and AUC (0.9981) on a held-out dataset.

Replicationbiological Sample sizeSample counts are given (106 normal and 132 cancer samples; 238 total; 13,487 genes; training/test splits described), but no formal power or sample-size calculation is reported. Groupscolorectal cancer vs. normal tissue Pairingunpaired Randomization/blindingna Dispersionnone Exact p-valuesno Effect sizesyes Confidence intervalsno Multiplicity correctionBenjamini-Hochberg-type FDR correction (described as 'FDR correction toolkit in R'; adj. p < 0.05)
Statistical tests used
Test Applied to n Assumptions
limma (moderated/linear model) for differential expression, with FDR correction identifying differentially expressed genes between cancer and normal tissue (1159 genes; thresholds adj. p < 0.05 and |log2FC| > 1) derived from samples in GSE8671, GSE9348, GSE23878 (text states 238 combined samples overall; exact n per comparison not explicitly stated) not stated
Pearson correlation coefficient (PCC) correlation between each module eigengene (ME) and cancer status (coded cancer=1, normal=0), reported with p-values in Table 2 173 samples used for WGCNA (as stated in abstract); exact n for correlation not explicitly stated not stated
Principal component analysis (PCA) visual assessment of batch effects before and after correction (Figure 1A,B) all gene expression data across the four datasets na
Empirical Bayes batch-effect adjustment (ComBat, sva package) removing batch effects across the four combined datasets prior to downstream analysis 238 combined samples not stated
Approaches that could also have been used
  • Differential expression was assessed with limma after the data were log2-transformed and treated as continuous microarray intensities.
    ↳ Could also: DESeq2 or edgeR (negative-binomial models) could also be used, and were mentioned in the text as common tools. — These are designed for count-based RNA-seq data; limma is well-matched to microarray intensities, so the choice aligns with the data type, while naming the alternatives clarifies when each family is typically applied.
  • Classifier performance was reported as a single accuracy (0.9692) and AUC (0.9981) on one held-out dataset (GSE37364).
    ↳ Could also: k-fold or nested cross-validation, with confidence intervals around accuracy/AUC, could also be reported. — Resampling-based estimates with intervals convey the variability of the performance estimate and how stable it is across data partitions, which complements a single split.
  • Batch effects were removed with the empirical Bayes ComBat method, with success judged visually via PCA.
    ↳ Could also: Alternatives such as singular value decomposition (SVD) or distance-weighted discrimination (DWD), both mentioned in the text, could also be applied, alongside quantitative diagnostics (e.g., guided PCA or a batch-association test). — Quantitative batch diagnostics would add a numeric check to complement the visual PCA assessment, and the alternative correction methods make different modeling assumptions.
  • The correlation between module eigengenes and cancer status (a 0/1 binary variable) was quantified with the Pearson correlation coefficient.
    ↳ Could also: A point-biserial correlation (a special case of Pearson for a binary variable) or logistic-regression / Wilcoxon-based association could also be used. — These framings make the binary nature of the trait explicit and can offer an odds-ratio or rank-based view of the same association.
  • Module-to-cancer PCC p-values (Table 2) are presented for multiple modules.
    ↳ Could also: An explicit multiplicity adjustment (e.g., Benjamini-Hochberg) across the set of modules could also be reported. — Applying the same FDR logic used for genes to the module-level tests would keep the family-wise/false-discovery framing consistent across both analyses.
  • Reported summaries focus on point estimates (fold changes, correlations, accuracy, AUC) without dispersion measures.
    ↳ Could also: Reporting SD, IQR, or 95% confidence intervals alongside key estimates could also be done. — Adding a measure of spread or interval conveys uncertainty, which is often preferred—especially given the relatively modest per-dataset sample sizes.
Software: R / limma · R / sva (ComBat empirical Bayes) · R / WGCNA (e.g., pickSoftThreshold, signedKME) · Variational autoencoder (custom; pipeline on GitHub: gutmicrobes/WGCNA-VAE) · Support vector machine (SVM) classifier

Result convergence & founder nodes

Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.

Citation network

Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.

Authors · 4
1Dongmei Ai 2Yuduo Wang 3Xiaoxin Li 4Hongfei Pan
Citations
40
Impact: medium
Foundation confidence
None of its references are in our reproducibility record yet — its foundation cannot be assessed.
Topics
Classifier (UML)Support vector machineComputer sciencePattern recognition (psychology)

No assessed neighbours yet — the network grows as more papers are assessed.

Data lineage

The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.

GPL570 GEO in Methods (http://purl.org/orb/Methods)
also used by 16 papers:
GSE23878 GEO in Figure (http://semanticscience.org/resource/SIO_000080)
no other assessed paper uses this yet
GSE37364 GEO in Results (http://purl.org/orb/Results)
no other assessed paper uses this yet
GSE8671 GEO in Results (http://purl.org/orb/Results)
no other assessed paper uses this yet
GSE9348 GEO in Results (http://purl.org/orb/Results)
no other assessed paper uses this yet

Downstream reach in the literature

280 downstream papers · 4 datasets

How widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.

This paper is currently under reproducibility review (see the verdict above). The map below shows where the data in question has propagated — so reuse can be traced, not so the downstream work is presumed affected.
GSE8671 GEO reused by 170 papers in the literature
Most-cited downstream papers:
GSE9348 GEO reused by 155 papers in the literature
Most-cited downstream papers:
GSE23878 GEO reused by 86 papers in the literature
Most-cited downstream papers:
GSE37364 GEO reused by 68 papers in the literature
Most-cited downstream papers:

What was reproduced

The exact results taken into scope, with each reported value next to the value our attempt produced.

Scope — pmid-32825264 (WGCNA-VAE colorectal cancer prediction)

Ai D, Wang Y, Li X, Pan H. Colorectal Cancer Prediction Based on Weighted Gene Co-Expression Network Analysis and Variational Auto-Encoder. Biomolecules 2020; 10(9):1207. PMCID PMC7563725. Repo: github.com/gutmicrobes/WGCNA-VAE @ 9e697d279f231b7793a929d30e75dcfbc71dff98 (only commit, 2020-12-26).

Pipeline (all results are pipeline-derived → all in scope)

Four GEO microarray series (GSE8671, GSE9348, GSE23878, GSE37364), 238 samples (106 normal / 132 CRC), Affymetrix HG-U133 Plus 2. The repo ships every stage's script and its intermediate output files, which lets us check each stage in isolation and check whether the shipped files reproduce the paper's numbers.

# Stage Script (repo) Input Reported output Determinism
S1 Batch correction normalize.R (ComBat/sva) data/merge.csv (13487×238) data/normalize.csv deterministic
S2 DEG (limma) Differential-gene/limma.R normalize.csv 1159 DEG (419↑/740↓), |log2FC|>1 & adj.P<0.05 deterministic
S3 WGCNA WGCNA/WGCNA.R c1.csv (13487×173 train) β=12, 12 modules, turquoise≈6865, 191 hub (GS&MM filter), 165 ∩ DEG deterministic*
S4 VAE vae/VAE.py (Keras) diff2T (1159 DEG × samples) 10-dim latent features (584-100-10-100-584, lr 5e-4, bs 20, ep 6) STOCHASTIC (no seed set)
S5 Classifier classifier/SVM.txt combine.csv (VAE feats+hub genes) accuracy 0.9692, AUC 0.9981 (linear SVC, train 173 / test 65) deterministic

*WGCNA module detection is deterministic; only a TOM-heatmap subsample uses set.seed(10) (cosmetic, not on the module/hub path).

In scope (attempted)

  • S1 ComBat: rerun on merge.csv, compare to shipped normalize.csv (corr).
  • S2 limma DEGs: rerun on shipped normalize.csv; compare count 1159/419/740 and gene identity vs shipped diff.csv/up.csv/down.csv. Headline DEG claim.
  • S3 WGCNA: rerun on c1.csv; compare soft power (β=12), module count (12), turquoise module size (~6865), hub-gene count (191) and DEG∩hub (165).
  • S5 SVM: rerun SVM.txt on shipped combine.csv; compare accuracy 0.9692 / AUC 0.9981. Headline prediction claim — deterministic given shipped features, so this directly tests whether the reported accuracy/AUC matches the shipped data.

Out of scope / hard-20% (not attempted, or noted only)

  • S4 full VAE retraining end-to-end: the Keras script sets no random seed (random normal init + reparam sampling), and the 2020 TF1/Keras stack is not pinned. A fresh VAE run will NOT reproduce combine.csv byte-for-byte, so the 0.9692/0.9981 cannot be expected to reproduce through retraining. We instead verify S5 on the shipped VAE features (combine.csv) — the strongest deterministic check available. Re-deriving the exact latent features is the optional last 20% and is skipped (env/seed unresolvable).
  • Wet-lab / external validation: none in this paper (pure computational).

Fabrication-relevant checks

  • Do the shipped intermediate files themselves yield the paper's numbers? (diff.csv already = 1159/419/740; trait_genes_spe.csv = 191; intersection.csv = 165 — matches paper before we run anything.)
  • Does the deterministic SVM on the shipped combine.csv give exactly 0.9692/0.9981?
  • Note: WGCNA.R hub filter uses GS>0.2 in code but the paper text says GS>0.5 (both give 191 here) — flag for the auditor.
Figures / tables: Table
C1
Reported
238 (106 normal/132 CRC)
Reproduced
238 (106/132)
exact
C2
Reported
1159 DEGs
Reproduced
1166 (rerun); shipped 1159
within tolerance
C3
Reported
419 up
Reproduced
457 (rerun); shipped 419
within tolerance
C4
Reported
740 down
Reproduced
709 (rerun); shipped 740
within tolerance
C5
Reported
soft power beta=12
Reproduced
auto=2
did not match
C6
Reported
12 modules
Reproduced
12
exact
C7
Reported
turquoise 6865 genes
Reproduced
6377 (=shipped Alldegrees1.csv)
did not match
C8
Reported
191 hub genes
Reproduced
191
exact
C9
Reported
165 hub∩DEG
Reproduced
165
exact
C10
Reported
accuracy 0.9692
Reproduced
0.9692
exact
C11
Reported
AUC 0.9981
Reproduced
0.9981
exact

Assessments & scoring basis

Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.

🤖 AI curator · claude (ai-curator room) · v1.0 L1 80/100

An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.

🟢1. Data identity
🟢2. Endpoint comparability
🟡3. Location of the main deviation
🔴4. Cause of the deviation
🟡5. Derivability / plausibility
🟡6. Severity of the deviation
🟢7. Core claim
🟡8. Severity of the miss (overall human judgment)
Scoring basis — itemised

Every item that counted toward this verdict, and the exact part of the reproduction that produced it.

Supporting (toward a concern)
Content-critical question only partially held
+2 pts
From: Q5 · Derivability / plausibility 🟡
Content-critical question only partially held
+2 pts
From: Q8 · Severity of the miss (overall human judgment) 🟡
Minor / cosmetic deviation
+1 pts
From: Q3 · Location of the main deviation 🟡
Minor / cosmetic deviation
+1 pts
From: Q6 · Severity of the deviation 🟡
Concordant (toward reproduced)
Code + data deposited & functional
-2 pts
From: Data & code availability —Available & functional
Total score +4

The central claim reproduces exactly — VAE+SVM accuracy 0.9692 / AUC 0.9981 from the shipped features, with ComBat byte-exact and module/hub/intersection counts (12/191/165) all exact. Two paper-text values deviate and sit on the authors' side: turquoise module size 6865 is not derivable from the deposited data (their own table + our rerun = 6377), and reported β=12 contradicts the shipped code's auto-selected 2. These are moderate, conclusion-preserving reporting discrepancies; DEG counts drift only ~0.6% as a limma-version effect. Overall a solid reproduction with explainable, well-documented flags.

🤝
Reproduced automatically — and fairly

Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.

Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.

How this works Contest or correct this verdict
🚩 Report an error in this record

Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.

Prefer email, or the form below not working? Contact us at [email protected].

Reproduction footprint

claude-opus-4-8

Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.

145 k
tokens (I/O) · 10.5 M incl. cache
17 min
runtime · 0.04 CPU-h
7.8 GB
peak RAM
1
HPC jobs
hummel
machine