Corpus 1,272 assessed · 1,173 scored · 643 reproduced ≥75 · 168 flagged ·∅ 74.1/100
← New search

Colorectal Cancer Prediction Based on Weighted Gene Co-Expression Network Analysis and Variational Auto-Encoder.

Biomolecules · 2020
PubMed 32825264 ↗ DOI 10.3390/biom10091207 ↗ pmid-32825264
L1 80/100 ★ ☆ ★ ★ 3/4
⚑ Flagged for review — a reproduced result did not match the reported value

Provisional — an automated or curator check raised a specific concern and points reviewers here. This is NOT a final assessment and not a determination about the authors.

Why this verdict

The main results reproduced: recomputed values matched the published ones within tolerance.

Reproduced on the brainbox compute brainarbeit.com
Scoring basis — itemised

Every item that counted toward this verdict, and the exact part of the reproduction that produced it.

Supporting (toward a concern)
Content-critical question only partially held
+2 pts
From: Q5 · Derivability / plausibility 🟡
Content-critical question only partially held
+2 pts
From: Q8 · Severity of the miss (overall human judgment) 🟡
Minor / cosmetic deviation
+1 pts
From: Q3 · Location of the main deviation 🟡
Minor / cosmetic deviation
+1 pts
From: Q6 · Severity of the deviation 🟡
Concordant (toward reproduced)
Code + data deposited & functional
-2 pts
From: Data & code availability —Available & functional
Total score +4
✓ What held up
  • ✓Same input data as the authors
  • ✓Reported values were directly comparable
  • ✓The central claim held under reproduction
What did not (or only partly)
  • 🟡A deviation arose in the data or preprocessing
  • 🔴A deviation was attributed to the published material
  • 🟡Reported values were not (fully) derivable from the shared data
  • 🟡The deviation was non-trivial in magnitude
  • 🟡Overall, the reproduction showed a material discrepancy
How its reproducibility compares
80/100
Reproducibility score
0.3 SD above mean
vs. all fields · 1173 studies
🎯 Scores higher than 56% of all assessed papers rank 484 of 1173 scored

A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.

▸Reproduction agent’s raw note

Described well enough to reproduce: YES. The repo (gutmicrobes/WGCNA-VAE @9e697d2) ships every pipeline stage's script AND its intermediate output, enabling stage-isolated checks. Headline result reproduced 1:1: linear SVC on the shipped combine.csv gives accuracy 0.9692 / AUC 0.9981 EXACTLY (deterministic). ComBat byte-exact (cor 1.0). WGCNA module count (12), hub genes (191) and DEG-intersection (165) EXACT. DEG counts within-tol (rerun limma 1166/457/709 vs reported 1159/419/740; shipped diff.csv = 1159 = paper; ~82% gene overlap; limma-version boundary effect). TWO paper-text values do not match the shipped pipeline and are flagged for the auditor: (a) turquoise module size reported 6865 but repo's own output AND our rerun = 6377; (b) soft power reported beta=12 but the shipped code auto-selects 2. NOT attempted (hard 20%): end-to-end VAE retraining — the Keras script sets no seed and the 2020 TF stack is unpinned, so latent features are not byte-reproducible; we verified S5 on the shipped VAE features instead.

💻 Code ↗ 🗄 Data: GSE8671

These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.

Assessment versions

Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.

  1. v1 current initial assessment Score 80
    assessed: 2026-06-15 ⛓ 06126848748d
✎ I am an author of this paper

Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.

Reason for the rerun

We email you a confirmation link first. The rerun is an objective re-measurement — it cannot change the verdict in your favour, only ask us to look again.

Provenance — full disclosure

When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.

Reproduced
2026-06-15
Rubric version
v1.0
Assessed by
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15
no human curator yet
Last updated
2026-08-05

Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.

Deep full-text extraction

Model: opus
Founding hypothesis

Can a two-step feature extraction method—combining hub genes identified by weighted gene co-expression network analysis (WGCNA) with a 10-dimensional representation learned by a variational autoencoder (VAE)—improve the accuracy of classifying colorectal cancer versus normal samples from microarray gene expression data?

Core claims
  • ★ Combining WGCNA-derived hub genes with a VAE-derived 10-dimensional representation as features for an SVM classifier achieves high accuracy (0.9692) and AUC (0.9981) for colorectal cancer prediction. finding
  • ★ A two-step feature extraction pipeline (WGCNA hub genes + VAE dimensionality reduction) is an effective method for CRC classification. method
  • ★ The MEturquoise module, containing 6865 genes, is the gene co-expression module most highly correlated with colorectal cancer. finding
  • ★ Ten hub genes (including GUCA2B, CDK4, TRIP13, EIF3B, CDH3, GUCA2A) selected by module membership, gene significance, and intramodular connectivity may serve as diagnostic/prognostic markers for CRC. finding
  • A VAE maps data to a distribution rather than a single function, effectively summarizing high-dimensional gene expression data into a low-dimensional representation. method
  • An analysis pipeline for WGCNA hub gene identification and VAE 10-dimensional representation is freely available on GitHub. resource
Experimental setups
Assay System Perturbation Readout Platform
microarray gene expression analysis (Affymetrix Human Genome U133 Plus 2.0) colorectal cancer and normal human tissue samples (238 samples; 132 CRC, 106 normal) none gene expression levels (13,487 genes) GPL570 Affymetrix Human Genome U133 Plus 2.0 Array (GEO datasets GSE8671, GSE9348, GSE23878, GSE37364)
differential gene expression analysis combined CRC vs normal microarray samples none differentially expressed genes (adj. p<0.05, |log2FC|>1) limma R package with FDR correction
weighted gene co-expression network analysis (WGCNA) combined CRC vs normal microarray samples (173 training samples) none co-expression gene modules, module eigengenes, hub genes (MM, GS, K.in) WGCNA R package (signedKME, pickSoftThreshold)
Gene Ontology and KEGG pathway enrichment analysis 165 differentially expressed hub-candidate genes from MEturquoise module none enriched GO terms and KEGG pathways —
variational autoencoder (VAE) dimensionality reduction 1159 differentially expressed genes from CRC/normal samples none 10-dimensional latent representation neural network (input/output 1159 nodes; hidden layers 584,100,10,100,584; lr 0.0005, batch 20, 6 epochs)
SVM classification CRC vs normal samples (train: GSE8671/GSE9348/GSE23878 = 173 samples; test: GSE37364 = 65 samples) none classification accuracy, AUC support vector machine (SVM)
principal component analysis (PCA) for batch effect assessment combined four GEO datasets batch effect correction (ComBat/empirical Bayes via sva) sample separation/clustering before and after correction sva R package (empirical Bayesian framework)
Key results
  • – SVM classifier using combined hub genes and VAE 10-dimensional features achieved test accuracy of 0.9692 0.9692
  • – SVM classifier achieved an AUC of 0.9981 0.9981
  • – SVM correctly classified all 65 samples in the test set 65/65
  • – 1159 significantly altered genes identified, 419 upregulated and 740 downregulated 419 up / 740 down
  • ▲ MEturquoise module had the highest Pearson correlation coefficient with cancer and the lowest p-value among modules
  • – 191 genes met GS>0.5 and MM>0.8; 165 of these showed significant differences between cancer and normal samples 191 / 165
  • – WGCNA initially produced 17 modules, merged into 12 modules (combining those with correlation >0.8) 17→12 modules
  • – All modules except midnight blue were significantly correlated with cancer
Key statistics
  • other accuracy = 0.9692 (SVM test set classification accuracy)
  • other AUC = 0.9981 (SVM test set AUC)
  • count 1159 differentially expressed genes (419 up, 740 down) (differential analysis after batch correction)
  • count 6865 genes (number of genes in MEturquoise module of interest)
  • count 238 samples (132 CRC, 106 normal); 13,487 genes (combined dataset size)
  • other β = 12 (soft threshold for scale-free network (SET.R.sq > 0.85))
  • fold_change |log2FC| > 1, adj. p < 0.05 (thresholds for differentially expressed gene selection)
  • count 191 genes (GS>0.5, MM>0.8); 165 significantly different (hub gene candidate filtering)

Statistical methods review

Model: opus

A neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.

This is a methods-development study that integrates four public microarray datasets (238 samples) to build a colorectal cancer classifier. The statistical pipeline combines batch-effect correction (empirical Bayes / ComBat via the sva package), differential expression analysis with the limma package (FDR-adjusted p < 0.05 and |log2FC| > 1), weighted gene co-expression network analysis (WGCNA) to identify gene modules, and Pearson correlation between module eigengenes and disease status to select a key module and hub genes. The selected features were then used in a variational autoencoder for dimensionality reduction and a support vector machine classifier, with performance reported as accuracy (0.9692) and AUC (0.9981) on a held-out dataset.

Replicationbiological Sample sizeSample counts are given (106 normal and 132 cancer samples; 238 total; 13,487 genes; training/test splits described), but no formal power or sample-size calculation is reported. Groupscolorectal cancer vs. normal tissue Pairingunpaired Randomization/blindingna Dispersionnone Exact p-valuesno Effect sizesyes Confidence intervalsno Multiplicity correctionBenjamini-Hochberg-type FDR correction (described as 'FDR correction toolkit in R'; adj. p < 0.05)
Statistical tests used
Test Applied to n Assumptions
limma (moderated/linear model) for differential expression, with FDR correction identifying differentially expressed genes between cancer and normal tissue (1159 genes; thresholds adj. p < 0.05 and |log2FC| > 1) derived from samples in GSE8671, GSE9348, GSE23878 (text states 238 combined samples overall; exact n per comparison not explicitly stated) not stated
Pearson correlation coefficient (PCC) correlation between each module eigengene (ME) and cancer status (coded cancer=1, normal=0), reported with p-values in Table 2 173 samples used for WGCNA (as stated in abstract); exact n for correlation not explicitly stated not stated
Principal component analysis (PCA) visual assessment of batch effects before and after correction (Figure 1A,B) all gene expression data across the four datasets na
Empirical Bayes batch-effect adjustment (ComBat, sva package) removing batch effects across the four combined datasets prior to downstream analysis 238 combined samples not stated
Approaches that could also have been used
  • Differential expression was assessed with limma after the data were log2-transformed and treated as continuous microarray intensities.
    ↳ Could also: DESeq2 or edgeR (negative-binomial models) could also be used, and were mentioned in the text as common tools. — These are designed for count-based RNA-seq data; limma is well-matched to microarray intensities, so the choice aligns with the data type, while naming the alternatives clarifies when each family is typically applied.
  • Classifier performance was reported as a single accuracy (0.9692) and AUC (0.9981) on one held-out dataset (GSE37364).
    ↳ Could also: k-fold or nested cross-validation, with confidence intervals around accuracy/AUC, could also be reported. — Resampling-based estimates with intervals convey the variability of the performance estimate and how stable it is across data partitions, which complements a single split.
  • Batch effects were removed with the empirical Bayes ComBat method, with success judged visually via PCA.
    ↳ Could also: Alternatives such as singular value decomposition (SVD) or distance-weighted discrimination (DWD), both mentioned in the text, could also be applied, alongside quantitative diagnostics (e.g., guided PCA or a batch-association test). — Quantitative batch diagnostics would add a numeric check to complement the visual PCA assessment, and the alternative correction methods make different modeling assumptions.
  • The correlation between module eigengenes and cancer status (a 0/1 binary variable) was quantified with the Pearson correlation coefficient.
    ↳ Could also: A point-biserial correlation (a special case of Pearson for a binary variable) or logistic-regression / Wilcoxon-based association could also be used. — These framings make the binary nature of the trait explicit and can offer an odds-ratio or rank-based view of the same association.
  • Module-to-cancer PCC p-values (Table 2) are presented for multiple modules.
    ↳ Could also: An explicit multiplicity adjustment (e.g., Benjamini-Hochberg) across the set of modules could also be reported. — Applying the same FDR logic used for genes to the module-level tests would keep the family-wise/false-discovery framing consistent across both analyses.
  • Reported summaries focus on point estimates (fold changes, correlations, accuracy, AUC) without dispersion measures.
    ↳ Could also: Reporting SD, IQR, or 95% confidence intervals alongside key estimates could also be done. — Adding a measure of spread or interval conveys uncertainty, which is often preferred—especially given the relatively modest per-dataset sample sizes.
Software: R / limma · R / sva (ComBat empirical Bayes) · R / WGCNA (e.g., pickSoftThreshold, signedKME) · Variational autoencoder (custom; pipeline on GitHub: gutmicrobes/WGCNA-VAE) · Support vector machine (SVM) classifier

Result convergence & founder nodes

Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.

Citation network

Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.

Authors · 4
1Dongmei Ai 2Yuduo Wang 3Xiaoxin Li 4Hongfei Pan
Citations
40
Impact: medium
Foundation confidence
None of its references are in our reproducibility record yet — its foundation cannot be assessed.
Topics
Classifier (UML)Support vector machineComputer sciencePattern recognition (psychology)

No assessed neighbours yet — the network grows as more papers are assessed.

Data lineage

The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.

GPL570 GEO in Methods (http://purl.org/orb/Methods)
also used by 16 papers:
GSE23878 GEO in Figure (http://semanticscience.org/resource/SIO_000080)
no other assessed paper uses this yet
GSE37364 GEO in Results (http://purl.org/orb/Results)
no other assessed paper uses this yet
GSE8671 GEO in Results (http://purl.org/orb/Results)
no other assessed paper uses this yet
GSE9348 GEO in Results (http://purl.org/orb/Results)
no other assessed paper uses this yet

Downstream reach in the literature

280 downstream papers · 4 datasets

How widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.

This paper is currently under reproducibility review (see the verdict above). The map below shows where the data in question has propagated — so reuse can be traced, not so the downstream work is presumed affected.
GSE8671 GEO reused by 170 papers in the literature
Most-cited downstream papers:
GSE9348 GEO reused by 155 papers in the literature
Most-cited downstream papers:
GSE23878 GEO reused by 86 papers in the literature
Most-cited downstream papers:
GSE37364 GEO reused by 68 papers in the literature
Most-cited downstream papers:

What was reproduced

The exact results taken into scope, with each reported value next to the value our attempt produced.

Scope — pmid-32825264 (WGCNA-VAE colorectal cancer prediction)

Ai D, Wang Y, Li X, Pan H. Colorectal Cancer Prediction Based on Weighted Gene Co-Expression Network Analysis and Variational Auto-Encoder. Biomolecules 2020; 10(9):1207. PMCID PMC7563725. Repo: github.com/gutmicrobes/WGCNA-VAE @ 9e697d279f231b7793a929d30e75dcfbc71dff98 (only commit, 2020-12-26).

Pipeline (all results are pipeline-derived → all in scope)

Four GEO microarray series (GSE8671, GSE9348, GSE23878, GSE37364), 238 samples (106 normal / 132 CRC), Affymetrix HG-U133 Plus 2. The repo ships every stage's script and its intermediate output files, which lets us check each stage in isolation and check whether the shipped files reproduce the paper's numbers.

# Stage Script (repo) Input Reported output Determinism
S1 Batch correction normalize.R (ComBat/sva) data/merge.csv (13487×238) data/normalize.csv deterministic
S2 DEG (limma) Differential-gene/limma.R normalize.csv 1159 DEG (419↑/740↓), |log2FC|>1 & adj.P<0.05 deterministic
S3 WGCNA WGCNA/WGCNA.R c1.csv (13487×173 train) β=12, 12 modules, turquoise≈6865, 191 hub (GS&MM filter), 165 ∩ DEG deterministic*
S4 VAE vae/VAE.py (Keras) diff2T (1159 DEG × samples) 10-dim latent features (584-100-10-100-584, lr 5e-4, bs 20, ep 6) STOCHASTIC (no seed set)
S5 Classifier classifier/SVM.txt combine.csv (VAE feats+hub genes) accuracy 0.9692, AUC 0.9981 (linear SVC, train 173 / test 65) deterministic

*WGCNA module detection is deterministic; only a TOM-heatmap subsample uses set.seed(10) (cosmetic, not on the module/hub path).

In scope (attempted)

  • S1 ComBat: rerun on merge.csv, compare to shipped normalize.csv (corr).
  • S2 limma DEGs: rerun on shipped normalize.csv; compare count 1159/419/740 and gene identity vs shipped diff.csv/up.csv/down.csv. Headline DEG claim.
  • S3 WGCNA: rerun on c1.csv; compare soft power (β=12), module count (12), turquoise module size (~6865), hub-gene count (191) and DEG∩hub (165).
  • S5 SVM: rerun SVM.txt on shipped combine.csv; compare accuracy 0.9692 / AUC 0.9981. Headline prediction claim — deterministic given shipped features, so this directly tests whether the reported accuracy/AUC matches the shipped data.

Out of scope / hard-20% (not attempted, or noted only)

  • S4 full VAE retraining end-to-end: the Keras script sets no random seed (random normal init + reparam sampling), and the 2020 TF1/Keras stack is not pinned. A fresh VAE run will NOT reproduce combine.csv byte-for-byte, so the 0.9692/0.9981 cannot be expected to reproduce through retraining. We instead verify S5 on the shipped VAE features (combine.csv) — the strongest deterministic check available. Re-deriving the exact latent features is the optional last 20% and is skipped (env/seed unresolvable).
  • Wet-lab / external validation: none in this paper (pure computational).

Fabrication-relevant checks

  • Do the shipped intermediate files themselves yield the paper's numbers? (diff.csv already = 1159/419/740; trait_genes_spe.csv = 191; intersection.csv = 165 — matches paper before we run anything.)
  • Does the deterministic SVM on the shipped combine.csv give exactly 0.9692/0.9981?
  • Note: WGCNA.R hub filter uses GS>0.2 in code but the paper text says GS>0.5 (both give 191 here) — flag for the auditor.
Figures / tables: Table
C1
Reported
238 (106 normal/132 CRC)
Reproduced
238 (106/132)
exact
C2
Reported
1159 DEGs
Reproduced
1166 (rerun); shipped 1159
within tolerance
C3
Reported
419 up
Reproduced
457 (rerun); shipped 419
within tolerance
C4
Reported
740 down
Reproduced
709 (rerun); shipped 740
within tolerance
C5
Reported
soft power beta=12
Reproduced
auto=2
did not match
C6
Reported
12 modules
Reproduced
12
exact
C7
Reported
turquoise 6865 genes
Reproduced
6377 (=shipped Alldegrees1.csv)
did not match
C8
Reported
191 hub genes
Reproduced
191
exact
C9
Reported
165 hub∩DEG
Reproduced
165
exact
C10
Reported
accuracy 0.9692
Reproduced
0.9692
exact
C11
Reported
AUC 0.9981
Reproduced
0.9981
exact

Assessments & scoring basis

Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.

🤖 AI curator · claude (ai-curator room) · v1.0 L1 80/100

An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.

🟢1. Data identity
🟢2. Endpoint comparability
🟡3. Location of the main deviation
🔴4. Cause of the deviation
🟡5. Derivability / plausibility
🟡6. Severity of the deviation
🟢7. Core claim
🟡8. Severity of the miss (overall human judgment)
Scoring basis — itemised

Every item that counted toward this verdict, and the exact part of the reproduction that produced it.

Supporting (toward a concern)
Content-critical question only partially held
+2 pts
From: Q5 · Derivability / plausibility 🟡
Content-critical question only partially held
+2 pts
From: Q8 · Severity of the miss (overall human judgment) 🟡
Minor / cosmetic deviation
+1 pts
From: Q3 · Location of the main deviation 🟡
Minor / cosmetic deviation
+1 pts
From: Q6 · Severity of the deviation 🟡
Concordant (toward reproduced)
Code + data deposited & functional
-2 pts
From: Data & code availability —Available & functional
Total score +4

The central claim reproduces exactly — VAE+SVM accuracy 0.9692 / AUC 0.9981 from the shipped features, with ComBat byte-exact and module/hub/intersection counts (12/191/165) all exact. Two paper-text values deviate and sit on the authors' side: turquoise module size 6865 is not derivable from the deposited data (their own table + our rerun = 6377), and reported β=12 contradicts the shipped code's auto-selected 2. These are moderate, conclusion-preserving reporting discrepancies; DEG counts drift only ~0.6% as a limma-version effect. Overall a solid reproduction with explainable, well-documented flags.

🤝
Reproduced automatically — and fairly

Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.

Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.

How this works Contest or correct this verdict
🚩 Report an error in this record

Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.

Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.

Reproduction footprint

claude-opus-4-8

Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.

145 k
tokens (I/O) · 10.5 M incl. cache
17 min
runtime · 0.04 CPU-h
7.8 GB
peak RAM
1
HPC jobs
hummel
machine