Using random walks to identify cancer-associated modules in expression data.
Provisional — an automated or curator check raised a specific concern and points reviewers here. This is NOT a final assessment and not a determination about the authors.
Part of the results reproduced; minor but material deviations remained.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Any deviation was negligible
- 🟡Could not use the authors’ exact input data
- 🔴Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
▸Reproduction agent’s raw note
The Walktrap-GM code repository (github.com/petrochilos/walktrap-gm) is genuinely and irrecoverably deleted - confirmed via live 404s, GitHub API, and Wayback Machine forensics showing only HTML wrapper pages were ever crawled, never the binary R package (walktrap.gm_1.0.tar) that held the actual implementation - so the paper's core computational pipeline (network construction, random-walk module detection, significance testing, GSEA/benchmark validation) could not be executed or reproduced by any means available, and this room is recorded as a genuine drop per HARD RULE 7 rather than a fabricated result. Independently, all three GEO datasets the paper relies on were profiled with real matrix-level QC: GSE7390 (198 samples, GPL96) and GSE8671 (64 samples, GPL570) match the paper's stated sample counts exactly; GSE14520's GPL571 platform-subset shows 43 samples versus the paper's stated 44 (a minor, well-evidenced discrepancy), and the series has grown to 488 samples overall via a later GPL3921 cohort addition unrelated to the original paper. All findings are provisional, pending human review.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-07-30
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-07-31no human curator yet
- Last updated
- 2026-07-31
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusThe paper asks whether a random-walk-based community detection algorithm (Walktrap), adapted to genomic modules and applied to an interactome whose edges are weighted by differential expression, can identify densely connected gene modules significantly associated with cancer phenotypes. It tests the premise that mining dense subgraphs of significant genes in biological interaction networks reveals functionally relevant, cancer-associated modules and candidate genes.
- ★ Walktrap-GM, a random-walk community detection algorithm adapted with stopping criteria (maximum modularity, maximum size, maximum module score), identifies modules significantly enriched with cancer genes in expression-weighted interaction networks. method
- ★ Significant modules highlight functionally cancer-related gene groups and candidate therapeutic targets, including transcription factors (SPIB, RPS6KA2, RPS6KA6), cell-cycle regulators (BRSK1, WEE1, CDC25C), cell-cycle/proliferation modulators (CBLC, IRS2), and MAP-kinase pathway genes (MAPK9, DUSP1, DUSP9, RIPK2). finding
- ★ Compared with the highly cited module-finding tools jActiveModules and Matisse, Walktrap-GM shows strong performance in discovering modules enriched with known cancer genes. finding
- ★ Restricting maximum module size (200 nodes) yields smaller modules that permit more specific functional annotation and easier interpretation than larger maximum sizes (200-500), which produced modules too general in functional annotation. finding
- ★ An integrated interactome built from KEGG and HPRD human interactions, weighted by the square of the mean absolute fold-change of adjacent nodes, serves as the substrate for semi-supervised module discovery. resource
- ★ Module significance is scored by comparing a module's cumulative activity against a bootstrap random distribution generated by permuting fold-change values per module size. method
- Absolute fold-change values are more robust edge-weight factors than p-values because they have a more discrete range of values and stable dispersion. method
- The random walk is computationally more efficient than edge-betweenness, spectral methods, or spanning trees for detecting communities in this setting. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Gene expression microarray (public dataset reanalysis, GSE14520) | Human hepatocellular carcinoma tissue, 22 paired tumor and non-tumor profiles | none (tumor vs. paired non-tumor tissue comparison) | Normalized log-transformed fold-change and p-values of differential expression between paired normal and disease tissue | Affymetrix HG-U133A 2.0 array |
| Gene expression microarray (public dataset reanalysis, GSE7390) | Human breast tumors from 198 lymph node-negative patients (TRANSBIG) | none (high-risk vs. low-risk prognosis comparison via Adjuvant!Online clinical risk index) | Normalized log-transformed fold-change and p-values of differential expression between high-risk and low-risk samples | Affymetrix HG-U133A array |
| Gene expression microarray (public dataset reanalysis, GSE8671) | Human sporadic colonic adenoma and normal mucosa from 32 colonoscopy patients | none (paired adenoma vs. paired normal mucosa comparison) | Normalized log-transformed fold-change and p-values of differential expression between paired normal and adenoma tissue | Affymetrix HG-U133A 2.0 array |
| Differential expression statistical analysis | All three GEO expression datasets (HCC, BC, CCA) | none | Log odds of differential expression by empirical Bayes; p-values corrected by Benjamini-Hochberg FDR | R/Bioconductor with GEOquery and limma packages |
| Interaction network construction | Human interactome from KEGG (KGML-parsed metabolic and signaling relations) and HPRD protein-protein interactions | none | Nodes and unique pairwise interactions; global topology statistics (diameter, density, degree, betweenness, closeness, shortest path length, clustering coefficient) | KEGG, HPRD |
| Random walk community detection (Walktrap-GM) | Expression-weighted global interaction network for each of HCC, CCA, BC | Algorithmic configuration: walk length t = 3; stopping criteria of maximum modularity, maximum module size 200 nodes, or maximum module score | Module partitions and module scores; test statistic of cumulative module activity against a permutation distribution | Walktrap (Pons and Latapy) implemented in iGraph |
| Pathway overrepresentation / functional annotation | Gene lists from top-scoring modules | none | Overrepresented curated pathways at significance threshold p <= 0.01 with minimum overlap of two genes | ConsensusPathDB querying KEGG, WikiPathways, PID, HumanCyc, Reactome |
| Gene Set Enrichment Analysis (GSEA) and benchmarking against module-finding tools | Each cancer phenotype (HCC and CCA disease vs. normal; BC high risk vs. low risk); modules compared against jActiveModules and Matisse using an OMIM-derived cancer gene list | none | Enriched gene sets ranked by normalized enrichment score (NES) and overlap of top 10 NES gene sets with module annotation; enrichment of modules for known cancer genes | GSEA with all canonical pathways (c3.cp v2.5) from MSigDB, 1000 permutations by phenotype, collapse to gene symbols = TRUE; jActiveModules; Matisse; OMIM |
- – Walktrap-GM identifies modules significantly enriched with cancer genes across the three cancer datasets, outperforming or performing strongly relative to jActiveModules and Matisse.
- – Best scoring partitions were found for each study under a maximum cluster size of 200 nodes. 200 nodes maximum cluster size
- – Significant modules contain interacting transcription factors SPIB, RPS6KA2 and RPS6KA6.
- – Significant modules contain cell-cycle regulatory genes BRSK1, WEE1 and CDC25C.
- – Significant modules contain modulators of the cell cycle and proliferation, CBLC and IRS2.
- – Significant modules contain genes regulating and participating in the MAP-kinase pathway: MAPK9, DUSP1, DUSP9, RIPK2.
- – The assembled interactome shows small-world properties: small average path length and high betweenness, so nodes are generally reachable in relatively few steps. average shortest path length 4.1281
- – Maximum module sizes tested between 200 and 500 nodes generally produced modules that were too general in functional annotation and less informative. 200-500 nodes tested
- count 10,882 nodes and 70,385 interactions (Global interaction network after removing duplicate nodes and edges)
- count 10,642 nodes and 62,407 interactions (Largest connected cluster of unique pairwise interactions extracted for analysis)
- count 32,563 unique interactions (KEGG); 39,240 protein-protein interactions (HPRD) (Source interaction counts parsed from KGML files and downloaded from HPRD)
- other diameter = 15; density = 0.0011; average node degree = 11.72; average node betweenness = 16723; average edge betweenness = 3759; closeness = .2454; average shortest path length = 4.1281; global clustering coefficient = 0.1314 (Global topology statistics of the interaction network)
- count 22 hepatocellular tumors with 22 paired non-tumor; 198 breast tumors (good = 47, poor = 151); 32 paired sporadic adenoma with 32 paired normal (Sample counts for GSE14520, GSE7390 and GSE8671)
- count 1239 cancer-associated genes from 6995 gene references queried (OMIM-derived cancer gene list used to evaluate tool performance)
- count 5000 permutations (Bootstrap random distribution of cumulative activity estimates per module size for module significance scoring)
- pvalue p <= .01 (Default significance threshold for ConsensusPathDB overrepresentation analysis (minimum overlap of two genes))
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The study applies a network-based random walk algorithm (Walktrap-GM) to three public microarray datasets — paired hepatocellular carcinoma (HCC) and colorectal adenoma (CCA) tumor/normal comparisons, and a breast cancer (BC) high-risk vs low-risk prognosis comparison — estimating gene-level differential expression with limma's empirical Bayes framework and correcting for multiple testing using the Benjamini-Hochberg false discovery rate. Network edge weights are derived from fold-change values, and the significance of resulting modules is assessed by comparing observed cumulative activity scores to a permutation-generated (5000 permutations) null distribution. Functional relevance of top-scoring modules is further evaluated with GSEA (1000 phenotype permutations) and ConsensusPathDB overrepresentation analysis (p ≤ .01), and Walktrap-GM's module-finding performance is compared against two other network tools (jActiveModules, Matisse) by enrichment for OMIM-curated cancer genes.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Empirical Bayes moderated t-statistic (limma) for differential expression | Paired tumor vs non-tumor (HCC) and paired adenoma vs normal (CCA) comparisons; high-risk vs low-risk prognosis comparison (BC) | 22 paired samples (HCC); 32 paired samples (CCA); 198 samples, 47 good/151 poor prognosis (BC) | not stated |
| Benjamini-Hochberg false discovery rate correction | Applied to log-odds differential expression p-values from limma across all three datasets | — | not stated |
| Permutation test comparing module cumulative activity to a bootstrap null distribution (test statistic (μ0²-μ1²)/σerr²) | Significance scoring of Walktrap-GM network modules (size > 3 nodes) in each dataset | 5000 permutations per module size | not stated |
| GSEA weighted Kolmogorov-Smirnov-like enrichment statistic with phenotype permutation | Enrichment of canonical pathway gene sets (MSigDB c3.cpv2.5) comparing disease vs normal (HCC, CCA) and high-risk vs low-risk (BC) | 1000 permutations | not stated |
| Overrepresentation analysis (ConsensusPathDB) | Functional/pathway annotation of top-scoring modules against curated pathway databases (KEGG, WikiPathways, PID, HumanCyc, Reactome) | Minimum overlap of two genes; significance threshold p ≤ .01 | not stated |
-
Differential expression significance was assessed using limma's empirical Bayes moderated t-statistic, applied to paired comparisons in HCC/CCA and a two-group comparison in BC.↳ Could also: A standard paired or unpaired t-test, or a non-parametric alternative such as the Wilcoxon signed-rank or Mann-Whitney U test — could also be used to compare expression values directly; a non-parametric test would relax the distributional assumptions underlying the moderated t-statistic, which can be a useful property with smaller sample sizes such as the 22 paired HCC profiles.
-
Multiple-testing correction for differential expression was performed using the Benjamini-Hochberg FDR method.↳ Could also: Bonferroni correction or Storey's q-value approach — would offer an alternative balance between false-positive and false-negative control; Bonferroni is more conservative, while the q-value method estimates the proportion of true nulls and can increase power relative to BH in some datasets.
-
Module-level significance was evaluated by comparing observed cumulative activity to a permutation-based null distribution built from 5000 permutations per module size.↳ Could also: A parametric approximation based on the known distribution of the summary statistic, or extending the GSEA-style phenotype-permutation framework directly to modules — could also provide a significance estimate; a parametric approach would reduce computation time, while a shared permutation framework across module-finding and GSEA analyses would keep the null-model construction consistent throughout the paper.
-
Pathway overrepresentation for top-scoring modules was assessed with ConsensusPathDB using a fixed significance threshold (p ≤ .01) for the pathway family tested.↳ Could also: Applying a false discovery rate correction (e.g., Benjamini-Hochberg) across the set of tested pathways — would also account for the number of pathways queried simultaneously, a common consideration when many gene sets are tested in parallel.
-
Results such as fold-change values, module scores, and network topology statistics are summarized largely as point estimates without confidence intervals or dispersion measures.↳ Could also: Reporting bootstrap- or permutation-derived confidence intervals alongside the point estimates — would also convey the precision/uncertainty of these estimates, complementing the significance values already reported.
-
GSEA was run with 1000 phenotype permutations to generate normalized enrichment scores for each cancer comparison.↳ Could also: A larger number of permutations (e.g., 10,000) or an analytic approximation to the permutation null distribution — could also refine the precision of estimated p-values, particularly for gene sets in the extreme tails of the enrichment score distribution.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a code-availability drop, not a discrepancy finding. The paper's method existed solely as walktrap.gm_1.0.tar in github.com/petrochilos/walktrap-gm, which was deleted between 2022-04-25 and 2024-03-08 and was never archived in binary form (Wayback crawled only HTML wrappers; no CRAN/Bioconductor mirror), so the network topology (10,882/70,385 nodes/interactions; LCC 10,642/62,407) and the Walktrap-t=3 module detection, permutation testing and GSEA/jActiveModules/Matisse benchmarks were correctly not attempted. What was checkable holds up well: GSE7390 (n=198) and GSE8671 (n=64) match exactly, and GSE14520's GPL571 subset is off by a single sample (43 vs 44) amid post-publication series growth to 488 — negligible and attributable to GEO curation drift, on neither our nor the authors' analytic side. The blocking cause sits on the authors'/artifact side (published method no longer executable), but there is no evidence of non-derivability-by-fabrication or of a failed result — the central claim is simply untestable, which caps q7 and q8 at yellow rather than red.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.