Topologically inferring pathway activity toward precise cancer classification via integrating genomic and metabolomic data: prostate cancer as a case.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡A deviation arose in the data or preprocessing
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce: YES. Ideal P16 case - the authors' own R package DRWPClassGM (github.com/chunquanli/DRWPClassGM @ 2cb01c8) is self-contained, shipping the paper's processed data as .rda (GSE8511 training + GSE3325 independent test, metabolomic, pathway gene-sets, gene-metabolite graph) plus a runnable README example. Ran shipped code on shipped data on «our HPC» (R 4.1, RWeka 0.4.46 with bundled weka.jar, openjdk 11). Needed three non-algorithmic modern-R shims (Matrix rBind/cBind are now defunct stubs; force stringsAsFactors=TRUE so Weka gets a nominal class; read AUC from RWeka's structured object instead of the package's Weka-version-specific text parsing that returned NA). RESULT: the paper's CENTRAL CLAIM reproduces 1:1 - DRW-GM gives near-perfect prostate-cancer classification (AUC 0.976-1.000) integrating genomic+metabolomic data; all 4 reported AUC point estimates land within |delta|<=0.032 (1.0000/1.0000/1.0000/0.9762 vs reported 0.9684/0.9992/0.9899/0.9958). DIFFERENT in detail: our run is fully deterministic (sd=0, 1 pathway feature/fold) because on the very small held-out folds (2-3 samples/class) the top pathway separates perfectly -> AUC saturates at 1.0 -> greedy stops; the paper reports sd 0.012-0.083, implying a Weka build that produced non-saturating fold AUCs. This is an environment/tiny-sample-saturation effect, not fabrication - reported values are derivable from the shipped artifact and sit in the same band. NOT ATTEMPTED (80/20): GSE32269/GSE35988 test sets (not shipped in package, need separate GEO + unshipped preprocessing); competing-method baseline rows; full 100-partition run (unnecessary - result deterministic).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 76assessed: 2026-06-16 ⛓ d3a1eced9ab1
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-16
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan integrating matched genomic (gene expression) and metabolomic data via a directed random walk on a global gene–metabolite pathway graph yield more reproducible pathway activities and more accurate, robust cancer classification than methods using single-omics data, using prostate cancer as a case?
- ★ DRW-GM integrates gene expression and metabolomic profiles via directed random walk on a global gene–metabolite pathway graph to weight genes by topological importance and infer reproducible pathway activities method
- ★ Joint analysis of genomic and metabolomic data yields more accurate and robust cancer classification than single-omics pathway-based methods (DRW, PAC, Mean, Median) and gene-based classifiers finding
- ★ Incorporating differential metabolites and the gene–metabolite graph improves estimation of genes' topological importance compared to gene–gene graphs (DRW-GM > DRW-GM-NM > DRW) finding
- ★ DRW-GM identifies biologically informative risk metabolic pathways (e.g., steroid hormone biosynthesis, arginine and proline metabolism, purine metabolism) and topologically important differential genes/metabolites relevant to prostate cancer prognosis and therapy resource
- Hub genes have consistent but low differential expression, so weighting genes by topological importance promotes robust pathway activity mechanism
- Topological importance rankings of genes are not correlated with p-value rankings across datasets finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| gene expression microarray profiling | human prostate tissue (Benign, PCA, Mets) dataset GSE8511 (training) | none | gene expression levels / differential gene p-values used for pathway activity inference | — |
| metabolomic profiling | human prostate tissue (PCA and Mets samples) | none | metabolite concentrations / differential metabolites (e.g., cholesterol, sarcosine, l-Proline) | — |
| gene expression microarray profiling (independent test sets) | human prostate tissue datasets GSE3325, GSE32269, GSE35988 | none | classification AUC/accuracy for Benign-PCA and PCA-Mets | — |
| computational pathway activity inference and classification (DRW-GM) | global KEGG gene–metabolite graph (1353 genes, 1489 metabolites, 9824 edges) | none | logistic regression classifier AUC/accuracy; topological weight vector W∞ | KEGG metabolic pathways |
- – Within-dataset Benign-PCA classification AUC (accuracy) for DRW-GM AUC 0.9684 (90.12%), SD 0.0834
- – Within-dataset PCA-Mets classification AUC (accuracy) for DRW-GM AUC 0.9992 (95.81%), SD 0.0123
- – Cross-dataset Benign-PCA AUCs for DRW-GM on GSE3325, GSE32269, GSE35988 0.9899 (93.60%), 0.8522 (81.96%), 0.9836 (84.15%)
- – Cross-dataset PCA-Mets AUCs for DRW-GM on GSE3325, GSE32269, GSE35988 0.9958 (91.60%), 0.9011 (60.89%), 0.9994 (95.00%)
- – DRW-GM AUCs ranked first in 5 of 6 paired training-test classifications, outperforming DRW, PAC, Mean, Median, Genes 5/6
- ▼ Genes method PCA-Mets AUC on GSE32269 was much lower than DRW-GM, showing individual gene markers less robust 0.7492 vs 0.9011
- ▲ PNP upregulated in Mets samples p = 9.58 × 10^-3
- – Cholesterol concentrations significantly differed between PCA and Mets samples in steroid hormone biosynthesis pathway p = 3.05 × 10^-3
- other 1353 genes, 1489 metabolites, 9824 edges (size of global directed gene–metabolite pathway graph)
- other AUC 0.9684 / 0.9992 (DRW-GM within-dataset Benign-PCA and PCA-Mets AUCs (GSE8511))
- other SD 0.0834 / 0.0123 (DRW-GM smallest standard deviations of AUCs, indicating robustness)
- pvalue 9.58 × 10^-3 (PNP upregulation in Mets samples)
- pvalue 3.05 × 10^-3 (Wilcox rank-sum test) (cholesterol difference between PCA and Mets metabolomic profiles)
- pvalue 4.40 × 10^-2 (CYP11A (EC:1.14.15.6) in steroid hormone biosynthesis pathway)
- count 1327/1500 (CYP1A1 selection frequency as gene marker in PCA-Mets case)
- count 105/1500 (DRW-GM) vs 31/1500 (DRW) (selection frequency of arginine and proline metabolism pathway)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a methods/classification study that develops a directed random walk approach (DRW-GM) integrating gene expression and metabolomic data to infer pathway activities, which are then used as features in logistic regression classifiers for two-class prostate cancer comparisons (Benign vs PCA, PCA vs Mets). Classification performance was assessed by AUC, with within-dataset evaluation using 100 repeats of five-fold cross-validation (mean of 1500 AUCs) and cross-dataset evaluation using 100 repeated random training partitions (mean of 500 AUCs), and results compared against several existing methods. Differential genes and metabolites were assessed with per-feature significance tests (t-test for genes; Wilcoxon rank-sum test for metabolites), and exact p-values were reported for selected features.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Student's t-test (differential gene expression, p-value ranking) | ranking differential genes within pathways and comparing p-value vs topological rankings (e.g., PNP p = 9.58 × 10⁻³) | — | not stated |
| Wilcoxon rank-sum test (Mann-Whitney) | comparing metabolite concentrations between PCA and Mets (e.g., cholesterol p = 3.05 × 10⁻³; CYP11A p = 4.40 × 10⁻²) | — | not stated |
| Area under the ROC curve (AUC) as classification performance metric | Benign–PCA and PCA–Mets classification in within-dataset and cross-dataset experiments (Tables 1 and 2) | mean of 1500 AUCs (within-dataset; 100×five-fold CV) and mean of 500 AUCs (cross-dataset; 100 random partitions) | na |
| Logistic regression classifier | building classifiers from pathway activity features | — | not stated |
-
Per-feature significance for genes was assessed with a t-test and reported without a stated multiple-testing correction across the many genes tested.↳ Could also: A multiple-testing correction such as Benjamini-Hochberg FDR (or Bonferroni for stricter control) applied across the family of gene tests. — An FDR adjustment would characterize the expected proportion of false positives when ranking many genes, which is a common companion to large-scale per-gene testing.
-
Differential gene significance used a parametric t-test while metabolite significance used a nonparametric Wilcoxon rank-sum test.↳ Could also: Using a single consistent framework (e.g., a rank-based test such as Mann-Whitney for both, or a moderated t-statistic via limma for expression data). — A unified or variance-moderated approach can be helpful with small per-group sample sizes and makes the testing strategy uniform across data types.
-
Classifier performance was summarized primarily by mean AUC with its standard deviation across cross-validation/partition repeats.↳ Could also: Reporting a 95% confidence interval (or interquartile range) alongside the mean, and a formal comparison (e.g., DeLong's test or paired resampling test) between methods' AUCs. — Interval estimates and a paired statistical comparison would quantify the uncertainty in, and the difference between, method performances rather than relying on point estimates alone.
-
Overall performance was evaluated with AUC and accuracy.↳ Could also: Additional metrics such as sensitivity/specificity, precision-recall AUC, F1, or Matthews correlation coefficient, especially where class sizes differ. — Complementary metrics convey performance under class imbalance and clarify the trade-offs that a single summary metric may not capture.
-
Dispersion of AUCs was conveyed with standard deviation.↳ Could also: Reporting a 95% confidence interval derived from the resampling distribution. — A CI directly communicates the range of plausible performance values and is often preferred when summarizing resampled estimates.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
Cholesterol concentrations are significantly altered between primary and metastatic prostate cancer within the steroid hormone biosynthesis pathway (p = 3.05 × 10⁻³).metabolomics human prostate tissue mixed 2015×1papers★ This paper is the founder (earliest)
-
Individual gene markers achieve substantially lower cross-dataset AUC (0.75) than the DRW-GM pathway-metabolome integrative method (0.90) for PCA vs metastasis classification on GSE32269, indicating reduced robustness.microarray human prostate tissue down 2015×1papers★ This paper is the founder (earliest)
-
PNP is significantly upregulated in metastatic prostate cancer relative to primary tumors (p = 9.58 × 10⁻³).microarray human prostate tissue up 2015×1papers★ This paper is the founder (earliest)
-
DRW-GM integrative genomic-metabolomic classifier achieves AUC 0.97 (accuracy 90%) for benign vs prostate cancer discrimination in within-dataset evaluation (GSE8511).other human prostate tissue 2015×1papers★ This paper is the founder (earliest)
-
DRW-GM ranks first in AUC in 5 of 6 paired training-test prostate cancer classification tasks, outperforming DRW, PAC, Mean, Median, and individual gene-based methods.other human prostate tissue 2015×1papers★ This paper is the founder (earliest)
-
DRW-GM integrative classifier achieves near-perfect AUC 0.9992 (accuracy 96%) for primary prostate cancer vs metastasis discrimination in within-dataset evaluation (GSE8511).other human prostate tissue 2015×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
Downstream reach in the literature
86 downstream papers · 1 datasetsHow widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.
- Intracrine Androgens and AKR1C3 Activation Confer Re... 2015 · 211 cites
- ERG induces androgen receptor-mediated regulation of... 2013 · 175 cites
- Androgen receptor is the key transcriptional mediato... 2014 · 157 cites
- Pre-existing Castration-resistant Prostate Cancer-li... 2022 · 125 cites
- MUC1-C regulates lineage plasticity driving progress... 2020 · 119 cites
- Activation of MAPK Signaling by CXCR7 Leads to Enzal... 2019 · 107 cites
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-26286638 (DRW-GM, Sci Rep 2015, srep13192)
Paper
Topologically inferring pathway activity toward precise cancer classification via integrating genomic and metabolomic data: prostate cancer as a case. Liu W. et al. Method DRW-GM = Directed Random Walk on a gene–metabolite graph → pathway activity profiles → Weka classifier (Logistic) → cancer-status classification.
Code & data artifact
- Repo: https://github.com/chunquanli/DRWPClassGM (renamed from chunquanli/DRW-GM;
GitHub repo id 30226999, default branch
master, pushed 2015-03-01, public, not archived). - It is a self-contained R package that ships the paper's OWN processed data as
.rdaobjects:GProf8511(GSE8511 mRNA),GProf3325(GSE3325 mRNA),MProf(metabolomic),pathSet(pathway gene sets),dGMGraph(gene–metabolite graph). README gives a complete runnable example. → ideal P16 case: run shipped code on shipped data. - Deps: R, igraph, Matrix, RWeka (bundles its own weka.jar via RWekajars → no separate Weka install needed despite README), samr. Java needed for RWeka.
In scope (pipeline-derived, attempted)
The README example IS the reproduction. It trains fit.DRWPClassGM on GSE8511
(normal/Benign vs PCA) and evaluates on GSE3325 → maps directly to:
- T2_BPCA_3325 (HEADLINE): cross-dataset DRW-GM Benign–PCA, train GSE8511 → test GSE3325, reported AUC 0.9899 ± 0.0294 (acc 93.60%), Table 2.
- T1_BPCA: within-dataset 5-fold CV DRW-GM Benign–PCA on GSE8511, reported
AUC 0.9684 ± 0.0834 (acc 90.12%), Table 1. (=
fit$AUCover repeated partitions.)
Paper reports the mean over 100 repeated random partitions (500 / 1500 AUCs). We repeat the shipped example N times with seeds and report mean ± sd — honest 1:1 of the quantity the shipped code+data produce.
Out of scope / not attempted (the hard 20%)
- PCA–Mets rows (T1_PCAMets, T2_PCAMets_3325): require Mets-class labels in the
bundled objects; attempted only if
$Metsfields exist (checked at run time). - GSE32269 / GSE35988 test sets (Table 2 cols 2–3): NOT shipped in the package data → would need separate GEO download + the authors' unshipped preprocessing. Skipped (80/20).
- Competing methods (PAC, Mean, Median, Genes, DRW, DRW-GM-NM): comparison baselines, not the paper's contribution; not reproduced.
- Exact 100×/1500-AUC reproduction of the published means: stochastic + the fit keeps the best-of-folds classifier; we approximate with N repetitions.
Reproduction substrate
All compute on «our HPC» (SLURM, conda env on «infra»). «host» holds results only. «infra» workdir: «path»
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Ran the authors' own self-contained R package (DRWPClassGM @2cb01c8) on its shipped .rda data, so input identity and endpoint comparability are 1:1. All four reported AUCs reproduce within |Δ|≤0.032 and the central near-perfect classification claim holds fully — values are clearly derivable from the shared artifact, no fabrication concern. The only real deviation is on our/environment side: the published per-fold variance is not reproduced because modern RWeka saturates fold AUCs to 1.0 (sd=0) on the very small held-out folds. This is a benign metric-version/tiny-sample-saturation effect, hence solid-with-explainable-deviation rather than a clean 1:1.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.