Topologically inferring pathway activity toward precise cancer classification via integrating genomic and metabolomic data: prostate cancer as a case.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡A deviation arose in the data or preprocessing
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce: YES. Ideal P16 case - the authors' own R package DRWPClassGM (github.com/chunquanli/DRWPClassGM @ 2cb01c8) is self-contained, shipping the paper's processed data as .rda (GSE8511 training + GSE3325 independent test, metabolomic, pathway gene-sets, gene-metabolite graph) plus a runnable README example. Ran shipped code on shipped data on «our HPC» (R 4.1, RWeka 0.4.46 with bundled weka.jar, openjdk 11). Needed three non-algorithmic modern-R shims (Matrix rBind/cBind are now defunct stubs; force stringsAsFactors=TRUE so Weka gets a nominal class; read AUC from RWeka's structured object instead of the package's Weka-version-specific text parsing that returned NA). RESULT: the paper's CENTRAL CLAIM reproduces 1:1 - DRW-GM gives near-perfect prostate-cancer classification (AUC 0.976-1.000) integrating genomic+metabolomic data; all 4 reported AUC point estimates land within |delta|<=0.032 (1.0000/1.0000/1.0000/0.9762 vs reported 0.9684/0.9992/0.9899/0.9958). DIFFERENT in detail: our run is fully deterministic (sd=0, 1 pathway feature/fold) because on the very small held-out folds (2-3 samples/class) the top pathway separates perfectly -> AUC saturates at 1.0 -> greedy stops; the paper reports sd 0.012-0.083, implying a Weka build that produced non-saturating fold AUCs. This is an environment/tiny-sample-saturation effect, not fabrication - reported values are derivable from the shipped artifact and sit in the same band. NOT ATTEMPTED (80/20): GSE32269/GSE35988 test sets (not shipped in package, need separate GEO + unshipped preprocessing); competing-method baseline rows; full 100-partition run (unnecessary - result deterministic).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 76assessed: 2026-06-16 ⛓ d3a1eced9ab1
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-16
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-09-19
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetIntegrating matched gene expression and metabolomic profiles via a directed random walk on a reconstructed gene–metabolite pathway graph (DRW-GM) will more accurately evaluate the topological importance of genes and yield more reproducible pathway activities, thereby improving the accuracy and robustness of cancer classification compared to methods based on genomics alone.
- ★ DRW-GM evaluates gene topological importance by directed random walk on a global gene–metabolite pathway graph integrating gene expression and metabolomic profiles to infer pathway activities method
- ★ DRW-GM achieves more accurate and robust classification (higher AUC, lower variance) than existing pathway-based methods (DRW, PAC, Mean, Median) and gene-based classification (Genes) in both within-dataset and cross-dataset prostate cancer classification finding
- ★ Incorporating metabolomic information (vs. genomic information alone, i.e., DRW) improves the evaluation of gene topological importance and thus classification accuracy finding
- ★ A global directed gene–metabolite pathway graph was constructed from KEGG metabolic pathways, comprising 1353 genes, 1489 metabolites, and 9824 edges resource
- Hub genes are critical to pathway function but tend to show low, consistent expression changes that are overshadowed by noisier non-hub genes, motivating topology-weighted pathway activity inference mechanism
- ★ DRW-GM identifies biologically informative risk pathways and topologically important differential genes/metabolites (e.g., steroid hormone biosynthesis pathway, CYP1A1, PNP, SMS, cholesterol, l-Proline) relevant to prostate cancer prognosis and metastasis finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| gene expression microarray profiling | human prostate tissue (Benign, PCA, Mets) - dataset GSE8511 | disease status (Benign vs PCA; PCA vs Mets) | gene expression levels used as classification features | — |
| metabolomic profiling | matched human prostate tissue samples | disease status | metabolite concentrations / differential metabolites | — |
| pathway activity inference and classification (directed random walk + logistic regression classifier, within-dataset) | GSE8511 gene expression and matched metabolomic profiles | none (comparison of DRW-GM vs DRW, PAC, Mean, Median, Genes methods) | AUC and accuracy from five-fold cross-validation (100 repeats, 1500 AUCs) | — |
| cross-dataset classification validation | GSE8511 (training) vs GSE3325, GSE32269, GSE35988 (independent test sets) | disease status classification (Benign vs PCA; PCA vs Mets) | AUC and accuracy on independent test sets (100 repeats, 500 AUCs) | — |
| t-test for differential gene expression / p-value ranking | prostate cancer gene expression datasets | disease status | gene p-values (e.g., PNP p=9.58×10-3) | — |
| Wilcoxon rank-sum test for differential metabolites | prostate metabolomic profiles (PCA vs Mets) | disease status | metabolite concentration difference p-values (e.g., cholesterol p=3.05×10-3) | — |
- ▲ DRW-GM yielded the highest within-dataset AUCs (0.9684 Benign-PCA; 0.9992 PCA-Mets) among all compared methods AUC 0.9684 / 0.9992
- ▼ DRW-GM produced the smallest standard deviation of AUCs within-dataset, indicating robustness SD 0.0834 / 0.0123
- ▲ In cross-dataset experiments across three independent test sets, DRW-GM ranked first in AUC in five of six training-test classification comparisons AUC up to 0.9994
- – CYP1A1 (steroid hormone biosynthesis pathway) was the most frequently selected gene marker in the PCA-Mets case 1327/1500 selections
- ▲ PNP was upregulated in Mets samples and had a strongly elevated topological importance ranking relative to its p-value ranking p=9.58×10-3; rank improved from 386 to 66
- – Cholesterol concentrations differed significantly between PCA and Mets metabolomic profiles, consistent with its top-ranked position in the steroid hormone biosynthesis pathway p=3.05×10-3
- ▲ The arginine and proline metabolism pathway was selected far more frequently as a classifier feature by DRW-GM than by DRW 105/1500 vs 31/1500
- ▲ The purine metabolism pathway was selected far more frequently as a classifier feature by DRW-GM than by DRW 222/1500 vs 39/1500
- other AUC 0.9684 (accuracy 90.12%) (within-dataset Benign-PCA classification, DRW-GM)
- other AUC 0.9992 (accuracy 95.81%) (within-dataset PCA-Mets classification, DRW-GM)
- other SD 0.0834 and 0.0123 (standard deviation of within-dataset AUCs, DRW-GM)
- pvalue 9.58 × 10-3 (PNP expression upregulation in Mets samples)
- pvalue 3.05 × 10-3 (cholesterol concentration difference between PCA and Mets (Wilcoxon rank-sum test))
- pvalue 4.40 × 10-2 (CYP11A differential expression p-value)
- count 1327/1500 (frequency CYP1A1 selected as gene marker in PCA-Mets classifiers)
- count 1353 genes, 1489 metabolites, 9824 edges (size of the global directed gene-metabolite pathway graph built from KEGG)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a methods/classification study that develops a directed random walk approach (DRW-GM) integrating gene expression and metabolomic data to infer pathway activities, which are then used as features in logistic regression classifiers for two-class prostate cancer comparisons (Benign vs PCA, PCA vs Mets). Classification performance was assessed by AUC, with within-dataset evaluation using 100 repeats of five-fold cross-validation (mean of 1500 AUCs) and cross-dataset evaluation using 100 repeated random training partitions (mean of 500 AUCs), and results compared against several existing methods. Differential genes and metabolites were assessed with per-feature significance tests (t-test for genes; Wilcoxon rank-sum test for metabolites), and exact p-values were reported for selected features.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Student's t-test (differential gene expression, p-value ranking) | ranking differential genes within pathways and comparing p-value vs topological rankings (e.g., PNP p = 9.58 × 10⁻³) | — | not stated |
| Wilcoxon rank-sum test (Mann-Whitney) | comparing metabolite concentrations between PCA and Mets (e.g., cholesterol p = 3.05 × 10⁻³; CYP11A p = 4.40 × 10⁻²) | — | not stated |
| Area under the ROC curve (AUC) as classification performance metric | Benign–PCA and PCA–Mets classification in within-dataset and cross-dataset experiments (Tables 1 and 2) | mean of 1500 AUCs (within-dataset; 100×five-fold CV) and mean of 500 AUCs (cross-dataset; 100 random partitions) | na |
| Logistic regression classifier | building classifiers from pathway activity features | — | not stated |
-
Per-feature significance for genes was assessed with a t-test and reported without a stated multiple-testing correction across the many genes tested.↳ Could also: A multiple-testing correction such as Benjamini-Hochberg FDR (or Bonferroni for stricter control) applied across the family of gene tests. — An FDR adjustment would characterize the expected proportion of false positives when ranking many genes, which is a common companion to large-scale per-gene testing.
-
Differential gene significance used a parametric t-test while metabolite significance used a nonparametric Wilcoxon rank-sum test.↳ Could also: Using a single consistent framework (e.g., a rank-based test such as Mann-Whitney for both, or a moderated t-statistic via limma for expression data). — A unified or variance-moderated approach can be helpful with small per-group sample sizes and makes the testing strategy uniform across data types.
-
Classifier performance was summarized primarily by mean AUC with its standard deviation across cross-validation/partition repeats.↳ Could also: Reporting a 95% confidence interval (or interquartile range) alongside the mean, and a formal comparison (e.g., DeLong's test or paired resampling test) between methods' AUCs. — Interval estimates and a paired statistical comparison would quantify the uncertainty in, and the difference between, method performances rather than relying on point estimates alone.
-
Overall performance was evaluated with AUC and accuracy.↳ Could also: Additional metrics such as sensitivity/specificity, precision-recall AUC, F1, or Matthews correlation coefficient, especially where class sizes differ. — Complementary metrics convey performance under class imbalance and clarify the trade-offs that a single summary metric may not capture.
-
Dispersion of AUCs was conveyed with standard deviation.↳ Could also: Reporting a 95% confidence interval derived from the resampling distribution. — A CI directly communicates the range of plausible performance values and is often preferred when summarizing resampled estimates.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
Cholesterol concentrations are significantly altered between primary and metastatic prostate cancer within the steroid hormone biosynthesis pathway (p = 3.05 × 10⁻³).metabolomics human prostate tissue mixed 2015×1papers★ This paper is the founder (earliest)
-
Individual gene markers achieve substantially lower cross-dataset AUC (0.75) than the DRW-GM pathway-metabolome integrative method (0.90) for PCA vs metastasis classification on GSE32269, indicating reduced robustness.microarray human prostate tissue down 2015×1papers★ This paper is the founder (earliest)
-
PNP is significantly upregulated in metastatic prostate cancer relative to primary tumors (p = 9.58 × 10⁻³).microarray human prostate tissue up 2015×1papers★ This paper is the founder (earliest)
-
DRW-GM integrative genomic-metabolomic classifier achieves AUC 0.97 (accuracy 90%) for benign vs prostate cancer discrimination in within-dataset evaluation (GSE8511).other human prostate tissue 2015×1papers★ This paper is the founder (earliest)
-
DRW-GM ranks first in AUC in 5 of 6 paired training-test prostate cancer classification tasks, outperforming DRW, PAC, Mean, Median, and individual gene-based methods.other human prostate tissue 2015×1papers★ This paper is the founder (earliest)
-
DRW-GM integrative classifier achieves near-perfect AUC 0.9992 (accuracy 96%) for primary prostate cancer vs metastasis discrimination in within-dataset evaluation (GSE8511).other human prostate tissue 2015×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
Downstream reach in the literature
86 downstream papers · 1 datasetsHow widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.
- Intracrine Androgens and AKR1C3 Activation Confer Re... 2015 · 211 cites
- ERG induces androgen receptor-mediated regulation of... 2013 · 175 cites
- Androgen receptor is the key transcriptional mediato... 2014 · 157 cites
- Pre-existing Castration-resistant Prostate Cancer-li... 2022 · 125 cites
- MUC1-C regulates lineage plasticity driving progress... 2020 · 119 cites
- Activation of MAPK Signaling by CXCR7 Leads to Enzal... 2019 · 107 cites
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-26286638 (DRW-GM, Sci Rep 2015, srep13192)
Paper
Topologically inferring pathway activity toward precise cancer classification via integrating genomic and metabolomic data: prostate cancer as a case. Liu W. et al. Method DRW-GM = Directed Random Walk on a gene–metabolite graph → pathway activity profiles → Weka classifier (Logistic) → cancer-status classification.
Code & data artifact
- Repo: https://github.com/chunquanli/DRWPClassGM (renamed from chunquanli/DRW-GM;
GitHub repo id 30226999, default branch
master, pushed 2015-03-01, public, not archived). - It is a self-contained R package that ships the paper's OWN processed data as
.rdaobjects:GProf8511(GSE8511 mRNA),GProf3325(GSE3325 mRNA),MProf(metabolomic),pathSet(pathway gene sets),dGMGraph(gene–metabolite graph). README gives a complete runnable example. → ideal P16 case: run shipped code on shipped data. - Deps: R, igraph, Matrix, RWeka (bundles its own weka.jar via RWekajars → no separate Weka install needed despite README), samr. Java needed for RWeka.
In scope (pipeline-derived, attempted)
The README example IS the reproduction. It trains fit.DRWPClassGM on GSE8511
(normal/Benign vs PCA) and evaluates on GSE3325 → maps directly to:
- T2_BPCA_3325 (HEADLINE): cross-dataset DRW-GM Benign–PCA, train GSE8511 → test GSE3325, reported AUC 0.9899 ± 0.0294 (acc 93.60%), Table 2.
- T1_BPCA: within-dataset 5-fold CV DRW-GM Benign–PCA on GSE8511, reported
AUC 0.9684 ± 0.0834 (acc 90.12%), Table 1. (=
fit$AUCover repeated partitions.)
Paper reports the mean over 100 repeated random partitions (500 / 1500 AUCs). We repeat the shipped example N times with seeds and report mean ± sd — honest 1:1 of the quantity the shipped code+data produce.
Out of scope / not attempted (the hard 20%)
- PCA–Mets rows (T1_PCAMets, T2_PCAMets_3325): require Mets-class labels in the
bundled objects; attempted only if
$Metsfields exist (checked at run time). - GSE32269 / GSE35988 test sets (Table 2 cols 2–3): NOT shipped in the package data → would need separate GEO download + the authors' unshipped preprocessing. Skipped (80/20).
- Competing methods (PAC, Mean, Median, Genes, DRW, DRW-GM-NM): comparison baselines, not the paper's contribution; not reproduced.
- Exact 100×/1500-AUC reproduction of the published means: stochastic + the fit keeps the best-of-folds classifier; we approximate with N repetitions.
Reproduction substrate
All compute on «our HPC» (SLURM, conda env on «infra»). «host» holds results only. «infra» workdir: «path»
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Ran the authors' own self-contained R package (DRWPClassGM @2cb01c8) on its shipped .rda data, so input identity and endpoint comparability are 1:1. All four reported AUCs reproduce within |Δ|≤0.032 and the central near-perfect classification claim holds fully — values are clearly derivable from the shared artifact, no fabrication concern. The only real deviation is on our/environment side: the published per-fold variance is not reproduced because modern RWeka saturates fold AUCs to 1.0 (sd=0) on the very small held-out folds. This is a benign metric-version/tiny-sample-saturation effect, hence solid-with-explainable-deviation rather than a clean 1:1.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at [email protected].
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.