Inference of Subpathway Activity Profiles Reveals Metabolism Abnormal Subpathway Regions in Glioblastoma Multiforme.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values are derivable from the shared data
- ✓The central claim held under reproduction
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡The deviation was non-trivial in magnitude
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce. The crmSubpathway tool's documented pipeline (KEGG metabolic KGML -> k-clique K=4 -> GSVA Gaussian activity matrix -> limma diff -> adj.P<0.05) is BIT-FOR-BIT deterministic: re-running SubpathwayMatrix rebuilt the shipped Spwmatrix with max abs diff 0 (C1) and CalculateDF rebuilt DFspw to machine epsilon ~1e-16 (C2), using era-matched GSVA 1.36.0 / R 4.0.5 on «our HPC». Applying the same tool + shipped Metspwlist to the paper's OWN data GSE4290 (P16) reproduced the exact cohort 77 GBM + 23 normal (C4a) and the paper's specific highlighted finding that sterol-biosynthesis subpathway 00100_6 is down-regulated in GBM (logFC -0.546, adj.P 3.3e-6; all 00100_* down) (C4c). The headline '25 GBM-related subpathways' is NOT 1:1 (C4b): on GSE4290 alone we get 197 significant subpathways, because the paper's 25 is the intersection across THREE datasets - a stricter filter, and the larger single-dataset count is the expected direction. NOT attempted (the ~20%): reconstructing the exact 3-dataset intersection, IDH subgroup analyses (Table S2), survival/HR, and all wet-lab validation (Figs 4-5). No fabrication signal; only cosmetic doc-drift (man page 3000 vs 2564 genes; min.sz doc default 10 vs code 1).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 89assessed: 2026-06-15 ⛓ ea779d3fbf82
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator headless) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-09-19
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetMetabolic subpathway activity, rather than whole-pathway or single-gene analysis, can accurately identify abnormal metabolic targets in glioblastoma (GBM), and the sterol biosynthesis intermediate (S)-2,3-Epoxysqualene—occupying a key position in subpathway 00100_6—has functional relevance to GBM cell behavior.
- ★ A GSVA-based method for constructing a metabolic subpathway activity score matrix accurately identifies abnormal GBM metabolic targets method
- ★ 25 metabolic subpathways are significantly abnormal in GBM, identified via intersection of three independent gene expression datasets finding
- ★ (S)-2,3-Epoxysqualene occupies the central/key position of the sterol biosynthesis subpathway 00100_6, suggesting high association with GBM mechanism
- ★ (S)-2,3-Epoxysqualene inhibits U87-MG glioblastoma cell viability finding
- ★ (S)-2,3-Epoxysqualene arrests U87-MG cells in G0/G1 phase and induces apoptosis finding
- ★ LSS gene expression within subpathway 00100_6 is significantly associated with GBM patient survival finding
- ★ The crmSubpathway R package was developed and made freely available for disease-related metabolic subpathway identification resource
- GBM-related subpathways cluster into two functional classes showing distinct up- and down-regulation patterns finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| k-clique network-based subpathway mining from KEGG pathway XML data | human metabolic pathway data (computational) | none | metabolic subpathway gene sets | crmSubpathway / iSubpathwayMiner R package |
| GSVA-based subpathway activity scoring and differential expression (limma) | GBM patient tumor tissue vs normal (GSE4290, GSE16011, TCGA) | none (disease vs normal) | subpathway activity score differences (adjusted p<0.05) | GSVA and limma R packages |
| survival analysis (Cox proportional hazards, Kaplan-Meier, log-rank test) | TCGA GBM patient cohort | none (stratified by LSS expression level) | hazard ratio and survival curve | R survival and survminer packages |
| CCK8 cell viability assay | U87-MG glioblastoma cell line | (S)-2,3-Epoxysqualene at varying concentrations, 2-12 h | optical absorbance at 450 nm | Cell Counting Kit 8 (Dojindo) |
| wound-healing motility assay | U87-MG glioblastoma cell line | (S)-2,3-Epoxysqualene, 100 nmol/L, 12 h | migration distance / wound closure | phase-contrast microscope, Image J software |
| flow cytometry cell cycle analysis (PI staining) | U87-MG glioblastoma cell line | (S)-2,3-Epoxysqualene | cell cycle phase distribution | FACSCalibur flow cytometer, ModFit LT 3.2 |
| flow cytometry apoptosis assay (Annexin V-FITC/PI double staining) | U87-MG glioblastoma cell line | (S)-2,3-Epoxysqualene | apoptosis rate | flow cytometer (BD) |
- – 25 metabolic subpathways identified as significantly GBM-related across three independent datasets
- – LSS in metabolic subpathway 00100_6 significantly associated with GBM patient prognosis HR = 1.5
- ▼ (S)-2,3-Epoxysqualene inhibits U87-MG cell activity/viability
- ▲ (S)-2,3-Epoxysqualene arrests U87-MG cells in G0/G1 phase
- ▲ (S)-2,3-Epoxysqualene induces apoptosis in U87-MG cells
- – Optimal number of clusters for the 25 GBM-related subpathways determined to be two (K-means and consensus clustering)
- – First class of subpathways up-regulated and second class down-regulated in GBM patients
- – (S)-2,3-Epoxysqualene affects migration/wound-healing behavior of U87-MG cells
- count 427 (total GBM patients pooled across the three transcriptomic datasets)
- other 81% (GBM represents 81% of malignant brain and CNS tumors)
- other 65% (GBM incidence accounts for 65% of all glioma types)
- other HR = 1.5 (LSS gene expression hazard ratio for survival in TCGA GBM data set)
- pvalue adjusted p-value < 0.05 (threshold used to determine significantly different metabolic subpathways)
- count 25 (number of GBM-related metabolic subpathways identified)
- count GSE4290: 77 GBM, 23 normal (sample sizes for microarray dataset GSE4290)
- count TCGA GBM: 155 GBM, 5 normal (RNA-seq) (sample sizes for TCGA RNA-seq dataset)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper proposes a computational pipeline using GSVA to score metabolic subpathway activity across three independent GBM transcriptome datasets (total n=427 patients plus 39 normal controls), then applies limma differential analysis within each dataset and takes the intersection of significantly dysregulated subpathways (adjusted p<0.05) as the final GBM-associated set. Prognostic significance of a key gene (LSS) was assessed by univariate Cox regression and Kaplan-Meier survival analysis with a log-rank test in the TCGA cohort. In vitro validation of (S)-2,3-Epoxysqualene effects on U87-MG cells used CCK8, wound-healing, and flow cytometry assays, with between-group comparisons by unpaired Student's t-tests on triplicate experiments reported as mean±SD.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| limma moderated t-test (empirical Bayes) | Differential subpathway activity scores: GBM vs. normal in each of the three datasets (GSE4290, GSE16011, TCGA GBM) independently | GSE4290: 77 GBM + 23 normal; GSE16011: 159 GBM + 8 normal; TCGA GBM: 155 GBM + 5 normal | not stated |
| Log-rank test | Survival analysis comparing high-expression vs. low-expression LSS groups in TCGA GBM cohort (median split) | 155 GBM patients (TCGA) | not stated |
| Univariate Cox proportional-hazards model | Hazard ratio estimation for LSS gene expression in TCGA GBM survival analysis | 155 GBM patients (TCGA) | not stated |
| Unpaired Student's t-test | Cell viability (CCK8), wound-healing migration distance, cell cycle phase proportions, and apoptosis rates: (S)-2,3-Epoxysqualene-treated vs. DMSO control U87-MG cells | Triplicate experiments (exact per-comparison n not specified) | not stated |
| Kolmogorov-Smirnov-like random walk statistic (GSVA enrichment scoring) | Metabolic subpathway activity scoring for each sample across all three datasets | 466 samples total across three datasets | not stated |
| K-means clustering with WSS, average silhouette width, gap statistics, and CDF of consensus clustering | Determining optimal number of clusters for subpathway activity matrix (GSE4290 dataset) | 100 samples (GSE4290) | na |
-
Significantly dysregulated subpathways were identified by taking the intersection of independent per-dataset limma findings (adjusted p<0.05) across three datasets↳ Could also: A formal meta-analysis (e.g., Fisher's combined p-value method, or random-effects meta-analysis of limma log-fold-change estimates) pooling all three datasets could also be used — Meta-analytic pooling uses quantitative effect-size information from all datasets simultaneously, potentially increasing power to detect modest but consistent effects and providing a single ranked effect estimate with uncertainty quantification, rather than a binary present/absent intersection
-
Survival groups were defined by median split of LSS expression in the TCGA GBM cohort↳ Could also: An optimal cutpoint approach (e.g., maximally selected rank statistics via the survminer cutpoint function) or treating expression as a continuous term in the Cox model could also be used — Median split is straightforward and widely used; an optimal cutpoint identifies a biologically or clinically more meaningful threshold; treating expression as continuous avoids dichotomization entirely and retains full statistical information in the hazard model
-
Multiple unpaired Student's t-tests were applied across several in vitro endpoints (viability at multiple concentrations and time points, migration, cell cycle phase fractions, apoptosis fractions)↳ Could also: A two-way ANOVA (concentration × time) for the CCK8 dose-response data and one-way ANOVA with Dunnett's or Tukey HSD post-hoc tests for other endpoints could also be used — ANOVA with post-hoc correction controls the family-wise error rate across related comparisons; Dunnett's test is specifically designed for comparing multiple treatment conditions against a single control, and two-way ANOVA can model interaction between concentration and time
-
The standard limma pipeline (designed for microarray data) was applied to TCGA RNA-seq FPKM values after GSVA-based rank transformation↳ Could also: limma-voom (with mean-variance weighting of count-derived data), DESeq2, or edgeR could also be applied when the input is RNA-seq count data — limma-voom, DESeq2, and edgeR explicitly model the count-based (negative-binomial) distribution and overdispersion of RNA-seq data; the GSVA rank-transformation applied here partially addresses distributional concerns, and the paper notes the kcdf normalization, but describing the FPKM-to-GSVA-to-limma pipeline in detail would clarify the workflow
-
The univariate Cox model was used to assess the prognostic association of LSS expression and HR=1.5 was reported without a confidence interval↳ Could also: A multivariable Cox model adjusting for known GBM prognostic covariates (e.g., age, IDH status, MGMT methylation) and reporting the 95% CI for the HR could also be used — Multivariable adjustment assesses whether the LSS association with survival is independent of established clinical and molecular prognostic factors; reporting the CI conveys the precision of the HR estimate, which is particularly informative for a cohort of 155 patients where point estimates can be imprecise
-
Cluster validity was assessed using WSS, silhouette width, gap statistics, and consensus CDF curve, all applied to a single dataset (GSE4290)↳ Could also: External cluster validation by applying the cluster assignments derived from GSE4290 to the GSE16011 or TCGA subpathway matrices (e.g., via silhouette or prediction strength) could also be used — Internal validity indices (WSS, silhouette) measure cohesion and separation within the training dataset; external or cross-dataset validation would directly assess whether the cluster structure generalizes to independent cohorts, complementing the visual heatmap reproducibility check the paper presents
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-33072547 (crmSubpathway / GBM metabolic subpathways)
Paper: Han et al. 2020, Front Oncol 10:1549. "Inference of Subpathway Activity Profiles Reveals Metabolism Abnormal Subpathway Regions in Glioblastoma Multiforme." Tool/code: https://github.com/hanjunwei-lab/crmSubpathway (commit a590d8f). Paper data: GEO GSE4290 (Sun et al; GPL570 HG-U133 Plus 2.0; 77 GBM + 23 normal).
The crmSubpathway pipeline (from Methods + repo)
- Subpathway mining — 86 KEGG human metabolic pathway KGML files
(
inst/extdata/hsaMetspwXML.zip) → undirected enzyme network → k-clique (K=4) decomposition (k_clique()) → de-duplication (redup()) → list of metabolic subpathways. Shipped precomputed as data objectMetspwlist. - Activity matrix —
SubpathwayMatrix(Gexp, Metspwlist)runs GSVA (method="gsva",kcdf="Gaussian", min.sz=1, max.sz=Inf) → g-subpathway × n-sample enrichment matrix. Shipped precomputed (on demo data) asSpwmatrix. - Differential analysis —
CalculateDF(Spwmatrix, class, "cancer", "control")runs limma (lmFit → contrasts.fit(case-control) → eBayes → topTable) → per-subpathway logFC / P / adj.P.Val. Significant = adj.P.Val < 0.05. Shipped precomputed asDFspw. - Visualisation (
plotNetGraph,plotspwheatmap) — out of scope (figures only).
In scope (pipeline-derived, attempted)
- C1 — Re-run step 2
SubpathwayMatrix(Geneexp, Metspwlist)and check it reproduces the shippedSpwmatrix(determinism / tool 1:1). DemoGeneexpis the package's simulated 3000-gene × 40-sample matrix (man page: "Simulated gene expression data") — not GSE4290. - C2 — Re-run step 3
CalculateDF(Spwmatrix, …)and check it reproduces shippedDFspw(determinism / tool 1:1); report # significant subpathways in the demo. - C3 — Structural counts: # KEGG metabolic pathways (86), # metabolic subpathways
in
Metspwlist/ rows ofSpwmatrix(the "g" subpathways scored). - C4 (P16, paper data) — Apply the same tool + shipped
Metspwlistto the paper's actual data GSE4290 (GBM vs normal): build the subpathway activity matrix (GSVA) and runCalculateDF; count significant metabolic subpathways at adj.P.Val<0.05. Compare honestly to the paper's headline "25 GBM-related metabolic subpathways."
Out of scope / not attempted (the hard ~20%)
- The paper's "25" is explicitly the intersection of the significant subpathways across THREE datasets (GSE4290 + two others). We reproduce GSE4290 alone, so the raw GSE4290 count is expected to be larger than 25; we report it as-is rather than reverse-engineering the exact 3-dataset intersection.
- IDH-mutant subgroup analyses (Table S2: 13 / 7 subpathways), survival/HR (Fig 3D), and all wet-lab validation (CCK8 p=0.0002, cell-cycle, apoptosis — Figs 4–5): out of scope (manual/experimental, not pipeline-derived).
- Regenerating
Metspwlistfrom scratch viak_clique()on all 86 KGML files is feasible but redundant — the shippedMetspwlistis that output; we use it and separately sanity-check k_clique on the 2 demo XMLs.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The crmSubpathway pipeline reproduces exactly: shipped Spwmatrix (297×40, max|diff|=0) and DFspw (~1e-16, 82 sig) are bit-for-bit regenerable, and on the paper's own GSE4290 the cohort (77/23) and the highlighted sterol-biosynthesis down-regulation (00100_6, logFC=-0.546, adj.P=3.3e-6) reproduce cleanly. The one non-1:1 item is the abstract's '25 GBM-related subpathways', which is the intersection of three datasets; we ran GSE4290 alone (197 significant), so the larger count is the expected direction and the deviation is on our side (scope not reconstructed), not the authors'. No fabrication signal — all shipped objects are derivable and only cosmetic doc-drift exists. Overall: a solid reproduction whose central claim holds, with a single explainable, scope-limited deviation → yellow.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at [email protected].
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.