Inference of Subpathway Activity Profiles Reveals Metabolism Abnormal Subpathway Regions in Glioblastoma Multiforme.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values are derivable from the shared data
- ✓The central claim held under reproduction
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡The deviation was non-trivial in magnitude
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce. The crmSubpathway tool's documented pipeline (KEGG metabolic KGML -> k-clique K=4 -> GSVA Gaussian activity matrix -> limma diff -> adj.P<0.05) is BIT-FOR-BIT deterministic: re-running SubpathwayMatrix rebuilt the shipped Spwmatrix with max abs diff 0 (C1) and CalculateDF rebuilt DFspw to machine epsilon ~1e-16 (C2), using era-matched GSVA 1.36.0 / R 4.0.5 on «our HPC». Applying the same tool + shipped Metspwlist to the paper's OWN data GSE4290 (P16) reproduced the exact cohort 77 GBM + 23 normal (C4a) and the paper's specific highlighted finding that sterol-biosynthesis subpathway 00100_6 is down-regulated in GBM (logFC -0.546, adj.P 3.3e-6; all 00100_* down) (C4c). The headline '25 GBM-related subpathways' is NOT 1:1 (C4b): on GSE4290 alone we get 197 significant subpathways, because the paper's 25 is the intersection across THREE datasets - a stricter filter, and the larger single-dataset count is the expected direction. NOT attempted (the ~20%): reconstructing the exact 3-dataset intersection, IDH subgroup analyses (Table S2), survival/HR, and all wet-lab validation (Figs 4-5). No fabrication signal; only cosmetic doc-drift (man page 3000 vs 2564 genes; min.sz doc default 10 vs code 1).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 89assessed: 2026-06-15 ⛓ ea779d3fbf82
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator headless) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusThe authors hypothesize that constructing a metabolic subpathway activity score matrix from gene expression data can accurately identify abnormal metabolic subpathway regions associated with glioblastoma multiforme (GBM), and that targeting a key node ((S)-2,3-Epoxysqualene) in an identified subpathway will affect GBM cell behavior.
- ★ A metabolic subpathway activity score matrix method combining the k-clique algorithm and GSVA can accurately identify disease-related metabolic abnormal subpathways in GBM method
- ★ 25 metabolic subpathways were identified as significantly abnormal (GBM-related) across three independent datasets finding
- ★ (S)-2,3-Epoxysqualene, located at the central region of the sterol biosynthesis subpathway (00100_6) and substrate of LSS, may have high association with GBM and regulate the whole pathway mechanism
- ★ (S)-2,3-Epoxysqualene inhibits U87-MG cell activity, arrests the cell cycle in G0/G1 phase, and induces apoptosis finding
- ★ Gene LSS (lanosterol synthase) in metabolic subpathway 00100_6 significantly impacts GBM patient survival and is a potential therapeutic target finding
- ★ The R package crmSubpathway was developed to perform disease-related metabolic subpathway identification and is freely available on GitHub resource
- The 25 GBM-related metabolic subpathways cluster into two robust classes (first class up-regulated, second class down-regulated) reproducible across datasets finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Gene expression profiling by microarray (computational analysis) | GBM patient tissue (GSE4290: 77 GBM, 23 normal) | none | metabolic subpathway activity scores / differential subpathway activity | DNA microarray (GEO GSE4290) |
| Gene expression profiling by microarray (computational analysis) | GBM patient tissue (GSE16011: 159 GBM, 8 normal) | none | metabolic subpathway activity scores / differential subpathway activity | DNA microarray (GEO GSE16011) |
| Gene expression profiling by RNA-seq (computational analysis) | GBM patient tissue (TCGA-GBM: 155 GBM, 5 normal) | none | metabolic subpathway activity scores / differential subpathway activity | RNA-seq (GDC TCGA, htseq_fpkm) |
| Survival analysis (Kaplan-Meier, Cox proportional-hazards) | TCGA GBM patients with clinical data | none (stratified by median LSS gene expression) | hazard ratio and survival probability | — |
| Cell viability assay (CCK8) | U87-MG human glioblastoma cell line | (S)-2,3-Epoxysqualene treatment (various concentrations, 2-12 h) | optical absorbance at 450 nm (cell viability) | Cell Counting Kit 8 (Dojindo) |
| Wound-healing motility assay | U87-MG human glioblastoma cell line | 100 nmol/L (S)-2,3-Epoxysqualene | migration distance over 12 h | phase-contrast microscope; ImageJ |
| Flow cytometry cell cycle analysis (PI staining) | U87-MG human glioblastoma cell line | (S)-2,3-Epoxysqualene treatment | cell cycle phase distribution (DNA content) | FACSCalibur (BD); ModFit LT 3.2 |
| Flow cytometry apoptosis analysis (Annexin V-FITC/PI) | U87-MG human glioblastoma cell line | (S)-2,3-Epoxysqualene treatment | apoptotic cell fraction | FACSCalibur (BD) |
- – 25 significant metabolic subpathways identified as GBM-related (intersection across three datasets) 25 subpathways
- ▲ LSS gene in subpathway 00100_6 significantly associated with GBM patient prognosis HR = 1.5
- ▼ (S)-2,3-Epoxysqualene inhibits U87-MG cell activity (CCK8)
- – (S)-2,3-Epoxysqualene arrests U87-MG cell cycle in G0/G1 phase
- ▲ (S)-2,3-Epoxysqualene induces U87-MG cell apoptosis
- ▼ (S)-2,3-Epoxysqualene inhibits migration of GBM cells (wound-healing)
- – First class subpathways up-regulated and second class subpathways down-regulated in GBM, robust across datasets
- other HR = 1.5 (hazard ratio for LSS gene (subpathway 00100_6) in TCGA GBM survival analysis)
- count 25 (number of GBM-related metabolic subpathways identified)
- count 427 patients (total patients across three independent GBM datasets)
- count GSE4290: 77 GBM / 23 normal (microarray dataset sample sizes)
- count GSE16011: 159 GBM / 8 normal (microarray dataset sample sizes)
- count TCGA GBM: 155 GBM / 5 normal (RNA-seq dataset sample sizes)
- other adjusted p-value < 0.05 (significance threshold for differential metabolic subpathways (limma))
- other GBM represents 81% of malignant brain/CNS tumors; 65% of all gliomas (epidemiological background on GBM incidence)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper proposes a computational pipeline using GSVA to score metabolic subpathway activity across three independent GBM transcriptome datasets (total n=427 patients plus 39 normal controls), then applies limma differential analysis within each dataset and takes the intersection of significantly dysregulated subpathways (adjusted p<0.05) as the final GBM-associated set. Prognostic significance of a key gene (LSS) was assessed by univariate Cox regression and Kaplan-Meier survival analysis with a log-rank test in the TCGA cohort. In vitro validation of (S)-2,3-Epoxysqualene effects on U87-MG cells used CCK8, wound-healing, and flow cytometry assays, with between-group comparisons by unpaired Student's t-tests on triplicate experiments reported as mean±SD.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| limma moderated t-test (empirical Bayes) | Differential subpathway activity scores: GBM vs. normal in each of the three datasets (GSE4290, GSE16011, TCGA GBM) independently | GSE4290: 77 GBM + 23 normal; GSE16011: 159 GBM + 8 normal; TCGA GBM: 155 GBM + 5 normal | not stated |
| Log-rank test | Survival analysis comparing high-expression vs. low-expression LSS groups in TCGA GBM cohort (median split) | 155 GBM patients (TCGA) | not stated |
| Univariate Cox proportional-hazards model | Hazard ratio estimation for LSS gene expression in TCGA GBM survival analysis | 155 GBM patients (TCGA) | not stated |
| Unpaired Student's t-test | Cell viability (CCK8), wound-healing migration distance, cell cycle phase proportions, and apoptosis rates: (S)-2,3-Epoxysqualene-treated vs. DMSO control U87-MG cells | Triplicate experiments (exact per-comparison n not specified) | not stated |
| Kolmogorov-Smirnov-like random walk statistic (GSVA enrichment scoring) | Metabolic subpathway activity scoring for each sample across all three datasets | 466 samples total across three datasets | not stated |
| K-means clustering with WSS, average silhouette width, gap statistics, and CDF of consensus clustering | Determining optimal number of clusters for subpathway activity matrix (GSE4290 dataset) | 100 samples (GSE4290) | na |
-
Significantly dysregulated subpathways were identified by taking the intersection of independent per-dataset limma findings (adjusted p<0.05) across three datasets↳ Could also: A formal meta-analysis (e.g., Fisher's combined p-value method, or random-effects meta-analysis of limma log-fold-change estimates) pooling all three datasets could also be used — Meta-analytic pooling uses quantitative effect-size information from all datasets simultaneously, potentially increasing power to detect modest but consistent effects and providing a single ranked effect estimate with uncertainty quantification, rather than a binary present/absent intersection
-
Survival groups were defined by median split of LSS expression in the TCGA GBM cohort↳ Could also: An optimal cutpoint approach (e.g., maximally selected rank statistics via the survminer cutpoint function) or treating expression as a continuous term in the Cox model could also be used — Median split is straightforward and widely used; an optimal cutpoint identifies a biologically or clinically more meaningful threshold; treating expression as continuous avoids dichotomization entirely and retains full statistical information in the hazard model
-
Multiple unpaired Student's t-tests were applied across several in vitro endpoints (viability at multiple concentrations and time points, migration, cell cycle phase fractions, apoptosis fractions)↳ Could also: A two-way ANOVA (concentration × time) for the CCK8 dose-response data and one-way ANOVA with Dunnett's or Tukey HSD post-hoc tests for other endpoints could also be used — ANOVA with post-hoc correction controls the family-wise error rate across related comparisons; Dunnett's test is specifically designed for comparing multiple treatment conditions against a single control, and two-way ANOVA can model interaction between concentration and time
-
The standard limma pipeline (designed for microarray data) was applied to TCGA RNA-seq FPKM values after GSVA-based rank transformation↳ Could also: limma-voom (with mean-variance weighting of count-derived data), DESeq2, or edgeR could also be applied when the input is RNA-seq count data — limma-voom, DESeq2, and edgeR explicitly model the count-based (negative-binomial) distribution and overdispersion of RNA-seq data; the GSVA rank-transformation applied here partially addresses distributional concerns, and the paper notes the kcdf normalization, but describing the FPKM-to-GSVA-to-limma pipeline in detail would clarify the workflow
-
The univariate Cox model was used to assess the prognostic association of LSS expression and HR=1.5 was reported without a confidence interval↳ Could also: A multivariable Cox model adjusting for known GBM prognostic covariates (e.g., age, IDH status, MGMT methylation) and reporting the 95% CI for the HR could also be used — Multivariable adjustment assesses whether the LSS association with survival is independent of established clinical and molecular prognostic factors; reporting the CI conveys the precision of the HR estimate, which is particularly informative for a cohort of 155 patients where point estimates can be imprecise
-
Cluster validity was assessed using WSS, silhouette width, gap statistics, and consensus CDF curve, all applied to a single dataset (GSE4290)↳ Could also: External cluster validation by applying the cluster assignments derived from GSE4290 to the GSE16011 or TCGA subpathway matrices (e.g., via silhouette or prediction strength) could also be used — Internal validity indices (WSS, silhouette) measure cohesion and separation within the training dataset; external or cross-dataset validation would directly assess whether the cluster structure generalizes to independent cohorts, complementing the visual heatmap reproducibility check the paper presents
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-33072547 (crmSubpathway / GBM metabolic subpathways)
Paper: Han et al. 2020, Front Oncol 10:1549. "Inference of Subpathway Activity Profiles Reveals Metabolism Abnormal Subpathway Regions in Glioblastoma Multiforme." Tool/code: https://github.com/hanjunwei-lab/crmSubpathway (commit a590d8f). Paper data: GEO GSE4290 (Sun et al; GPL570 HG-U133 Plus 2.0; 77 GBM + 23 normal).
The crmSubpathway pipeline (from Methods + repo)
- Subpathway mining — 86 KEGG human metabolic pathway KGML files
(
inst/extdata/hsaMetspwXML.zip) → undirected enzyme network → k-clique (K=4) decomposition (k_clique()) → de-duplication (redup()) → list of metabolic subpathways. Shipped precomputed as data objectMetspwlist. - Activity matrix —
SubpathwayMatrix(Gexp, Metspwlist)runs GSVA (method="gsva",kcdf="Gaussian", min.sz=1, max.sz=Inf) → g-subpathway × n-sample enrichment matrix. Shipped precomputed (on demo data) asSpwmatrix. - Differential analysis —
CalculateDF(Spwmatrix, class, "cancer", "control")runs limma (lmFit → contrasts.fit(case-control) → eBayes → topTable) → per-subpathway logFC / P / adj.P.Val. Significant = adj.P.Val < 0.05. Shipped precomputed asDFspw. - Visualisation (
plotNetGraph,plotspwheatmap) — out of scope (figures only).
In scope (pipeline-derived, attempted)
- C1 — Re-run step 2
SubpathwayMatrix(Geneexp, Metspwlist)and check it reproduces the shippedSpwmatrix(determinism / tool 1:1). DemoGeneexpis the package's simulated 3000-gene × 40-sample matrix (man page: "Simulated gene expression data") — not GSE4290. - C2 — Re-run step 3
CalculateDF(Spwmatrix, …)and check it reproduces shippedDFspw(determinism / tool 1:1); report # significant subpathways in the demo. - C3 — Structural counts: # KEGG metabolic pathways (86), # metabolic subpathways
in
Metspwlist/ rows ofSpwmatrix(the "g" subpathways scored). - C4 (P16, paper data) — Apply the same tool + shipped
Metspwlistto the paper's actual data GSE4290 (GBM vs normal): build the subpathway activity matrix (GSVA) and runCalculateDF; count significant metabolic subpathways at adj.P.Val<0.05. Compare honestly to the paper's headline "25 GBM-related metabolic subpathways."
Out of scope / not attempted (the hard ~20%)
- The paper's "25" is explicitly the intersection of the significant subpathways across THREE datasets (GSE4290 + two others). We reproduce GSE4290 alone, so the raw GSE4290 count is expected to be larger than 25; we report it as-is rather than reverse-engineering the exact 3-dataset intersection.
- IDH-mutant subgroup analyses (Table S2: 13 / 7 subpathways), survival/HR (Fig 3D), and all wet-lab validation (CCK8 p=0.0002, cell-cycle, apoptosis — Figs 4–5): out of scope (manual/experimental, not pipeline-derived).
- Regenerating
Metspwlistfrom scratch viak_clique()on all 86 KGML files is feasible but redundant — the shippedMetspwlistis that output; we use it and separately sanity-check k_clique on the 2 demo XMLs.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The crmSubpathway pipeline reproduces exactly: shipped Spwmatrix (297×40, max|diff|=0) and DFspw (~1e-16, 82 sig) are bit-for-bit regenerable, and on the paper's own GSE4290 the cohort (77/23) and the highlighted sterol-biosynthesis down-regulation (00100_6, logFC=-0.546, adj.P=3.3e-6) reproduce cleanly. The one non-1:1 item is the abstract's '25 GBM-related subpathways', which is the intersection of three datasets; we ran GSE4290 alone (197 significant), so the larger count is the expected direction and the deviation is on our side (scope not reconstructed), not the authors'. No fabrication signal — all shipped objects are derivable and only cosmetic doc-drift exists. Overall: a solid reproduction whose central claim holds, with a single explainable, scope-limited deviation → yellow.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.