Enhancing chemotherapy response prediction via matched colorectal tumor-organoid gene expression analysis and network-based biomarker selection.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH TO RUN: yes for the consensus-WGCNA core (authors' own R/Rmd code @ TransBioInfoLab/Organoid-Prediction 1f0ba48 + public GEO data). VERDICT: PARTIAL, mixed 1:1. (1) The shipped consensus-WGCNA output (results/module_stats_tan_salmon.xlsx) is internally consistent with the paper EXACTLY: filtering |cons_kME|>=0.5 over the 124 tan+salmon genes yields exactly 35 hub genes, and all 7 Table-2 biomarker genes are within those 35 -> no internal fabrication in that artifact. (2) Re-running the consensus WGCNA from RAW GEO (GSE171680 tissue, GSE171681 organoid, GSE64392 organoid+IC50) in a fresh conda env reproduces the MODULE ARCHITECTURE within tolerance (module sizes within ~8%: turquoise 197 vs 214, blue 192 vs 190, brown 169 vs 157, yellow 149 vs 144, green 146 vs 141; 15 vs 16 modules; 19 5-FU samples EXACT; 15819 common genes vs code-comment 15836). (3) The headline tan/salmon tissue-organoid eigengene correlations (0.7/0.5) and the exact hub-gene/biomarker MEMBERSHIP do NOT reproduce 1:1 by label -- but WGCNA color labels are size-rank artifacts (not stable IDs), the strongest module correlation that does appear is 0.747 (~0.7 magnitude recovered), and 6/7 biomarker genes co-cluster into a single module (turquoise) in the fresh run, so biomarker co-modularity largely holds. NOT ATTEMPTED (hard ~20%): model training (RF/Ridge/Ensemble) and the 6-cohort survival validation (Table 3 log-rank p-values) -- requires 6 additional dataset-specific preprocessing pipelines AND the function get_top_genes_in_module() which is invoked but UNDEFINED anywhere in the repo (code gap); the paper reports no AUC/CV numbers in main text. FLAG: paper text cites organoid data as GSE64932 but the code uses GSE64392 (different accession) -- treated as a typo. No fabrication concluded; the gaps are WGCNA label-instability + version/annotation drift + unshipped intermediates + a missing function, not non-derivable numbers.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 78assessed: 2026-06-14 ⛓ cf36ba86abef
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-09-19
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetIntegrating matched colorectal tumor and organoid gene expression data via a consensus gene co-expression network approach can identify robust biomarkers that improve prediction of chemotherapy (5-FU-based) response compared to traditional single-source methods.
- ★ A consensus WGCNA approach combining matched tumor-organoid and independent organoid drug-response expression data identifies gene modules and hub genes predictive of 5-FU chemotherapy response method
- ★ The tan and salmon gene modules show significant eigengene concordance between matched organoid and tissue expression and are associated with survival outcomes finding
- ★ An ensemble model (random forest gene selection + ridge regression) trained on organoid hub genes outperforms ridge and random forest alone for predicting drug response and patient survival stratification finding
- ★ A 7-gene organoid-derived drug-response signature yields a patient-specific drug-resistance score that stratifies overall survival in independent validation cohorts resource
- ★ The matched tumor-organoid biomarker discovery approach demonstrates superior predictive accuracy over traditional methods when tested on independent datasets finding
- 35 hub genes from the significant modules were characterized functionally via KEGG, Reactome, and GO over-representation analysis resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Microarray gene expression + drug sensitivity assay (IC50) | Colorectal cancer organoids (GSE64932, n=19 tested with 5-FU) | 5-fluorouracil (5-FU) drug treatment | IC50 drug sensitivity values | — |
| RNA-seq gene expression | Matched patient colorectal tumor tissue (GSE171680, n=87) | none | Overall survival (OS) and recurrence-free survival (RFS) outcomes | — |
| RNA-seq gene expression | Matched colorectal tumor-derived organoids (GSE171681, n=87) | none | Overall survival (OS) and recurrence-free survival (RFS) outcomes, paired with tissue | — |
| Consensus Weighted Gene Co-expression Network Analysis (WGCNA) | Combined tumor tissue, matched organoid, and independent organoid datasets | none | Gene modules and hub genes (module membership, kME) | WGCNA R package (blockwiseConsensusModules) |
| Cox proportional hazards regression | Patient tissue expression (module eigengenes) | none | Association between module eigengenes and OS/RFS | — |
| Machine learning model training (ridge, random forest, ensemble) | Organoid gene expression hub genes | 5-FU (IC50 as response) | Predicted drug response / AUC / cross-validation error | glmnet, randomForestSRC (R packages) |
| Functional enrichment / over-representation analysis | 35 selected hub genes | none | Enriched KEGG, Reactome, GO pathways (FDR) | clusterProfiler R package, MSigDB |
| Kaplan-Meier survival analysis and log-rank test | Six independent validation cohorts (GSE39582, GSE17538, GSE106584, GSE72970, GSE87211, TCGA-COAD) | 5-FU-based chemotherapy | Overall survival stratified by drug-resistant score group | MaxStat R package for cut-point determination |
- – Tan module eigengenes highly correlated between matched organoid and tissue expression R²=0.7, p=2.2×10^-16
- – Salmon module eigengenes significantly correlated between matched organoid and tissue expression R²=0.5, p=1.4×10^-6
- – Ensemble model achieved the lowest 3-fold CV error and highest AUC with significant log-rank test results among the three models tested
- – Seven genes were selected by the optimal ensemble model as drug-response related genes for the patient-specific drug-resistance score 7 genes
- – 3637 common genes among the top 50% most variable across all three datasets were used for consensus WGCNA 3637 genes
- – 35 hub genes were selected from the significant modules for downstream functional enrichment analysis 35 genes
- correlation R²tan=0.7, p=2.2×10^-16 (Tan module eigengene concordance between paired organoid and tissue expression)
- correlation R²salmon=0.5, p=1.4×10^-6 (Salmon module eigengene concordance between paired organoid and tissue expression)
- pvalue <0.05 (Threshold for Cox regression coefficient significance (OS or RFS) used to select the three candidate modules)
- count 3637 (Common genes among top 50% most variable used for consensus WGCNA across three datasets)
- count 7 (Genes selected by the optimal ensemble model as drug-response related genes)
- count 35 (Hub genes selected from significant modules for functional enrichment analysis)
- other |MM| ≥ 0.5 (Consensus module membership threshold used to select hub genes)
- count 19 (Organoid samples from GSE64932 tested with 5-FU used for IC50 drug-response modeling)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The study applies Consensus Weighted Gene Co-expression Network Analysis (WGCNA) across three colorectal cancer datasets (two paired tumor-organoid RNA-seq sets, n=87, and one organoid microarray set with 5-FU IC50 values, n=19) to identify gene modules and hub genes associated with chemotherapy response. Module significance was evaluated by Cox proportional hazards regression against OS/RFS, and organoid-tissue concordance by Spearman correlation; an ensemble ridge/random-forest model was then trained on organoid IC50 data. Validation across six independent patient cohorts used Kaplan-Meier survival analysis and log-rank tests, with per-dataset risk-score cutpoints derived by the MaxStat method.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Cox proportional hazards regression | Association of module eigengenes with patient OS and RFS for module significance selection | 87 (GSE171680/GSE171681 paired datasets) | not stated |
| Spearman correlation | Concordance of module eigengenes between paired organoid and tissue expressions; also gene-IC50 and gene-paired expression association tests for comparison gene-selection approaches | 87 for paired eigengene concordance; 19 for gene-IC50 correlation (GSE64932) | not stated |
| Log-rank test | OS differences between derived drug-resistant and drug-sensitive groups in GSE171680 (model selection) and across six independent validation datasets | varies by dataset (5-FU-treated subsets: n=83 to n=203) | not stated |
| Hypergeometric distribution (over-representation analysis) | Pathway enrichment for 35 hub genes across KEGG, Reactome, and GO (MSigDB) | — | not stated |
| 3-fold cross-validation (repeated 100 times), MSE minimization | Model selection among ridge, random forest, and ensemble models trained on organoid IC50 data | 19 (GSE64932 organoid samples with 5-FU IC50) | na |
| AUC (binary classification) | Discrimination of each candidate model on GSE171680 using OS dichotomized as binary outcome | 87 (GSE171680) | na |
-
The optimal risk-score cutpoint was determined by the MaxStat (maximally selected rank statistic) method applied separately within each validation dataset↳ Could also: Apply a single cutpoint fixed from the training set (or a pre-specified rule such as the median) uniformly across all validation datasets — Deriving the cutpoint within each test set introduces an optimization step at evaluation time; a fixed pre-specified cutpoint removes this and provides a more straightforward estimate of out-of-sample performance
-
Module significance was assessed by Cox regression at P<0.05 across multiple modules, with no multiple-testing correction at the module-selection step↳ Could also: Apply Benjamini-Hochberg FDR or Bonferroni adjustment across the number of modules tested — Adjusting for the number of modules screened would control the expected proportion of false-positive module selections, a common practice when simultaneously evaluating many candidate units
-
3-fold cross-validation was used for model selection with n=19 organoid training samples↳ Could also: Use leave-one-out cross-validation (LOOCV) or 5-fold CV for this sample size — With very small n, LOOCV or larger-fold CV tends to produce less biased MSE estimates than 3-fold CV; the 100-repeat strategy mitigates variance but does not reduce systematic bias from small fold size
-
Model discrimination was summarized as AUC using OS converted to a binary outcome in GSE171680↳ Could also: Report time-dependent AUC (e.g., at 1, 3, and 5 years) or Harrell's C-statistic — Dichotomizing a time-to-event outcome discards censoring information; time-dependent AUC and C-statistic are designed for survival data and make full use of follow-up time, providing a more appropriate discrimination measure
-
Concordance between paired organoid and tissue module eigengenes was quantified by Spearman correlation (reported as R²)↳ Could also: Additionally report the intraclass correlation coefficient (ICC) or a Bland-Altman analysis — Spearman correlation captures monotonic agreement but not absolute agreement or systematic bias; ICC and Bland-Altman analysis distinguish random disagreement from systematic offset in paired measurements
-
Pathway enrichment used over-representation analysis (hypergeometric test) on the 35 selected hub genes as a binary gene list↳ Could also: Apply Gene Set Enrichment Analysis (GSEA) using all module genes ranked by their consensus module membership score — ORA requires a hard membership threshold and may miss moderate but consistent signals distributed across many genes; GSEA uses the full ranked list and can detect enrichment even when no single gene crosses a fixed cutoff
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
Downstream reach in the literature
198 downstream papers · 2 datasetsHow widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.
- Experimentally derived metastasis gene expression pr... 2010 · 600 cites
- Single-cell dissection of transcriptional heterogene... 2011 · 547 cites
- Characterization of the immunophenotypes and antigen... 2015 · 448 cites
- <i>Fusobacterium nucleatum</i> promotes colorectal c... 2019 · 414 cites
- CDX2 as a Prognostic Biomarker in Stage II and Stage... 2016 · 370 cites
- FXR Regulates Intestinal Cancer Stem Cell Proliferat... 2019 · 364 cites
- Colorectal cancer susceptibility loci as predictive... 2018 · 101 cites
- CircIL4R activates the PI3K/AKT signaling pathway vi... 2021 · 98 cites
- The aging-related risk signature in colorectal cance... 2021 · 92 cites
- MYH9 Promotes Growth and Metastasis via Activation o... 2019 · 73 cites
- MYH9-dependent polarization of ATG9B promotes colore... 2021 · 67 cites
- FGF19-Induced Inflammatory CAF Promoted Neutrophil E... 2023 · 57 cites
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-39754813
Paper: Zhang et al. 2025, Transl Oncol — "Enhancing chemotherapy response
prediction via matched colorectal tumor-organoid gene expression analysis and
network-based biomarker selection." DOI 10.1016/j.tranon.2024.102238.
Repo: https://github.com/TransBioInfoLab/Organoid-Prediction @ 1f0ba48
(authors' own code; R/Rmarkdown).
Pipeline (what the paper computes)
- Consensus WGCNA across 3 expression sets — GSE171680 (CRC tissue, 87 pts),
GSE171681 (matched organoids, 87 pts), GSE64392 (organoid biobank w/ drug IC50,
van de Wetering 2015).
blockwiseConsensusModules(power=12, signed, minModuleSize=30). → 16 modules (incl. grey); two key modules tan & salmon chosen by the tissue↔organoid eigengene correlation. - Hub-gene selection — genes in tan+salmon with |consensus kME (MM)| ≥ 0.5 → 35 hub genes.
- Organoid drug-response models (RF / Ridge / Ensemble) trained on GSE64392 5-FU mIC50 using hub genes → 7-gene ensemble biomarker (Table 2).
- Survival validation of the score on 6 independent CRC cohorts (GSE39582, GSE17538, TCGA-COAD, GSE106584, GSE72970, GSE87211) → log-rank p-values (Table 3).
In scope (attempted)
- Module structure: # modules, module sizes (turquoise 214 / blue 190 / brown 157 / yellow 144 / green 141). Regenerated by rerunning consensus WGCNA on raw GEO data (SLURM 2175604).
- tan/salmon tissue↔organoid eigengene correlation (R²_tan=0.7, R²_salmon=0.5).
- 35 hub genes (|MM|≥0.5) and 7-gene biomarker membership — verified
directly against the shipped
results/module_stats_tan_salmon.xlsx, and recomputed from the fresh WGCNA run.
Out of scope / not attempted (the hard ~20%)
- Steps 3–4 (model training + 6-cohort survival validation, Table 3 p-values).
Reasons: (a) requires preprocessing 6 additional datasets, each with its own
bespoke clinical/drug parsing; (b) the function
get_top_genes_in_module()invoked inWGCNA_main.Rmd/organoid_model.Rmdis not defined anywhere in the repo (code gap); (c) several inputs (TCGA-COAD drug table, GSE171682 clinical supplement) needed for survival are not the low-hanging, clearly pinnable headline numbers. The paper reports only log-rank p-values (no AUC/CV numbers in the main text), and those are downstream of the full model. - Wet-lab / organoid culture / IC50 measurement — not computational.
Notable data-citation discrepancy (flag)
Paper main text cites the organoid-IC50 dataset as GSE64932 (and training tissue/organoid as GSE171680/GSE171681). The authors' own code downloads GSE64392 (van de Wetering "Living Organoid Biobank", GPL16686) and uses GSE171680/GSE171681. GSE64932 is a different accession. Treated here as a typo in the paper; reproduction uses the accession the code actually uses (GSE64392).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The consensus-WGCNA core reproduces well: from the shipped module_stats_tan_salmon.xlsx, |cons_kME|>=0.5 yields exactly 35 hub genes with all 7 Table-2 biomarkers as a clean subset, and a from-raw rerun recovers module sizes within ~8% and the 19 5-FU samples exactly. Residual deviations are on the technical/our-method side — WGCNA size-rank label instability (tan-by-label 0.39 vs 0.7, but 0.747 reappears under another label), R^2-vs-Spearman metric drift, annotation version drift (15819 vs 15836 genes), and a re-implemented missing function — not non-derivable or fabricated numbers. There are minor authors'-side defects (an undefined get_top_genes_in_module() and a GSE64932/GSE64392 citation typo) but they don't undermine the shared values. The predictive/survival core (Table 3) was not attempted, so the overall conclusion is solid-but-limited rather than fully confirmed.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at [email protected].
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.