Artificial Intelligence Meets Whole Slide Images: Deep Learning Model Shapes an Immune-Hot Tumor and Guides Precision Therapy in Bladder Cancer.
The main result did not reproduce in this reproduction attempt. Where our recomputation produced values that differ from the published ones, those discrepancies are listed below. This is a single automated attempt — not peer review and not a finding of error or misconduct — and differences can also arise from data access, undocumented parameters or the computing environment. The verdict can be contested via “report an error”.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🔴Could not use the authors’ exact input data
- 🔴Reported values were only indirectly comparable
- 🔴A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🔴Reported values were not (fully) derivable from the shared data
- 🔴The deviation was non-trivial in magnitude
- 🔴The central claim did not (fully) hold under reproduction
- 🔴Overall, the reproduction showed a material discrepancy
▸Reproduction agent’s raw note
DROP (docs_insufficient; secondary no_code). Not described well enough to reproduce, and not 1:1. The repo (github.com/YihengJiang0912/AI-meets-WSI @ 1e3c9224) is exactly ONE file ('R script', 17 KB) -- no README, no data, no environment, 1 branch, 0 releases (verified via GitHub trees API, recursive). That script is downstream-bioinformatics boilerplate: each section load()s a preprocessed input (pre_Survival/Diff/ROC/GSVA/mutation/Lasso/Cox/pRRophetic.rda = 8 files) and reads external files (TCGA.BLCA.maf, GISTIC outputs, CIBERSORT sig_matrix+mixture_file). NONE of these inputs are shipped -- not in the repo, not in the 9.2 MB supplementary (figures+tables only), not on GEO. It errors on its first line load('pre_Survival.rda'). The paper's central contribution -- the whole-slide-image deep-learning clustering model (acc 92.38%, AUC 0.99) and the survival CNN (AUC 0.95) -- has NO code or weights anywhere. GSE32894 is public and resolves, but every reported number is conditioned on the DL-derived patient grouping (clusters C0/C1/C2, AI-score high/low) for which there is no code path from raw public data, so NO reported value is regenerable and NO clean 1:1 data point exists even via a third-party tool on the paper's own data. Decision made at the control plane (no «our HPC» compute spent: running a script that load()s 8 nonexistent files is a deterministic failure -- not worth burning a SLURM job). Data-availability statement defers to 'contact the corresponding authors'. NOT ATTEMPTED: the DL models (no_code) and every R-script section (inputs absent). Fabrication: reported numbers are not independently verifiable from released artifacts; this is flagged for the human reviewer, not asserted.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessmentassessed: 2026-06-15 ⛓ cd04219a98b7
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a deep learning model applied to whole slide H&E histopathology images construct a clinically meaningful molecular subtype (WSI cluster) and a quantitative AI score that predict prognosis, depict the tumor microenvironment immunophenotype, and guide precision therapy in bladder cancer?
- ★ A deep learning WSI cluster (three-class mini batch K-means on Inception V3 features) is associated with overall survival (P<0.001) and is an independent prognostic predictor (P=0.031) in BLCA. finding
- ★ A neural-network rerecognition of TME features combined with clinical features yields a robust 3-year survival prediction model (accuracy 86%, AUC 0.95). method
- ★ An AI score quantifies the WSI cluster logic and predicts classic BLCA molecular subtypes (AUC=0.838). resource
- ★ High AI score shapes an immune-hot tumor microenvironment, supporting use of immune checkpoint blockade, chemotherapy, and ERBB therapy for WSI cluster1 (high AI score) patients. mechanism
- ★ Deep learning can predict prognosis and aid precision medicine for BLCA directly from H&E histology, more economically than RNA-Seq. finding
- The three WSI subtypes (C0/C1/C2) differ in immune infiltration, mutation profiles, and enriched pathways (e.g., C2 enriched in T cell activation/cytokine signaling; C0-C2 differ in ECM organization and PI3K-Akt). finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Deep learning feature extraction and clustering of whole slide images (WSI) | TCGA-BLCA MIBC patients (363 patients, 435 H&E WSI) | none | Inception V3 2048-dim features clustered (mini batch K-means/hierarchical/GMM) into survival-associated WSI clusters | openslide, ImageNet-pretrained Inception V3 |
| Transfer-learning image classification with Grad-CAM visualization | TCGA-BLCA WSI (three WSI clusters as labels) | none | classification accuracy, AUC, F1, specificity, sensitivity, precision; class activation maps | Inception V3, SGD optimizer (lr 0.001), cross-entropy loss, 6-fold CV |
| Neural-network rerecognition of TME features / BPNN survival prediction | BLCA cohort (12,000 labeled image blocks; 120 cases for prognosis) | none | TME category classification and 3-year survival prediction (accuracy, AUC, specificity, sensitivity) | ImageNet-pretrained neural networks, BPNN |
| Bulk RNA-seq based DEG and functional enrichment analysis (GO/KEGG) | TCGA-BLCA and external GEO cohorts | none | differentially expressed genes, enriched GO/KEGG pathways | DESeq2, clusterProfiler |
| AI score construction (Cox/LASSO + PCA on prognostic DEGs) | TCGA cohort and external testing cohorts (GSE32894, GSE13507, IMvigor210, E-MTAB-4321, GSE135222, GSE100797, Nathanson2017pre) | none | AI score (PC1 sum) and median-based high/low stratification; survival | — |
| Immune/TME deconvolution and scoring | TCGA-BLCA gene expression profiles | none | IPS, ESTIMATE stromal/immune scores, CIBERSORT/xCell/MCP-counter cell fractions, ssGSEA cancer-immunity cycle | ESTIMATE, CIBERSORT, MCP-counter, xCell R packages |
| Molecular subtype classification and treatment-response signature scoring | TCGA-BLCA and external cohorts | none | CIT/Lund/MDA/TCGA/Baylor/UNC/Consensus subtypes; ERBB2/ERCC2/ATM/RB1/FANCC mutation proportions; atezolizumab response signatures | ConsensusMIBC, BLCAsubtyping R packages |
| Somatic mutation frequency analysis | TCGA-BLCA WSI clusters | none | frequencies of 13 significantly mutated genes per cluster (e.g., KDM6A, ARID1A, FGFR3, PIK3CA, RB1) | permutation test |
- – WSI cluster associated with overall survival P<0.001
- – WSI cluster is an independent prognostic predictor by multivariate Cox P=0.031
- – Three-class mini batch K-means WSI cluster had best prognostic separation P=0.005
- – Inception V3 transfer-learning classifier distinguished the three WSI clusters accuracy 92.38%, AUC 0.99
- – 3-year survival prediction model combining image, TME, and clinical features AUC=0.95, accuracy 86%, specificity 0.8353, sensitivity 0.8923
- – AI score predicts classic BLCA molecular subtypes AUC=0.838
- – C1 showed higher effector cell scores and lower suppressor cell scores (immune-hot) effector P=0.0066; suppressor P=0.0017
- ▲ C1 showed higher stromal and immune scores by ESTIMATE stromal P<0.001; immune P=0.0091
- pvalue P<0.001 (WSI cluster association with survival)
- pvalue P=0.031 (WSI cluster as independent prognostic predictor)
- pvalue P=0.005 (three-class mini batch K-means best OS prognosis)
- other AUC=0.99, accuracy 92.38% (WSI cluster classification (6-fold CV))
- other AUC=0.95, accuracy 86.00% (3-year survival prediction model)
- other AUC=0.838 (AI score predicting classic molecular subtypes)
- count 3,792 DEGs (3,198 up, 544 down) (high AI score (n=181) vs low AI score (n=181), |log2FC|>1 & FDR<0.05)
- count KDM6A 35%, ARID1A 31%, FGFR3 21% (C0); PIK3CA 25%, RB1 21% (C1) (mutation enrichment by WSI cluster)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper constructs an unsupervised deep learning WSI cluster (mini-batch K-means applied to 2048-dimensional Inception V3 features) from TCGA BLCA histopathology slides and validates it across multiple external cohorts. Survival differences across clusters were assessed by Kaplan-Meier and multivariate Cox regression; differential gene expression was quantified with DESeq2 followed by GO/KEGG enrichment; immune cell infiltration was estimated with ESTIMATE, CIBERSORT, MCP-counter, and Xcell. A continuous AI score was derived from the first principal component of prognostic DEGs selected by univariate Cox and LASSO regression, with discriminative performance evaluated by time-dependent ROC/AUC.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Kaplan-Meier survival analysis with implied log-rank test | Overall survival comparison across WSI clusters (C0, C1, C2) and high vs. low AI score groups; cluster selection among 70 candidate solutions | 363 TCGA-BLCA patients (training); external cohorts evaluated separately | not stated |
| Multivariate Cox proportional hazards regression | Independent prognostic value of WSI cluster for overall survival | null | not stated |
| Univariate Cox regression | Initial selection of prognostic DEGs for AI score construction | null | not stated |
| LASSO regression | Further penalized selection of prognostic AI gene signature from univariate Cox-selected DEGs | null | not stated |
| DESeq2 Wald test (negative binomial GLM) | Differential gene expression between WSI cluster subgroups and between high vs. low AI score groups; FDR-adjusted P < 0.05 and |log2FC| > 1 threshold applied | 181 vs. 181 for AI score comparison; WSI cluster contrast n not stated per pair | not stated |
| Student's t-test | Continuous and ordered categorical variable comparisons across groups | null | not stated |
| Kruskal-Wallis test | Immune cell infiltration population comparisons across WSI clusters; continuous and ordered categorical variables | null | not stated |
| Wilcoxon rank-sum test | Continuous and ordered categorical variable comparisons | null | not stated |
| Pearson chi-squared test | Disordered categorical variable comparisons | null | not stated |
| Fisher exact test | Disordered categorical variable comparisons (small expected counts) | null | not stated |
| Permutation test | Mutation frequency comparisons between WSI clusters | null | not stated |
| Pearson and Spearman correlation | Correlation matrices (specific variable pairs not detailed in main text) | null | not stated |
| Time-dependent ROC / AUC (timeROC) and standard ROC (pROC) | Predictive performance of AI score (AUC = 0.838) and 3-year survival model (AUC = 0.95); AI score vs. classic molecular subtypes | null | na |
| ssGSEA | Cancer-immunity cycle step scores per WSI cluster and AI cluster | null | na |
-
The optimal clustering solution (3-class mini-batch K-means) was selected by comparing survival curves across 70 candidate solutions spanning 2–5 groups and three algorithms, without a pre-specified stability or internal-validity criterion↳ Could also: Consensus clustering (e.g., ConsensusClusterPlus) with cophenetic correlation, silhouette width, or gap statistic could also be used to determine the number of clusters prior to survival assessment — A pre-specified stability-based selection criterion reduces the probability that the chosen solution reflects a chance survival separation in one cohort, and makes the cluster-count decision reproducible and independent of the outcome
-
High and low AI score groups were defined by splitting at the median, converting a continuous predictor into a binary one for Kaplan-Meier analysis↳ Could also: A continuous Cox proportional hazards model or a restricted cubic spline Cox model could also characterize the survival-AI score relationship without dichotomization — Median dichotomization discards information about the shape of the dose-response relationship, reduces statistical power, and the chosen cutpoint may not be optimal or reproducible across cohorts; a continuous model retains full information
-
The AI score was derived as the sum of PC1 loadings from PCA applied to LASSO- and univariate Cox-selected DEGs↳ Could also: A penalized Cox-derived linear risk score (LASSO-Cox or elastic-net-Cox with cross-validated lambda) could also produce a continuous prognostic index directly optimized on survival — A Cox-penalized score directly optimizes the partial likelihood for survival rather than variance explained, and cross-validated coefficients provide an internal estimate of out-of-sample performance on the training data
-
Multiple pairwise immune cell infiltration comparisons across three WSI clusters used Kruskal-Wallis tests; a post-hoc correction method for the follow-up pairwise tests is not stated↳ Could also: Dunn's test or pairwise Wilcoxon tests with Holm or Bonferroni correction could also serve as a named post-hoc procedure following a significant Kruskal-Wallis omnibus test — Without a stated post-hoc method, the family-wise error rate across pairwise comparisons for the many immune cell populations tested is not controlled; a named procedure makes the false-positive rate transparent to readers
-
Cox regression results are reported as P values only; hazard ratios with confidence intervals are not presented in the main text↳ Could also: Reporting hazard ratios with 95% confidence intervals for each covariate in the Cox models is also standard (e.g., per REMARK and TRIPOD reporting guidelines) — Hazard ratios with CIs communicate both the direction and magnitude of association and their precision, enabling readers to assess clinical relevance and compare findings with other published cohorts
-
The 3-year survival neural network was evaluated on 120 cases retained after excluding those with missing clinical data, survival < 3 years, or block prediction accuracy < 80%↳ Could also: Reporting model performance on all eligible cases alongside the filtered subset (as a sensitivity analysis) could also characterize how the selection criteria affect performance estimates — Evaluating only on cases passing quality filters may produce optimistic performance estimates if excluded cases are systematically harder to classify; documenting both sets helps readers gauge real-world applicability
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
Downstream reach in the literature
172 downstream papers · 2 datasetsHow widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.
- Comprehensive Molecular Characterization of Muscle-I... 2017 · 1,754 cites
- Identification of distinct basal and luminal subtype... 2014 · 1,327 cites
- Meta-Analysis of the Luminal and Basal Subtypes of B... 2016 · 279 cites
- Siglec15 shapes a non-inflamed tumor microenvironmen... 2021 · 277 cites
- Genomic Subtypes of Non-invasive Bladder Cancer with... 2017 · 232 cites
- An EMT-related gene signature for the prognosis of h... 2020 · 158 cites
- Siglec15 shapes a non-inflamed tumor microenvironmen... 2021 · 277 cites
- An EMT-related gene signature for the prognosis of h... 2020 · 158 cites
- Inhibition of the CCL2 receptor, CCR2, enhances tumo... 2020 · 141 cites
- Assessment of Luminal and Basal Phenotypes in Bladde... 2020 · 132 cites
- PHGDH Inhibits Ferroptosis and Promotes Malignant Pr... 2022 · 127 cites
- 5mC regulator-mediated molecular subtypes depict the... 2021 · 65 cites
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-36245985
Paper: Jiang et al. 2022, J Oncol. "Artificial Intelligence Meets Whole Slide Images: Deep Learning Model Shapes an Immune-Hot Tumor and Guides Precision Therapy in Bladder Cancer." DOI 10.1155/2022/8213321 · PMCID PMC9553530.
Code: https://github.com/YihengJiang0912/AI-meets-WSI
@ commit 1e3c9224a6b3367d19e8c82d2609ab750e878f50 (pushed 2022-08-09, only commit).
Data accession (registry): GEO GSE32894 (public; resolves, 1 GDS hit).
What the repo actually contains
A single file named R script (17,086 bytes, 540 lines). No README, no data, no
environment spec, no DL-model code, 1 branch (main), 0 releases. Verified via
GitHub trees API (recursive) → exactly one blob.
The script is a grab-bag of downstream-bioinformatics snippets, each of which
load()s a preprocessed .rda object that is not shipped and reads external
files that are not shipped:
| Section (line) | Reads / loads (NOT in repo) |
|---|---|
| Survival (L5) | pre_Survival.rda |
| Differential analysis (L36,44) | pre_Diff.rda, inputFile (undefined), outDiff (undefined) |
| MultitimeROC (L105) | pre_ROC.rda |
| CIBERSORT (L227-228) | sig_matrix, mixture_file (verbatim public Newman et al. source, incl. its next anti-redistribution watermark) |
| GSVA/KEGG/GO/Hallmark/PCA (L344) | pre_GSVA.rda, exprjyh, metajyh |
| Mutation (L425,433) | pre_mutation.rda, TCGA.BLCA.maf, all_lesions.conf_99.txt, amp/del_genes.conf_99.txt, scores.gistic |
| LASSO (L440) | pre_Lasso.rda |
| Cox / Forest (L463) | pre_Cox.rda |
| pRRophetic (L530) | pre_pRRophetic.rda |
In scope vs out of scope (pipeline-derived results)
ALL reported quantitative results are downstream of, and conditioned on, the deep-learning WSI clustering model (defines clusters C0/C1/C2) and the survival-prediction CNN (defines the AI score, high/low). These are the paper's central contribution.
- Out of scope — no code shipped (
no_code): the WSI deep-learning models themselves (WSI clustering: acc 92.38%, AUC 0.99, F1 0.9167; 3-yr survival CNN: AUC 0.95, acc 86.00%). Neither the model code nor weights are in the repo or supplementary. Not attempted. - Nominally in scope but un-runnable (
docs_insufficient): every R-script section above. Each requires a preprocessed.rdainput (the DL-derived patient grouping + expression/mutation matrices) that is not shipped, has no public download, and has no code path from the raw public data (GSE32894/TCGA-BLCA) to produce it. No README/entrypoint/parameters tie the script to any input. The script therefore cannot run as shipped (it would error on the firstload('pre_Survival.rda')).
Third-party-tool route (P16) — considered, not feasible here
Applying an existing third-party tool to the paper's own public data would be equally valid, BUT every reported number (DEG counts, KM p-values, time-ROC AUCs, LASSO 28-gene signature, Cox HRs, CIBERSORT fractions, GSVA pathways, pRRophetic IC50s) is defined relative to the DL-derived groups. Without the cluster/AI-score assignments (not shipped, not derivable without the absent DL model), there is no reported value computable from public data alone to compare 1:1 against. So no clean reproducible data point exists.
Decision
Drop — docs_insufficient (with a strong secondary no_code for the headline
DL model). Determinable at screening (control-plane) without «our HPC» compute: the
shipped script deterministically fails for lack of its 8 .rda inputs, and the
central DL pipeline has no code. Recording the drop per SCREENING.md; not burning
compute on an un-runnable artifact (80/20).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a clean docs-insufficient / no-code drop: the repo is a single 17 KB R script that load()s 8 preprocessed .rda inputs and external MAF/GISTIC/CIBERSORT files that are shipped nowhere, and the headline WSI deep-learning clustering model (acc 92.38%, AUC 0.99) and survival CNN (AUC 0.95) have no code or weights at all. Although GSE32894 is public, every reported value (e.g. AI-score time-ROC AUC 0.838, LASSO 28 genes at λ=0.0741, DEGs 530/497/342) is conditioned on the DL-derived C0/C1/C2 / AI-score groups, for which there is no code path from raw data — so no reported number is independently verifiable from released artifacts. The defect sits on the authors'/release side (incomplete deposit), not on our methodology. We flag this as not independently verifiable rather than asserting fabrication, since the authors may hold the data privately.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.