IRSN-23 gene diagnosis enhances breast cancer subtype classification and predicts response to neoadjuvant chemotherapy: new validation analyses.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- ✓Overall, the reproduction was clean
- Every checked point held up.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough -> 1:1 reproduction. The paper's lead pipeline result (OUH-cohort pCR enrichment in IRSN-23 Gp-R vs Gp-NR without anti-HER2 = 29% vs 1%, P=1.70E-5) was reproduced EXACTLY from the authors' own shipped supclust DLDA model applied to their shipped OUH data (repo SNlaboratory/irsn23 @ db0f6b4), run on «our HPC» with R 4.5.3 + supclust 1.1-1. pCR rates 29%/1% match exactly; the P-value matches to the same order of magnitude (test-convention difference). Training/Validation reference tables also reproduced deterministically. NOT attempted (hard ~20%): the external GEO meta-analysis (N=1282, GSE25066 + 9 other series), pooled odds ratios 3.23-5.72 (Fig 1B), AUC/DeLong, GSEA, subtype heatmaps -- the repo ships only the fitted model + OUH data, not the per-GEO normalization/scoring pipeline, so that would require rebuilding the whole external pipeline. Repo's Validation.data.Robj is corrupted (bad magic 'X') but is not needed for the reproduced claims. No fabrication flags.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 71assessed: 2026-06-14 ⛓ e898702cc6d3
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-09-19
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe study tests whether the IRSN-23 immune-related gene signature reproducibly classifies breast cancer patients into chemotherapy-sensitive (Gp-R) and less-sensitive (Gp-NR) groups and predicts pathological complete response (pCR) to neoadjuvant chemotherapy, and how it impacts breast cancer subtype classification.
- ★ IRSN-23 Gp-R patients have significantly higher pCR rates than Gp-NR patients without anti-HER2 therapy, across the OUH cohort and multiple independent public datasets finding
- ★ IRSN-23 predictive performance is reproducible across different microarray platforms (Affymetrix U133, X3P, Agilent, Illumina) finding
- ★ In patients receiving anti-HER2 therapy, pre-treatment IRSN-23 does not significantly predict pCR, but reassessment two weeks post-treatment does finding
- ★ IRSN-23 combined with Oncotype Dx or PAM50 refines breast cancer subtype classification based on tumor microenvironment (offensive factor PAM50, defensive factor IRSN-23) finding
- ★ The immune subtype identified via IRSN-23 is correlated with better prognosis after NAC finding
- IRSN-23 is a 23-probe (19-gene) immune-related signature developed by DLDA supervised analysis, patented, and available on GitHub resource
- The 23-probe IRSN-23 score correlates highly (r=0.99) with the 19-gene aggregated version used to adapt the signature to other platforms method
- TMB and HRD scores derived from WES were compared against IRSN-23 as predictors of chemotherapy sensitivity method
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| DNA microarray (gene expression) | breast tumor tissue, OUH patients (n=146) | NAC (paclitaxel-FEC) ± trastuzumab | IRSN-23 immune score and pCR after NAC | Affymetrix Human Genome U133 Plus 2.0 (GPL570) |
| DNA microarray (gene expression) | breast tumor tissue, GSE25066 public dataset (n=488) | NAC | IRSN-23 classification and pCR rate | GPL96/GPL570/GPL571 |
| DNA microarray (gene expression) | breast tumor tissue, public GEO datasets (GSE28844, GSE42822, GSE140494, GSE37946, GSE66399) | NAC ± anti-HER2 therapy | IRSN-23 classification and pCR rate | GPL570 |
| DNA microarray (gene expression) | breast tumor tissue, GSE4779 (n=102) | NAC | IRSN-23 classification and pCR rate | Affymetrix Human X3P Array (GPL1352) |
| DNA microarray (gene expression) | breast tumor tissue, GSE21974, GSE34138, GSE130788 | NAC ± anti-HER2 therapy | IRSN-23 classification and pCR rate | Agilent/Illumina (GPL6480, GPL6884) |
| RNA-Seq | breast tumor tissue, OUH patients (n=43) | none | concordance of IRSN-23 score with microarray-derived score | — |
| Whole exome sequencing | breast tumor tissue, OUH patients (n=81) | none | TMB and HRD score (NtAI, LST, HRD-LOH) | SureSelect Human All Exon V6 (Agilent) |
| Immunohistochemistry / FISH | breast tumor tissue, OUH patients | none | ER, PR, HER2 status, TILs, Foxp3+ Tregs, CD8+ T cells, IL17F+ cells | — |
- ▲ OUH cohort without trastuzumab: pCR rate higher in Gp-R than Gp-NR 29% vs 1%, P=1.70E-5
- ▲ Pooled validation datasets without anti-HER2 therapy: pCR rate higher in Gp-R than Gp-NR N=1103, 40% vs 12%, P=2.02E-26
- ▲ Pooled validation datasets with anti-HER2 therapy: pCR rate higher in Gp-R than Gp-NR N=304, 49% vs 35%, P=0.017
- ▲ GSE4779 (Affymetrix X3P array): pCR rate higher in Gp-R than Gp-NR 54% vs 24%, P=3.53E-03
- ▲ Agilent (GSE21974) and Illumina (GSE34138) datasets confirmed higher pCR in Gp-R than Gp-NR P=3.22E-03 and P=3.63E-05 respectively
- – GSE130788: pre-treatment IRSN-23 not significant for pCR, but post-treatment reassessment (2 weeks) was significant pre-treatment P=0.084 (ns); post-treatment P=0.025
- – 23-probe IRSN-23 score correlated strongly with 19-gene IS version in OUH internal validation r=0.99 (n=59)
- – Immune subtype identified via IRSN-23 associated with better prognosis after NAC
- correlation r=0.99 (23-probe IRSN-23 score vs 19-gene IS, OUH internal validation)
- pvalue P=1.70E-5 (OUH Gp-R vs Gp-NR pCR rate without trastuzumab (abstract))
- pvalue P=2.35E-05 (OUH Gp-R vs Gp-NR pCR rate without anti-HER2 therapy (results section))
- pvalue P=2.02E-26 (pooled validation without anti-HER2 therapy, Gp-R 40% vs Gp-NR 12% pCR)
- pvalue P=0.017 (pooled validation with anti-HER2 therapy, Gp-R 49% vs Gp-NR 35% pCR)
- pvalue P=8.39E-13 (GSE25066 Gp-R 38% vs Gp-NR 10% pCR)
- count 1282 cases (1261 analyzed after exclusion) (external validation set from 8 GEO datasets)
- pvalue P=3.53E-03 (GSE4779 (Affymetrix X3P array) Gp-R 54% vs Gp-NR 24% pCR)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The study validates the IRSN-23 immune gene signature for predicting pathological complete response (pCR) to neoadjuvant chemotherapy in breast cancer using an institutional dataset (n=146) and independent public GEO datasets (N=1282). Group comparisons (Gp-R vs Gp-NR) used chi-square or Fisher's exact tests across multiple individual and pooled datasets; Kaplan-Meier survival curves were compared with log-rank tests; and AUC comparisons employed the DeLong test. All tests were two-tailed at α=0.05, and results were reported primarily as pCR proportions with exact p-values using R version 3.6.3.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| chi-square test or Fisher's exact test (two-tailed) | pCR rate comparisons between Gp-R and Gp-NR in each individual dataset and in pooled analyses | varies by dataset: OUH without anti-HER2 n=112; GSE25066 n=488; GSE28844 n=32; GSE42822 n=66; GSE140494 n=91; GSE4779 n=102; GSE21974 n=32; GSE34138 n=178; pooled without anti-HER2 N=1103; pooled with anti-HER2 N=304 | not stated |
| DeLong test | comparison of area under the curve (AUC) values | — | not stated |
| log-rank test | comparison of Kaplan-Meier distant recurrence-free survival curves by breast cancer subtype | — | not stated |
| Pearson's correlation coefficient | concordance between 23-probe and 19-gene IRSN-23 scores (OUH internal validation n=59) and as the linkage method for unsupervised hierarchical clustering (heat maps) | n=59 for probe/gene score concordance; n=43 for microarray vs RNA-Seq concordance | not stated |
| Gene Set Enrichment Analysis (GSEA) via DAVID Functional Annotation Bioinformatics | genes with principal component loadings greater than 0.10 in absolute value | — | na |
-
pCR rates between Gp-R and Gp-NR were compared using chi-square or Fisher's exact tests, reporting only p-values↳ Could also: logistic regression reporting odds ratios (or risk ratios) with 95% confidence intervals — odds ratios with CIs quantify both the magnitude and precision of the association rather than only its statistical significance, and are directly amenable to formal meta-analytic pooling across the contributing datasets
-
pooled analysis combined raw patient counts across independent datasets by simple aggregation into a single 2×2 table↳ Could also: conduct a random-effects meta-analysis pooling log odds ratios across studies using inverse-variance weighting (e.g., DerSimonian-Laird) — a formal meta-analysis accounts for between-study heterogeneity, yields a pooled effect estimate with a CI, and allows assessment of heterogeneity (e.g., I²) across the contributing datasets, which differ in platform, chemotherapy regimen, and patient mix
-
numerous separate hypothesis tests were performed across multiple datasets and subgroups without a stated multiplicity adjustment↳ Could also: apply Benjamini-Hochberg false discovery rate (FDR) correction or Bonferroni correction across the family of tests — when many comparisons are made, FDR or family-wise correction controls the expected proportion of false positives and helps readers calibrate which findings are most robust
-
Kaplan-Meier survival differences between subtypes were assessed with log-rank tests↳ Could also: Cox proportional hazards regression reporting hazard ratios with 95% CIs — hazard ratios quantify the magnitude of the survival difference and permit adjustment for clinical covariates such as tumor stage, ER status, and treatment, which is informative given the heterogeneous patient populations
-
concordance between microarray-derived and RNA-Seq-derived IRSN-23 scores was assessed with Pearson's correlation (n=43)↳ Could also: use intraclass correlation coefficient (ICC) or Bland-Altman analysis — ICC directly quantifies agreement rather than linear association; Bland-Altman plots additionally visualize systematic bias and limits of agreement across the measurement range, which is relevant when assessing whether the two platforms can be used interchangeably
-
pCR proportions were reported as point estimates without uncertainty bounds↳ Could also: report 95% confidence intervals around each proportion (e.g., Wilson or Clopper-Pearson intervals) — CIs convey estimation uncertainty alongside the point estimate and are especially informative for small-n subgroups such as GSE28844 (n=32) or GSE21974 (n=32), where the point estimate alone may be imprecise
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
- CoINcIDE: A framework for discovery of patient... L1 87/100
- A curated collection of transcriptome datasets... L1 62/100
- An NMF-Based Methodology for Selecting Biomark... L1 84/100
- Unveiling prognostics biomarkers of tyrosine m...⚑ L1 51/100 ⚑
- Meta-analysis of gene expression profiles of l... L1 78/100
- Colorectal Cancer Prediction Based on Weighted...⚑ L1 80/100 ⚑
- Curation of over 10 000 transcriptomic studies... L1 80/100
- Construction and Validation of an Immune Infil...⚑ L1 51/100 ⚑
- Identification of a novel 10 immune-related ge...
- Exploration of the shared diagnostic genes and... L1 76/100
- Molecular Classification Models for Triple Neg... L1 86/100
- Predicting Bone Metastasis Using Gene Expressi... L1 62/100
- Autoencoder Networks Decipher the Association... L1 74/100
- Comprehensive analysis of a novel RNA modifica... L1 71/100
- Discovery and validation of molecular patterns... L1 83/100
- Comparative profiling of skeletal muscle model... L1 64/100
- CoINcIDE: A framework for discovery of patient... L1 87/100
- A curated collection of transcriptome datasets... L1 62/100
- PulmonDB: a curated lung disease gene expressi...⚑ L1 53/100 ⚑
- Meta-analysis of gene expression profiles of l... L1 78/100
- Curation of over 10 000 transcriptomic studies... L1 80/100
- Screening of Diagnostic Biomarkers and Immune...⚑ L1 48/100 ⚑
- Predicting Bone Metastasis Using Gene Expressi... L1 62/100
Downstream reach in the literature
100 downstream papers · 1 datasetsHow widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.
- Survival analysis across the entire transcriptome id... 2021 · 751 cites
- Epithelial-mesenchymal transition spectrum quantific... 2014 · 604 cites
- A genomic predictor of response and survival followi... 2011 · 543 cites
- Differential response to neoadjuvant chemotherapy am... 2013 · 532 cites
- Inhibition of fatty acid oxidation as a therapy for... 2016 · 438 cites
- Aerobic glycolysis tunes YAP/TAZ transcriptional act... 2015 · 358 cites
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-40128415 (IRSN-23 new validation analyses)
Paper: Sota Y et al. IRSN-23 gene diagnosis enhances breast cancer subtype classification and predicts response to neoadjuvant chemotherapy: new validation analyses. Breast Cancer (2025). PMID 40128415 · PMCID PMC11993443 · DOI 10.1007/s12282-025-01687-6.
Method (from Methods + repo): IRSN-23 = a 23-probe (19-gene) immune-related
signature scored with DLDA (Diagonal Linear Discriminant Analysis) via the R
package supclust. The pre-fit classifier and the OUH cohort data are shipped
in the authors' repo https://github.com/SNlaboratory/irsn23 (pinned commit
db0f6b4f2229464dfe07453696460153afd01d30). The repo's How to Use IRSN-23.Rmd
loads the model + OUH.data and tabulates predicted group (Gp-R = responder /
Gp-NR = non-responder) against the actual pCR result.
In scope (pipeline-derived, reproducible from shipped artifacts)
| result | pipeline | source data | status |
|---|---|---|---|
| OUH prospective cohort: pCR rate Gp-R vs Gp-NR, without anti-HER2 (paclitaxel-FEC) | supclust DLDA IRSN.23.model$predict → 2×2 vs pCR |
OUH.data (shipped in repo) |
REPRODUCED |
| OUH Training confusion table (Sota 2014 Ann Oncol, re-shipped) | same | OUH.data$Training |
reproduced (deterministic) |
| OUH Validation confusion table (Sota 2014 Ann Oncol, re-shipped) | same | OUH.data$Validation |
reproduced (deterministic) |
| significance of OUH Gp-R vs Gp-NR pCR difference (P value) | Fisher/chi-square on the 2×2 | derived from above | reproduced (within order of magnitude) |
Out of scope / NOT attempted (the hard ~20%)
- External GEO meta-analysis (N = 1282, GSE25066 + 9 other series), pooled odds ratios 3.23–5.72 (Fig 1B), AUC/DeLong comparisons, GSEA, subtype-class heatmaps. The repo ships only the fitted model + the OUH cohort; it does not ship the per-GEO download/normalization/probe-mapping pipeline used for the 10 external datasets. Reproducing those would require independently reconstructing the entire normalization + scoring pipeline for 10 microarray series across 3 platforms (GPL96/570/571, GPL1352, GPL6480, GPL6884) — that is the unspecified, high-effort 20% and is intentionally not attempted.
- Wet-lab / clinico-pathological / patent items: out of scope (non-pipeline).
Headline reproduced claim: the OUH-cohort pCR enrichment in Gp-R (the paper's lead quantitative result, abstract + Results), computed end-to-end from the shipped model on the shipped data.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The paper's lead result — OUH-cohort pCR enrichment in IRSN-23 Gp-R vs Gp-NR without anti-HER2 (29% vs 1%, P=1.70E-5) — reproduces 1:1 from the authors' own shipped DLDA model applied to their shipped OUH data; pCR rates match exactly and significance holds (P~1.4E-5/2.4E-5, a pure test-convention difference). No deviation sits on the authors' side or in the core computation, and there are no fabrication flags. Caveats are scope/integrity only: the external GEO meta-analysis (Fig 1B, OR 3.23-5.72) was not attempted because its per-series pipeline isn't shipped, and Validation.data.Robj is corrupted — neither affects the cleanly reproduced headline.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at [email protected].
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.