CoINcIDE: A framework for discovery of patient subtypes across multiple datasets.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- 🟡Could not use the authors’ exact input data
- 🟡A deviation arose in the data or preprocessing
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce the data-processing pipeline 1:1. CoINcIDE (Planey & Gevaert, Genome Med 2016) ships its method as the R/Bioconductor pkg Coincide (github.com/kplaney/CoINcIDE @5845d379) plus dissertation analysis scripts, with breast data in the author's curatedBreastData Bioconductor pkg. Built a conda R env on «our HPC» and reproduced THREE clear pipeline outputs: (C1) the package's own shipped unit test for the gene-expression NA-filter+KNN-impute step reproduced EXACTLY (99 genes, 29 patients; imputation error 0.00503 vs shipped 0.00517, within-tol because the error-estimation sample() is unseeded). (C2) the paper's headline '17 datasets' — mergeDatasetList(minNumGenes=10000,minNumPatients=40) on curatedBreastData printed 'Found 17 passing the input thresholds' -> 17 EXACT, and the kept GEO study IDs match Table 1. (C3) total samples 2237 vs reported 2235 (within-tol, Δ=2/0.09%; curatedBreastData is now v2.30.0/2024 vs the 2016 curation). NOT attempted (hard 20%): C4, the full meta-cluster discovery (PAM50 centroid clustering -> CoINcIDE computeEdges with 500 null permutations -> community detection -> '5 breast meta-clusters'), and the ovarian analyses, GSEA/druggable-gene/survival meta-features (the Broad MSigDB gene-list RData is NOT in the repo -> data_unavailable for those), and the simulation figures. Honesty note: the author's ExpressionSet wrapper procExprSet() crashes on modern Biobase (dim-changing S4 assignments + a column-vs-row pData indexing bug 2016 Biobase tolerated); we ran the author's OWN lower-level functions (filterAndImputeSamples, collapseDupProbes, removeDupPatients, filterGenesByVariance) on plain matrices — the identical computation without the broken S4 bookkeeping — then the genuine mergeDatasetList. Faithful re-run, not a re-implementation. No fabrication signs: every reproduced value is derivable from the shipped data+code and matches the paper to within curation drift.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 87assessed: 2026-06-14 ⛓ 63c2143bb991
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan patient disease subtypes be discovered as replicable 'meta-clusters' across multiple high-dimensional molecular datasets without requiring between-dataset batch-correction transformations, by finding consensus across dataset-specific clusterings rather than clustering a single concatenated matrix?
- ★ CoINcIDE is a methodological framework that discovers replicable patient subtypes (meta-clusters) across multiple datasets by finding consensus across dataset-specific clusterings, requiring no between-dataset transformations. method
- ★ CoINcIDE extends the unsupervised meta-analysis paradigm by building on the In-Group-Proportion (IGP) metric to compute cluster-cluster similarities and assemble a network whose communities are the final subtypes. method
- ★ curatedBreastData is a high-quality curated database collection of 24 breast cancer gene expression datasets (over 2,500 samples) with linked outcome and treatment information, released as an R package. resource
- ★ CoINcIDE validates known breast cancer subtypes (e.g., PAM50 subtypes) across multiple datasets. finding
- ★ CoINcIDE discovers novel breast and ovarian cancer subtypes with prognostic significance across multiple datasets. finding
- ★ CoINcIDE identifies novel hypothesized ovarian cancer therapeutic targets across multiple datasets. finding
- Avoiding concatenation/batch correction preserves true between-dataset signaling variances (e.g., latent subtypes from different clinical trials) and retains per-dataset consensus information that concatenation loses. mechanism
- Both CoINcIDE and curatedBreastData are distributed as R packages. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Gene expression microarray clustering (consensus k-means clustering analysis) | Human breast cancer primary tumor tissue (curatedBreastData: 2,235 pre-treatment samples, 17 datasets, 15 studies, 6 platforms) | none | Dataset-specific patient clusters / meta-cluster subtype assignments | Affymetrix U133A/U133 Plus 2.0, Agilent, Arcturus, AFFY Human Phase3, MLRG (GEO; ~11,000 genes avg) |
| Gene expression microarray clustering (CoINcIDE meta-cluster analysis) | Human ovarian cancer primary tumor tissue (curatedOvarianData: 3,105 samples, 24 datasets, 10 platforms) | none | Meta-cluster subtype assignments, prognostic significance, hypothesized therapeutic targets | Illumina HumanRef-8, Agilent, Affymetrix, Operon, ABI (GEO/TCGA; ~15,000 genes avg) |
| Semi-supervised clustering using PAM50 centroids (CoINcIDE evaluation) | Human breast cancer datasets (17 datasets) | none | Patient assignment to 5 PAM50 subtypes (Normal, Basal, Luminal A, Luminal B, HER2) via Pearson correlation | PAM50 50-gene centroid set (min 35/50 genes; UNC Microarray Database) |
| De novo PAM50 gene-set clustering (CoINcIDE) | Human breast cancer datasets | none | De novo dataset-specific clusters / meta-clusters | PAM50 gene set |
| Simulated clustered-dataset benchmark (Eigen decomposition simulation) | Simulated gene expression datasets derived from a real 4-tissue-type dataset (7 sets x 10 datasets, 200 simulated genes) | Added random normal noise (mean 0, SD 0 to 2.4) | True positive rate and false positive rate of CoINcIDE meta-cluster edges (same-tissue connections) | — |
| Concatenated-matrix clustering comparison | Human breast cancer datasets (17 datasets, minimum 35-gene PAM50 set) | none | Replicability and biological intuitiveness of discovered subtypes vs CoINcIDE | — |
- – CoINcIDE re-discovers known PAM50 breast cancer subtypes under semi-supervised conditions across multiple datasets
- – CoINcIDE discovers breast and ovarian cancer subtypes with prognostic significance across multiple datasets
- – CoINcIDE identifies novel hypothesized ovarian cancer therapeutic targets across multiple datasets
- – Final meta-cluster patient assignments are robust to small changes in the mean similarity threshold (first breast cancer analysis gave the same results at thresholds 0.15 and 0.25)
- – Simulation benchmark across 7 dataset sets with increasing noise yielded measurable TPR/FPR for CoINcIDE meta-clusters
- count over 2,500 breast cancer gene expression samples (curatedBreastData collection size)
- count 2,235 pre-treatment patient tumor sample microarrays (breast cancer samples after filtering (17 datasets, 15 studies, 6 platforms))
- count 3,105 samples (ovarian cancer samples (24 datasets, 10 platforms) after filtering)
- other sample sizes ranged from 46 to 286; average ~11,000 genes per platform (breast cancer dataset characteristics)
- other sample sizes ranged from 28 to 578; average ~15,000 genes per platform (ovarian cancer dataset characteristics)
- other recommended averaged nearest neighbor fraction of 0.7 and P value threshold of 0.01 (CoINcIDE edge-assignment thresholds)
- count 500 null centroid iterations (suggested iterations to compute IGP-based P value)
- other similarity thresholds of 0.15 and 0.25 (breast PAM50); 0.5 (ovarian) (mean similarity thresholds chosen from density curves)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a methods/framework paper presenting CoINcIDE, an unsupervised pipeline that derives patient subtypes by clustering each dataset separately and then finding consensus 'meta-clusters' across datasets without batch correction. Within-dataset clustering uses consensus clustering (Hartigan-Wong k-means with resampling and a PAC score); cross-dataset cluster similarity is quantified with Pearson-correlation-based centroid assignment, a nearest-neighbor fraction, and a mean similarity metric, with a permutation P value built on the In-Group-Proportion (IGP) null distribution. Edges passing user thresholds form a network, and meta-clusters are detected with the Girvan-Newman community detection algorithm. Performance is assessed on simulated datasets via averaged true/false positive rates, and the framework is applied to curated breast and ovarian gene-expression collections.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Permutation/null-distribution P value based on the In-Group-Proportion (IGP) method (clusterRepro), comparing real cluster-centroid assignments to null centroid sets | Step 2 cluster-cluster similarity significance for each cluster pair across datasets | 500 null centroid iterations | not stated |
| Pearson's correlation (centroid assignment and mean-correlation similarity metric) | assigning patients to nearest cluster centroid and computing cluster-cluster similarity; also for PAM50 subtype assignment | — | not stated |
| Consensus clustering with Hartigan-Wong k-means (one random start, 90% resampling) and a rounded Proportion of Ambiguous Clusters (PAC) score | Step 1 single-dataset clustering | — | not stated |
| Girvan-Newman community detection (igraph) | Step 4 meta-cluster identification from the cluster similarity network | — | na |
| True positive rate / false positive rate computation | simulation evaluation of CoINcIDE meta-clusters | averaged over 50 simulation iterations per set; 10 datasets per set | na |
-
Significance of cluster-cluster similarity was assessed with a per-pair permutation P value thresholded at 0.01 across many cross-dataset comparisons.↳ Could also: A Benjamini-Hochberg FDR or other multiple-comparison adjustment could also be applied across the family of cluster-cluster tests. — Applying a multiplicity correction would also characterize the expected proportion of spurious edges when many pairs are tested, complementing the fixed per-pair threshold.
-
Method performance was summarized as TPR and FPR averaged over 50 simulation iterations.↳ Could also: One could also report dispersion (SD/IQR) or 95% confidence intervals around these averaged rates, and/or ROC/AUC across thresholds. — Adding a measure of variability or a full ROC curve would also convey the stability of performance across simulation replicates and threshold choices.
-
Within-dataset clustering used consensus clustering with Hartigan-Wong k-means and the PAC score to choose cluster number.↳ Could also: Model-based clustering (e.g., Gaussian mixtures with BIC) or hierarchical/PAM clustering with silhouette/gap statistics could also be used. — These alternatives would also provide principled, distributional or internal-validity criteria for selecting cluster number and could be compared as a sensitivity check.
-
Cluster similarity and centroid assignment relied on Pearson's correlation.↳ Could also: Spearman correlation or cosine/Euclidean-based distances could also serve as the similarity metric. — Rank-based or alternative distance metrics would also be more robust to outliers and monotonic-but-nonlinear relationships, offering a check on metric sensitivity.
-
Network-level thresholds (similarity, nearest-neighbor fraction, P value) were chosen via inspection of similarity density curves and recommended defaults.↳ Could also: A formal sensitivity analysis sweeping thresholds, or stability-selection across resamples, could also be reported systematically. — Systematically tabulating how meta-clusters change across the threshold grid would also document robustness of the final subtypes to user-chosen parameters.
-
Meta-clusters were detected using the Girvan-Newman community detection algorithm.↳ Could also: Modularity-based methods such as Louvain/Leiden or label propagation could also be used for community detection. — These alternatives would also scale efficiently and provide a comparison point for the stability of detected communities.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
CoINcIDE breast cancer meta-cluster patient assignments are stable across similarity thresholds 0.15–0.25microarray human breast-cancer none 2016×1papers★ This paper is the founder (earliest)
-
CoINcIDE semi-supervised clustering re-discovers all five PAM50 breast cancer subtypes across 17 independent microarray datasetsmicroarray human breast-cancer none 2016×1papers★ This paper is the founder (earliest)
-
CoINcIDE de novo meta-cluster analysis identifies prognostically significant patient subtypes across 24 ovarian cancer microarray datasetsmicroarray human ovarian-cancer 2016×1papers★ This paper is the founder (earliest)
-
CoINcIDE ovarian cancer meta-cluster analysis yields novel hypothesized therapeutic targets from multi-dataset gene expression integrationmicroarray human ovarian-cancer 2016×1papers★ This paper is the founder (earliest)
-
CoINcIDE meta-cluster edge recovery achieves quantifiable TPR and FPR across simulated multi-dataset benchmarks with increasing noise levelsother simulated 2016×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
- A curated collection of transcriptome datasets... L1 62/100
- An NMF-Based Methodology for Selecting Biomark... L1 84/100
- Unveiling prognostics biomarkers of tyrosine m...⚑ L1 51/100 ⚑
- Meta-analysis of gene expression profiles of l... L1 78/100
- Colorectal Cancer Prediction Based on Weighted...⚑ L1 80/100 ⚑
- Curation of over 10 000 transcriptomic studies... L1 80/100
- Construction and Validation of an Immune Infil...⚑ L1 51/100 ⚑
- Identification of a novel 10 immune-related ge...
- Exploration of the shared diagnostic genes and... L1 76/100
- IRSN-23 gene diagnosis enhances breast cancer... L1 71/100
- Molecular Classification Models for Triple Neg... L1 86/100
- Predicting Bone Metastasis Using Gene Expressi... L1 62/100
- Autoencoder Networks Decipher the Association... L1 74/100
- Comprehensive analysis of a novel RNA modifica... L1 71/100
- Discovery and validation of molecular patterns... L1 83/100
- Comparative profiling of skeletal muscle model... L1 64/100
- A curated collection of transcriptome datasets... L1 62/100
- PulmonDB: a curated lung disease gene expressi...⚑ L1 53/100 ⚑
- Meta-analysis of gene expression profiles of l... L1 78/100
- Curation of over 10 000 transcriptomic studies... L1 80/100
- Screening of Diagnostic Biomarkers and Immune...⚑ L1 48/100 ⚑
- IRSN-23 gene diagnosis enhances breast cancer... L1 71/100
- Predicting Bone Metastasis Using Gene Expressi... L1 62/100
Downstream reach in the literature
194 downstream papers · 2 datasetsHow widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.
- The consensus molecular subtypes of colorectal cance... 2015 · 3,929 cites
- Cooperation between Constitutive and Inducible Chemo... 2019 · 677 cites
- Meta-analysis of gene expression profiles in breast... 2008 · 645 cites
- Role of the TMPRSS2-ERG gene fusion in prostate canc... 2008 · 557 cites
- Proteomic profiling identifies dysregulated pathways... 2012 · 438 cites
- The splicing factor SRSF1 regulates apoptosis and pr... 2012 · 358 cites
- Lin28 promotes transformation and is associated with... 2009 · 692 cites
- Cooperation between Constitutive and Inducible Chemo... 2019 · 677 cites
- A collagen-remodeling gene signature regulated by TG... 2014 · 292 cites
- LAYN Is a Prognostic Biomarker and Correlated With I... 2019 · 262 cites
- Molecular profiling uncovers a p53-associated role f... 2010 · 214 cites
- Functional genomics identifies five distinct molecul... 2013 · 186 cites
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-26961683 (CoINcIDE; Planey & Gevaert, Genome Med 2016)
Paper: CoINcIDE: A framework for discovery of patient subtypes across multiple
datasets. PMID 26961683 · PMCID PMC4784276 · DOI 10.1186/s13073-016-0281-4.
Method code: https://github.com/kplaney/CoINcIDE (R/Bioconductor pkg Coincide
- dissertation analysis scripts). Data pkg:
curatedBreastData(Bioconductor experiment-data pkg authored by same author),curatedOvarianData(Bioconductor).
CoINcIDE = "Clustering INtra and Inter DatasEts": per-dataset consensus clustering → cluster centroids → significant edges between cluster centroids across datasets (null permutation test) → community detection on the network → "meta-clusters" (patient subtypes reproducible across datasets).
In scope (pipeline-derived; attempted)
| id | result (reported) | pipeline | tractability |
|---|---|---|---|
| C1 | filterAndImputeSamples() self-test: 99/100 genes pass NA filter, 29/30 patients pass, imputation avg-abs-error ≈ 0.0052 |
Coincide pkg testthat unit test on a synthetic matrix; shipped expected output tests/testthat/createTestTrainSetsOutput.txt. Fully deterministic counts (impute seed fixed rng.seed=362436069). |
HIGH — self-contained, fast |
| C2 | 17 breast datasets used (Table 1 / Results; processing filename literally ..._17_studies...) |
curatedBreastData → procExprSetList(minVar=.001) → mergeDatasetList(minNumGenes=10000, minNumPatients=40) keeps datasets ≥40 patients & ≥10k genes |
HIGH — deterministic filter on shipped data |
| C3 | 2,235 breast pre-treatment tumor samples (Results) | sum of samples across the 17 retained datasets | HIGH — deterministic |
| C4 | 5 breast PAM50 meta-clusters (Fig 3a / Fig 5a) | PAM50 centroid clustering per dataset → CoINcIDE computeEdges (500 null iters, p<0.01) → community detection |
LOW (hard 20%) — full multi-step pipeline, 10-yr-old code, hours; attempt only as stretch |
Out of scope (not attempted, with reason)
- Ovarian analyses (24 datasets, 3/6 meta-clusters; Fig 7): needs
curatedOvarianData+ full pipeline — same hard-20% class as C4; one disease suffices for a clear data point. - GSEA / druggable-gene / survival meta-feature results: require Broad MSigDB
gene-list RData object that is NOT in the repo (README: "if the RData object
is not in the Github repo, it's because I was nervous about its size … reach out
to obtain it"). →
data_unavailablefor those specific results; not pipeline- reproducible from shipped artifacts. - Simulation-study figures: synthetic-data sanity checks, not a reported empirical claim with a single pinnable value.
- Wet-lab / manual interpretation (subtype biological labels): non-pipeline.
Heavy-compute note
All compute on «our HPC» («infra» SLURM) via «host»; conda prefix env built in-job
on «infra». «host» holds only small results + pointers.
Repo: github.com/kplaney/CoINcIDE @ default branch master (last push 2016-08-28).
Pinned commit SHA resolved at run time and recorded in code/code.json.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
A faithful re-run (author's own Coincide functions on plain matrices to bypass a broken modern-Biobase S4 wrapper): the processing self-test is exact (99 genes/29 patients) and the headline data scale reproduces 1:1 (17 datasets exact, 2237 vs 2235 samples within 0.09%). The two deviations are both technical/expected — curatedBreastData version drift (2024 vs 2016) and an unseeded sample() in the imputation-error estimate. The central 5-meta-cluster discovery (C4) was not attempted (data-prep confirmed, clustering result not), so overall a solid, non-suspicious partial.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.