Enhancing cell subpopulation discovery in cancer by integrating single-cell transcriptome and expressed variants.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Any deviation was negligible
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
REPRODUCED (within tolerance). scCluster is the authors' own tool and the repo ships the Patel glioblastoma demo (gene expression + precomputed eSNP + true labels) PLUS the expected output (ACC 0.9853/NMI 0.9583/AMI 0.9575/ARI 0.9652 = Suppl. Table S2) with a fixed seed (manual_seed(44)) -- an ideal reproduction target. Running the authors' code (commit 1a7578e) on their shipped data on a «our HPC» H100 («job») reproduced ACC 0.9797 / NMI 0.9505 / AMI 0.9496 / ARI 0.9530 with the estimated cluster count matching exactly (7). All four metrics land within <=0.0122 absolute (<1.3%) of reported, uniformly ~0.6-1.2% lower -- the expected signature of a documented hardware deviation (README torch==2.0.1 raises 'no kernel image' on H100/sm_90, so torch==2.1.2/cu121 was used) plus GPU fp non-determinism. No fabrication concern: inputs, expected output and seed are all shipped, and an independent run lands on essentially the same numbers. The prior run's CPU NaN blocker was confirmed GPU-specific (GPU pretraining ran clean, no NaN); the only fix needed was adding setuptools<80 (louvain imports pkg_resources). NOT attempted (the deliberate 20%): regenerating eSNP from raw reads via Trimmomatic->STAR->GATK for Gerber=GSE81383/Chung. All grades are PROVISIONAL -- a human reviewer signs off.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-14 ⛓ e4a23f56389e
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-23
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetIntegrating expressed genetic variants (eSNPs) derived from raw scRNA-seq reads together with gene expression profiles, rather than relying on gene expression alone, improves the identification/stratification of cell subpopulations in cancer scRNA-seq data.
- ★ scCluster, an end-to-end deep clustering model integrating gene expression and expressed variant (eSNP) features, stratifies cell subpopulations in cancer scRNA-seq data. method
- ★ scCluster outperforms state-of-the-art clustering methods on multiple real-world cancer scRNA-seq datasets. finding
- ★ Incorporating expressed variant (eSNP) features alongside gene expression substantially enhances stratification of cell subpopulations compared to gene expression alone. finding
- ★ Expressed SNPs (eSNPs) detected from scRNA-seq reads act as signatures of expressed variants that can serve as unique features for classifying cell subtypes, since genetic variants confer distinct functional properties to specific tumor cell subpopulations and drive tumor heterogeneity. mechanism
- ★ scCluster uses a two-stage optimization: pre-training with a denoising ZINB-based dual-modal autoencoder plus deep embedding clustering (DEC), followed by fine-tuning with deep soft K-means clustering (DSK) and cross-instance guided contrastive clustering (C3). method
- Source code, demo data, and illustrations of scCluster are freely available on GitHub. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Expressed variant (eSNP) detection via adapted GATK variant-calling workflow (QC, alignment, duplicate marking, BQSR, HaplotypeCaller, joint genotyping, variant recalibration) | raw scRNA-seq reads, human cancer datasets (aligned to hg38) | none | expressed SNP (eSNP) profile/genotype calls | GATK, Trimmomatic, STAR, Picard, samtools |
| Gene expression quantification from scRNA-seq reads | raw scRNA-seq reads, human cancer datasets | none | gene expression count matrix per cell | FeatureCounts |
| Deep clustering of integrated gene expression and eSNP latent representations (scCluster) | real-world cancer scRNA-seq datasets | none | cell subpopulation cluster assignments | scCluster (custom ZINB dual-modal autoencoder + DEC/DSK/C3 deep learning framework) |
- – scCluster surpasses state-of-the-art methods across multiple real-world cancer scRNA-seq datasets in subpopulation identification performance.
- ▲ Adding eSNP features to gene expression substantially improves stratification of cell subpopulations relative to gene expression alone.
- other eSNPs with >10% missing data removed (eSNP quality filtering criterion in pre-processing pipeline)
- other minimum allele frequency below 5% removed (eSNP quality filtering criterion in pre-processing pipeline)
- other Louvain clustering resolution parameter = 0.8 (default resolution used to determine cluster number when not predefined)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
scCluster is an unsupervised deep clustering framework evaluated on real-world cancer scRNA-seq datasets; it integrates gene expression profiles and expressed variant (eSNP) features through a ZINB-based dual-modal autoencoder combined with deep embedding clustering (DEC), deep soft K-means (DSK), and cross-instance guided contrastive clustering (C3). Model optimization proceeds in two stages (pre-training then fine-tuning) via sequential minimization of four loss functions. The provided text covers introduction and methods only; the results section, specific clustering-metric values, and comparison methodology are not present in the excerpt, so reporting and inferential details cannot be extracted.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Zero-Inflated Negative Binomial negative log-likelihood (ZINB reconstruction loss) | Pre-training and fine-tuning of dual-modal autoencoder for both gene expression and eSNP modalities | — | not stated |
| Kullback-Leibler divergence minimization (Deep Embedding Clustering, DEC) between soft assignment Q and auxiliary distribution P | Pre-training (epochs 201-400) and first 20 epochs of fine-tuning, applied to gene expression and eSNP latent representations | — | not stated |
| Deep soft K-means clustering (DSK) loss — weighted squared distance from cell latent vector to centroid | First 20 epochs of fine-tuning stage for final cell-type cluster assignment | — | not stated |
| Cross-instance guided contrastive clustering (C3) loss based on cosine similarity with threshold ζ | Final 30 epochs of fine-tuning to pull together cells with similar joint gene-expression and eSNP profiles | — | not stated |
| Louvain community detection (graph-based clustering) at resolution 0.8 | Cluster number estimation for datasets without a predefined K | — | not stated |
-
The model uses the ZINB distribution to handle zero-inflation and overdispersion in scRNA-seq count data↳ Could also: A plain Negative Binomial (NB) model without zero-inflation, or a hurdle model, could also accommodate overdispersed counts — The appropriateness of ZINB vs. NB depends on the degree of excess zeros relative to NB predictions; a likelihood-ratio test or comparison of AIC between ZINB and NB fits could empirically support the distributional choice
-
When K is not predefined, the Louvain algorithm at a fixed resolution of 0.8 is used to estimate the number of clusters↳ Could also: The gap statistic, silhouette score sweep, or elbow method on within-cluster sum of squares could also be used to estimate K — Louvain resolution is sensitive to its hyperparameter; complementary stability-based estimates (e.g., silhouette averaged over a resolution range) would characterize how robustly K is identified across parameter choices
-
eSNP features are filtered by a hard threshold: >10% missing values excluded, minor allele frequency <5% excluded↳ Could also: Probabilistic imputation (e.g., mean imputation, low-rank matrix factorization, or MAGIC) before or instead of hard missingness filtering could also be applied — Hard removal of sites with >10% missing data may discard informative rare variants; imputation preserves more features while reducing missing-data bias, with the tradeoff of introducing imputation uncertainty
-
The C3 contrastive loss defines positive pairs using a preset cosine-similarity threshold ζ applied globally across all cells↳ Could also: K-nearest-neighbor or mutual K-NN graph construction could also define positive pairs for contrastive learning — Graph-based positive-pair construction adapts to local density and avoids the need to tune a single global similarity threshold that may behave differently across heterogeneous datasets
-
Benchmark comparisons are reported in the abstract as qualitative superiority (surpasses state-of-the-art) without inferential statistics visible in the provided excerpt↳ Could also: Bootstrap confidence intervals or permutation tests on clustering metrics (e.g., ARI, NMI) across multiple datasets could also accompany the performance comparisons — Formal uncertainty quantification on clustering metric differences would allow readers to assess whether observed improvements exceed expected variation, particularly when dataset-level sample sizes differ substantially
-
Model hyperparameters (ϕ1, ϕ2, γ, ζ, τ, α, Louvain resolution) are described but their selection process is not detailed in the provided text↳ Could also: Cross-validation or held-out stability analysis could also be used to select or report sensitivity to these hyperparameters — Reporting hyperparameter sensitivity (e.g., how clustering metrics vary with ζ or resolution) helps readers understand whether results are robust to these choices across different datasets
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Reproduction scope — pmid-41647537
Paper: Wang T, Mai D, et al. Enhancing cell subpopulation discovery in cancer by integrating single-cell transcriptome and expressed variants. Fundam Res 2025. PMID 41647537 · DOI 10.1016/j.fmre.2025.01.001 Code: https://github.com/duoduomai/scCluster (authors' own tool; P16 N/A — own code) Data accession (registry): GEO GSE81383 = the Gerber melanoma dataset.
What scCluster does (pipeline)
A deep-learning clustering method that fuses two modalities from scRNA-seq: (1) gene-expression counts and (2) expressed SNP (eSNP) features called from the same reads. Two-stage training: ZINB dual-modal autoencoder + deep embedding clustering (300 pretrain epochs), then soft-K-means / contrastive fine-tuning. Outputs cluster labels evaluated against true labels by ACC, NMI, ARI, AMI.
Pipeline-derived results in the paper
| Result | Dataset | Reported (scCluster) | Pipeline | In scope? |
|---|---|---|---|---|
| Clustering metrics | Patel (glioblastoma, 543 cells, 7 clust) | ACC 0.9853, NMI 0.9583, AMI 0.9575, ARI 0.9652 (repo README = Table S2) | scCluster on shipped demo data (eSNP precomputed) | YES — primary |
| Clustering metrics | Gerber (melanoma=GSE81383, 307 cells, 4 clust) | ACC 0.987, NMI 0.961, ARI 0.962, AMI 0.961 (main text) | scCluster + full GATK eSNP calling from raw reads | partial — see below |
| Clustering metrics | Chung (breast, 550 cells, 13 clust) | best on all metrics (Table S2) | scCluster + GATK eSNP | out (no shipped data) |
| eSNP calling | all | — | Trimmomatic→STAR→GATK HaplotypeCaller (pipeline/*.sh) | out of scope (the hard 20%) |
Decision (80/20)
Reproduce the Patel result by running the authors' own scCluster on the demo data
they ship (demo_data/Patel.7z = expression, Patel_snp.zip = eSNP,
Patel_truelabels.csv). This is the clearly-specified, low-hanging output: the repo
ships both inputs and the expected metric values, the random seed is fixed in source
(torch.manual_seed(44)), and clustering is deterministic on CPU.
Not attempted (the ~20%): regenerating eSNP features from raw FASTQ via the GATK pipeline for Gerber/GSE81383 or Chung. That needs raw reads + STAR + HaplotypeCaller and a reference genome; it is the heavy, under-specified part and is explicitly out of scope. The shipped Patel eSNP matrix lets us reproduce the clustering result without it. We therefore reproduce the authors' code on the authors' provided data — a faithful 1:1 of the clustering claim, not the upstream variant-calling.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a partial, infrastructure-limited reproduction, not a discrepancy. The authors ship everything needed — Patel expression, precomputed eSNP, true labels, the expected Suppl. Table S2 output (ACC 0.9853/NMI 0.9583/AMI 0.9575/ARI 0.9652), and a fixed manual_seed(44) — and the ZINB pretrain ran with monotonically decreasing loss. The reproduction stalled purely on our compute side (H100 queue saturation, torch==2.0.1 vs sm_90, CPU NaN overflow), so no final value was produced and nothing was fabricated. Derivability looks likely but is unconfirmed (q5 yellow); the core claim is neither confirmed nor refuted (q7 yellow); there is no fabrication concern on the authors' side.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.