HArmonized single-cell RNA-seq Cell type Assisted Deconvolution (HASCAD).
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Any deviation was negligible
- 🟡Reported values were only indirectly comparable
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
HASCAD is described well enough to reproduce its shipped inference 1:1. The authors' repo (github.com/holiday01/HASCAD @ 6dd61b5) ships the complete pretrained model: 3 ensemble DNN checkpoints, network code, the inference notebook, the 1158-gene reference, the 15 cell-type labels, a worked example input, AND the documented expected output (Result/result.png). Re-running the exact inference on «our HPC» via SLURM (deterministic, eval mode, torch 1.13.1 vs pinned 1.10.0) reproduced ALL 15 example cell fractions to the exact rounded percentage shown in Result/result.png (grade=exact, C1), and the shipped network.py matches the architecture described in Methods (C2). This is a clean, fully-specified, deterministic reproduction of the repo's primary computational output. CAVEAT (honesty): C1 is self-referential -- the repo reproducing its own shipped figure -- so it confirms runnability and internal consistency, not the paper's external benchmark numbers. The paper's headline scientific claims (C3 100% pure-cell accuracy on GSE141498 [Table 5], C4 SD=0.07 on GSE107572 [Fig 8], C5 10-fold CV PCC>0.79 [Fig 5], C6 TCGA-LIHC survival p=0.019 [Fig 10]) were NOT attempted: each requires downloading external bulk datasets plus re-running the authors' Harmony-Symphony R preprocessing (or full retraining) -- the optional hard ~20%, deliberately not chased per the 80/20 rule. No fabrication signal within attempted scope.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 67assessed: 2026-06-15 ⛓ 70f6f1cb5adf
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a deep neural network trained on batch-effect-corrected (Harmony-Symphony harmonized) scRNA-seq-derived simulated bulk RNA-seq accurately deconvolve the fractions of more than 10 immune cell types from bulk RNA-seq, and does removing batch effects in the reference scRNA-seq improve cell composition deconvolution?
- ★ HASCAD, a DNN-based cell composition deconvolution model, predicts the fractions of up to 15 immune cell types from bulk RNA-seq. method
- ★ Removing batch effects in reference scRNA-seq datasets via a Harmony-Symphony strategy benefits the cell composition deconvolution task. finding
- ★ HASCAD outperforms and is more suitable for analyzing bulk RNA-seq than the widely used CCD methods CIBERSORTx and quanTIseq. finding
- ★ In TCGA-LIHC liver cancer samples, predicted abundances of Treg and effector CD8 T cells are significantly associated with patients' overall survival. finding
- HASCAD can classify the cell type from pure bulk RNA-seq and predict cell composition of PBMC bulk RNA-seq. resource
- An ensemble of three parallel DNN modules with MSE+PCC composite loss improves deconvolution over Scaden-style architectures. method
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| scRNA-seq (training reference) | Human PBMC datasets PBMC6K (3' v1), PBMC8K (3' v2), PBMC5GEX (5') | none | Cell type-specific gene expression profiles for 15 immune cell types used to simulate bulk RNA-seq | 10x Chromium Single Cell 3' v1, 3' v2, and 5' paired-end chemistry |
| Simulated bulk RNA-seq deconvolution benchmark | In silico bulk RNA-seq simulated from harmonized scRNA-seq | none | Predicted vs ground-truth cell fractions (MSE, Pearson correlation) | — |
| Bulk RNA-seq deconvolution benchmark | Three human PBMC RNA-seq datasets | none | Predicted cell fractions compared against CIBERSORTx and quanTIseq | — |
| Bulk RNA-seq survival analysis | TCGA-LIHC liver hepatocellular carcinoma patient samples | none | Association of predicted Treg and effector CD8 T cell abundance with overall survival | — |
| DNN model training | Harmony-Symphony corrected scRNA-seq-derived simulated bulk RNA-seq | Harmony-Symphony batch correction vs no correction | Cell fraction prediction performance (MSE loss + PCC loss) | Adam optimizer, lr 0.0001, batch size 64, early stop after 20 epochs |
- ▲ HASCAD predicts fractions of 15 immune cell types, exceeding Scaden's limit of no more than 10 cell types 15 cell types
- ▲ Harmony-Symphony batch-effect removal in reference scRNA-seq improved cell composition deconvolution performance
- ▲ HASCAD is more suitable for bulk RNA-seq analysis than CIBERSORTx and quanTIseq in benchmarks
- – Predicted Treg and effector CD8 T cell abundance significantly associated with TCGA-LIHC overall survival
- count 15 immune cell types predicted (Number of immune cell types HASCAD deconvolves)
- count 3 scRNA-seq datasets (PBMC6K, PBMC8K, PBMC5GEX) (Reference PBMC scRNA-seq datasets used for training)
- other learning rate 0.0001, batch size 64, early stop after 20 epochs (DNN training hyperparameters)
- count 57 HCC patients (Tu et al. IHC analysis of ICOS+FOXP3+ Tregs cited in introduction)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
HASCAD is a deep neural network ensemble for cell composition deconvolution (CCD) trained on simulated bulk RNA-seq derived from three Harmony-Symphony batch-corrected PBMC scRNA-seq datasets. Model performance across 15 immune cell types was quantified using mean square error (MSE) and Pearson correlation coefficient (PCC) and benchmarked against CIBERSORTx and quanTIseq on both simulated and real PBMC datasets. The model was then applied to TCGA-LIHC data to evaluate associations between predicted immune cell abundances and patient overall survival, with results described as statistically significant; however, the specific inferential tests and full results section are not fully visible in the provided text excerpt.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Mean square error (MSE) | Benchmarking HASCAD vs. CIBERSORTx and quanTIseq on simulated and real PBMC bulk RNA-seq; also used as a component of the DNN training loss function | — | na |
| Pearson correlation coefficient (PCC) | Benchmarking HASCAD vs. CIBERSORTx and quanTIseq on simulated and real PBMC bulk RNA-seq; also used as a component of the DNN training loss function (PCC loss = 1 - r) | — | not stated |
| Survival analysis test (specific test not named in provided text excerpt) | Association of predicted Treg and effector CD8 T cell abundance with overall survival in TCGA-LIHC liver cancer cohort | — | not stated |
-
Model accuracy was benchmarked using MSE and PCC as the primary metrics across all cell types↳ Could also: Spearman rank correlation and root mean square error (RMSE) could also be reported as complementary metrics — Spearman correlation does not assume linearity and is less sensitive to outliers in predicted fractions, which can be common for rare cell types with near-zero true fractions; RMSE is on the same scale as the cell fraction values, which can aid interpretability compared to MSE
-
Ground-truth benchmarking relied primarily on simulated bulk RNA-seq generated by averaging single-cell profiles from the same scRNA-seq reference datasets↳ Could also: Experimentally mixed PBMC samples with independently measured ground-truth fractions (e.g., flow cytometry-validated dilution series) could also be used as a benchmark — Experimentally constructed mixtures with orthogonally measured cell fractions provide a validation that is not internally circular with the scRNA-seq training reference, which complements simulation-based evaluation
-
Batch effects in the reference scRNA-seq were addressed using Harmony followed by Symphony for query projection↳ Could also: Other integration methods such as ComBat-seq, Scanorama, scVI, or BBKNN could also have been applied and compared — Different batch-correction algorithms make distinct assumptions about batch-effect structure; a comparison of HASCAD trained on data corrected by alternative methods would characterize how sensitive downstream CCD performance is to the specific correction strategy chosen
-
The DNN was trained with a combined loss function of MSE and PCC (summed with equal weight)↳ Could also: Divergence-based loss functions such as Kullback-Leibler divergence or Jensen-Shannon divergence could also be considered for compositional outputs — Cell fractions are compositional data (non-negative, sum-to-one simplex-constrained); divergence measures explicitly designed for probability distributions may more naturally reflect the geometry of the output space than a sum of MSE and correlation losses
-
Survival associations for predicted cell types in TCGA-LIHC were reported as significant using an unspecified test↳ Could also: Multivariable Cox proportional hazards regression with predicted cell fractions as continuous covariates adjusted for clinical confounders (e.g., tumor stage, age) could also be applied — Multivariable Cox regression estimates the independent prognostic contribution of each predicted cell type while controlling for clinical variables, and yields hazard ratios with confidence intervals as interpretable effect-size estimates
-
Survival or association analyses appear to have been conducted across multiple predicted cell types simultaneously↳ Could also: A false-discovery rate correction such as Benjamini-Hochberg FDR could also be applied across the family of per-cell-type tests — Testing associations across 15 predicted cell types increases the number of simultaneous comparisons; FDR correction is a standard approach for controlling the expected proportion of false discoveries in such multi-hypothesis settings
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-37907883 (HASCAD)
Paper: HArmonized single-cell RNA-seq Cell type Assisted Deconvolution (HASCAD).
Chiu YJ, Ni CE, Huang YH. BMC Med Genomics 2023. DOI 10.1186/s12920-023-01674-w.
Code: https://github.com/holiday01/HASCAD @ commit 6dd61b5 (2022-11-25).
Data accession: GSE141498 (registry); paper also uses GSE107572, TCGA-LIHC.
What HASCAD is
A bulk-RNA-seq immune-cell deconvolution method: an ensemble of 3 parallel DNN
modules (each 3 hidden layers, node counts between 32 and 1024, softmax output),
trained on Symphony-harmonized PBMC scRNA-seq. Predicts fractions of 15 immune
cell types. The repo ships the 3 pretrained checkpoints (Model/model_pt/ checkpoint{1,2,3}.pt), the network definition (Model/network.py), the inference
notebook (Model/main.ipynb), the 1158-gene reference list (Source/gene.csv), the
15 cell-type labels (Source/celltype.csv), a worked example input
(Source/Example.csv), and the documented expected output (Result/result.png).
In scope (pipeline-derived, attempted)
- C1 — shipped-example deconvolution (Result/result.png). Load the 3 shipped
pretrained checkpoints, run the ensemble on
Source/Example.csvexactly asmain.ipynbdoes, obtain the 15 cell fractions. The repo'sResult/result.pngis the documented expected output → directly comparable. Deterministic (eval mode, no dropout). Pipeline: HASCAD inference (PyTorch). - C2 — model architecture vs Methods. Verify the shipped
network.pymatches the architecture described in the paper Methods (ensemble of 3 DNN, 3 hidden layers each, node range 32–1024, softmax, averaged outputs). Static check.
Out of scope / not attempted (the hard ~20%)
- Table 5 — pure-cell-type accuracy on GSE141498 (HASCAD 100% for B/CD4T/DC/ monocytes). Needs GEO download + pseudobulk construction + Harmony-Symphony R preprocessing (R 4.1, Symphony pkg, external reference download) + gene mapping.
- Figure 8 — human-PBMC benchmark on GSE107572 (SD of error 0.07; PCCs > 0.6). Same Harmony-Symphony preprocessing dependency on external bulk data.
- Figure 5 — 10-fold CV PCC > 0.79. Requires retraining on the 3 PBMC scRNA-seq datasets (no training data/splits shipped).
- Figure 10 — TCGA-LIHC survival (log-rank p = 0.019). Needs TCGA-LIHC bulk + clinical/survival data + downstream survival analysis.
Reason for deferral: each needs downloading external bulk datasets and re-running the authors' Harmony-Symphony R preprocessing (or full retraining) — beyond the low-hanging, clearly-specified inference output. Per the 80/20 rule these are the optional hard tail; C1 is the clean, deterministic, fully-specified result.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The reproduction cleanly and deterministically reproduces HASCAD's shipped example (all 15 cell fractions exact, sum=1.0) and confirms the architecture matches Methods (C2) — no deviation, no fabrication signal in attempted scope. However, C1 is self-referential (the repo regenerating its own Result/result.png), so it establishes runnability and internal consistency, not the paper's external claims. Every headline scientific result (Table 5 100% accuracy, Fig 8 SD=0.07, Fig 5 PCC>0.79, Fig 10 survival p=0.019) was NOT attempted because it needs external bulk data plus the authors' Harmony-Symphony preprocessing or retraining. This is our scope choice / methodology gap, not an authors' defect — hence overall yellow: solid for what was run, but the central conclusions remain unverified.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.