Digital sorting of complex tissues for cell type-specific gene expression profiles.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Successfully reproduced the paper's core computational benchmark (Figure 1, GSE19830 liver/brain/lung tissue-mixture deconvolution) using the DSA R package (v1.0) on «our HPC» HPC: estimated mixing proportions correlate with true proportions at r=0.977, and deconvolved cell-type-specific expression profiles correlate with pure reference profiles at r=0.96-0.98 across all three tissues, consistent with the paper's qualitative 'highly correlated' claims (no exact numeric targets exist in the paper text for stricter grading). The GSE19830 dataset (42 samples, rat, matching the paper's stated triplicate liver/brain/lung mixture design) was independently downloaded and profiled and matches expectations fully. The main technical obstacle - the quadprog R package (required for the QP-constrained deconvolution step) failing to compile due to a missing gfortran on the default compute-node PATH - was resolved by locating an alternate gfortran binary under a different pkgsrc environment and prepending it to PATH, after the intended 'matching' compiler module (env/gcc-13.3.0_openmpi-5.0.7) turned out to conflict with the default loaded environment module and still lacked gfortran itself. All artifacts (JSON summary, bundled input CSVs, estimated-weight CSVs, dataset profile) are saved under the results directory for human audit.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-07-28
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-07-31no human curator yet
- Last updated
- 2026-07-31
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan cell type-specific gene expression profiles be extracted from heterogeneous (mixed) tissue samples without prior knowledge of cell type frequencies or in vitro reference expression profiles, using only sets of marker genes known to be highly expressed in each cell type?
- ★ The Digital Sorting Algorithm (DSA) deconvolves mixed tissue expression into cell type-specific profiles using only marker genes, without requiring prior knowledge of cell type frequencies or in vitro pure-cell profiles. method
- ★ DSA accurately estimates cell type proportions in mixed samples from marker genes alone. finding
- ★ DSA is an unbiased estimation algorithm for signal reconstruction and deconvolution, estimating absolute expression levels rather than marker-relative values. finding
- ★ DSA fold-change estimates are more accurate than PSEA, which is biased by differing marker gene expression levels across cell types. finding
- ★ Deconvolved profiles support specific and sensitive differential gene expression analysis between cell types. finding
- ★ Cell types present above 10% frequency are accurately estimated; 1-10% is estimable with larger error; below 1% fails. finding
- ★ DSA-estimated tumor-associated macrophage (TAM) percentage in Hodgkin's lymphoma predicts progression-free survival better than CD68 as a marker. finding
- DSA is implemented as an R package with sample benchmark data and cell type-specific markers (https://github.com/zhandong/DSA). resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Microarray gene expression profiling (benchmark tissue mixtures) | Liver, brain and lung tissue from a single rat, homogenized and mixed in 11 different proportions in triplicates, plus pure tissues | Defined RNA/tissue mixing at known proportions | Genome-wide expression; estimated vs. true cell type proportions and expression profiles; correlation and mean absolute difference | Affymetrix expression array (GSE19830) |
| Microarray gene expression profiling (immune cell line mixtures) | IM-9, Raji, Jurkat and THP-1 human cell lines mixed in different proportions | Defined cell mixing at known proportions | Estimated cell type frequencies and deconvolved per-cell-type expression profiles vs. pure cell profiles | Microarray (GSE11057) |
| In silico simulation of mixed blood samples from microarray reference profiles | 6 immune cell types (eosinophil, myeloid dendritic, mature B-cells, granulocyte, naïve B-cells, basophils) | Simulated mixing with randomly sampled decreasing weights, proportions from 60% to 0.1% | Estimated vs. measured transcriptomes; MSE and weight vs. signal-to-noise ratio (SNR) | Microarray reference data (GSE24759) |
| ROC / AUC analysis of differential expression detection | Deconvolved profiles from rat tissue benchmark, immune cell line mixtures, and simulated blood | None (computational evaluation) | True positive rate vs. false positive rate for genes with >2-fold change; AUC | R package 'ROCR' (CRAN) |
| Comparative deconvolution benchmarking (DSA vs. PSEA) | Rat liver-brain-lung benchmark mixtures | None (algorithm comparison) | Estimated fold change between liver and brain vs. true fold change | — |
| Microarray-based deconvolution of clinical tumor samples with survival analysis | Human Hodgkin's lymphoma tumors | None (observational) | Estimated TAM percentage; Cox proportional hazard model, dichotomization at median TAM percentage, log rank test for progression-free survival | Microarray (GSE17920); TAM markers from GSE18404 |
| Gene Ontology enrichment analysis | Genes highly expressed in DSA-extracted TAMs vs. tumor transcriptome (Hodgkin's lymphoma) | None | Enriched biological process terms | — |
| Marker gene number robustness analysis (random subsampling) | Rat liver-brain-lung benchmark mixtures; markers sampled from TIGER list | Random sampling of marker genes, 100 repetitions | Correlation and mean absolute difference between estimated and pure cell-specific expression vs. number of markers | — |
- – DSA-estimated mixing proportions recapitulate the true percentage of liver, brain and lung in the rat mixtures.
- – Deconvolved liver, brain and lung expression profiles were highly correlated with true profiles measured on pure tissue; error was smaller for highly expressed genes.
- – ROC analysis of deconvolved immune cell line profiles yielded high AUC values, indicating accurate identification of differentially expressed genes. AUC 0.8 or higher
- – PSEA-estimated fold changes between liver and brain were all negative because liver marker genes are expressed at higher levels than brain markers, whereas DSA fold changes correlated highly with the truth.
- – Cell types above 10% frequency were accurately estimated; >1% but <10% estimated with larger errors; <1% failed to be identified. proportions ranging from 60% to 0.1%
- ▲ MSE of deconvolution increases as SNR decreases with lower cell type frequency; best cut-off point observed around SNR 45. SNR cut-off ~45
- ▼ DSA-estimated TAM percentage in Hodgkin's lymphoma was negatively associated with progression-free survival, outperforming CD68 as a TAM marker. 30-fold lower p-value than CD68; hazard ratio 2.7 (TAM high vs. TAM low)
- ▲ Genes highly expressed in DSA-extracted TAMs were significantly enriched for response to wounding, defense response, and inflammatory response.
- – DSA is robust to the number of marker genes used and requires only several markers for accurate deconvolution. 100 repetitions of random marker sampling
- other AUC 0.8 or higher (ROC analysis of differential expression using deconvolved immune cell line (IM-9 vs. Jurkat and related) profiles)
- other hazard ratio 2.7 (TAM high vs. TAM low group, progression-free survival in Hodgkin's lymphoma)
- pvalue 30-fold lower p-value than using CD68 (Survival association of DSA-estimated TAM percentage vs. CD68 marker)
- other best cut-off point around 45 (Signal-to-noise ratio threshold below which MSE increases dramatically)
- count 11 different proportions, in triplicates (Rat liver/brain/lung benchmark mixture design (GSE19830))
- count 6 different immune cell types (Simulated blood samples with proportions from 60% to 0.1%)
- count 100 repetitions (Random marker gene subsampling to test robustness to marker number)
- count Five tumor associated macrophage (TAM) marker genes (TAM markers selected by comparing mouse mammary tumor macrophages to normal splenic macrophages)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a computational methods paper introducing a Digital Sorting Algorithm (DSA) for deconvolving mixed-tissue gene expression into cell-type-specific profiles, validated on benchmark microarray mixing experiments (rat liver/brain/lung; immune cell lines) and on a clinical Hodgkin's lymphoma dataset. Performance was assessed mainly through correlation between estimated and true expression profiles, ROC/AUC analysis for detecting differentially expressed genes, and a Cox proportional hazards model with log-rank test for a survival association; results were reported largely through figures (scatter plots, ROC curves, correlation trends) rather than tabulated inferential statistics.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Cox proportional hazards model | Association between estimated TAM (tumor-associated macrophage) percentage and progression-free survival in Hodgkin's lymphoma (Figure 6a) | — | not stated |
| Log-rank test | Comparison of progression-free survival between TAM-high and TAM-low groups, dichotomized at the median TAM percentage (Figure 6a) | — | not stated |
| ROC/AUC analysis | Assessing sensitivity/specificity of detecting >2-fold differentially expressed genes from DSA-deconvolved profiles (Figure 1e-f, Figure 2c, Figure 5a-c, Additional file 5: Figure S3) | — | not stated |
| Correlation and mean absolute difference (comparison of estimated vs. true expression) | Accuracy of DSA-estimated cell-type expression/fold-change vs. true pure-tissue expression, including across 100 repetitions of random marker sampling (Figure 1b-d, Figure 3a, Additional file 2: Figure S1) | 100 repetitions (for marker-number analysis) | not stated |
| Gene Ontology (GO) enrichment analysis | Identifying biological processes enriched among genes highly expressed in DSA-extracted TAM profiles (Figure 6b) | — | not stated |
-
Differential expression detection was evaluated using ROC/AUC analysis against a fixed >2-fold change threshold as the reference 'true positive' definition.↳ Could also: A model-based differential expression framework (e.g., limma, DESeq2, or a moderated t-test) with FDR-adjusted p-values as the ground truth for differential calls — This would let the accuracy assessment reflect a statistically defined significance threshold in addition to a fold-change magnitude, which can be complementary when evaluating deconvolution performance.
-
The survival benefit of the DSA-based TAM estimate over a CD68 marker-based estimate was reported as a relative p-value comparison ('30-fold lower') rather than as two exact p-values or a formal statistical test of difference between the models.↳ Could also: Reporting both exact p-values with confidence intervals for each Cox model, or a formal model-comparison statistic (e.g., likelihood ratio test, C-index comparison) — This would allow readers to directly quantify and compare the precision and magnitude of each model's association with survival.
-
GO enrichment of TAM-associated genes was described as 'highly significantly enriched' without a stated correction method for the many gene-ontology categories tested.↳ Could also: An explicit multiple-testing correction (e.g., Benjamini-Hochberg FDR) with a stated significance cutoff, as commonly used in GO enrichment tools — Explicit correction and threshold reporting make the enrichment magnitude and stringency transparent and reproducible for readers.
-
Patients were dichotomized into TAM-high and TAM-low groups at the median for the log-rank test.↳ Could also: Modeling TAM percentage as a continuous covariate in the Cox model (in addition to the categorical log-rank comparison), or exploring alternative cutoff selection methods (e.g., maximally selected rank statistics) — Continuous modeling avoids information loss from dichotomization and can reveal whether the association holds across the full range of TAM percentages, complementing the group-based log-rank result.
-
The comparison between DSA and PSEA fold-change accuracy was presented through visual/correlation comparison against the true fold change rather than a formal statistical test of the difference in accuracy between the two methods.↳ Could also: A paired statistical comparison of residuals or bootstrap confidence intervals on the correlation coefficients for each method — This would provide a quantitative, testable measure of how distinguishable the two methods' accuracy levels are, complementing the visual comparison.
-
Variability across the 100 repetitions of random marker sampling (for assessing robustness to marker gene number) was summarized as a trend in correlation and mean absolute difference.↳ Could also: Reporting dispersion across repetitions explicitly (e.g., SD, IQR, or 95% CI bands/boxplots at each marker-count level) — Explicit dispersion measures would let readers assess the consistency of DSA's robustness claim across repetitions, not just the central trend.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The paper's central Figure 1 benchmark reproduces cleanly: running the authors' own published DSA package on the public GSE19830 rat liver/brain/lung mixture series recovers mixing proportions at r=0.9768 and deconvolved cell-type profiles at r=0.9733/0.9639/0.9789 (liver/brain/lung), fully consistent with the paper's 'recapitulate the true percentage' and 'highly correlated'. The dataset itself checks out independently (42 samples, 9 pure + 33 mixture = 11 recipes x triplicate, n_reported = n_observed). Two caveats keep this from a 1:1 green: the paper states no numeric r anywhere, so agreement can only be judged against prose (q2 yellow), and the pipeline input was the authors' bundled triplicate-averaged mix.signals rather than a from-scratch RMA of the raw CEL files, with Figure 3 (PSEA comparison) out of scope — both deviations are on our side as scoping choices, not authors' defects. Nothing here is fabrication-suspect; the 0.96-0.98 range is convincingly imperfect, and the incidental QC finding (unconstrained LM yields 5064/93297 NaNs vs zero for QP) independently corroborates the paper's methodological recommendation.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.