SpaGene: A Deep Adversarial Framework for Spatial Gene Imputation.
The main results reproduced, with only marginal, non-material deviations.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH and reproduces ~1:1. Authors' own code (SpaGene @589ae46, seed=0) run on the paper's own public data (Zenodo 3967291, md5 verified) on a «our HPC» H100 (g004). Of the 8 imputation pairs, the 5 whose data is in the public deposit all reproduced; both pinned-value pairs match within tolerance: STARmap_AllenVISp PCC 0.2005 vs 0.205 and RMSE 1.258 vs 1.254; osmFISH_Zeisel SSIM 0.441 vs 0.447 (osmFISH shared-gene count 32 = paper exact). Reproduced 5-pair mean SSIM 0.400 ~ paper 8-pair 0.403 (RMSE 1.190 vs 1.167), independently supporting the headline. Every reproduced value is IDENTICAL to 5 decimals to an independent prior run on a different toolchain (conda/py3.10 vs this venv/py3.12) -> strong determinism evidence. NOT attempted: 3 pairs whose data is registration-gated or absent from the deposit -> data-access limits. One honest discrepancy: STARmap shared-gene count 418 vs reported 242 (metrics still reproduce -> likely unstated gene filter, flagged). No fabrication indicators. Undocumented prep step (Zenodo *_scvi.csv -> .pkl) reconstructed and verified vs codes/data_utils.py. This re-run was triggered because the janitor reclaimed the prior «infra» work dir; rebuilt entirely SLURM-only (conda libmamba broken on compute nodes -> venv+pip; shared «infra» quota volatile -> build+train on compute-node-local /tmp).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 72assessed: 2026-06-19 ⛓ a50a92d90de7
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-22
- Rubric version
- not recorded
- Assessed by
- —
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper tests whether a deep adversarial encoder-decoder-translator framework (SpaGene) can more accurately impute unmeasured genes in spatial transcriptomics (ST) data by leveraging reference scRNA-seq (SC) data than existing integration methods.
- ★ SpaGene improves average PCC and SSIM and reduces RMSE compared to 6 baseline methods (SpaGE, gimVI, Tangram, VISTA, spRefine, stDiff) across 8 diverse ST-SC dataset pairs under gene-holdout evaluation. finding
- ★ SpaGene architecture uses 2 encoder-decoder pairs, 2 translators, and 2 discriminators to translate latent representations between ST and SC domains and impute missing gene expression. method
- ★ Translation is anchored by an identity constraint restricted to shared genes between ST and SC data to align domains without forcing data-space correspondence on unshared genes. mechanism
- ★ Representation learning is decoupled from adversarial alignment by pretraining modality-specific autoencoders and freezing them during translator/discriminator training, improving stability and reducing oversmoothing. mechanism
- ★ Application of SpaGene to lung tumor tissue (NanoString Lung9 rep1) reveals spatial patterns consistent with immune cell enrichment at tumor boundaries and restricted myeloid cell presence in adjacent normal regions. finding
- SpaGene performance improves with greater shared gene overlap between ST and SC data and degrades progressively as ST data sparsity increases. finding
- SpaGene does not generate strong false-positive spatial autocorrelation patterns, as most imputed genes show only modest shifts in Moran's I relative to measured expression. finding
- SpaGene shows competitive recovery of spatially variable genes (via SPARK-X AUPRC) compared to other imputation methods. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| ST-SC gene imputation benchmark (gene-holdout cross-validation) | 8 dataset pairs: MERFISH_Moffitt, NanoString_GSE, osmFISH_AllenSSp, osmFISH_AllenVISp, osmFISH_Zeisel, seqFISH_AllenVISp, STARmap_AllenVISp, Xenium_breast_GSE_breast | computational gene holdout | Pearson correlation coefficient (PCC), structural similarity index (SSIM), root mean squared error (RMSE), Wasserstein distance, Jensen-Shannon divergence | — |
| 5-fold cross-validation gene imputation | NanoString Lung9 rep1 lung tumor tissue with SC reference | gene holdout | PCC and spatial expression patterns of STMN1, NDRG1, CD163 | NanoString CosMX Spatial Molecular Imager (SMI) |
| spatially variable gene (SVG) detection with SPARK-X | 8 ST-SC dataset pairs | none | area under precision-recall curve (AUPRC) for SVG recovery | SPARK-X |
| shared gene overlap ablation | osmFISH_Zeisel and NanoString_GSE dataset pairs | retaining 10%, 25%, 50%, 75%, or 100% of shared genes for training | PCC, SSIM, RMSE | — |
| ST data sparsity ablation | osmFISH_Zeisel and NanoString_GSE dataset pairs | masking 0%, 25%, 50%, 75%, or 90% of nonzero ST expression values | PCC, SSIM, RMSE | — |
| spatial autocorrelation comparison (Moran's I) | MERFISH_Moffitt dataset, 141 held-out genes | none | ΔMoran's I between imputed and measured gene expression | — |
- ▲ SpaGene achieved average PCC of 0.303 across 8 datasets, higher than all 6 baselines (37.1-79.3% improvement) 37.1% to 79.3% higher PCC
- ▲ SpaGene achieved average SSIM of 0.403 across 8 datasets, higher than all 6 baselines 14.2% to 39.9% higher SSIM
- ▼ SpaGene achieved average RMSE of 1.167 across 8 datasets, lower than all 6 baselines 5.4% to 9% lower RMSE
- ▼ SpaGene achieved lowest average Wasserstein distance and JS divergence across 8 datasets Wasserstein = 0.256, JS = 0.544
- ▲ On NanoString Lung9 rep1, SpaGene achieved average PCC of 0.246, higher than SpaGE, gimVI, Tangram, VISTA, spRefine, and stDiff PCC 0.246 vs 0.119-0.198 for baselines
- ▲ On MERFISH_Moffitt dataset pair, SpaGene achieved average PCC of 0.381, higher than all baselines PCC 0.381 vs 0.119-0.345 for baselines
- – Spatial autocorrelation shift (ΔMoran's I) between imputed and measured expression was generally modest across held-out genes median ΔMoran's I = 0.022, mean = 0.0145, 78.7% genes within |Δ| ≤ 0.10
- ▲ Lung tumor tissue application showed immune cell enrichment at tumor boundaries and restricted myeloid cell presence in adjacent normal regions
- mean average PCC = 0.303 (SpaGene overall PCC across 8 dataset pairs)
- mean average SSIM = 0.403 (SpaGene overall SSIM across 8 dataset pairs)
- mean average RMSE = 1.167 (SpaGene overall RMSE across 8 dataset pairs)
- fold_change 37.1% to 79.3% higher PCC than baselines (SpaGene vs 6 baseline methods, average PCC)
- other Wasserstein = 0.256, JS divergence = 0.544 (SpaGene lowest average distribution-based metrics across 8 datasets)
- count 141 held-out genes (Moran's I spatial autocorrelation analysis on MERFISH_Moffitt)
- mean median ΔMoran's I = 0.022, mean ΔMoran's I = 0.0145 (spatial autocorrelation shift between imputed and measured expression)
- other 78.7% of genes within |ΔMoran's I| ≤ 0.10 (proportion of genes without strong false-positive spatial pattern inflation)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
SpaGene is evaluated in a computational benchmarking study in which 8 ST–SC dataset pairs serve as the unit of replication (n = 8). Performance is characterized across five metrics (PCC, SSIM, RMSE, Wasserstein distance, JS divergence), summarized as averages and displayed as box plots. To assess whether SpaGene consistently outperforms each of six baseline methods, paired t-tests and Wilcoxon signed-rank tests were applied across the eight dataset pairs, with FDR correction applied to the resulting p-values (Table S2). Spatial autocorrelation fidelity of imputed genes was additionally assessed using Moran's I on 141 held-out genes from the MERFISH_Moffitt dataset.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Paired t-test | Comparison of SpaGene vs each of 6 baseline methods across 8 dataset pairs per performance metric | n = 8 dataset pairs | not stated |
| Wilcoxon signed-rank test | Same paired comparisons as the t-test, applied as a non-parametric complement | n = 8 dataset pairs | not stated |
| SPARK-X (nonparametric spatial variability test) | Identification of spatially variable genes (SVGs) from measured spatial data; used as reference set for AUPRC-based SVG recovery evaluation | null | not stated |
| Moran's I (spatial autocorrelation statistic) | Assessment of spatial pattern fidelity (ΔMoran's I) for imputed vs measured expression in MERFISH_Moffitt dataset | n = 141 held-out genes | na |
-
Paired t-tests and Wilcoxon signed-rank tests were conducted with n = 8 dataset pairs as the statistical unit↳ Could also: A permutation test or bootstrap confidence interval on the mean performance difference could also be used — With n = 8, the normality assumption underlying the paired t-test is difficult to verify; permutation or bootstrap approaches make no distributional assumption and are often preferred in small-n computational benchmarking studies
-
FDR correction was applied, but the specific procedure and the scope of the test family are not stated↳ Could also: An explicitly named procedure (e.g., Benjamini-Hochberg) applied to a clearly defined family—for example, 6 methods × 5 metrics = 30 tests—could also be used — Naming the FDR algorithm and defining the family of tests makes the correction fully reproducible and allows readers to evaluate whether the family definition is appropriate for the inference being drawn
-
Pearson correlation coefficient (PCC) served as the primary gene-level performance metric↳ Could also: Spearman rank correlation or the concordance correlation coefficient (CCC) could also be used — Spearman correlation is robust to outliers and non-normal expression distributions; CCC captures both precision and accuracy (systematic bias) jointly and is commonly used in method-comparison studies where agreement rather than linear association alone is the quantity of interest
-
Percentage improvements over baselines were reported as the effect-size summary↳ Could also: A standardized effect size such as Cohen's d or Hedges' g computed from the paired differences could also be reported — Standardized effect sizes quantify practical significance on a common scale independently of the metric's original units, making it easier to compare the magnitude of gains across PCC, SSIM, and RMSE simultaneously
-
Each of the 8 dataset pairs was treated as an independent replicate and a separate paired test was run per baseline method↳ Could also: A linear mixed-effects model with method as a fixed effect and dataset pair as a random effect could also be used — A mixed-effects framework enables simultaneous inference across all methods, explicitly partitions dataset-level variability, and avoids multiplying the number of individual tests while naturally accommodating the repeated-measures structure
-
Spatial pattern fidelity was summarized by the shift in Moran's I (ΔMoran's I) but without a formal significance test for individual genes↳ Could also: A paired Wilcoxon test or sign test on per-gene Moran's I values between measured and imputed data could also be reported alongside the descriptive summary — A formal test would allow a probabilistic statement about whether the observed median shift (0.022) is distinguishable from zero across the gene set, complementing the descriptive percentage of genes within a threshold
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.