High performance imputation of structural and single nucleotide variants using low-coverage whole genome sequencing.
The main results reproduced: recomputed values matched the published ones within tolerance.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
PARTIAL reproduction. Paper: Atlantic-salmon low-coverage WGS imputation of SNVs+SVs via GLIMPSE v1.1.1, reference panel = 365 wild individuals (3,156,297 SNVs + 15,461 SVs). The headline imputation-accuracy numbers (SNV/SV PPV/recall @1-4x, in/out panel) are NOT reproducible and were not attempted (well-founded, verified at code level): the reference-panel VCF that every accuracy number depends on is not deposited by this paper, and reconstructing it requires re-running two upstream studies' SV/SNP calling pipelines. What IS reproducible 1:1 from the paper's OWN deposited data is out-panel sequencing depth, and it matches: raw depth = bases/genome = 15.80x vs reported 15.7x (within-tol, n=18 of 20 deposited); mapped depth via the paper's own aligner (BWA-MEM) = 15.82x read-span at 99.64% mapping rate over 7 samples (within-tol). Two of two attempted, deposit-derivable claims reproduced within tolerance. No completeness claim.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 85assessed: 2026-06-20 ⛓ 838db735bb31
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-22
- Rubric version
- not recorded
- Assessed by
- —
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper tests whether combined imputation of structural variants (SVs) and single nucleotide variants (SNVs) using low-coverage whole genome sequencing (WGS) against a species-level reference panel (via GLIMPSE) can achieve high-accuracy genotyping in Atlantic salmon, both within and external to the reference panel.
- ★ SNVs are imputed with high accuracy and recall across all tested WGS depths (1-4x), including in samples external to the reference panel. finding
- ★ Supplementing LD-based imputation with SV genotype likelihoods (GLs) from low-coverage WGS increases SV imputation accuracy but trades off against recall, with best performance requiring 3-4x depth. finding
- ★ Combining LD-only and SV-GL-supplemented imputation strategies captures 84% of reference panel deletions with 87% accuracy at only 1x depth. finding
- ★ SV length affects imputation performance, with provision of SV GLs greatly enhancing accuracy for the longest SVs in the dataset. finding
- ★ GLIMPSE, originally validated only for SNV imputation, was modified into a workflow that jointly imputes SNVs and SVs from low-coverage WGS against a phased reference panel. method
- ★ A merged reference panel of high-confidence SVs (deletions, duplications, inversions) and SNVs was built from 365 wild Atlantic salmon individuals spanning diverse European populations. resource
- Low-coverage WGS imputation against a reference panel offers a cost-effective strategy to achieve genome-wide SNV and SV coverage, with potential to enhance resolution of GWAS. mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Low-coverage whole genome sequencing with genotype imputation (GLIMPSE) | Atlantic salmon, in-panel wild individuals (n=98) from North/South Norway rivers | down-sampling of WGS depth to 1x, 2x, 3x, 4x (leave-one-out from reference panel) | positive predictive value (PPV) and recall rate for imputed SNV/SV genotypes vs reference genotypes | GLIMPSE v1.1.1 |
| Low-coverage whole genome sequencing with genotype imputation (GLIMPSE) | Atlantic salmon, out-panel farmed individuals (n=20, Hendrix Genetics Landcatch strain) | down-sampling of WGS depth to 1x, 2x, 3x, 4x; samples external to reference panel | PPV and recall rate for imputed SNV/SV genotypes vs gold-standard genotype calls | GLIMPSE v1.1.1 |
| SV genotyping from short-read WGS | Atlantic salmon, in-panel and out-panel down-sampled BAM files | none (genotyping of existing reference panel SVs) | SV genotype likelihoods (GLs), converted to phred-scaled likelihoods (PLs) for GLIMPSE input | SVTyper via Smoove/Lumpy pipeline |
| Whole genome sequencing and de novo SV calling | Atlantic salmon, out-panel farmed individuals (n=20) | none | high-confidence deletions, duplications, inversions; overlap with reference panel SVs | Illumina NovaSeq 6000; BWA alignment; Smoove/Lumpy v0.2.13; SV-plaudit curation; Bedtools intersect |
| Whole genome sequencing and SNV calling | Atlantic salmon, out-panel farmed individuals (n=20) | none | high-confidence SNV genotype calls used as 'true' genotypes for imputation comparison | DeepVariant (WGS model); GLnexus joint calling; BCFtools; Vcftools filtering |
| Population structure and relatedness analysis (PCA, genomic relationship matrix) | Atlantic salmon, merged SNV data from out-panel and reference panel individuals | none | PC1 vs PC2 clustering; mean genomic relationships between out-panel and North Norway/South Norway/whole reference panel | PLINK |
- ▲ SNV imputation achieved high accuracy and recall across all four WGS depths tested, for both in-panel and out-panel samples.
- – Adding SV genotype likelihoods to the imputation pipeline improved SV imputation accuracy relative to LD-only imputation, but reduced the proportion of SVs recalled.
- ▲ Best SV imputation performance using SV GLs required 3-4x sequencing depth.
- ▲ Merging LD-only and SV-GL-supplemented imputation approaches captured the majority of reference panel deletions with high accuracy even at minimal (1x) sequencing depth. 84% recall, 87% accuracy at 1x depth
- ▲ Provision of SV GLs markedly improved imputation accuracy specifically for the longest SVs in the dataset.
- count n = 365 (wild Atlantic salmon individuals comprising the reference panel)
- mean mean WGS depth = 8x (min 4.2x, max 17.5x) (sequencing depth of reference panel individuals)
- count 13,999 deletions, 1221 duplications, 241 inversions (curated high-confidence SVs in the reference panel)
- count 3,156,297 SNVs (high quality SNVs in the reference panel)
- count n = 98 (in-panel samples used for leave-one-out imputation benchmarking)
- mean mean depth 7.8x, s.d. 1.49x (WGS depth of in-panel samples)
- count n = 20 (out-panel farmed Hendrix Genetics Landcatch individuals)
- other 84% of reference panel deletions captured with 87% accuracy (combined imputation strategy performance at 1x depth)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This bioinformatic benchmarking study evaluated the GLIMPSE v1.1.1 imputation pipeline for joint SNV and SV imputation in Atlantic salmon using low-coverage WGS (1×–4× depth) against a reference panel of 365 wild individuals. Primary performance metrics were Positive Predictive Value (PPV) and recall rate, computed by comparing imputed genotypes to higher-coverage reference calls. In-panel performance was assessed via leave-one-out cross-validation (n=98), while out-panel performance used 20 farmed individuals as an independent external validation set. Results were reported descriptively across WGS depths, posterior probability cutoffs, variant classes, SV length bins, and imputation strategies, with no formal hypothesis tests or p-values.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Positive Predictive Value (PPV = correct genotype calls / (correct + incorrect genotype calls)) | Primary accuracy metric across all depths (1×–4×), posterior probability cutoffs (0.60, 0.75, 0.90), SV types (deletions, duplications, inversions), SV length bins, and imputation strategies (with/without SV GLs) | 98 in-panel samples (leave-one-out); 20 out-panel samples | na |
| Recall rate (proportion of reference panel variants imputed above posterior cutoff) | Complement to PPV across the same depth, cutoff, and strategy combinations | 98 in-panel samples; 20 out-panel samples | na |
| Principal Components Analysis (PCA via PLINK eigendecomposition of genetic covariance matrix) | Population structure visualization: out-panel vs. reference panel samples plotted on PC1 vs. PC2 | 365 reference panel + 20 out-panel samples | not stated |
| Genomic relationship matrix (PLINK) | Mean genomic relationships of 20 out-panel samples against North Norway, South Norway, and full reference panel sub-groups | 365 reference panel + 20 out-panel samples | not stated |
-
PPV and recall were reported as point estimates aggregated across samples at each condition, without uncertainty quantification↳ Could also: Bootstrap confidence intervals around PPV and recall, or per-sample distributions (e.g., boxplots across the 98 leave-one-out iterations), could also be reported alongside point estimates — Reporting variance across individuals would convey how consistently the method performs and whether observed differences between conditions exceed within-condition variability, helping distinguish stable improvements from noise
-
A proxy genetic map was constructed by assuming a uniform recombination rate of 1 Mb per cM across all chromosomes↳ Could also: An empirical, sex-averaged or sex-specific recombination map derived from pedigree linkage data or crossover inference could also be used if available for Atlantic salmon — An empirical map captures heterogeneity in recombination rate across the genome; more accurate local LD decay estimates can improve haplotype phasing and imputation accuracy, particularly in recombination hotspot or coldspot regions
-
A single imputation tool (GLIMPSE v1.1.1) was evaluated across all conditions↳ Could also: Additional low-coverage WGS imputation tools such as STITCH, BEAGLE (low-depth GL mode), or Minimac4 could also be benchmarked on the same reference panel and samples — A multi-tool comparison on identical data would contextualize GLIMPSE's performance and help determine whether the findings are method-specific or generalizable across imputation algorithms
-
SV length was divided into six fixed categorical bins for analysis of imputation accuracy by length↳ Could also: A continuous modeling approach (e.g., LOESS regression of PPV on log-transformed SV length) or quantile-based binning could also characterize the length–accuracy relationship — Continuous smoothing avoids sensitivity to arbitrary bin boundary placement and can reveal non-linear or threshold effects across the full length distribution more flexibly
-
Leave-one-out cross-validation removed one individual at a time from the 98 in-panel samples↳ Could also: A population-level hold-out design (removing all individuals from one or more river populations at a time) could also be used to assess cross-population generalization — Removing individuals from populations that remain well-represented in the training panel may produce optimistic performance estimates; population-level hold-outs would provide a more conservative benchmark matching the out-panel validation scenario
-
The merged imputation strategy combined outputs from pipelines with and without SV GLs using hard 'logical' or 'non-logical' filtering rules based on posterior likelihood↳ Could also: A soft ensemble approach averaging posterior genotype probabilities from both pipelines before genotype calling could also be used to combine the two strategies — Averaging posteriors propagates uncertainty from both pipelines and may produce better-calibrated genotype calls than a hard winner-take-all selection rule, potentially smoothing the PPV–recall trade-off
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-40155798
Paper: Gundappa MK et al. (2025) High performance imputation of structural and
single nucleotide variants using low-coverage whole genome sequencing. Genet Sel
Evol. PMID 40155798 · PMCID PMC11951665 · DOI 10.1186/s12711-025-00962-6.
Code: https://github.com/manugundappa/SV_imputation_pipeline (commit
6d9b14fa2270e082ea38c27fcd643f4ad8541abb, last push 2023-06-28).
Species/system: Atlantic salmon (Salmo salar), reference genome ICSASG_v2
(GCF_000233375.1, ~2.967 Gb).
What the paper does (pipeline overview)
Low-coverage WGS imputation of SNVs and structural variants (deletions, duplications, inversions) using GLIMPSE v1.1.1 against a reference panel of 365 wild individuals (3,156,297 SNVs + ~15.5k SVs). Tests on 98 in-panel and 20 out-of-panel (farmed) samples, downsampled to 1×/2×/3×/4×. Accuracy reported as PPV/recall vs posterior-probability cutoffs, plus a GLIMPSE+SVtyper "merged" variant for SV genotype likelihoods. Tools named: BWA, samtools v1.12, bcftools v1.12, DeepVariant, GLnexus, SMOOVE/Lumpy, SVTyper, Picard v2.23.8, PLINK, vcftools, GLIMPSE v1.1.1.
In scope (pipeline-derived) vs out of scope
| Reported result | Pipeline | In scope? | Note |
|---|---|---|---|
| Out-panel sample sequencing depth (mean 15.7×±1.1×) | read QC / alignment depth | YES (attempted) | derivable from PRJNA917857 (deposited) + genome size; verifiable with BWA+samtools |
| SNV imputation PPV/recall @1–4× (in/out panel) | GLIMPSE + concordance R | NO (not attempted) | requires the 365-sample reference panel VCF, which is not deposited by this paper |
| SV (DEL/DUP/INV) imputation PPV/recall | GLIMPSE(+SVtyper) | NO | same blocker; plus SV input VCFs (allSV.vcf.gz) not shipped |
| Reference-panel composition (SNV/SV counts) | bcftools merge of upstream VCFs | NO | input SNP+SV VCFs from Bertolotti/Sinclair-Waters not shipped; filtToHQSVs.R references local files |
Reproducibility blockers (why the headline results are not attempted)
- Reference panel not deposited. Data-availability statement: only the 20 out-panel samples (PRJNA917857) are deposited by this paper. The 365-sample reference panel (SNV+SV genotypes) — the critical input to every imputation accuracy number — points to two prior studies (Bertolotti et al. [29], Sinclair-Waters et al. [30]) and is not provided as a usable merged VCF.
- Code is cluster submission scripts, not a runnable pipeline. The repo ships
Sun-Grid-Engine/
qsubsubmit*.shwrappers + R scripts with hardcoded local paths, hardcoded VCF header-skip line numbers (vroom(skip=232762)), hardcoded sample IDs, and no pinned environment (noenvironment.yml/Dockerfile/ versions), no test data, and no expected-output fixtures. - Per HARD RULE 3 (80/20) these constitute the hard "last 20%" — reconstructing the reference panel from two upstream papers is a separate multi-study effort.
What we DO reproduce (clear, deposited-data-only data point)
Out-panel sample sequencing depth. Using only the paper's deposited reads (PRJNA917857, 18 public runs) and the ICSASG_v2 genome size we reproduce the reported mean depth, and we additionally align one representative sample on «our HPC» with BWA-MEM + samtools (third-party tools on the paper's own data, HARD RULE 2 / P16) to confirm the mapped depth is consistent. This is one honest, auditable 1:1 comparison; we make no completeness claim about the imputation results.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
<synthetic>Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.