Genetic polyploid phasing from low-depth progeny samples.
The main results reproduced, with only marginal, non-material deviations.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
PARTIAL/IN-PROGRESS. WhatsHap polyphasegenetic (frozen Zenodo 6519173, commit b2f9b825) built + run on «our HPC» over Zenodo 5645743 region VCFs (chr3/4/5) at C=0 and C=2, then whatshap compare vs deposited hifiasm ground truth. RESULT: (1) single phasing block per region EXACTLY reproduced (1 block, all 6 runs). (2) #phased variants close: chr3 4522 vs 3934, chr4 3447 vs 3127, chr5 5355 vs 5096 (~10-15% high). (3) error rates same order: chr4 C=0 SER 0.32% vs 0.26% (near-exact), chr5 1.26% vs 0.82%, chr3 2.91% vs 0.97% (3x). chr3 truth had duplicate positions (compare warning) -> refinement «job» re-compares vs synchronize_alt_alleles + dedup truth to test if that closes the gap. 193 progeny confirmed (ped.txt). Build needed 3 fixes: <cstdint> include, TMPDIR, Debian-multiarch linker (compiler_compat/ld removed).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-25
- Rubric version
- not recorded
- Assessed by
- —
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper tests whether the Mendelian inheritance signal contained in a large panel of low-depth (~6x) progeny sequencing samples can be combined with two deep-sequenced parental samples to accurately phase the parental haplotypes of an autopolyploid species (tetraploid potato), circumventing the read-connectivity limitations of purely read-based polyploid phasing.
- ★ WH-PPG phases polyploid parental samples by scoring informative variant pairs with a Bayesian log-likelihood model of progeny allele depths, clustering alleles by co-occurrence likelihood, and assigning clusters to haplotypes via interval scheduling method
- ★ WH-PPG produces high-quality sparse haplotype phasings with Hamming error rates below ~3% on validated regions containing several thousand variants finding
- ★ WH-PPG scales to whole genomes, phasing each of the 12 potato chromosomes in a total of 300 CPU hours (max 40h for one chromosome) finding
- ★ Simplex-nulliplex variants are the most informative and confidently phasable variant type because they carry a unique, easy-to-trace allele mechanism
- ★ Phasing continuity from genetic phasing is limited by the recombination rate rather than by read connectivity, enabling longer haplotype blocks than read-based approaches finding
- A reduced progeny population of about 60 samples still yields sufficiently low error rates compared to using the full 193 samples finding
- ★ Including simplex-simplex and duplex-nulliplex variants increases the fraction of phased variants but substantially raises the error rate (roughly tripling Hamming rate) finding
- WH-PPG is released as part of the widely-used WhatsHap phasing suite resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Illumina short-read whole-genome sequencing + GATK variant calling | Solanum tuberosum, parental samples 'Altus' and 'Colomba' plus 193 progeny samples | none | genotypes and allele depths at heterozygous biallelic variants | Illumina 250bp paired-end reads; GATK |
| PacBio HiFi long-read sequencing + de novo assembly (hifiasm) | Solanum tuberosum parental sample 'Altus' | none | ground-truth 4-haplotype assembly for 3 validation regions (chr3, chr4, chr5) | hifiasm v0.13 |
| WH-PPG genetic phasing algorithm applied to 3 validation regions | Solanum tuberosum chromosome 3, 4, 5 regions (~300kb each) | variant-type inclusion (simplex-nulliplex only vs. + simplex-simplex vs. + duplex-nulliplex) | switch error rate (SER), wrong genotype rate (WGR), Hamming rate (HR), number of phased variants | WhatsHap polyphase genetic (WH-PPG) |
| Parental coverage downsampling with GATK variant recalling, then WH-PPG phasing | Solanum tuberosum chromosome 3, 4, 5 regions | parental read coverage downsampled to 50% and 25% | SER, WGR, HR, number of phased variants | GATK; WH-PPG |
| Progeny subsampling and WH-PPG phasing | Solanum tuberosum, subsamples of 15-150 progeny drawn from 193 | progeny sample size reduction (10 random subsamples per size) | SER, HR, number of phased variants | WH-PPG |
| Whole-chromosome WH-PPG phasing runs | all 12 Solanum tuberosum chromosomes | none | runtime (CPU hours), peak memory, fraction of simplex-nulliplex variants phased | WH-PPG, single-core runs |
| HiFi read alignment to reference to estimate read-based phasing block sizes | Solanum tuberosum reference genome (Solyntus V1.1) | none | estimated block size (N50/N90) achievable by read-based phasing per chromosome | HiFi reads aligned to reference |
- ▼ Hamming rate below 1% on chromosome 4 and 5 validation regions using default (simplex-nulliplex only) settings <1%
- ▲ Chromosome 3 region shows higher Hamming rate, close to 3% 2.89%
- – WH-PPG phases about 25-41% of bi-allelic variants using only simplex-nulliplex variants, rising to 36-61% when simplex-simplex and duplex-nulliplex variants are added, but at the cost of roughly tripling the Hamming rate 25-41% -> 36-61%
- – Whole-genome phasing completed in 300 total CPU hours; chromosome 3 had the highest runtime (40h) and chromosome 6 the highest peak memory (44GB) 300 CPU hours; 40h max; 44GB max
- ▲ Genome-wide, about 80% of simplex-nulliplex variants (occurring roughly every 100bp) were phased ~80%
- ▼ Reducing parental coverage to 25% led to about a 20% overall decline in number of phased variants, without a clear consistent change in error rates ~20% decline
- ▲ Error rates increase and phased-variant counts drop as progeny pool size decreases, with samples below 60 progeny falling off notably from the rest
- ▼ Estimated read-based phasing block N50 is less than 20% of chromosome length (except chromosomes 7 and 10) and N90 less than 5%, indicating genetic phasing yields substantially longer blocks N50<20%; N90<5%
- other SER 0.97%, WGR 3.46%, HR 2.89%, 3,934 phased (chr3 region, simplex-nulliplex only, default settings)
- other SER 0.26%, WGR 1.37%, HR 0.63%, 3,127 phased (chr4 region, simplex-nulliplex only, default settings)
- other SER 0.82%, WGR 0.90%, HR 0.92%, 5,096 phased (chr5 region, simplex-nulliplex only, default settings)
- count 193 progeny samples (total progeny panel used for genetic phasing)
- other ~6x average progeny sequencing depth; >300x parental depth (sequencing coverage of progeny vs. parental samples)
- other 24x average HiFi coverage per haplotype (HiFi read coverage for Altus ground-truth assembly)
- other 300 CPU hours total; 40h max (chr3); 44GB max memory (chr6) (computational resources for whole-genome phasing)
- fold_change duplex-nulliplex inclusion raises HR from 2.89% to 8.88% (chr3) (effect of adding duplex-nulliplex variants on chr3 error rate)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a computational/methods paper describing a new algorithm (WhatsHap polyphase genetic, WH-PPG) for genetic phasing of polyploid genomes, combining a Bayesian log-likelihood model, allele clustering, and interval scheduling. Evaluation was performed by computing descriptive accuracy metrics (switch error rate, wrong genotype rate, Hamming rate) on three manually selected validation regions with HiFi-read-derived ground truth, and by repeating a progeny-subsampling experiment 10 times to observe how accuracy degrades with smaller sample sizes. Results are reported as point values or means across regions/subsamples, without formal hypothesis testing, p-values, or confidence intervals.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Bayesian log-likelihood scoring model (used to classify whether allele pairs co-occur on the same haplotype, based on Mendelian segregation) | Scoring of simplex-nulliplex (and other) variant pairs from progeny allele depths, feeding the clustering/interval-scheduling steps (Figure 1, Figure 5) | 193 progeny samples per parental pair, as stated | stated (Mendel's law of segregation is explicitly invoked as the modeling assumption) |
-
The progeny pool-size experiment (Figure 2) reports only the mean SER/HR/phased-variant count across 10 random subsamples per pool size.↳ Could also: Reporting the spread across the 10 replicates (e.g., SD, range, or a boxplot) or a bootstrap confidence interval — This would convey how variable the accuracy metrics are at each pool size, letting readers judge how consistently the method performs across resamples, not just the central tendency.
-
The parental coverage downsampling to 50% and 25% was performed as a single run per region rather than repeated, as the authors themselves note when discussing the resulting non-monotonic error pattern.↳ Could also: Repeating the downsampling multiple times (e.g., several random 50%/25% subsamples of parental reads per region) and summarizing the resulting distribution of error rates — Multiple replicates would help distinguish a stable coverage effect from single-run sampling variability, directly addressing the uncertainty the authors flag ('owing to a single downsample experiment instead of multiple ones').
-
Differences in error metrics (SER, WGR, HR) across variant-type inclusion settings (simplex-nulliplex only vs. + simplex-simplex vs. + duplex-nulliplex) are compared descriptively across the three validation regions without a formal statistical test.↳ Could also: A paired non-parametric test (e.g., Wilcoxon signed-rank test) treating region as the pairing unit, or a mixed-effects model with region as a random effect — This would let the observed increase in error rate with additional variant types be summarized with a test statistic and p-value in addition to the raw metric values, which can be useful when extending the comparison to more regions or datasets.
-
Accuracy metrics (SER, WGR, HR) are reported as single point values per region/condition with no uncertainty quantification.↳ Could also: Bootstrap resampling of variants within each validation region to construct confidence intervals around SER/WGR/HR — Since each region already contains thousands of variants, within-region bootstrap resampling could provide a sense of estimation precision for the reported error rates without requiring additional ground-truth regions.
-
The Bayesian log-likelihood model's allele-pair classifications (same vs. different haplotype) are evaluated only through downstream phasing accuracy, not through direct classifier performance metrics.↳ Could also: Reporting classifier-style metrics (e.g., precision/recall, ROC-AUC) for the same/different-haplotype calls on a held-out labeled subset of variant pairs — This would isolate how well the core scoring model performs at its classification task, separately from the effects of the later clustering and interval-scheduling steps.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-35692633
Paper: Schrinner, Serra Mari, Finkers, Arens, Usadel, Marschall, Klau. "Genetic polyploid phasing from low-depth progeny samples." iScience 25(6):104461, 2022.
Method/tool: WhatsHap polyphasegenetic (WH-PPG) — a new subcommand in the WhatsHap suite. It phases two deep-sequenced tetraploid parents using Mendelian signals from a large panel of low-depth progeny. Algorithm: Bayesian log-likelihood allele co-occurrence model → weighted cluster-editing → interval scheduling (ILP via PuLP/CBC) to assign alleles to k=4 haplotypes.
Code (P16 note: authors' own tool, in WhatsHap)
- Frozen software: Zenodo 10.5281/zenodo.6519173 —
whatshap-genetic-polyphase-frozen-2022-04-08.zip(28.5 MB), WhatsHap commitb2f9b825bb99e45d322fe87558d5caa7cea99d13, branchgenetic-polyphase(Apr 8 2022). conda env fromwhatshap-dev.yaml;pip install -e .. - Experiment scripts: github.com/AlBi-HHU/genetic-polyploid-phasing-suppl-material (Snakefile-ASM / Snakefile-Genetic / Snakefile-Whole + helper py scripts).
- NOTE: registry
code_url=broadinstitute/gatkis a text-mining FALSE POSITIVE. GATK 4.1.8 is only an upstream variant caller (HaplotypeCaller -ploidy 4); the paper's method is WhatsHap, not GATK.
Data
- Zenodo 5645743 "VCF data for genetic polyploid phasing" (the small, ready
inputs for the phasing experiment): per-region parent + progeny decluttered
VCFs and
chNN.hifiasm.vcf.gzHiFi-assembly ground-truth VCFs for chr 3/4/5, plusped.txt(pedigree, 193 progeny + 2 parents). - SRA PRJNA718240 = parental WGS (Altus, Colomba). PRJEB48582 = 193 progeny low-depth WGS. PRJNA778192 = Altus HiFi reads (for ground truth).
- Reference: Solyntus V1.1 potato genome (public).
IN SCOPE (pipeline-derived, reproducible)
- Phasing-accuracy table (core result). Run
whatshap polyphasegeneticon each region (chr3 60Mx, chr4 71Mx, chr5 56Mx) at complexity C=0 (simplex-nulliplex only) and C=2 (+ simplex-simplex, duplex-nulliplex), full 193 progeny,--scoring-window 250 --ratio-cutoff 0 --distrust-parent-genotypes -p 4 --sample Altus_470bp. Thenwhatshap compare ... -p 4 --tsv-pairwiseagainst the HiFi ground truth (chNN.combined.vcf.gz, derived fromchNN.hifiasm.vcf.gz). Metrics: switch error rate, Hamming rate, #phased variants, #blocks. - Number of phased variants / single-block property per region (paper states each region phases into one block).
- (stretch) Progeny-count scaling (Fig 2): subsample ped to 150/125/.../15 progeny ×10 runs, accuracy vs #progeny.
OUT OF SCOPE (not attempted / heavier)
- Building the HiFi ground truth from raw assembly graph (Snakefile-ASM: gfa →
minimap2 → paftools) — we use the deposited
chNN.hifiasm.vcf.gzinstead. - Whole-chromosome results (Snakefile-Whole) — needs full parental+progeny VCFs from NCBI (variant-call from raw WGS); heavier, attempted only if time permits.
- Upstream raw-read alignment + GATK variant calling (provided pre-computed as the decluttered region VCFs on Zenodo).
- Wet-lab / sequencing steps (out of scope by definition).
Reproduction unit chosen
The Snakefile-Genetic "phasing results" path on the Zenodo region VCFs is the clean, self-contained ~80% target: small inputs, deterministic core phasing, direct numeric comparison to the paper's accuracy table.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.