Sequencing of human genomes with nanopore technology.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🔴Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce the CODE ARTIFACT, not the whole-genome numbers. SEW (the authors' shipped phasing tool, bioconda r-sew 1.0.1, repo tag 1.0.1) installs and runs cleanly on «our HPC» and correctly phases its bundled example (chr10, 50 het SNPs) into haplotypes; an independent VCF-vs-truth switch-error check = 0% (confirmed with no-phasefile control, so not circular), and WhatsHap 2.8 on the same input also = 0% — reproducing the paper's headline switch-error METRIC and its SEW-vs-WhatsHap COMPARISON (qualitatively SEW ~= WhatsHap). NOT 1:1 on the paper's reported magnitudes: the example is 50 simulated SNPs, far easier than NA12878 chr22, so it cannot reproduce 1.84%/1.91% nor the sub-2% gap. NOT ATTEMPTED (out of 80/20 scope): 273.4 Gb yield, read N50/coverage (basecalling of raw ONT), SNV F1 93.4% (bwa mem + FreeBayes), exact chr22 switch error, clinical SAMD9L phasing (EGA controlled access) - because the raw->BAM/variant pipeline is 'available on request' (not public) and processing 273 Gb is the hard 20%. No fabrication flags; the gap is a reproducibility-surface limitation, not evidence of fabrication.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 75assessed: 2026-06-16 ⛓ 4eef7179109c
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-16
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan Oxford Nanopore Technologies' MinION long-read sequencer, despite its high per-base error rate, serve as a viable primary method for routine clinical whole-genome sequencing, and can novel analytical methods (especially reference panel-free phasing) raise SNV-calling accuracy to clinically useful levels?
- ★ A novel single-sample, reference panel-free, read-based phasing algorithm built on the STITCH model improves nanopore SNV calling from modest baseline levels. method
- ★ Phasing- and annotation-based filtering substantially improves SNV-calling accuracy (F1 from 88% to 93.4% genome-wide). finding
- ★ Consensus SNV-calling error rates from ONT data remain substantially higher than short-read methods, limited by systematic errors (notably homopolymer/deletion alignment artefacts) rather than random error. finding
- ★ In a clinical sample with ataxia-pancytopenia syndrome, two non-synonymous de novo SAMD9L variants were identified and directly phased to the same paternal haplotype. finding
- ★ Long ONT reads enable phasing accuracy (switch error rate) comparable to SNP-array phasing with very large reference panels. finding
- ★ False-positive variant calls are disproportionately enriched in putative loss-of-function and highly constrained genes, posing a clinical interpretation risk at current FDR. finding
- Sniffles applied to ONT reads detects large structural variants (>100 nt) including ONT-specific calls not supported by other technologies. method
- Albacore v2.0.2 base-caller achieves lowest substitution/deletion error rates and best variant calling after phasing-derived filtering compared to other base-callers. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Whole-genome nanopore sequencing (PCR-based library, ~6 kb size selection) | GM12878/NA12878 lymphoblastoid reference cell line (female CEPH/Utah) | none | reads, yield, read length, error rates, coverage depth | Oxford Nanopore MinION, R9.4 flow cells (73 flow cells, 8 instruments); base-called with Albacore v2.0.2, trimmed with Porechop 0.2.2 |
| SNV calling and benchmarking | NA12878 genome (chr22 and full autosomes) | none | F1 score, FDR, FNR vs GIAB gold-standard truth set | FreeBayes variant caller |
| Read-based haplotype phasing | NA12878 genome (chr22) | none | switch error rate vs parentally-phased truth (NA12891/NA12892) | Novel STITCH-based phasing algorithm and WhatsHap |
| Large structural variant (>100 nt) discovery | NA12878 genome (chr22) | none | number of large variants called, validation vs truth set / Illumina / PacBio | Sniffles |
| Base-caller comparison | NA12878 ONT reads | none | per-read error rates and variant-calling accuracy | Albacore 2.0.2, Metrichor, and two other base-callers |
| Simulated sequencing dataset analysis | In silico NA12878 model (idealized random per-read errors, no amplification bias) | none | F1/FDR/FNR across error rates and depths | — |
| Whole-genome nanopore sequencing | Clinical individual with ataxia-pancytopenia syndrome and severe immune dysregulation | none | identification and phasing of de novo SAMD9L variants | Oxford Nanopore MinION |
- – Genome-wide consensus SNV accuracy of 99.9% with pre-phasing filtering (QUAL + contamination), full autosomes 99.9% accuracy; F1 88.3%, FDR 10.9%, FNR 12.5%
- ▲ Phasing + heuristics filtering improved full-autosome SNV calling F1 93.4%, FDR 5.3%, FNR 7.8%
- ▲ Restricting to sites with >60x coverage further improved accuracy F1 93.6%, FDR 6.1%, FNR 6.6%
- ▼ Read-based phasing achieved low switch error rate on chr22, lower than WhatsHap 1.84% (all SNVs) and 0.80% (read-spanned adjacent SNPs) vs WhatsHap 1.91%/0.90%
- – Two non-synonymous de novo SAMD9L variants phased to the same paternal haplotype
- ▲ False positives enriched among putative pathogenic LoF variants relative to true positives FP 69/45219 (0.15%) vs TP 173/788782 (0.02%)
- – Simulated idealized data achieves much higher accuracy than observed, indicating systematic errors dominate simulated F1 99% (FDR/FNR 1%) vs observed ~10 points lower
- – Sniffles discovered 82 large variants on chr22; 22 in truth set, 21 PacBio-supported, 31 ONT-specific, 8 apparent false positives 82 variants discovered
- count 45,740,123 reads; total yield 273.4 Gb (Total reads/yield across 73 R9.4 flow cells for NA12878)
- mean mean read length 6373 bp; mean per-base coverage 81.7x (NA12878 sequencing metrics)
- other substitution rate 12.7%, deletion rate 4.7%, insertion rate 3.2% (Mean per-read error rates of aligned ONT reads)
- other F1 93.4%, FDR 5.3%, FNR 7.8% (Best full-autosome SNV calling after phasing + heuristics filtering)
- count FDR of 5.3% corresponds to ~140 thousand FP variant calls (Whole-genome false-positive burden)
- other switch error rate 0.80% (our method) vs 0.90% (WhatsHap) (Phasing of read-spanned adjacent heterozygous SNPs on chr22)
- count 42,631,376 of 42,924,782 high-quality reads aligned (99.3%); 37,859,481 (88.8%) uniquely mapped (Alignment to GRCh37 for NA12878)
- count constrained genes pLI>0.90: 17 FP vs 20 TP; non-constrained pLI<=0.10: 46 FP vs 122 TP (FP enrichment in constrained genes for LoF variants)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a genomics technology-evaluation study that benchmarks Oxford Nanopore MinION whole-genome sequencing against established truth sets rather than testing biological hypotheses with classical inferential statistics. Performance is reported descriptively using classification accuracy metrics (F1 score, false discovery rate, false-negative rate, switch error rate, consensus accuracy) for SNV calling, phasing, and large-variant detection, with parameters tuned on chromosome 22 and then applied genome-wide. Comparisons between methods (base-callers, phasing algorithms, filtering approaches) are presented as point-estimate differences in these metrics, and enrichment of false positives among loss-of-function/constrained-gene variants is reported as raw counts and proportions.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Classification performance metrics (F1 score, FDR, FNR) vs GIAB truth set | SNV discovery in NA12878, chromosome 22 and full autosomes (Table 1, Fig. 2) | — | na |
| Consensus accuracy (percent) vs GIAB reference | Whole-genome SNV calling accuracy (99.9%) | — | na |
| Switch error rate vs trio-phased truth set | Phasing accuracy comparison of WhatsHap vs novel method, chromosome 22 | — | na |
| Raw count / proportion enrichment comparison | FP vs TP enrichment among putative LoF variants and constrained (pLI) genes (e.g. 69/45219 vs 173/788782) | counts of variant calls as stated | not stated |
| Simulation-based evaluation of performance across error rates and depths | Idealized-model NA12878 simulations (Supplementary Figs. 15–20) | — | stated (idealized random per-read substitution error, no amplification bias) |
| Visual/manual adjudication of calls against orthogonal platforms | Large-variant (>100 nt) calls on chromosome 22 vs PacBio/Illumina reads | 82 discovered variants | na |
-
Method and platform comparisons (e.g. WhatsHap switch error 1.91% vs the novel method 1.84%; base-caller comparisons) are reported as point-estimate differences in metrics.↳ Could also: Reporting these metrics with bootstrap or binomial confidence intervals, or a paired comparison across genomic intervals/flow cells. — Interval estimates would convey the uncertainty around each metric and help readers gauge how large or stable the observed differences are, which is especially informative when two values are close.
-
Variant-calling parameters were tuned to maximize F1 on chromosome 22 and then applied to the remaining autosomes.↳ Could also: A cross-validation or held-out chromosome scheme, or repeating the tuning across multiple folds. — An explicit train/test split or cross-validation would characterize how well tuned parameters generalize and quantify any optimism from selecting parameters on the same metric being reported.
-
Enrichment of false positives among LoF and constrained-gene variants is presented as raw counts and proportions (e.g. 17 FP vs 20 TP).↳ Could also: A Fisher's exact test or chi-square test with an odds ratio and confidence interval for the enrichment. — A formal association statistic plus effect size would put a number on the strength and uncertainty of the enrichment alongside the descriptive counts, particularly where counts are small.
-
Per-read and per-flow-cell error/coverage characteristics are summarized primarily as means (e.g. mean substitution rate 12.7%) and distributions in figures.↳ Could also: Accompanying central estimates with a dispersion measure such as SD, IQR, or range across flow cells. — An explicit spread measure would convey flow-cell-to-flow-cell variability directly in the text, complementing the distributional figures.
-
Large-variant calls were adjudicated by visual inspection against orthogonal platforms.↳ Could also: Computing sensitivity/precision against the consensus truth set with confidence intervals, or independent re-scoring by multiple blinded reviewers with an agreement statistic. — Quantified validation metrics and inter-reviewer agreement would make the manual adjudication step more reproducible and easier to compare across studies.
-
Differences between observed and simulated performance are interpreted by comparing point estimates (e.g. a 10-percentage-point F1 gap).↳ Could also: Running multiple simulation replicates and summarizing the distribution of outcomes. — Replicated simulations would provide a sense of Monte Carlo variability around the simulated metrics, clarifying how much of the observed gap exceeds simulation noise.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
Read-based phasing of nanopore reads yields a low switch error rate on chr22 (1.84%), lower than WhatsHap.long-read-DNA human na12878 down 2019×1papers★ This paper is the founder (earliest)
-
Nanopore false-positive calls are enriched among putative pathogenic loss-of-function variants relative to true positives (0.15% vs 0.02%).long-read-DNA human na12878 up 2019×1papers★ This paper is the founder (earliest)
-
Nanopore whole-genome sequencing achieves 99.9% genome-wide consensus SNV accuracy on the full autosomes after pre-phasing quality/contamination filtering.long-read-DNA human na12878 2019×1papers★ This paper is the founder (earliest)
-
Haplotype phasing plus heuristic filtering improves full-autosome nanopore SNV calling (F1 to 93.4%).long-read-DNA human na12878 up 2019×1papers★ This paper is the founder (earliest)
-
Sniffles discovers 82 large structural variants (>100 nt) on chr22 from nanopore reads, most validated against truth/PacBio sets.long-read-DNA human na12878 2019×1papers★ This paper is the founder (earliest)
-
Nanopore sequencing identifies and phases two non-synonymous de novo SAMD9L variants to the same paternal haplotype in a patient with ataxia-pancytopenia syndrome.long-read-DNA human 2019×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-31015479
Paper: Bowden R, Davies RW, et al. Sequencing of human genomes with nanopore technology. Nat Commun 2019. PMID 31015479 · PMC6478738 · DOI 10.1038/s41467-019-09637-5.
Shipped code: https://github.com/Genomicsplc/SEW — SEW ("reference-panel-
free long-read phasing"), an R/C++ package by the same author (RW Davies),
derived from STITCH. On bioconda as r-sew v1.0.1. This is the only publicly
runnable code artifact from the paper.
Data: ENA PRJEB30620 (NA12878 ONT, 273.4 Gb raw); EGA EGAS00001003469 (clinical sample, controlled access). Code availability statement: SEW phasing code is public; "The code used to analyse the data ... are available on request from the authors."
In scope (attempted)
| Result | Pipeline | Feasible? |
|---|---|---|
| SEW long-read phasing tool runs & phases haplotypes | SEW (shipped) on its bundled example (chr10, 50 het SNPs, 1 sample, truth phasefile) | YES — small, deterministic, self-contained |
| Switch error rate metric (paper's headline phasing accuracy measure) | SEW output vs truth phasefile → switch error rate; WhatsHap on same input for the SEW-vs-WhatsHap comparison the paper reports | YES on the example data (mirrors the method, not the NA12878 numbers) |
Out of scope (NOT attempted, with reason)
| Result | Why dropped from attempt |
|---|---|
| Yield 273.4 Gb, read N50/mean 6,373 bp, coverage 81.7× | Derived from raw ONT signal across 73 flow cells (273 Gb). Basecalling = instrument/wet-lab-adjacent; not in repo. Out of 80/20. |
| Mapping (bwa mem 0.7.12), SNV F₁ 93.4% (FreeBayes v1.0.2) | Requires the full 273 Gb dataset + GIAB truth comparison; analysis code "available on request" (not public). Heavy 20%. |
| NA12878 chr22 switch error 1.84% / 0.80% (SEW) vs 1.91% / 0.90% (WhatsHap) | Exact numbers need chr22 ONT reads (basecalled, bwa-mapped) + het calls + GIAB truth. The raw→BAM pipeline is not shipped; processing 273 Gb is the hard 20%. We reproduce the metric & the SEW-vs-WhatsHap comparison on the tool's bundled example instead, and record the reported numbers for human audit. |
| Clinical SAMD9L phasing (199 kb block, 33 reads) | EGA controlled-access data + clinical specifics; out of scope. |
Approach
Run the authors' shipped SEW tool (the published, citable code artifact) on its own example data on «our HPC»; compute switch error rate vs the bundled truth, and run WhatsHap on the same input to reproduce the paper's qualitative claim that SEW's switch error is comparable to / slightly better than WhatsHap. This is a faithful 1:1 reproduction of the code artifact and its core metric, explicitly NOT a reproduction of the NA12878 whole-genome numbers (different, much larger data; basecalling + mapping code not public).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The shipped SEW tool reproduces cleanly: it installs, phases all 50 bundled het sites, and an independent VCF-vs-truth check gives 0/49 switch error (confirmed non-circular via a no-phasefile control), with WhatsHap 2.8 also at 0/49 — so the metric and the SEW≈WhatsHap comparison reproduce qualitatively. The gap is a reproducibility-surface limitation on the data/code side: the paper's chr22 numbers (1.84%/0.80%), 273.4 Gb yield, 81.7x coverage and 93.4% SNV F1 require a basecalling+mapping+variant pipeline that is only 'available on request' plus EGA-controlled clinical data, and the curator self-chose an easier toy example under 80/20 scope. Deviation is moderate and explainable (different/easier input, not different logic) with no fabrication signal — every reproduced value is regenerable from the public tool. Overall yellow: a solid code-artifact reproduction whose absolute magnitudes simply weren't checkable.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.