Evaluating sequence data quality from the Swift Accel-Amplicon CFTR Panel.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough -- exceptionally so. A Scientific Data descriptor of the Swift Accel-Amplicon CFTR panel with verbatim command lines and tool versions (bwa 0.7.17, samtools, Picard 2.1.0, GATK 3.5, primerclip). Reproduced LARGELY 1:1 from publicly deposited data on «our HPC» via FRESH compute («job»: rebuilt conda env, re-downloaded the 14 figshare BAMs + 14 VCFs, recomputed -- the prior pass's «infra» workdir had been reclaimed by the janitor, and every recomputed value is IDENTICAL to the prior pass). Results: all 14 Table-1 read counts EXACT (primary reads = 2x SRA read pairs of the per-sample demultiplexed runs); Table-1 mean coverage all 14 within 0.4% (median ratio 1.003) and %>20x within 0.02% (Run1=100.00 exact) via samtools depth over the 30 Supplementary target intervals; Table-7 per-exon coverage median ratio 1.0036; Table-2 amplicon sizes 30/30 exact; and the entire Online-only Table 1 (119 called variants, 13 samples) matches the authors' deposited GATK-3.5 VCFs with 119/119 position+allele concordance, all Table-3 pathogenic variants present, VCF headers confirming the paper's exact GATK params. NOT fully faithful in ONE respect: the primerclip primer-trimming step could not be re-executed because the panel masterfile is no longer public, and the figshare BAMs are raw pre-primerclip bwa output -- coverage was thus computed on pre-trim BAMs yet still reproduces within 0.3-0.4% because the densely-tiled panel compensates clipped primers with neighbouring inserts. NOT attempted: Table 6 (out of scope), Table 2 %GC-per-exon (boundaries unpublished), an independent GATK re-call (needs the trimmed BAM/masterfile). No fabrication indicated: every checkable headline number is derivable from deposited data and matches. Grades are provisional pending human audit.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 89assessed: 2026-06-16 ⛓ c0e2858f02fc
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-24
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-18no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper evaluates the technical performance (coverage, on-target rate, mutation detection) of the Swift Biosciences Accel-Amplicon CFTR Capture Panel using CFTR-positive DNA samples to determine its suitability for clinical CF carrier screening.
- ★ The Accel-Amplicon CFTR panel generates sequencing data with high coverage depth and near 100% on-target reads. finding
- ★ Coverage depth per exon correlates with GC content of the exon, with depth highest as GC% approaches 50%. finding
- ★ The number of amplicons per exon correlates with exon size (R2=0.9766). finding
- ★ The assay detected all expected pathogenic CFTR mutations in the seven CF-positive samples and no pathogenic variants in the normal control, with concordant results across repeated/inter- and intra-run samples. finding
- ★ FASTQ, BAM, and VCF files from 14 processed samples (7 unique CFTR-positive + 1 normal control) are provided as a public dataset resource. resource
- The Swift Accel-Amplicon protocol is a one-day, one-tube reaction covering 87 amplicons spanning all CFTR coding regions, 5'/3'UTR, and four intronic regions. method
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Amplicon sequencing (targeted NGS) | Human genomic DNA (Coriell cell line samples and one patient sample), CFTR gene | none (samples selected for known CFTR mutation genotypes) | Coverage depth, % reads on target, variant calls | Illumina MiSeq Nano Reagent Kit V2 (300-cycle) |
| Multiplex PCR / amplicon library preparation | Human genomic DNA | none | Library concentration, amplicon generation | Swift Biosciences Accel-Amplicon CFTR Panel (AL-55048) |
| qPCR library quantification | Adapted DNA libraries | none | Library concentration (nM) | KAPA Library Quantification Kit |
| Variant calling / bioinformatic analysis | Aligned BAM files (hg19) | none | SNVs/indels (VCF), on-target metrics | BWA, SAMtools, Picard, GATK HaplotypeCaller, Swift primerclip |
| Sanger sequencing | PCR products flanking CFTR variants | none | Confirmation of pathogenic variant calls | Applied Biosystems 3500 Genetic Analyzer, BigDye Terminator v1.1 |
| DNA quantification | Genomic DNA samples | none | DNA concentration (ng/mL) | Qubit dsDNA HS Assay Kit |
| Sequence quality assessment | FASTQ files from sequencing runs | none | Base quality scores (Q30/Q38), cluster density | MultiQC |
- – Mean coverage depth was 5753x in run 1 (4 samples) and 1344x in run 2 (10 samples)
- – Percent reads on target ranged 98-99% across samples 98-99%
- ▲ Amplicon number per exon correlated with exon size R2=0.9766
- – All CFTR mutations detected in CF-positive samples; no pathogenic variants in normal control; repeated samples concordant
- – Run 1 samples had 100% of targeted region >20x coverage; run 2 samples had <20x coverage only at one 3'UTR region with no known pathogenic variants 100.00% vs 99.71-99.73%
- – Cluster density and Q30 scores were high in both runs 807±1 k/mm2 (98.08% Q30) run1; 534±8 k/mm2 (98.05% Q30) run2
- ▼ Samples 5, 6, 7 had no coverage at a common TG-repeat deletion site in intron 8
- correlation R2 = 0.9766 (amplicon number vs exon size)
- mean 5753x (mean coverage depth, run 1)
- mean 1344x (mean coverage depth, run 2)
- other 98.32-99.29% (run1), 99.14-99.26% (run2) (percent of reads on target per sample)
- other 807 ± 1 k/mm2 (cluster density, run 1)
- other 534 ± 8 k/mm2 (cluster density, run 2)
- other 98.08% / 98.05% (percent of reads with Q30 score or higher, run1/run2)
- other 22.92% (population frequency of TG repeat deletion in intron 8 per gnomAD)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a descriptive technical validation study evaluating the sequencing performance of the Swift Accel-Amplicon CFTR Panel across 8 unique samples (14 sample runs including replicates) in two sequencing runs. The primary statistical approach is descriptive: coverage depth, on-target read percentages, and base quality scores are summarized as means and percentages by sample and by exon. A linear R² correlation is reported for amplicon number versus exon size, and GC content versus coverage depth is assessed graphically. No formal inferential hypothesis tests are employed; inter- and intra-run concordance of variant calls is assessed qualitatively.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Linear regression / R² correlation | Number of amplicons vs. exon size (Fig. 1); R² = 0.9766 reported | 27 exons/amplicon regions (introns 1, 12, and 22 excluded per text) | not stated |
| Graphical/visual correlation with trendline | GC content per exon vs. mean coverage depth for Run 1 and Run 2 separately (Fig. 2c,d) | 27 exon/amplicon regions per run | not stated |
| Qualitative concordance assessment | Inter-run and intra-run reproducibility of variant calls and coverage (Technical Validation section) | 4 samples repeated across both runs; Sample 4 run in triplicate in Run 2 | na |
-
Inter- and intra-run reproducibility of variant calls and coverage depth was described qualitatively as 'concordant' without a numeric reproducibility metric↳ Could also: A quantitative metric such as intraclass correlation coefficient (ICC) or Bland-Altman analysis could also be applied to per-exon or per-sample coverage depth across repeated runs — Quantitative reproducibility metrics provide a bounded numeric estimate of run-to-run variability, which allows laboratories evaluating the panel to compare it against pre-specified clinical acceptance thresholds
-
The relationship between GC content and coverage depth was presented graphically with a narrative description; no correlation coefficient was reported for this specific relationship↳ Could also: A Pearson or Spearman correlation coefficient with a 95% confidence interval could also be reported for GC content vs. coverage depth across exons — A formal correlation coefficient would quantify the GC-bias effect size and facilitate comparison with other amplicon-based CFTR panels; Spearman would be robust to potential non-linearity in the relationship
-
Per-exon and per-sample coverage depth was reported as means only, without a measure of spread across exons within a run↳ Could also: Reporting SD, IQR, or min–max range of per-exon coverage alongside the mean would also convey within-run amplicon balance — Measures of spread around mean coverage communicate the degree of amplicon uniformity, a key performance characteristic for clinical amplicon panels used in carrier screening
-
Cluster density was reported as mean ± an unlabeled dispersion value (807 ± 1 and 534 ± 8 k/mm²)↳ Could also: Explicitly labeling the dispersion statistic as SD, SEM, or range would also clarify the precision of the measurement — Knowing the dispersion type allows readers to interpret measurement precision and to compare cluster density performance across sequencing platforms or reagent lots
-
The number of samples (n = 8 unique) was not accompanied by a formal power calculation or reference to an analytical validation guideline specifying minimum specimen requirements↳ Could also: A reference to CAP, ACMG, or CLSI analytical validation frameworks specifying minimum positive and negative sample counts needed to estimate sensitivity and specificity within defined confidence bounds could also be provided — Citing an established validation framework contextualizes the chosen sample count within recognized clinical laboratory standards and helps readers assess coverage of the CFTR mutation spectrum
-
Variant detection accuracy was confirmed by Sanger sequencing used as a reference method, with agreement reported qualitatively↳ Could also: Positive percent agreement (PPA) and negative percent agreement (NPA) with exact 95% Wilson score confidence intervals could also be calculated to formally characterize variant detection performance — PPA/NPA with confidence intervals is the standard clinical diagnostic accuracy framework recommended by CLSI EP12 and FDA guidance for next-generation sequencing assays, and would make the performance claim quantitatively comparable across studies
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-31913291
Paper: Leung et al. 2020, Scientific Data 7:17. "Evaluating sequence data quality from the Swift Accel-Amplicon CFTR Panel." DOI 10.1038/s41597-019-0339-4. PMCID PMC6949293.
Type: Data Descriptor. Validation of a commercial targeted-amplicon NGS panel (Swift Accel-Amplicon CFTR, 87 amplicons spanning CFTR coding region + UTRs on chr7, hg19) on 8 samples (1 normal control NA12878 + 7 CF-positive), sequenced in 2 MiSeq Nano v2 runs. Sample 4 run in triplicate in run 2 → 14 sample-instances.
Pipeline (Methods, verbatim command lines)
Reference: hg19. Tools (Code-availability versions): SAMtools 1.6-2-gf068ac2 · Picard 2.1.0 · BWA 0.7.17-r1188 · GATK 3.5-0-g36282e4 · Java 1.8 · primerclip (swiftbiosciences/primerclip, the paper's central tool).
bwa mem ${FASTA} ${S}_R1.fastq.gz ${S}_R2.fastq.gz -U 17 -M -t 32 > ${S}_bwa.sam
samtools sort -n ${S}_bwa.sam -o ${S}_bwa_nsorted.sam
primerclip Accel-Amplicon_CFTR_masterfile.txt ${S}_bwa_nsorted.sam ${S}_bwa_primertrimmed.sam
picard AddOrReplaceReadGroups I=...sam O=...bam SO=coordinate RGID=snpID LB=swift SM=${S} PL=illumina PU=miseq
samtools index ${S}_bwa_primertrimmed.bam
# CollectTargetedPcrMetrics with AI/TI intervals built from cftr_180313_nonmerged_targets_5col.bed
picard CollectTargetedPcrMetrics I=...bam O=...txt AI=fullintervals TI=noprimerintervals R=${FASTA} PER_TARGET_COVERAGE=...
# variant calling
GATK -T HaplotypeCaller -R ${FASTA} -I ...bam -stand_call_conf 20 -stand_emit_conf 20 -mbq 20 -L CFTR_merged_5col.bed -o ...vcf
IN SCOPE (pipeline-derived; we attempt)
- Table 1 — Number of reads (14 cells): raw read count = 2 × SRA read pairs. Derivable directly from deposited FASTQ. PRIMARY, low-hanging.
- Table 1 — Read % on target / Mean coverage depth / % targeted region >20x (14 cells each): from BWA→primerclip→Picard CollectTargetedPcrMetrics over the target BED. Requires the CFTR masterfile (primer coords) for faithful primer trimming.
- Table 2 / Table 7 — per-exon mean coverage (30 exon-bins × runs/samples): Picard PER_TARGET_COVERAGE. Requires masterfile.
- Online-only Table 1 — variants per VCF (GATK 3.5 HaplotypeCaller). Requires masterfile + merged BED. Stretch goal.
- Table 2 derived — %GC per exon, amplicon/exon sizes, R² coverage~GC (Fig 2c,d): computable from coordinates + reference.
OUT OF SCOPE (not pipeline-derived)
- Table 6 — cluster density, %Q30, library nM: Illumina instrument QC / wet-lab, not reproducible from a bioinformatic pipeline.
- Table 3 — sample manifest / Coriell IDs: reference, not computed.
- Table 4 — primer sequences: panel design (manufacturer), not computed.
- Fig 3 — IGV/Sanger visualization: manual.
- Table 5 — SRA filenames: provenance (used to map samples→accessions).
Data mapping (SRP193469, demultiplexed per-sample accessions)
All 14 "Number of reads" cells matched EXACTLY to 2× ENA read pairs of the per-sample SRA runs (SRR10500893-900, SRR10541760-66, SRR8945286-89). See read_count_map.tsv. The raw multiplexed uploads SRR8945290-93 lump multiple sample/run instances together (e.g. SRR8945291 = Sample 2 in BOTH runs) and are NOT splittable from the deposited FASTQ alone — but the authors ALSO deposited the clean per-sample runs, so this does not block reproduction.
Key dependency / risk
Accel-Amplicon_CFTR_masterfile.txt (primer coordinates for primerclip) is NOT in the repo or paper supplementary; it was hosted on swiftbiosci.com (Swift was acquired by IDT in 2021). Target BED = Supplementary File 1 (MOESM1, 30 amplicon intervals) — obtained. If the masterfile cannot be retrieved, primer-trimming- dependent claims (mean coverage, per-exon coverage, variants) are downgraded to partial; read counts + target intervals remain fully reproducible.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
A Scientific Data descriptor reproduced essentially 1:1 from publicly deposited data: Table-1 read counts (14/14 exact), mean coverage (within 0.4%), %>20x (within 0.02%, Run1=100.00 exact), Table-2 amplicon sizes (27/27), Table-7 per-exon coverage (median 1.0036), and all 119 Online-Table-1 variant positions match the authors' deposited VCFs. The only deviations are small and fully explainable on our side: coverage computed on raw pre-primerclip BAMs (~0.3% high) and a read-level vs base-level on-target metric — both because the CFTR primer masterfile is no longer public (vendor defunct), a data-availability limitation, not an authors' defect. No fabrication is indicated and the descriptor's core quality claims hold; graded yellow overall only because the primerclip step was non-reproducible and one metric matched partially.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.