An improved assembly of the pearl millet reference genome using Oxford Nanopore long reads and optical mapping.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- ✓Overall, the reproduction was clean
- 🟡Could not use the authors’ exact input data
- 🟡A deviation arose in the data or preprocessing
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
STRONG REPRODUCTION of pearl millet genome assembly (Salson et al 2023, G3). All readily-specified pipeline-derived results reproduce with standard third-party tools on the paper's deposited data (ENA ERR10627707 + assembly GCA_947561735): NanoFilt -l5000 -q10 filtered set EXACT to the read (2,640,214 reads / 57.6 Gb / N50 25.2 kb / mean 21.8 kb = C3-C6); deposited assembly EXACT (total 1.85 Gb C9, chromosomes 1.78 Gb C10, 7 chromosomes C11, N 0.286% vs 0.29% C12); BUSCO 5.4.3 poales_odb10 EXACT 98.4% complete (C15). C1/C2 PARTIAL: the deposited 23DB.fastq holds 5.28M reads/97.5 Gb vs the paper's 6.26M/108 Gb 'generated' (deposit is likely guppy pass-only; the ~1M absent reads are exactly those NanoFilt removes, so the assembly-input set still reproduces exactly). DID NOT attempt: Guppy basecalling (no fast5 signal deposited); Bionano optical-map hybrid scaffolding C13/C14 (proprietary Bionano Solve); RagTag/Racon/Medaka/Hapo-G polishing (cost/depends-on-scaffolding). De-novo Flye 2.9.6 (C7/C8) attempted as a stretch on the 57.6 Gb filtered reads, limited by the 12h/job walltime cap. No fabrication indicators: every checked number is derivable from the deposited data. All verdicts provisional (human-checkable).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-30
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-07-25no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe previous short-read-based pearl millet (Tift 23D2B1-P1-P5) reference genome was incomplete, fragmented, and had ~200 Mb unplaced sequence; combining Oxford Nanopore long-read sequencing with Bionano optical mapping can substantially improve the completeness, contiguity, and accuracy of this reference genome assembly.
- ★ Combining ONT long reads with Bionano optical maps produced a substantially more complete and contiguous pearl millet Tift 23D2B1-P1-P5 assembly than the prior short-read assembly. finding
- ★ The new assembly adds approximately 200 Mb of sequence at the chromosome level compared to the old reference genome. finding
- ★ The new assembly adds more than 100 Mb around the centromeric region of chromosome 7. finding
- ★ The new assembly shows higher gene completeness (BUSCO) than the old assembly. finding
- ★ A hybrid scaffolding pipeline (Flye ONT assembly, Bionano hybrid scaffolding, BiSCoT, TGS Gap-Closer, Hapo-G polishing, RagTag chromosome construction) was used to build the improved assembly. method
- Optical map alignments can be used to identify misassemblies, including in centromeric regions, in the prior reference genome. method
- Centromeric regions were localized using a 137 bp satellite repeat specific to pearl millet centromeres. method
- ★ The improved Tift 23D2B1-P1-P5 assembly is made available as a community resource to support structural variant and genomics research in pearl millet. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Oxford Nanopore long-read whole-genome sequencing | Pearl millet (Pennisetum glaucum), Tift 23D2B1-P1-P5 cultivar | none | long-read sequence data for de novo genome assembly | Oxford Nanopore PromethION (SQK-LSK109), guppy v6.0.6 basecalling |
| Bionano optical mapping | Pearl millet, Tift 23D2B1-P1-P5 and PMiGAP257/IP-4927 genotypes | none | optical maps for hybrid scaffolding and structural validation | Bionano Genomics Saphyr system, DLE-1 DLS labeling, Bionano Solve v3.5.1/v3.3 |
| Illumina short-read whole-genome sequencing (reused public data) | Pearl millet, Tift 23D2B1-P1-P5 | none | short reads for contig/scaffold polishing and error correction | Illumina (NCBI SRA SRP063925), bwa-mem2 v2.2.1, Hapo-G v1.3 |
| Gene completeness assessment (BUSCO) | Pearl millet genome assemblies (old vs new) | none | percentage of complete conserved orthologs | BUSCO v5.4.3, Poales odb10 dataset (4,896 genes) |
| LTR assembly index (LAI) repeat-space quality assessment | Pearl millet genome assemblies (old vs new) | none | LAI score reflecting repetitive/LTR region assembly quality | LTR_FINDER_parallel, LTRharvest, LTR_retriever v2.8.7 |
| Base accuracy estimation | Pearl millet new genome assembly | none | consensus base accuracy | Merqury v1.3 |
| BLAST alignment of centromeric satellite sequence | Pearl millet new genome assembly chromosomes | none | positions of centromere-specific 137 bp repeat | BLAST v2.9.0+ |
| Comparative whole-genome and optical map alignment | Old vs new pearl millet assemblies; PMiGAP257/IP-4927 optical maps | none | structural continuity/misassembly comparison between assemblies | minimap2 v2.24, D-genies v1.4, Bionano Access v3.7 with RefAligner |
- ▲ New assembly chromosome total length increased to 1,778,181,882 bp from 1,564,537,551 bp in the old reference. ~214 Mb increase
- ▲ Complete BUSCO score improved in the new assembly relative to the old assembly. 98.4% vs 94.4%
- ▲ Scaffold N50 increased dramatically in the new assembly. 85,795,566 bp vs 884,945 bp (~100-fold)
- ▼ Percentage of Ns in chromosomes greatly reduced in the new assembly. 0.3% vs 13.5%
- ▲ Contig N50 increased substantially after ONT assembly compared to the old assembly. 1,209,791 bp vs 18,180 bp (67-fold)
- ▲ Optical maps aligning to old reference chromosome 7 were much larger than the chromosome's annotated size, identifying it as an outlier. 128 Mb larger than expected
- – Correlation between old reference chromosome lengths and cumulative optical map sizes was only marginally significant across all 7 chromosomes but strong and significant after excluding chromosome 7. r=0.54 (P=0.059) vs r=0.90 (P=0.015)
- ▲ Hybrid scaffolding of ONT contigs with optical maps produced 72 scaffolds with markedly improved contiguity. N50 = 86 Mb, ~100-fold increase
- correlation r=0.54, P=0.059 (old reference chromosome length vs cumulative optical map length, all 7 chromosomes)
- correlation r=0.90, P=0.015 (same correlation excluding chromosome 7 outlier)
- fold_change 67-fold (contig N50 of new ONT assembly vs old assembly)
- fold_change ~100-fold (scaffold N50 of new hybrid-scaffolded assembly vs old assembly)
- other 98.4% vs 94.4% (complete BUSCO score, new vs old assembly (Poales odb10, 4,896 genes))
- other 0.3% vs 13.5% (percentage of Ns in chromosomes, new vs old assembly)
- count 60X mean depth (108 Gb), 32X used for assembly (57.6 Gb) (ONT reads generated and used for Tift 23D2B1-P1-P5 assembly)
- other 1,778,181,882 bp vs 1,564,537,551 bp (total chromosome length, new vs old assembly)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a genome assembly report rather than a hypothesis-driven experimental study, so most results (N50, BUSCO completeness, LAI, percentage of Ns, assembly length) are reported as descriptive genome-quality metrics without formal statistical testing. The one inferential statistic used is a Pearson correlation (R's cor.test()) relating the cumulative length of optical maps assigned to each chromosome to the corresponding chromosome length in the old reference genome, computed once across all seven chromosomes and again after excluding chromosome 7 as an identified outlier. Results were reported as correlation coefficients with exact p-values and visualized with a smoothed regression line (ggplot2 geom_smooth()).
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Pearson correlation (cor.test()) | Correlation between cumulative optical-map length per chromosome and chromosome length in the old reference genome, all 7 chromosomes | n = 7 chromosomes | not stated |
| Pearson correlation (cor.test()), repeated after outlier exclusion | Same comparison as above, with chromosome 7 excluded as an outlier | n = 6 chromosomes | not stated |
-
The paper used Pearson's correlation to relate chromosome lengths to cumulative optical-map lengths, then excluded chromosome 7 as an outlier and recomputed the correlation on the remaining 6 chromosomes.↳ Could also: Spearman's rank correlation or Kendall's tau — These rank-based measures are less sensitive to a single extreme value, which could allow all 7 chromosomes to be retained in one analysis while still describing the strength of association.
-
An exact p-value is reported for the Pearson correlation (e.g., P = 0.059 and P = 0.015) but no confidence interval around the correlation coefficient is given.↳ Could also: A 95% confidence interval for r (e.g., via Fisher's z-transformation, as produced automatically by R's cor.test()) — A CI would convey the precision of the estimated association alongside the point estimate, which can be informative when working with a small number of chromosomes.
-
The correlation analysis is based on a very small number of data points (7, or 6 after exclusion), one per chromosome.↳ Could also: A robust regression or resampling-based approach (e.g., bootstrap confidence intervals) — With few observations, robust or resampling methods can characterize uncertainty in the association without relying on distributional assumptions such as bivariate normality.
-
Chromosome 7 was identified as an outlier and removed prior to recomputing the correlation.↳ Could also: A robust correlation/regression estimator (e.g., Kendall's tau or a robust linear model with down-weighted outliers) — Down-weighting rather than excluding an observation retains all data points while still reducing the influence of atypical values.
-
Genome-quality metrics (BUSCO completeness, N50, LAI, %Ns, total assembly length) are presented as single point-value comparisons between the old and new assemblies without associated variability estimates.↳ Could also: Bootstrap or subsampling-based resampling of contigs/genes to generate an interval around metrics like BUSCO completeness — Where feasible, this could convey how much these summary metrics might vary across resampled subsets of the assembly, complementing the single reported values.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-36891809
Paper: Salson et al. 2023, G3. "An improved assembly of the pearl millet reference genome using Oxford Nanopore long reads and optical mapping." DOI 10.1093/g3journal/jkad051. PMID 36891809.
Organism: Pearl millet (Cenchrus americanus / Pennisetum glaucum), cultivar Tift 23D2B1-P1-P5 ("23DB"). ~1.78–1.85 Gb genome, 7 chromosomes.
Pipeline (Methods §) and reproducibility scope
| Step | Tool (paper version) | In scope? | Notes |
|---|---|---|---|
| Basecalling | Guppy 6.0.6, dna_r9.4.1_450bps_hac | OUT | needs raw fast5 (not deposited as signal); deposited reads are already basecalled fastq |
| Read filtering | NanoFilt 1.0 (-l 5000 -q 10) |
IN | deterministic; deposited reads ERR10627707 |
| ONT assembly | CulebrONT 2.1.0 → Flye 2.9 | IN (stretch) | 1.8 Gb plant genome, 32X — needs big-mem node; expensive but feasible on big (2.3 TB) |
| Polishing | Racon 1.5.0 ×2, Medaka 1.6.1, Hapo-G 1.3 | OUT (cost) | polishing changes bases not contig structure; very expensive; deprioritised |
| Optical map scaffolding | Bionano Solve 3.3 hybridScaffold (-B 2 -N 2), BiSCoT 2.3.3, TGS-GapCloser 1.2.0 |
OUT | Bionano Solve is proprietary; optical-map raw data needs Bionano Access; not reproducible here |
| Chromosome assembly | RagTag 2.1.0 (guide = old reference) | OUT (depends on scaffolding output) | |
| QC: BUSCO | BUSCO 5.4.3, Poales odb10 (4,896 genes) | IN | run on the deposited final assembly GCA_947561735 |
| QC: assembly stats | (N50, total len, N%, chr count) | IN | computed on deposited assembly + on our Flye contigs |
| QC: Merqury QV | Merqury 1.3 + Illumina k-mers | OUT (cost/optional) | needs Illumina SRP063925 + meryl; possible stretch |
| QC: LAI | LTR_FINDER_parallel 1.0.5 + genometools 1.5.9 | OUT (cost/optional) | possible stretch |
| Gene annotation | (mapping prev. transcripts) | OUT | wet-lab/annotation, not core assembly |
In-scope reproduction targets (claims)
- Read-level (faithful, cheap-moderate): raw read count/bases; filtered read count, N50, mean length, total bases. Directly from deposited ERR10627707.
- Assembly-level on the deposit (auditability of what was deposited): total length, chromosome cumulative size, chromosome count, N content, scaffold N50, BUSCO completeness. From GCA_947561735.
- De novo Flye contigs (stretch): contig count, contig N50 — re-run Flye 2.9
on filtered reads. Heavy; attempted on
bigpartition.
Out of scope (stated, not attempted)
Basecalling (no signal data), Bionano optical-map scaffolding (proprietary software), RagTag chromosome build (depends on scaffolding), full Racon/Medaka/ Hapo-G polishing (cost), gene annotation (wet-lab/manual). Merqury QV and LAI are optional stretch QC.
Note (P16): the listed "Code" repo (nanoporetech/medaka) is only the polisher; the assembly itself is Flye via CulebrONT. Applying these standard third-party tools to the paper's deposited data is a valid reproduction.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Strong, essentially 1:1 reproduction. Using only third-party tools on the authors' deposited data (ENA ERR10627707 + GCA_947561735), the filtered read set reproduces exactly — 2,640,214 reads, 57,655,242,113 bp, N50 25,229 bp, mean 21,837 bp — and the assembly metrics match at the paper's precision (1,851,265,561 bp total; 1,778,181,882 bp on 7 chromosomes; 0.2855% vs 0.29% N; BUSCO poales_odb10 C:98.4%). The only genuine deviation is on the input/availability side and not the authors' analytical defect: the deposit holds 5,278,142 reads / 97.5 Gb against the reported 6,261,759 / 108 Gb generated, which is fully explained by a pass-only upload, since the ~1M absent reads are precisely those NanoFilt -l5000 -q10 removes. Two claims (C13 72 hybrid scaffolds, C14 86 Mb scaffold N50) remain unchecked purely because Bionano Solve is proprietary, and C7/C8 (3,641 Flye contigs, 1.2 Mb N50) were a stretch left pending under the 12 h walltime cap — none of these are evidence against the paper. No fabrication indicators: every checkable number is derivable from the shared data, and the mix of exact and rounded agreement is what genuine recomputation looks like.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.