polishCLR: A Nextflow Workflow for Polishing PacBio CLR Genome Assemblies.
The main results reproduced, with only marginal, non-material deviations.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
polishCLR is a Nextflow WORKFLOW/TOOL paper; its main text reports NO numbers (all quantitative results in Suppl. Table S1, full-genome H. zea, 3 FALCON input cases). The CENTRAL reproducible claim — the published workflow runs end-to-end on the published example data and polishes a PacBio-CLR assembly producing the documented metrics — is REPRODUCED: (R1) CI -stub-run reproduced for both steps (DAG valid); (R2) the REAL chr30 example data was polished end-to-end on «our HPC» («job», exit 0) executing every stage (pbmm2+gcpp Arrow, purge_dups, meryl/merqury QV, bbstat, BUSCO) and emitting all documented metrics before&after. On the chr30 test, QV was flat (20.32->20.29) and BUSCO 0.8% — both EXPECTED and explained: chr30 is a finished RefSeq reference (not a low-QV FALCON draft) and the absolute QV is depressed by the sparse test Illumina reads (151k pairs); a single chromosome holds ~0.8% of insecta_odb10 genes. The full-genome Table S1 QV-gain numbers (31.82->40.30 etc.) were NOT reproduced 1:1 — they require the full FALCON draft assemblies + full-coverage SRA reads (PacBio subreads-BAM recovery from SRA is non-trivial) and ~200 CPU-h/case, and the CPU-h are hardware-dependent. Honest partial: executability + real-data workflow reproduction confirmed; full-scale quantitative QV-gain a documented stretch. All grades PROVISIONAL for human sign-off.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-19 ⛓ eab34dc223c8
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-30
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThere is a need for a publicly available, flexible, and reproducible containerized workflow that implements current best practices for polishing (short-read error-correction of) genome assemblies generated from error-prone PacBio continuous long-read (CLR) data, runnable on conventional HPC or cloud environments.
- ★ polishCLR is a reproducible, containerized Nextflow workflow that implements best practices for polishing PacBio CLR genome assemblies. resource
- ★ polishCLR can be initiated from three distinct input cases: unresolved primary assembly without contigs (Case 1), haplotype-resolved but unpolished contigs (Case 2), or haplotype-resolved and long-read-polished contigs (Case 3). method
- ★ The workflow uses purge_dups with automatically estimated cutoffs from long-read coverage histograms to remove duplicated haplotypic sequence at contig ends. method
- ★ The workflow performs Arrow long-read polishing and two rounds of FreeBayes short-read polishing, with optional Merfin-based variant filtering that is applied by default only when CLR and Illumina reads originate from the same specimen. method
- BUSCO completeness metrics are generated before and after duplicate removal to ensure purge_dups cutoffs do not remove genic content. method
- The workflow automatically generates evaluation reports including Merqury k-mer completeness/consensus accuracy (QV) and BBMap genome size statistics (e.g., N50) at each major phase. method
- ★ polishCLR has been used to generate chromosome-scale assemblies for Helicoverpa zea and Pectinophora gossypiella as part of the Ag100Pest Initiative. finding
- ★ The workflow, its documentation, and test data (including a Chromosome 30 H. zea CLR/Illumina test dataset) are publicly available on GitHub, ReadTheDocs, and archived on Zenodo. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Long-read genome assembly polishing (Arrow/GCpp) | Helicoverpa zea (Chromosome 30 test dataset, GCA_022581195.1) | none | Polished contig VCF/FASTA, consensus correction | PacBio CLR reads via GCpp Arrow |
| Short-read genome assembly polishing (FreeBayes, 2 rounds) | Helicoverpa zea / Pectinophora gossypiella assemblies | none | Corrected consensus sequence (VCF to FASTA) | Illumina short reads, FreeBayes |
| Duplicate haplotype removal (purge_dups) | Primary and alternate haplotype contigs (H. zea, P. gossypiella) | none | Long-read coverage histogram used to estimate cutoffs; purged primary/alternate contig sets | purge_dups v1.2.5 |
| Genome completeness assessment (BUSCO) | Primary contigs before/after de-duplication | none | BUSCO completeness score (eukaryotic lineage default) | BUSCO |
| k-mer based completeness and consensus accuracy assessment (Merqury) | Assemblies throughout workflow phases | none | QV score, k-mer completeness | Merqury |
| Genome size distribution statistics | Assemblies throughout workflow phases | none | N50 and other size statistics | BBMap |
| Variant filtering by k-mer validation (Merfin) | Arrow- and FreeBayes-called variants | same-specimen vs different-specimen short/long read data | Filtered VCF (removal of over-polishing variants) | Merfin, meryl genome database |
| Reference-based misassembly evaluation (optional) | Assemblies with an available reference | none | Identification of mis-assemblies | QUAST-LG |
- – polishCLR was applied to produce a chromosome-scale genome assembly of a Bacillus thuringiensis Cry1Ac-resistant Helicoverpa zea strain.
- – polishCLR was applied to produce a chromosome-scale genome assembly of Pectinophora gossypiella (pink bollworm), deposited as GenBank accession GCA_024362695.1.
- – Runtimes and summaries for each of the three starting input cases were generated and reported in supplementary table S1.
- – Test CLR and Illumina read data for Chromosome 30 of H. zea were provided to allow users to benchmark workflow performance.
- other 5-15% error rate (Typical error rate of long-read sequencing technologies (especially indels) prior to polishing, motivating the need for the workflow)
- count at least seven Nextflow processes (Number of delineated Nextflow processes used to implement each Arrow polishing round (indexing, alignment, variant calling, VCF combination/filtering, FASTA conversion))
- count seven processes (Number of Nextflow processes used to implement FreeBayes polishing in Step 2)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software/methods paper describing polishCLR, a Nextflow workflow for polishing PacBio CLR genome assemblies. It does not report a hypothesis-driven experiment with statistical comparisons between groups; instead it describes pipeline architecture and demonstrates results using genome-assembly quality metrics (e.g., BUSCO completeness, Merqury QV/consensus accuracy, N50) on example arthropod datasets. No inferential statistical tests (e.g., t-tests, ANOVA) are reported.
-
The paper evaluates and reports assembly quality using descriptive metrics (BUSCO completeness, Merqury QV, N50) without formal statistical comparison across pipeline versions, input cases, or species.↳ Could also: A quantitative benchmarking approach with paired comparisons (e.g., before/after polishing QV values compared via a paired test, or bootstrapped confidence intervals on QV/BUSCO estimates) could also be used — This would let readers gauge whether observed improvements exceed the variability expected from sampling or run-to-run differences, complementing the single-value descriptive metrics reported
-
Runtime and output summaries across the three input cases are reported as example/demonstration data (referenced in supplementary table) rather than as replicated measurements.↳ Could also: Reporting runtimes and quality metrics across multiple independent replicate runs or datasets, with a measure of spread (e.g., range or SD), could also be used — This would give users of the workflow a sense of typical variability in runtime and output quality across different datasets or hardware environments, useful for planning resource use
-
Genome completeness and correctness are assessed primarily via BUSCO and Merqury single-value scores at each pipeline stage.↳ Could also: Complementary reference-based statistical evaluation (e.g., QUAST-LG misassembly counts summarized with confidence intervals, or k-mer-based error rate estimates with uncertainty bounds) could also be used where a reference genome is available — This would add a quantified uncertainty estimate around assembly correctness claims, which is useful when a validated reference genome exists for comparison
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-36792366 (polishCLR)
Paper: Chang J, Stahlke AR, Chudalayandi S, Rosen BD, Childers AK, Severin AJ. polishCLR: A Nextflow Workflow for Polishing PacBio CLR Genome Assemblies. Genome Biol Evol 2023;15(3):evad020. PMID 36792366 · PMCID PMC9985148 · DOI 10.1093/gbe/evad020.
Code: https://github.com/isugifNF/polishCLR (authors' own tool — own-repo, not P16). Data (reads): SRA BioProject PRJNA804956 (Helicoverpa zea PacBio CLR + Illumina). Example inputs (assemblies + small test reads): Ag Data Commons / figshare DOI 10.15482/USDA.ADC/1524676 (article 24667776):
- FALCON-stage primary/alt/cns contigs (p_ctg.fasta 413 MB, cns_p_ctg.fasta 419 MB, a_ctg_all.fasta 194 MB, all_h_ctg.fasta 103 MB, cns_h_ctg.fasta 101 MB, all_p_ctg.fasta 412 MB)
- chr30 reference: GCF_022581195.2_ilHelZeax1.1_chr30.fasta (6.4 MB)
- test reads: test.1.filtered.bam_.gz (2.57 GB, PacBio subreads), testpolish_R1/R2.fastq (~53 MB each, Illumina)
What the paper actually reports
This is a workflow/tool paper. The MAIN TEXT reports NO quantitative metrics (no BUSCO %, no Merqury QV, no N50 numbers). It states results live in supplementary Table S1 ("Runtimes and summaries from each of the three starting input cases"). The repo additionally ships authors' Nextflow execution reports/timelines (docs/case1_reports/, docs/case3_reports/, docs/report*.html).
The central reproducible CLAIM of the paper is therefore: "The published polishCLR Nextflow workflow runs end-to-end on the published example data and polishes a PacBio-CLR assembly, producing the documented quality metrics (BUSCO / Merqury QV / assembly stats), with completeness/QV not degrading after polishing."
Pipeline (all stages are bioinformatic → in scope)
Tools (environment.yml, channels conda-forge/bioconda/agbiome): pbmm2, pbgcpp (Arrow/gcpp), bwa-mem2, freebayes, bcftools, samtools, bamtools, meryl, merqury, merfin, purge_dups, busco=5.4.2, bbtools, pigz, parallel, python=3.7, openjdk=11.0.15, nextflow. Container csiva2022/polishclr:latest (not usable on «our HPC» — no docker/singularity; rebuild env via bioconda/conda).
Workflow modules (modules/*.nf): arrow.nf, purge_dups.nf, freebayes.nf, busco.nf, qv.nf (meryl/merqury), helper_functions.nf. Two steps:
- Step 1: pbmm2 align → Arrow(gcpp) polish (Case 1/2) → purge_dups dedup → BUSCO.
- Step 2: Arrow polish (gap-fill) → FreeBayes polish ×2 (bwa-mem2 + freebayes
- bcftools consensus, merfin-filtered when same specimen) → meryl/merqury QV + BUSCO.
IN SCOPE (pipeline-derived, will attempt)
- R1 Executability / DAG (stub-run). Reproduce the repo's own CI stubtest
(
nextflow run main.nf … -stub-run -profile local): proves the published process graph is internally consistent and runs end-to-end. Pipeline: Nextflow/polishCLR. - R2 Real chr30 polishing run (step1 → step2). Run the actual workflow on the
chr30 example assembly + test reads on «our HPC» SLURM. Produce:
- BUSCO completeness (complete/single/dup/frag/missing) before vs after
- Merqury QV before vs after
- Assembly stats (N50, # contigs, total length) before vs after Compare direction + magnitude vs paper claim / supplementary Table S1 / shipped case reports. Pipeline: full polishCLR (pbmm2+gcpp+purge_dups+bwa-mem2+ freebayes+meryl/merqury+busco).
- R3 Runtime/process-summary cross-check vs shipped docs/case*_reports (provisional; hardware differs from authors' Ceres/Atlas, so wall-times are not 1:1 — used only for qualitative "same processes ran / same scale").
OUT OF SCOPE (not pipeline-derived, or not feasible / not attempted)
- The upstream FALCON / FALCON-Unzip assembly itself (paper uses pre-made example contigs as input; assembling from raw reads is not what polishCLR does).
- Wet-lab sequencing (PacBio/Illumina library prep) — not computational.
- The three full-genome production cases at native scale (full p_ctg 413 MB + full-coverage CLR) if compute is prohibiti
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.