A chromosome-scale genome assembly and karyotype of the ctenophore Hormiphora californensis.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡A deviation arose in the data or preprocessing
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
PARTIAL reproduction, described well enough to reproduce the published outputs. Real «our HPC» compute («job») recomputed the assembly's contiguity + karyotype statistics 1:1 from the deposited UCSC_Hcal_v1 assembly: total 110.68 vs 110.6 Mb, 44 scaffolds (exact), N50 8.54 vs 8.5 Mb, 13 chromosomes (exact), top-13 99.49% vs 99.47%, gaps 2.81 vs 2.76/Mb, contigs 351 (exact, BUSCO definition) - all exact/within-tol. BUSCO completeness PARTIAL: 89.1% (eukaryota_odb10, BUSCO 5.7.1/miniprot) vs reported 96% (eukaryota_odb9, older BUSCO) - same high-completeness conclusion, gap is a documented lineage+algorithm version effect. KEY FINDING (mismatch, flagged for audit): the reported 14,265 protein-coding genes is NOT derivable from ANY deposited annotation file - all of them (release GFF, Hcalv1.1 GFF, transcript FASTA, Feb24 proteome) cluster at ~12,200 genes / ~17,700 transcripts. De novo assembly (wtdbg2->arrow->pilon->Dovetail HiRise->PurgeHaplotigs) NOT attempted: proprietary HiRise + non-deterministic. Datasets profiled: SRA PRJNA576068 (30 runs, all promised modalities present, RNA-seq ~28 Gbp corroborated, grade A); GitHub/Zenodo assembly+annotation deposit (assembly delivers exactly, annotation gene count does not - grade B).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 86assessed: 2026-06-18 ⛓ 9acc0006ae6a
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-25
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper investigates whether the ctenophore Hormiphora californensis can be karyotyped and chromosome-scale assembled to test whether it shares homologous chromosomes and karyotype with the ctenophore Pleurobrachia bachei, in support of using whole-chromosome comparisons for resolving early animal evolution and phylogenetics.
- ★ A chromosome-scale genome assembly of H. californensis spans 110 Mb in 44 scaffolds, with 99.47% of bases in 13 scaffolds resource
- ★ Chromosome micrographs and Hi-C heatmaps support a karyotype of 13 diploid chromosomes in H. californensis finding
- ★ Hi-C data reveal three large heterozygous inversions on chromosome 1, one sharing the same gene order as P. bachei finding
- ★ H. californensis and P. bachei share thirteen homologous chromosomes and the same karyotype of 1n=13 finding
- ★ Manually curated PacBio Iso-Seq-based annotation reveals complex gene structures, including nested genes and trans-spliced leader sequences finding
- TADs called via HOMER tend to occur near gene boundaries, supported by a permutation test finding
- Hi-C/PretextView-based combinatorial matrix inversion can distinguish misassemblies from true heterozygous inversions method
- Genome-wide single-nucleotide heterozygosity was estimated using sites at the modal Illumina WGS depth (178x) via the purpose-built chep package method
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| chromosome spread karyotyping / DAPI microscopy | H. californensis embryos (~64-cell stage) | none | chromosome number/karyotype | Leica DM5500 B microscope, DAPI |
| PacBio WGS CLR long-read sequencing | H. californensis individual Hc1 | none | genome sequence for de novo assembly | PacBio Sequel I |
| Illumina WGS sequencing / k-mer analysis | H. californensis (Hc1, Hc2) and P. bachei (SAMN00216730) | none | genome size and heterozygosity estimation | Illumina; jellyfish v2.2.10, GenomeScope2 |
| Hi-C sequencing / chromosome conformation capture | H. californensis (Hc1, Hc3) and P. bachei | none | chromosome-scale scaffolding, inversion detection, TAD calling | Illumina; Dovetail HiRise, HOMER, HiCExplorer, Cooler |
| Chicago library sequencing | H. californensis individual Hc1 | none | long-range scaffolding | Illumina |
| Illumina RNA-seq (TruSeq stranded) | H. californensis individual Hc1 | none | de novo transcriptome assembly, annotation support | Illumina; Trinity v2.5.1 |
| PacBio Iso-Seq (full-length transcript sequencing) | H. californensis individual Hc1 | none | manually curated gene/transcript models, nested genes, trans-splicing | PacBio Sequel II |
| Variant calling and phasing from mapped PacBio/Illumina reads | H. californensis individual Hc1 | none | diploid SNP/indel variants, haplotype phasing | minimap2, BWA-MEM, freebayes, HapCUT2 |
- – Final assembly spans 110 Mb across 44 scaffolds, with 99.47% of bases in 13 scaffolds 110 Mb; 13 scaffolds
- – Karyotype confirmed as 13 diploid chromosomes by micrographs and Hi-C heatmaps 13 chromosomes
- – Three heterozygous inversions detected on chromosome 1, spanning 73% of the chromosome 73%
- – H. californensis and P. bachei found to share 13 homologous chromosomes (1n=13) 13 homologous chromosomes
- – Predicted 1C genome size of H. californensis (96-98 Mb) is close to P. bachei (97.5 Mb) 96-98 Mb vs 97.5 Mb
- – K-mer spectra of both species show two peaks, with the lower-coverage peak larger than the higher-coverage peak, consistent with high heterozygosity
- – Deep sequencing coverage obtained for Hc1: PacBio WGS CLR, Illumina WGS, and combined Chicago/Hi-C reads 247x PacBio; 573x Illumina; 1956x Chicago+Hi-C
- – Mean read length for PacBio CLR and Iso-Seq data was consistent 2.7 kb
- other 110 Mb assembly, 44 scaffolds, 99.47% of bases in 13 scaffolds (genome assembly completeness)
- other 96-98 Mb (H. californensis); 97.5 Mb (P. bachei) (genome size estimation from k-mer spectra)
- other 247.7x PacBio WGS CLR; 573.5x Illumina WGS; 1956.4x Chicago+Hi-C (physical sequencing coverage for individual Hc1)
- other 2.7 kb mean read length (PacBio Sequel I CLR and Sequel II Iso-Seq reads)
- count 3 heterozygous inversions spanning 73% of chromosome 1 (Hi-C based inversion characterization)
- count 2.5 million Iso-Seq transcripts; 28 Gbp Illumina RNA-seq reads (transcriptomic data used for annotation)
- other 178x modal Illumina WGS mapping depth used for heterozygosity calling (whole-genome heterozygosity estimation method)
- other minimum 3 consecutive genes, up to 5 intervening genes, max 30 kb distance (microsynteny block detection parameters)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This genome report presents a chromosome-scale assembly and karyotype of the ctenophore Hormiphora californensis, relying primarily on computational genomics methods rather than classical inferential statistics. Genome size and heterozygosity were estimated from k-mer frequency spectra (GenomeScope2) and corroborated by variant-calling-based methods (vcftools, ANGSD). The sole formal significance test is a permutation test (1000 iterations) evaluating whether TAD boundaries co-localize with gene boundaries more than expected by chance. Results are communicated as assembly metrics (scaffold count, N50, physical coverage, BUSCO scores) and descriptive comparative-genomic observations.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Permutation test (1000 permutations, median distance statistic) | Testing significance of co-localization between TAD boundaries and nearest gene boundaries | — | not stated |
| k-mer spectrum analysis via GenomeScope2 | Genome size estimation and heterozygosity estimation for H. californensis and P. bachei | — | not stated |
| vcftools --het and ANGSD realSFS | Heterozygosity estimation in Hc1, Hc2, and P. bachei individual SAMN00216730 | — | not stated |
| BUSCO (Eukaryota v3) completeness scoring via gVolante | Completeness assessment of protein models, de novo transcriptome, and genome assembly | — | na |
| blastp / tblastn e-value threshold filter (e-value < 1e-10) | Inclusion criterion for P. bachei-derived gene models in annotation round 3 | — | not stated |
| HOMER de novo motif discovery | 1.5 kb flanking regions around TAD boundaries; background from genomic regions with minimal TAD separation score change | — | not stated |
-
Genome size was estimated from k-mer frequency spectra (GenomeScope2) using Illumina WGS reads↳ Could also: Flow cytometry (propidium iodide or DAPI staining against a reference standard) could also be used to provide an independent, sequencing-independent genome size estimate; smudgePlots could complement GenomeScope2 for ploidy inference in highly heterozygous genomes — Flow cytometry provides a direct physical measurement of nuclear DNA content without relying on sequencing-depth modeling assumptions; smudgePlots offer additional visualization of heterozygous k-mer pairs that can corroborate or refine ploidy calls independently of the GenomeScope2 model
-
The significance of TAD boundary co-localization with gene boundaries was assessed with a custom permutation test (1000 iterations, median distance statistic)↳ Could also: Dedicated genomic interval permutation frameworks such as GAT (Genomic Association Tester) or bedtools shuffle with chromosome-stratified randomization and a reported fold-enrichment could also quantify this association — Frameworks like GAT account for non-uniform gene density and assembly gaps when constructing the null distribution; reporting a fold-enrichment alongside the permutation p-value would also communicate the magnitude of the association, not only its statistical significance
-
Assembly and annotation completeness was evaluated with BUSCO against the broad Eukaryota v3 lineage gene set↳ Could also: A more taxonomically specific BUSCO lineage (e.g., Metazoa) and/or Merqury (reference-free, k-mer-based quality-value and completeness estimation) could also be applied — A Metazoa-level BUSCO set contains genes more relevant to a ctenophore than the pan-Eukaryota set and may capture lineage-specific completeness more sensitively; Merqury avoids dependence on a universal single-copy gene catalogue and directly assesses assembly-read concordance
-
Heterozygosity was estimated independently by three methods (GenomeScope2 k-mer spectrum, vcftools --het, ANGSD realSFS), plus a custom depth-filtered SNP counting approach at the modal coverage (178x)↳ Could also: Reporting confidence intervals or standard errors for each heterozygosity estimate (e.g., bootstrap intervals from the k-mer or ANGSD approach) could also accompany the point estimates — Because different estimation methods can yield slightly different values due to model assumptions, confidence intervals would allow readers to judge consistency across methods and assess sampling uncertainty, which is informative for subsequent comparative genomic inferences
-
No multiple-testing correction is described for the HOMER de novo motif discovery applied across TAD boundary regions↳ Could also: An FDR-based correction (e.g., Benjamini-Hochberg) applied across all motifs tested, as is standard in HOMER's own enrichment output, could also be reported explicitly — When many sequence motifs are tested simultaneously against a background set, an FDR threshold clarifies what fraction of reported motifs are expected to be true positives and aids cross-study reproducibility
-
Gene model inclusion in annotation round 3 used a fixed blastp e-value threshold (1e-10) as the sole statistical filter↳ Could also: A percent-identity and alignment-length filter applied jointly with the e-value, or a reciprocal-best-hit criterion, could also serve as an inclusion criterion — E-value alone is sensitive to database size and query/subject length; combining it with percent identity and coverage thresholds, or requiring reciprocal best hits, reduces the inclusion of spurious or highly divergent matches in the final annotation
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-34545398
Paper: A chromosome-scale genome assembly and karyotype of the ctenophore Hormiphora californensis. G3 (Bethesda) 2021. DOI 10.1093/g3journal/jkab302.
Code/data: GitHub conchoecia/hormiphora (@master, pushed 2020-10-09) +
Zenodo 10.5281/zenodo.4074309; raw reads SRA BioProject PRJNA576068;
mito MN544300/MN544301; figshare 10.25387/g3.15170382.
(BRIEF lists PacificBiosciences/gcpp — that is only the PacBio polishing tool
used in the pipeline, not the analysis repo. Per BRIEF rule P16 a third-party
tool is a valid code artifact; the reproducible outputs live in conchoecia/hormiphora.)
In scope — pipeline-derived results we attempt (run on «our HPC»)
These are deterministic computations over the deposited assembly FASTA
(annotation/raw_files/UCSC_Hcal_v1.fa.gz) and annotation GFF
(annotation/Hcv1a1d20200309_release/Hcv1a1d20200309.gff.gz):
| id | reported | where (paper) | how we reproduce |
|---|---|---|---|
| asm_size | 110.6 Mb | Abstract/Results | sum scaffold lengths (seqkit/python) |
| n_scaffolds | 44 | Abstract/Results | count sequences (excl. mito) |
| scaffold_N50 | 8.5 Mb | Results | N50 over scaffolds |
| n_contigs | 351 | Results | count contigs (split scaffolds on N-runs) |
| n_chromosomes | 13 (n=13) | Karyotype | # scaffolds ≥1 Mb / 13 longest |
| top13_pct | 99.47% | Results | sum(13 longest)/total |
| top13_range | 10.3–6.4 Mb | Results | min/max of 13 longest |
| gap_density | 2.76 gaps/Mb | Results | (#N-runs within scaffolds)/Mb |
| busco_complete | 96% (303 eukaryotic genes) | Results | BUSCO genome mode, eukaryota lineage |
| n_genes | 14,265 protein-coding genes | Results | count gene features in release GFF |
BUSCO note: the paper reports "96% (303 Eukaryotic genes)" → BUSCO eukaryota lineage (odb9 has 303 BUSCOs). Modern BUSCO ships eukaryota_odb10 (255 BUSCOs); we run odb10 and compare the % completeness (also run metazoa_odb10 for context). Lineage-version difference is documented, not hidden.
Out of scope — not attempted (and why)
- De novo assembly 1:1 (wtdbg2 v2.4 → arrow v2.2 → pilon v1.22 → Dovetail HiRise vAug2019 scaffolding → Purge Haplotigs v1.0.4). HiRise is a proprietary Dovetail Genomics service (no public binary) and the pipeline is non-deterministic at this scale; cannot be reproduced bit-for-bit. We instead verify the published assembly's contiguity/completeness statistics (above).
- Karyotype micrographs / FISH / cytogenetics — wet-lab (out of scope by rule 2).
- Read-mapping rates (95.32% PacBio subreads; 99.02% Iso-Seq FLNC to 13 largest) — would require downloading the full raw read sets (PacBio CLR 247×, Iso-Seq) from SRA and re-mapping. Possible but heavy; flagged as a stretch goal, attempted only if core results land and budget allows.
- Nested-intronic (NI) gene analysis (2,357 NI genes in 1,654 hosts) — depends on the authors' custom NI-detection script; recorded but not a primary target.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
All eight checked contiguity claims reproduce exact or within-tolerance directly from the publicly deposited assembly (44 scaffolds, 13 chromosomes, ~110.6 Mb, N50 8.5 Mb, 99.47% in top 13), so the data is identical and the core karyotype claim is fully confirmed. The single non-exact figure — #contigs 355 vs 351 — is a metric-definition (N-gap) difference on our side, not an authors' defect, and the size/N50/top13/gap-density gaps are pure rounding. The verdict is held at yellow overall only because the run is explicitly preliminary: BUSCO (96%) and gene count (14265) were not yet attempted. No fabrication concern — reported values are cleanly derivable from shared data.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.