HGA: de novo genome assembly method for bacterial genomes using high coverage short sequencing reads.
Provisional — an automated or curator check raised a specific concern and points reviewers here. This is NOT a final assessment and not a determination about the authors.
The main results reproduced, with only marginal, non-material deviations.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
PARTIAL (preliminary). HGA = real, runnable Velvet/SPAdes meta-assembler scored by QUAST vs the finished reference (P16 third-party-tool case); assigned dataset SRR522246 = GAGE-B R_sphaeroides_MiSeq assembles cleanly. Genome size/GC/completeness reproduce (C4). Headline contiguity NA50 246 Kbp NOT reached: best HGA NA50 = 165.8 Kbp (auN=240k is close). Baseline reproduces to right order (C2). Fresh clean re-run («job») re-stages everything and pushes a wider grid toward 246k. Out of scope: full 7-genome x 8-assembler grid, HiSeq, other organisms.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 58assessed: 2026-06-21 ⛓ e52dc2cd8e1b
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-24
- Rubric version
- not recorded
- Assessed by
- —
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper tests whether a hierarchical genome assembly (HGA) methodology—partitioning reads into disjoint subsets, assembling each independently, combining/merging the resulting contigs, and re-assembling them with the original reads—can further improve bacterial genome assembly quality (N50/NA50) beyond simply running a single assembler on very high coverage short-read data, as shown feasible by the GAGE-B study.
- ★ HGA leads to significant improvement in assembly quality (N50 and corrected N50) for all 7 evaluated GAGE-B bacterial datasets using most of the 8 evaluated assemblers finding
- ★ Partitioning reads into lower-coverage subsets reduces de Bruijn graph complexity (tips, bubbles, bulges, cycles), making assembly ambiguity resolution more efficient and producing contigs with fewer errors, especially for HiSeq data mechanism
- ★ minimus2 is misleading for combining partition contigs: despite improving NA50, it significantly increases duplication ratio and misassembly events by erroneously joining true but non-contiguous contigs finding
- ★ Velvet (kmer=31, expected coverage set to number of partitions) is the best-performing de Bruijn graph assembler for combining/assembling contigs from partitions method
- ★ SPAdes outperforms Velvet as the re-assembly step assembler for combining merged/combined contigs with the original reads method
- ★ Each partition must retain at least 10x coverage; there is no other constraint on the number of partitions produced method
- Re-assembling contigs with the same reads dataset that produced them does not improve, and can deteriorate, the resulting assembly finding
- String-graph-based assemblers (e.g., SGA) are ineffective for assembling contigs because overlaps between contigs can occur mid-sequence, not just at ends mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| de novo genome assembly (basic flow, HGA merged/combined-contig flows) | Bacillus cereus ATCC 10987 | read partitioning (p=1,2,4,8) vs no partitioning | N50, NA50, NGA50, genome fraction, duplication ratio, misassemblies, MP100K, IP100K | Illumina MiSeq, 250bp |
| de novo genome assembly (basic flow, HGA merged/combined-contig flows) | Mycobacterium abscessus 6G | read partitioning (p=1,2,4,8) vs no partitioning | N50, NA50, NGA50, genome fraction, duplication ratio, misassemblies, MP100K, IP100K | Illumina HiSeq 100bp and MiSeq 250bp |
| de novo genome assembly (basic flow, HGA merged/combined-contig flows) | Rhodobacter sphaeroides 2.4.1 | read partitioning (p=1,2,4,8) vs no partitioning | N50, NA50, NGA50, genome fraction, duplication ratio, misassemblies, MP100K, IP100K | Illumina HiSeq 101bp and MiSeq 251bp |
| de novo genome assembly (basic flow, HGA merged/combined-contig flows) | Vibrio cholerae CO1032(5) | read partitioning (p=1,2,4,8) vs no partitioning | N50, NA50, NGA50, genome fraction, duplication ratio, misassemblies, MP100K, IP100K | Illumina HiSeq 100bp and MiSeq 250bp |
| contigs-combining assembly comparison (Velvet vs minimus2) | contigs from partitioned assemblies of 4 bacterial datasets | assembler choice for combining contigs (Velvet kmer=31 vs minimus2) | NA50, duplication ratio, misassembly events | Velvet v1.2.10 / minimus2 |
| re-assembly step comparison (SPAdes vs Velvet) | merged/combined contigs plus original reads, tested on two genomes | re-assembler choice (SPAdes vs Velvet) | assembly quality metrics (N50/NA50 etc.) | SPAdes v3.0.0 / Velvet v1.2.10 |
| reads+reference genome co-assembly test | Mycobacterium abscessus HiSeq dataset assembled with M. abscessus genome sequence | kmer length sweep (21-91) | NA50 as percent of genome length | SPAdes assembler |
- ▲ For all 7 GAGE-B datasets, HGA (merged or combined contigs re-assembled with SPAdes) improved N50/NA50 compared to the basic single-pass assembly flow, using most of the 8 assemblers
- ▼ Average local misassemblies, global misassemblies, MP100K, and IP100K per partition were lower than the basic (unpartitioned) flow for most assemblers and genomes, especially HiSeq datasets
- ▲ Using minimus2 to combine contigs significantly increased duplication ratio and misassembly events relative to Velvet, despite an apparent NA50 improvement
- ▲ Assembling the M. abscessus HiSeq reads together with the reference genome itself at kmer=91 achieved NA50 equal to 99% of the genome length 99%
- ▲ SPAdes produced better re-assembly results than Velvet when re-assembling merged/combined contigs with reads
- – Partitions were viable down to approximately 10x coverage; splitting beyond 8 partitions was avoided because resulting coverage became too low (<10x) to assemble <10x
- other coverage range 100x–210x (sequencing coverage across the 7 GAGE-B datasets used)
- fold_change NA50 = 99% of genome length at kmer=91 (M. abscessus HiSeq reads co-assembled with reference genome)
- count 8 assemblers tested (Abyss v1.5.1, Cabog v7.0, Mira v4.0.2, MaSuRCA v2.2.1, SGA v0.10.13, SoapDenovo2 v2.04, SPAdes v3.0.0, Velvet v1.2.10) (assemblers evaluated with HGA methodology)
- count 7 GAGE-B bacterial datasets used (4 bacterial genomes, HiSeq and/or MiSeq reads)
- count partitions p = 2, 4, 8 (number of disjoint read partitions tested in HGA)
- other genome sizes 4.0–5.4 Mb, GC content 35–69% (characteristics of the 4 bacterial genomes assembled)
- other kmer lengths 21,31,...,91 (HiSeq) and 21,31,...,101 (MiSeq) (kmer sweep used for basic and HGA assembly flows)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a computational methods paper introducing the Hierarchical Genome Assembly (HGA) methodology for bacterial de novo genome assembly. The evaluation is entirely empirical and descriptive: HGA and a basic assembly baseline are compared across 8 assemblers and 7 GAGE-B datasets using assembly quality metrics (N50, NA50, NG50, NGA50, genome fraction, duplication ratio, misassembly counts, mismatch/indel rates per 100 kbp) computed by QUAST. No formal inferential statistics are applied; claims of 'significant improvement' are used informally to describe directional metric differences observed by inspection across assembler–dataset–kmer combinations.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| None — direct metric comparison by inspection (N50, NA50, corrected N50, and related QUAST metrics) | HGA flows vs. basic assembly flow across all 8 assemblers and 7 datasets | — | na |
-
Improvement of HGA over baseline was assessed by direct visual/tabular inspection of metric values across assembler–dataset combinations↳ Could also: A non-parametric paired test such as the Wilcoxon signed-rank test (or Friedman test for multiple assemblers simultaneously) could also compare HGA vs. basic assembly N50/NA50 values across the 7 datasets, treating each dataset as one observation — Formal testing would quantify whether the observed directional improvements exceed what might be expected by chance across the evaluation set, and would yield a p-value and effect size (e.g., rank-biserial correlation) to summarize the overall pattern
-
The best N50 over the full kmer range was selected and reported for each assembler–dataset combination↳ Could also: Reporting the median or full distribution of N50 over kmer values, or using a hold-out kmer for final comparison, would also characterize how sensitive conclusions are to kmer choice — Selecting the maximum over a parameter grid can favor the method with more configurations tested; reporting the distribution or a pre-specified kmer makes the comparison less dependent on post-hoc optimisation
-
Absolute metric values (N50, NA50) are reported for each condition↳ Could also: Reporting percentage improvement of HGA over baseline (e.g., (HGA_N50 − Basic_N50) / Basic_N50 × 100) would also characterize effect magnitude — Percentage or fold-change summaries make it easier to compare the size of gains across datasets that differ widely in genome size and baseline N50, without requiring mental arithmetic on raw numbers
-
Multiple assembly quality metrics (N50, NA50, genome fraction, duplication ratio, misassembly counts, MP100K, IP100K) are examined separately↳ Could also: A composite ranking or scoring approach (e.g., averaging per-metric ranks across datasets as in the GAGE-B study itself) could also aggregate performance across metrics into a single summary — Aggregate ranks reduce the number of comparisons to inspect and make it easier to identify assemblers that perform consistently well or poorly across the full metric set
-
Claims of improvement are made across all 8 assemblers and 7 datasets without any multiplicity adjustment↳ Could also: A Bonferroni or Benjamini–Hochberg correction applied to the family of assembler × dataset comparisons could also control the false discovery rate if formal tests were conducted — With many simultaneous comparisons, unadjusted inspection increases the chance that some apparent gains reflect sampling variability; multiplicity-aware analysis would clarify which improvements are robust across the evaluation family
-
Reads were partitioned into 2, 4, or 8 disjoint subsets without a systematic sensitivity analysis of how partition count affects downstream results↳ Could also: A factorial or regression analysis of metric outcomes as a function of partition count and kmer length would also characterise the parameter space and identify optimal or robust configurations — Systematic parameter sensitivity analysis (e.g., two-way interaction between partition count and kmer length on N50) would help users select settings for novel datasets without relying on empirical trial
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-26945881 (HGA de novo bacterial assembly, Alokaily 2016, BMC Genomics)
- Title: HGA: de novo genome assembly method for bacterial genomes using high coverage short sequencing reads.
- PMID 26945881 · PMCID PMC4779561 · DOI 10.1186/s12864-016-2515-7
- Code: https://github.com/aalokaily/Hierarchical-Genome-Assembly-HGA (Python 2.7,
HGA.py) - Brief-assigned data:
sra:SRR522246= Rhodobacter sphaeroides 2.4.1, Illumina MiSeq, 251 bp PE, ~100x, genome ~4.6 Mb (GAGE-B datasetR_sphaeroides_MiSeq).
What HGA is (pipeline)
HGA = a wrapper "meta-assembler" that runs existing assemblers in a hierarchy:
- Partition reads into P disjoint subsets (each >= 10x coverage).
- Partition assembly of each subset (Velvet or SPAdes) at k =
Pkmer. - Merge / Combine partition contigs (merged = concatenation; combined = Velvet re-assembly of merged at k=31).
- Re-assembly of the merged/combined contigs together with the whole reads using SPAdes at k =
Rkmer, producingHGA_merged/andHGA_combined/(SPAdescontigs.fasta/scaffolds.fasta). Quality is then scored with QUAST vs the finished reference (paper reports NA50, NG50, NGA50, #contigs, genome fraction, mis-assemblies, MP100K/IP100K). => This is the P16 case: a real, runnable bioinformatic pipeline. Reproducible.
In scope (pipeline-derived, attempted)
Assembly-quality metrics for the assigned dataset (R. sphaeroides MiSeq, SRR522246):
- C1 (headline): best HGA NA50 ≈ 246 Kbp vs GAGE-B best baseline NA50 ≈ 143 Kbp (Fig 2 / Additional file 1 Tables S3–S9, MiSeq panel). i.e. HGA roughly doubles contiguity over the best single-assembler GAGE-B result.
- C2 (support): direction/magnitude — HGA assembly of this dataset yields markedly larger NA50/N50 than a plain SPAdes/Velvet baseline run on the same reads.
- Secondary metrics we will also report (not all have a single clean paper number for this exact cell): #contigs, NG50/NGA50, genome fraction %, largest contig.
Out of scope (not attempted, why)
- The full 7-dataset × 8-assembler × k-mer × P grid (Tables S1–S9). 80/20: we reproduce the assigned dataset only; the full grid is the optional hard 20% (compute-heavy, many assembler builds).
- HiSeq datasets and the other organisms (B. cereus, M. abscessus, V. cholerae). Not the assigned accession.
- Any wet-lab / sequencing / manual-curation statements — none; paper is fully computational.
- Exact bit-identical NA50: assembler/QUAST versions and partition RNG differ; we grade on contiguity magnitude & HGA-vs-baseline improvement, not exact equality. Provisional grades only — human reviewer decides.
Method notes / params (from repo test scripts + Methods)
- Velvet 1.2.10 (
make MAXKMERLENGTH=111 LONGSEQUENCES=1), SPAdes 3.0.0; QUAST for scoring. - Repo recipe form:
HGA.py -velvet <dir> -spades <bin> -PA velvet -P12 reads -R12 reads -ins <I> -std <S> -P <parts> -Pkmer 31 -Rkmer 51 -t <n> -out HGA. - Paper MiSeq optimum: partition to ~12–25x per partition (P=4 or 8 from ~100x), Pkmer 21–31, Rkmer 61–81.
- GAGE-B R. sphaeroides MiSeq insert ≈ 220 bp (read from the trimmed read filenames
insert_<I>_*).
Reproduction strategy
Run on «our HPC» («infra») one or a few HGA configs (P=4/8, Pkmer 31, Rkmer in {51,61,81}) + a plain SPAdes baseline, QUAST all vs R. sphaeroides 2.4.1 reference, report NA50 etc., compare to C1/C2.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
<synthetic>Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.