SLDMS: A Tool for Calculating the Overlapping Regions of Sequences.
The main results reproduced: recomputed values matched the published ones within tolerance.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
REPRODUCED (1:1 on the verifiable claims). SLDMS = authors' own C/C++ overlap-region tool (commit 9b1a64d), reproduced on «our HPC» (SLURM «job», node n094). C1: clean build of both binaries + documented example overlap output (exact). C2 (headline claim 'accuracy above 99.99%'): reproduced EXACTLY at 100.00% for BOTH the exact (corr, 998/998) and fault-tolerant (raw, 488/488) versions, using the authors' shipped about-accuracy-proof framework with audit-visible parameters (genome 5Mb, read 10k, overlap 5k) and cross-checked with an independent per-line verifier. NOTE: the shipped cal_code_accuracy_raw.cpp has a line-width parsing bug (reads 13 tokens/line; raw output has 17) that makes it print 0.168033 as-is; fixing the skip count to 17 yields 100.00%, matching the independent verifier -- so the claim holds and the bug is in the verification script, not the tool. C3: SLDMS runs successfully on the paper's named real dataset SRR11606869 (Z. mays B73) -- a 40000-read/627Mbp subset mirroring the paper's '579 MB cut' -- producing 40000 overlap records in 114.02 s with 8.36 GiB peak RSS (a concrete data point; the paper reports no maize runtime number, Fig 7 is visual only). DATASET FLAG: ENA deposits SRR11606869 as PacBio '_subreads' (CLR), while the paper Table 1 calls it 'PacBio HiFi' -- a data-type discrepancy. NOT attempted: S1 runtime-vs-Canu/Flye (hardware-bound absolute numbers, not 1:1 reproducible by design).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 80assessed: 2026-06-21 ⛓ 5a94bde49f6f
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-21
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper tests whether a new method (SLDMS) based on suffix arrays and a monotonic stack can compute overlapping regions between sequencing reads more efficiently than existing genome assembly tools (Canu, Flye).
- ★ SLDMS is a novel method for computing overlapping regions of sequencing reads using suffix array (SA), longest common prefix (LCP) array, document array (DA), and a monotonic stack. method
- ★ SLDMS runs significantly faster than Canu and Flye when computing sequence overlap regions. finding
- ★ For datasets where sequencing errors are concentrated at both ends of reads, SLDMS running time is only about one-tenth of Canu and Flye. finding
- ★ SLDMS provides two versions: one for high-accuracy data using exact suffix matching, and one for error-containing data using fault-tolerant matching with a user-set parameter K. mechanism
- SLDMS uses gsufsort (based on the gSACA-K algorithm) to construct the SA, LCP, and DA arrays from the concatenated read data. method
- ★ The monotonic stack filters the suffix set using the LCP array to remove suffixes that cannot be optimal overlap candidates, improving computation speed. mechanism
- SLDMS provides top-K overlapping region results per read using a min-heap data structure for downstream genome assembly use. resource
- SLDMS accepts fastq and fasta formatted sequencing data as input and outputs overlap information for use by other assembly software. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Running-time benchmarking of overlap-region computation | sequencing read datasets with errors concentrated at both ends (e.g., long-read/third-generation data) | algorithm comparison (SLDMS vs Canu vs Flye) | running time | — |
| Suffix array / LCP array / document array construction | concatenated sequencing reads (fastq/fasta input) | none | SA, LCP, DA arrays | gsufsort (gSACA-K algorithm) |
| Overlap detection via exact suffix matching (version 1, no-mismatch mode) | PacBio-HiFi dataset / Sanger dataset (high-accuracy or corrected data) | none | overlapping regions between reads | — |
| Overlap detection via fault-tolerant matching (version 2, mismatch-tolerant mode) | third-generation long-read sequencing data with errors concentrated at read ends | parameter K (maximum allowed cut length at head/tail during matching) | overlapping regions between reads tolerant of end mismatches | — |
| Read error correction (upstream preprocessing) | PacBio third-generation reads, self-corrected or corrected using Illumina second-generation data | error correction (self-correction vs Illumina-assisted correction) | corrected read sequences | PBCR/Celera Assembler, LoRDEC |
- ▼ SLDMS running time is much less than that of Canu and Flye for computing sequence overlap regions.
- ▼ On data where sequencing errors mostly occur at both ends of reads, SLDMS running time is only about one-tenth of Canu and Flye. ~10-fold (one-tenth)
- fold_change one-tenth (SLDMS running time relative to Canu and Flye on data with sequencing errors concentrated at both ends of reads)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a computational methods/tool paper introducing SLDMS, a bioinformatics algorithm for computing sequence overlapping regions in genome assembly using suffix arrays and monotonic stacks. Performance evaluation consists of runtime comparisons between SLDMS and two existing tools (Canu and Flye) on sequencing datasets. No inferential statistical tests are described in the available text; evidence is presented as observed runtime differences (e.g., SLDMS running time approximately one-tenth of comparators on certain datasets). The paper does not report formal hypothesis testing, p-values, or measures of dispersion.
-
Runtime comparisons between SLDMS and comparator tools are summarized as single approximate ratios without reporting variance across runs↳ Could also: Report mean ± SD (or median ± IQR) wall-clock time across multiple independent runs of each tool on the same dataset — Repeated-run statistics would quantify run-to-run variability and allow readers to judge whether observed differences are consistent; a single-ratio summary cannot distinguish a stable speedup from a one-time observation
-
Runtime differences between tools are described verbally (e.g., 'much less than', 'about one-tenth') without formal hypothesis testing↳ Could also: Apply a non-parametric test such as the Wilcoxon rank-sum test (or paired Wilcoxon signed-rank test if the same dataset-run pairs are compared) across multiple replicate benchmark runs — Formal testing would let readers assess whether observed runtime differences exceed what could arise from system noise or scheduling variability, which is standard practice in algorithm benchmarking papers
-
Benchmarks appear to be run on a small number of specific datasets without reporting how many replicate timing measurements were taken per dataset↳ Could also: Specify the number of repeated timing runs per dataset (e.g., n = 5 independent runs) and report summary statistics for each, following benchmark best practices such as those in the Computer Systems benchmarking literature — Explicit replication of timing experiments reduces the influence of transient system load and makes the benchmark reproducible by others
-
Algorithm accuracy (overlap correctness) is described qualitatively for different error distributions without quantitative accuracy metrics↳ Could also: Report precision, recall, and F1-score for overlap detection against a ground-truth simulated dataset, with confidence intervals across multiple simulated replicates — Quantitative accuracy metrics with uncertainty bounds allow direct comparison with other tools and give users a principled basis for tool selection
-
Comparisons are made between SLDMS and exactly two other tools (Canu, Flye) with no stated rationale for excluding other commonly used assemblers↳ Could also: Include additional reference tools (e.g., Minimap2, wtdbg2) and report effect sizes (e.g., speedup ratio with bootstrapped 95% CI) across all comparators — A broader comparator set with effect-size reporting provides a more complete picture of where SLDMS sits in the landscape of available tools
-
The fault-tolerant version's accuracy is described as varying 'from one dataset to another' without quantifying that variability↳ Could also: Report accuracy as a function of the parameter K across a range of simulated error rates, using a factorial or dose-response design — A systematic parameter sweep would let users predict tool behavior for their own error distributions rather than relying on qualitative guidance
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-35046988 (SLDMS)
Paper: Chen Y, You D, Zhang T, Wang G. SLDMS: A Tool for Calculating the Overlapping Regions of Sequences. Front Plant Sci 2022. PMID 35046988 · PMCID PMC8761809 · DOI 10.3389/fpls.2021.813036.
Code: https://github.com/Dongliang-You/sldms — pinned commit
9b1a64d13b736a0300487993e6578d65edbf7953 (master, pushed 2023-02-18).
This is the authors' OWN tool (P16 N/A — own repo).
What SLDMS is. A C/C++ command-line tool that computes, for a collection of strings/reads, the overlapping regions between reads (intended as the overlap stage of an OLC genome assembler). Two modes:
--correxact-matching version--rawfault-tolerant version (allows head/tail trimming up to--tolerance K) Builds two binariessldms(32-bit index) andsldms-64(for inputs > 2 GB). Input: txt / fasta / fastq. Output:tempout.txt(per-read top-k overlaps).
In scope (pipeline-derived, computational → attempt)
| id | reported result | paper location | pipeline | feasibility |
|---|---|---|---|---|
| C1 | Tool builds (make → sldms, sldms-64) and runs the quick test ./sldms dataset/example.txt --corr --topk=10, producing the documented overlap output |
README "quick test"; repo | make; run binary |
EASY, deterministic |
| C2 | "the accuracy of the two versions of the SLDMS is above 99.99%" (exact --corr AND fault-tolerant --raw on simulated reads with known overlap positions) |
Results §"Accuracy Analysis" / Abstract | shipped about-accuracy-proof/ framework: create_DNA → GET_FASTA[raw] (+check file) → sldms → cal_code_accuracy{corr,raw} |
FEASIBLE — headline quantitative claim, fully shipped framework |
| C3 | SLDMS runs on the paper's real data SRR11606869 (Z. mays B73 PacBio-HiFi WGS) and reports a wall-clock runtime + peak memory (malloc_count) — a single clear data point | Table 1; Results / Fig 7 | download SRA → fasterq-dump → sldms --corr --time --verbose |
FEASIBLE (data point); absolute time is hardware-specific |
Stretch (attempt only if time; honesty over coverage)
| id | reported result | why hard |
|---|---|---|
| S1 | Runtime comparison "SLDMS faster than Flye on all datasets; faster than Canu on most; ~1/10 of Canu/Flye on error-end data" (Fig 7–9, Suppl. Tables 3–4) | Requires running Canu + Flye full assemblies on multi-GB HiFi sets (hours each); the absolute numbers are hardware-specific and NOT 1:1 reproducible by design. Direction (SLDMS faster) is checkable in principle but expensive. |
Out of scope (not pipeline-derived / not attempted)
- Hardware-specific absolute runtimes & figures plotted on the authors' machine (CPU/RAM-dependent; the paper itself states datasets were "cut" to fit their hardware) — comparisons are environment-bound, not reproducible 1:1.
- Algorithmic correctness proofs / design discussion (
about-accuracy-proofreasoning) — conceptual, not a data result.
Datasets the paper relies on (Table 1; profiled separately)
| accession | organism | platform | paper size |
|---|---|---|---|
| SRR11606869 | Zea mays "B73" | PacBio HiFi WGS | 579 MB (cut) — the RU's named dataset |
| SRR10971019 | E. coli K12 | PacBio (20 kb sheared) | 1,914 MB |
| SRR11606867 | F. × ananassa "Royal Royce" | PacBio HiFi | 1,131 + 2,190 MB |
| SRR11606870 | M. musculus C57BL/6J | PacBio HiFi | 1,567 + 2,760 MB |
Primary profiling target = SRR11606869 (the accession in the brief). Others noted but not all downloaded (multi-GB; profile primary + record the rest).
Reproduction strategy
All compute on «our HPC» (SLURM, «infra»). The accuracy framework ships placeholders
(you_path), (testdata_size), (coverage), (read length), (overlap) — we
fill them with a representative config consistent with the paper's described
"ultrahigh-accuracy" simulated sets (read counts 5k–20k, lengths 5k–20k bp;
Suppl. Tables 1–2
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.