StatsDB: platform-agnostic storage and understanding of next generation sequencing run metrics.
Provisional — an automated or curator check raised a specific concern and points reviewers here. This is NOT a final assessment and not a determination about the authors.
The main result did not reproduce in this reproduction attempt. Where our recomputation produced values that differ from the published ones, those discrepancies are listed below. This is a single automated attempt — not peer review and not a finding of error or misconduct — and differences can also arise from data access, undocumented parameters or the computing environment. The verdict can be contested via “report an error”.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓Reported values are derivable from the shared data
- 🟡A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
StatsDB (PMID 24627795) is an infrastructure/software paper reporting no quantitative benchmarks. Its documented Perl example command fails on the current TGAC/statsdb repo HEAD due to a genuine dead-code bug (add_header_scope), but once that call is locally patched out and a self-referential path bug in the shipped example metadata is corrected, the core FastQC-parsing-to-MySQL pipeline reproduces the shipped ground-truth values exactly (total sequences, GC%, sequence length, per-base quality means all match to full float precision). The Reports/consumer query API partially works (global analysis listing correct; two other demo calls returned empty, uninvestigated). The Java/Maven component cannot be built at all due to unrelated, confirmed-dead upstream Maven repositories/dependencies (repos.tgac.ac.uk and a fluxion-spi transitive dependency mirror), blocked further by modern Maven's HTTP-mirror policy -- an external repo-rot finding, not an environment-engineering shortcoming on our side.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-07-30
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-07-31no human curator yet
- Last updated
- 2026-07-31
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusSequencing centres generate large volumes of per-run QC metrics that are typically left as flat files scattered on disc, preventing coherent cross-run analysis; the paper asks whether a generic, platform- and tool-agnostic database with abstracting APIs can enable systematic storage and granular querying of next generation sequencing run metrics.
- ★ StatsDB is an open-source software package for storage and analysis of next generation sequencing run metrics, backed by an SQL (MySQL) database with Perl and Java APIs. resource
- ★ No currently available solution facilitates easy storage and access of QC statistics from sequencing runs, leading to many flat files stored on disc in multiple locations and a lack of coherent analysis. finding
- ★ The generic database schema enables storage of data from any QC tool designed for any sequencing platform, so it can be easily adapted beyond the supplied FastQC modules. method
- ★ API abstraction allows simpler, wider querying across multiple fields than manual dissection of individual reports, e.g. metrics about nucleotide bias in libraries using adaptor barcode X, across all runs on sequencer A, within the last month. method
- ★ Most functionality is embedded in the database as views and stored procedures, enabling lightweight APIs that behave consistently across languages and are easily portable to other languages. mechanism
- ★ The system is designed around two program types: parsers, which write QC tool output to the database, and consumers, which query and present the data to users. method
- The database is normalised to third normal form (3NF) and stores metrics on a per-base, per-partition or per-run basis, with auxiliary views (property, run, latest_run) to facilitate complex queries. method
- StatsDB is integrated into production use at TGAC, with reports accessed through the MISO LIMS system or a standalone GUI tool; use of the software requires no knowledge of SQL. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| FastQC read QC analysis (parsing of fastqc_data.txt plain-text output) | FASTQ sequence read files from next generation sequencing runs | none | per-base quality means, quality score counts, GC content, base content, over-represented sequences stored as analysis/per-position/per-partition values | FastQC |
| Relational storage and querying of QC metrics via stored procedures and views | MySQL 'statsdb' database (schema + stored_procedures SQL files) | none | run summaries grouped by instrument, run, lane, pair, barcode (general_summaries_for_run, summary_per_position_for_run, summary_value_with_comment) | MySQL |
| Programmatic data insertion and query via Perl API | Perl QCAnalysis and Reports.pm modules connecting via Perl DBI | none | add_general_value / add_position_value / add_partition_value insertions and property-hash-based queries | Perl DBI |
| Programmatic data insertion and query via Java API, including unit tests of database accessibility and schema correctness | Java statsdb-api Maven project (groupId uk.ac.tgac.statsdb, version 1.1), JDBC to MySQL | none | API load/query operations; command-line one-jar application processing report files or metadata tables (-t test mode writes nothing) | Maven / com.mysql.jdbc.Driver, jdbc:mysql://localhost:3306/statsdb |
| Installation/validation test run of the FastQC parser on supplied example data | Example metadata_test.txt and template_db.txt shipped with StatsDB | none | successful insertion of data into the statsdb database | parse_fastqc.pl |
| Interactive report visualisation for end users (consumers) | MISO LIMS system at TGAC and a standalone GUI tool | none | text and graphical summaries of run QC metrics | d3.js |
- ▲ Modern sequencing platforms generate enormous quantities of data, with short-read sequencers capable of generating hundreds of gigabases of sequence per run, and multiplexing allowing hundreds of samples per run. hundreds of gigabases per run; hundreds of samples per run
- – StatsDB supports three value scopes — analysis (global values such as total GC content), base_partition (per-base summaries allowing grouping of ranges) and sequence_cumulative (cumulative counts such as quality scores or read counts). 3 scopes
- – Four stored procedures are provided for listing/introspection (list_runs, list_selectable_properties, list_selectable_values_from_property, list_summary_per_scope) and three for report generation (general_summaries_for_run, summary_per_position_for_run, summary_value_with_comment). 4 query + 3 report procedures
- – Eleven controlled, platform-agnostic analysis properties (tool, encoding, chemistry, instrument, software, type, pair, sample_name, lane, run, barcode) define common denominators across platforms and analyses. 11 properties
- – Two APIs (Perl and Java) offer the same functionality, both calling stored procedures through an abstraction layer so parsers and consumers never access stored procedures directly. 2 APIs
- – The latest_run view selects the latest run from the run view, so if an analysis has been carried out more than once only the latest results are stored.
- – MISO, the open-source LIMS in development at TGAC, provides support for accessing data in StatsDB out-of-the-box.
- – Existing QC tools surveyed (FastQC, HTQC, PRINSEQ, NGSQC, QRQC, SAMStat, stsPlots, PacBioEDA) generate per-run statistics but none provide storage/access infrastructure; HTQC comprises six tools for analysis and trimming. HTQC: 6 tools
- count hundreds of gigabases of sequence data per run (throughput of next generation short-read sequencers)
- count 15,000 (example add_position_value call: reads in the analysis with a quality score of 30)
- mean 38.7 (example add_partition_value call: quality mean from position 10 to 14 (5 values))
- other 3NF (third normal form) (normalisation level of the StatsDB database schema)
- count 10 (example add_general_value call: count for over-represented common primer 'ACCTGATAT' in library A)
- count 100 (example add_general_value call: average_length of reads)
- count 6 (number of tools composing the HTQC toolkit)
- other BB/J010375/1 (BBSRC National Capability Grant funding StatsDB development at TGAC)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a bioinformatics software/tool paper describing StatsDB, a database system and API for storing and querying next-generation sequencing run quality-control metrics. The manuscript describes system architecture, database schema, and API usage rather than an experiment with statistical comparisons between groups; no hypothesis tests, effect estimates, or inferential statistics on experimental data are reported.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
No numerical discrepancy exists here. Every value the artifact can be held to — Total Sequences 13792597, Filtered 0, length min=max=251, %GC 42, quality_mean@pos1=32.73492018943205 — reproduced exactly to full stored float precision against ground truth embedded in the repo's own example FastQC report, so q5 is clean and there is no fabrication concern. The deviations are entirely executability failures on the authors'/maintainers' side: parse_fastqc.pl:34 calls add_header_scope(), which RunTable.pm has commented out as dead code, so the paper's verbatim example command crashes on current HEAD; the shipped metadata_test.txt points PATH_TO_FASTQC at itself, which after patching produces a silent empty insert rather than an error; and the Java/Maven module is unbuildable because repos.tgac.ac.uk and the fluxion-spi mirror are both dead. Two documented one-line patches were needed to reach the exact match, and two Reports-API demo calls (get_average_values, list_all_runs) returned empty — hence limited, not full, confirmation of the core claim. Overall yellow: scientifically sound and fully derivable, functionally degraded by code- and dependency-rot that our side did not cause and could not fix without altering the published artifact.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.