HTSQualC is a flexible and one-step quality control software for high-throughput sequencing data analysis.
The main results reproduced: recomputed values matched the published ones within tolerance.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
REPRODUCED. SOFTWARE/TOOL paper (P16): ran the authors' own published tool, bioconda htseqqc v1.0 == HTSQualC filter.py, on the paper's own demo FASTQ (SRR1805340 cotton SE, SRR2165176 sugarcane PE) with documented parameters via SLURM «job» on «our HPC» n152 (COMPLETED, exit 0). All 4 in-scope pipeline-derived claims reproduce: C4 exact (82,059,811 reads); C3 within-tol (5,190,762 filtered vs ~5M); C1 within-tol (250,861 filtered vs ~250K); C2 within-tol (451,127 removed vs ~451K, plus 20,698/19,140 adapter-trimmed vs ~20K) — near-exact against the supplementary values. No fabrication indicators: every reported number is derivable from running the shipped public tool on the public data. NOT attempted (out of scope): runtime/memory benchmarks (hardware-dependent), HTSQualC-vs-other-tools comparison (claim about other tools), tomato GBS 322-sample scaling demo (no per-sample numeric cited), Figure-3 qualitative panels. Grades are provisional pending human sign-off (see AUDIT.md).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-18 ⛓ 69ba757fcb33
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-25
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetAn integrated, flexible, one-step quality control software (HTSQualC) can overcome the limitations of existing standalone HTS QC tools by combining filtering, trimming, statistics, and visualization in a single run while supporting large-scale parallel batch analysis.
- ★ HTSQualC is a standalone, one-step QC software that performs filtering and trimming of raw HTS data in a single run resource
- ★ HTSQualC supports parallel computing (shared and distributed modes) to enable batch analysis of large numbers of samples method
- ★ HTSQualC completed QC of 322 tomato GBS samples in ~3 h using distributed computing versus ~31 h in shared mode finding
- ★ HTSQualC offers more integrated features (uncalled base filtering, automatic quality variant detection, FASTA output, GZIP input, batch analysis) than FastQC, NGS QC, QC-Chain, FASTX-Toolkit, and NGS QCbox finding
- ★ HTSQualC is available as a command-line tool and as a GUI integrated into the CyVerse Discovery Environment for non-bioinformatician biologists resource
- HTSQualC runs slower than fastp and FastQC because it is implemented in Python 3 rather than C/C++ or Java finding
- HTSQualC and fastp differ in quality-filtering algorithms, producing different numbers of filtered reads on the same dataset mechanism
- HTSQualC does not include duplicate read removal because this feature is tied to gene quantification/expression estimation method
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Single-end RNA-seq QC benchmarking | Cotton (dicot plant), SRA SRR1805340 | none (default HTSQualC parameters) | number of reads filtered, sequence quality metrics | Illumina HiSeq 2000 |
| Paired-end RNA-seq QC benchmarking | Sugarcane (monocot plant), SRA SRR2165176/77/78 | default vs customized filtering parameters (adapter, quality threshold, uncalled bases) | number of reads filtered/trimmed | Illumina HiScanSQ |
| Paired-end genotyping-by-sequencing (GBS) batch QC | Tomato, 322 genotypes (low-coverage whole genome sequencing) | shared vs distributed parallel computing mode (18 CPUs) | total run time for batch QC analysis | Illumina HiSeq 4000 |
| Runtime and memory benchmarking of QC tools | Illumina HTS dataset (unspecified sample) | tool comparison: HTSQualC vs FastQC vs FASTX-Toolkit vs fastp, default settings | processing time (min) and memory consumption (MB/GB) | Texas A&M HPRC compute node (64 GB RAM, 20 CPUs, Intel 2.5 GHz IvyBridge) |
| Quality filtering output comparison | Illumina dataset (~25 M reads) | HTSQualC (Phred <20 threshold) vs fastp (Phred <15 threshold) default filtering | number of reads filtered out | — |
- ▼ HTSQualC filtered out ~5 M reads from an 82,059,811-read single-end cotton dataset ~5 M reads removed
- ▼ 322 tomato GBS datasets processed in ~157 min (~3 h) distributed mode vs ~1855 min (~31 h) shared mode ~11.8-fold faster in distributed mode
- ▼ Default HTSQualC run on sugarcane paired-end data filtered ~250 K reads; customized run filtered ~451 K reads and trimmed ~20 K reads ~250K vs ~451K reads filtered
- – HTSQualC processed ~9 M paired-end reads in ~9 min and ~82 M single-end reads in ~40 min using 18 CPUs
- – Runtime comparison for default quality filtering: fastp 2 min, FastQC 3 min, HTSQualC 24 min, FASTX-Toolkit 33 min HTSQualC 12x slower than fastp
- – Memory usage: fastp ~420 MB, HTSQualC ~818 MB, FastQC 1.55 GB HTSQualC ~2x fastp, ~0.5x FastQC
- – On a ~25 M read dataset, fastp filtered out ~115 K reads while HTSQualC filtered out 4256 reads with default settings ~27-fold difference between tools
- – HTSQualC provides additional output metrics (sequence quality format, min/max/mean read lengths, average Phred quality) not included in fastp output
- count 322 samples (Tomato GBS batch QC analysis)
- other ~1,000,000 × 2 reads per sample (Average paired-end sequence reads per tomato GBS sample)
- other ~157 min (distributed) vs ~1855 min (shared) (Run time for 322-sample batch analysis, 18 CPUs)
- count 82,059,811 reads (Total single-end reads in cotton SRR1805340 dataset)
- fold_change ~5 M reads filtered (Cotton single-end dataset default QC)
- count 4256 reads filtered (HTSQualC) vs ~115 K (fastp) (Comparison on ~25 M read Illumina dataset, default settings)
- other runtime: HTSQualC 24 min, fastp 2 min, FastQC 3 min, FASTX-Toolkit 33 min (Tool runtime benchmarking, default settings)
- other memory: HTSQualC ~818 MB, fastp ~420 MB, FastQC 1.55 GB (Tool memory consumption benchmarking)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software methods paper presenting HTSQualC, a quality-control tool for Illumina high-throughput sequencing data. The evaluation is entirely descriptive and benchmarking-based: processing times, read counts filtered or trimmed, and memory consumption are reported as single observed measurements across several case-study datasets. No inferential statistical tests were applied; results are summarised as raw counts, elapsed times, and a qualitative feature-comparison table.
-
Processing times for each tool/mode were reported from a single benchmark run↳ Could also: Repeated benchmark runs (e.g., 5–10 replicates) with mean and SD or 95% CI of wall-clock time — A single timing observation conflates run-to-run variability (I/O scheduling, cache state, node load) with true performance differences; replicate runs would allow reporting uncertainty and support more reliable comparisons between tools
-
The 322-sample dataset reported an average of ~1 M reads per sample without a measure of spread↳ Could also: Reporting the mean together with SD, range, or IQR of per-sample read counts — Variability in read depth across the 322 samples could influence aggregate run-time estimates; a spread measure would help readers assess how representative the average is
-
Tool comparisons (Table 2) used a binary Yes/No feature matrix↳ Could also: A quantitative filtering-accuracy benchmark using a dataset with known ground-truth reads (e.g., simulated reads with injected artifacts), reporting sensitivity and specificity or F1-score for each tool — Binary feature presence does not capture differences in filtering stringency or accuracy; quantitative accuracy metrics would complement the feature table and let users assess quality trade-offs
-
Shared-mode vs distributed-mode speed was compared using a single 322-sample run for each↳ Could also: A scaling experiment across several dataset sizes (e.g., 10, 50, 100, 322 samples) to characterise how run time scales with sample count in each mode — A scaling curve would clarify whether the speedup from distributed mode is consistent or depends on dataset size, giving users better guidance for resource planning
-
Memory consumption was reported as a single peak value per tool (e.g., HTSQualC ~818 MB, FastQC 1.55 GB)↳ Could also: Profiling memory over time during a run and reporting peak plus a time-series or boxplot across replicates — A single peak figure from one run may not reflect typical usage; time-series profiling would reveal whether the peak is transient or sustained, which matters for node allocation decisions
-
The comparison of filtered read counts between HTSQualC and fastp was conducted on one dataset (~25 M reads) with default settings↳ Could also: Evaluating concordance across multiple datasets of varying quality profiles (e.g., Bland-Altman analysis or Pearson/Spearman correlation of per-sample filtered-read proportions) — A single-dataset comparison does not reveal whether the difference in filtering behavior is consistent across datasets; multi-dataset concordance analysis would help users understand when the two tools agree or diverge
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-34548573 (HTSQualC)
Paper: Bedre R, Avila C, Mandadi K. HTSQualC is a flexible and one-step quality control software for high-throughput sequencing data analysis. Sci Rep 2021. PMID 34548573 · PMCID PMC8455540 · DOI 10.1038/s41598-021-98124-3.
Tool / repo: https://github.com/reneshbedre/HTSQualC (entry point
ngsmodules/filter.py). Published as bioconda package htseqqc v1.0
(noarch, pure-Python, 2021-01-21). Pinned repo commit
27f19911edaac4fce6d1059ec78082aacca16913 (2021-07-21).
Nature of paper: This is a SOFTWARE/TOOL paper. The reproducible,
pipeline-derived results are the read-filtering / read-trimming counts produced
by running HTSQualC's filter.py on the demonstration FASTQ data with the
documented parameters. This is a P16 case (run the authors' own published tool on
the paper's own data per the described parameters).
Demonstration datasets
- SRR1805340 — Gossypium hirsutum (cotton), single-end Illumina HiSeq 2000. BioProject PRJNA275482. Paper Table 1: 82,059,811 total reads.
- SRR2165176 — Saccharum (sugarcane), paired-end. Used for default vs custom-parameter filtering demonstration (Supplementary File 1B / 1C). (SRR2165177/78 are additional sugarcane runs, not needed for the cited counts.)
IN SCOPE — pipeline-derived results we attempt to reproduce
| id | reported result | paper location | command |
|---|---|---|---|
| C4 | 82,059,811 total single-end reads (SRR1805340) | Table 1 / Methods | read count of downloaded FASTQ |
| C3 | ~5 M reads filtered out (SRR1805340, default qthr 20) | Fig 3 / Suppl File 1A | filter.py --p1 SRR1805340.fastq |
| C1 | ~250 K reads filtered out (SRR2165176, default qthr 20) | Suppl File 1B | filter.py --p1 SRR2165176_1 --p2 SRR2165176_2 |
| C2 | ~451 K reads filtered + ~20 K reads trimmed (SRR2165176, qthr 25, nb 5, adapters) | Suppl File 1C | filter.py --qthr 25 --nb 5 --adp AGATCGGAAGAGCACACGTCTGAACTCCAGTCA,AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT --p1 ... --p2 ... |
Note: reported filtering counts are stated as approximate ("~250 K", "~451 K", "~20 K", "~5 M") in the main text; exact per-metric values live in Supplementary File 1 (XLSX). We grade against the printed approximate values (within-tol).
OUT OF SCOPE (not attempted — not faithfully reproducible)
- Runtime / wall-clock and peak-memory benchmarks (Table 1): hardware- and load-dependent; not comparable across our SLURM nodes vs the authors' machine.
- Tool-vs-tool comparison (HTSQualC vs FastQC/Trimmomatic/fastp): a benchmarking claim about other tools, not a result of the HTSQualC pipeline itself.
- Tomato GBS 322-sample / 206 GB scaling demo: a throughput/scaling demonstration, not a specific reproducible numeric result (no per-sample counts cited).
- Figure-3 visual panels (quality/GC/composition plots): qualitative, regenerated as a by-product but not numerically graded.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.