HTSstation: a web application and open-access libraries for high-throughput sequencing data analysis.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- 🔴Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
GSE31570 (11 GEO samples: 6 expression microarray + 5 ChIP-seq/input) was fully profiled and its 5 in-scope HTS samples were downloaded from ENA and pushed through a modern equivalent of HTSstation's mapseq+chipseq pipeline (bowtie2 alignment, 85-96% alignment rate; MACS3 peak calling against a shared anterior-trunk input, since the original bioconda MACS2 build was broken on this cluster). The paper itself states no dataset-specific quantitative results for GSE31570 (it is a software-methods paper; the only checkable claim is the qualitative method description 'peak detection ... by running the MACS software' plus specific density-smoothing parameters we did not reimplement), so grading is necessarily qualitative: the reproduced ChIP-seq peaks show the expected bimodal active(H3K4me3)/repressive(H3K27me3) domain structure over the HoxA/HoxD clusters, and H3K4me3 signal is markedly denser in anterior trunk than forebrain, matching known posterior-specific Hox activation. NOT attempted: the original bbcflib Python2 code path itself, the paper's exact 100bp/75bp/29-window smoothing parameters, the microarray expression data, and the paper's other (out-of-scope) examples using different accessions (Kap1 RNA-seq, M. leprae SNPs).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-07-31
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-07-31no human curator yet
- Last updated
- 2026-07-31
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opus- ★ HTSstation is a web application suite coupling simple web forms to modular analysis pipelines for ChIP-seq, RNA-seq, 4C-seq and re-sequencing HTS applications, accessible at http://htsstation.epfl.ch. resource
- ★ Open-source components (bbcfutils programs and bbcflib libraries on GitHub) let developers build, configure, run and extend HTS analysis workflows and integrate third-party software via system calls. resource
- ★ The platform accommodates three user levels: web forms for biologists, per-module command-line Python programs for bioinformaticians, and downloadable libraries for programmers. method
- ★ Job definitions encode the experimental design (conditions and replicates), with result files named/displayed by the design matrix and groups joinable or splittable at any workflow step, minimizing sample mix-up. method
- ★ Workflows revolve around a Bowtie (Bowtie2 since September 2013) mapping module that computes genome-wide coverage profiles, optionally preceded by demultiplexing and followed by application-specific modules. method
- ★ Reproducibility and sharing are supported by job cloning, unique random execution keys transferring results between pipelines, obfuscated result URLs, and the Bein workflow manager storing execution parameters and file links in a database. method
- The client-server back-end is adapted to HPC with built-in task parallelization; large files are pulled by the server from public URLs rather than uploaded through the web interface. method
- Supporting applications are provided independently: GenRep for versioned genome references, GDV genome browser for visualization, and the Bioscript collection of statistical/data-mining algorithms. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| RNA-seq (mapping + differential isoform expression) | Mouse embryonic stem (ES) cells, wild-type and Kap1 knockout; mm9 genome | Kap1 (Trim28) knockout vs wild-type | Average read coverage per Ensembl exon, inferred transcript expression, differential expression (MA-plot) | Illumina GAII; Bowtie mapping; R package DESeq; Ensembl annotation |
| RNA-seq mapped to repetitive-element database | Mouse ES cells (same WT/Kap1-KO reads) mapped to Repbase repetitive DNA elements, rodent section version 16.08 | Kap1 knockout vs wild-type | Differential expression of repeat/ERV elements and read coverage profiles along element sequences | Repbase v16.08 (rodent); HTSstation mapping and post-processing plotting functions |
| Re-sequencing / SNP calling | Mycobacterium leprae, 5 complete strain genomes mapped to reference TN strain (RefSeq NC_002677.1); >400 genotyped samples | none (natural strain variation) | High-confidence SNP positions with genomic-context annotation; coverage depth | Illumina GAII reads from SRA URLs; SAMtools |
| 4C-seq (Circular Chromosome Conformation Capture with sequencing) | Mouse embryonic tissues: forebrain (FB) and anterior trunk (AT); Hoxd13 and Hoxd4 viewpoints; mm9 genome | none (tissue comparison / developmental state) | Normalized read counts per NlaIII restriction fragment, profile-corrected smoothed density (moving average of 29 fragments), domainogram-identified long-range interacting regions | Bowtie mapping; viewpoint-specific primer demultiplexing; pre-computed NlaIII restriction-fragment library; domainogram algorithm |
| ChIP-seq (histone modifications) with peak calling | Mouse forebrain and anterior trunk embryonic tissues; HoxD cluster and chromosome 2 | none | Read densities (reads extended to 100 bp, strand densities shifted 75 bp), PCR-duplicate-removed coverage, H3K4me3 peaks, heatmaps and H3K4me3-vs-H3K27me3 scatter plots over 2 kb intervals around gene starts | MACS peak caller; GDV genome viewer |
- ▲ Alox15, Col4a1 and Col4a2 are over-expressed in Kap1 KO ES cells relative to wild-type
- ▼ Kap1/Trim28 itself is the most down-regulated gene on the MA-plot, consistent with it being the knocked-out gene
- – Mapping to Repbase revealed induction of ERV elements in the KO condition: RLT1IAP highly over-expressed, MERVL slightly over-expressed, SINEB1 under-expressed
- – SNP analysis of 5 M. leprae strains against the TN reference yielded a list of 172 SNP positions including 64 non-synonymous coding variants 172 SNPs; 64 non-synonymous
- – 4C-seq profile correction has a localized effect and does not affect identification of potential interacting partners near the viewpoint
- – Combined 4C-seq and ChIP-seq views show distinct active and inactive chromatin compartments in the HoxD cluster with a boundary around Hoxd8, which shifts from inactive to active in anterior trunk
- – Heatmaps of ChIP-seq signal along HoxD genes show a clear division between inactive and active sections, split at Hoxd9
- – Scatter plot of H3K4me3 vs H3K27me3 across chromosome 2 gene starts shows two roughly equal populations of active and inactive genes plus a small number of H3K27me3-positive repressed genes, with HoxD genes spread across compartments and Hoxd9 intermediate
- count 47 M Illumina GAII sequencing reads in three FASTQ files (Kap1 RNA-seq: two KO replicates and one WT)
- count 172 SNP positions, including 64 non-synonymous coding variants (M. leprae re-sequencing of 5 strains vs TN reference)
- other average coverage between 0.3× and 2× (Mapping of 5 M. leprae strain read sets to reference genome)
- count more than 400 samples genotyped, including 5 complete genomes (M. leprae phylogeography study scope)
- other seed of 28 bp with 2 mismatches and a maximum of 5 hits per read (Default mapping module options used for Kap1 RNA-seq)
- other moving average of 29 fragments per window (4C-seq smoothing of normalized read counts per restriction fragment)
- other reads extended to 100 bp; strand-specific densities shifted downstream by 75 bp (ChIP-seq density computation relative to average sonication fragment size)
- count in excess of 500 jobs per year with an average of 15 distinct users per month (HTSstation portal usage over its two-year lifetime)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper is a software/methods description (HTSstation, a web platform for HTS data analysis) illustrated through several previously published biological use cases rather than a single hypothesis-driven statistical study. Differential expression between Kap1 knockout and wild-type mouse ES cells (RNA-seq, a small number of biological replicates) was assessed with the R package DESeq and displayed as an MA-plot highlighting specific genes and repetitive elements. SNP calling from Mycobacterium leprae resequencing data used SAMtools with an unspecified statistical-support filter to remove false positives, ChIP-seq peaks for H3K4me3 were detected with MACS, and 4C-seq long-range chromatin interactions were identified with a custom domainogram algorithm. Throughout, results are presented mainly as graphical summaries (MA-plots, heatmaps, scatter plots, coverage profiles) rather than as tables of test statistics, p-values, or effect-size estimates.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| DESeq (negative-binomial-based differential expression test) | Differential isoform/gene expression between Kap1 KO and WT ES cell RNA-seq (MA-plot, Figure 5A) | 2 KO replicates vs 1 WT replicate (stated) | not stated |
| DESeq (same method applied to repetitive-element read counts) | Differential expression of Repbase repetitive elements between KO and WT (Figure 5B) | same reads as above (2 KO vs 1 WT) | not stated |
| SAMtools-based genotype/quality filtering ('statistical support' for each polymorphism) | SNP calling across 5 M. leprae genomes vs reference strain TN | 5 genomes, 172 SNP positions reported | not stated |
| MACS peak calling | H3K4me3 ChIP-seq peak detection in forebrain and anterior trunk tissue (Figure 6B) | not stated (per-tissue sample, replicate number not given) | not stated |
| Domainogram algorithm | Identification of long-range interacting regions from 4C-seq profiles (Hoxd13/Hoxd4 viewpoints, Figure 6A) | not stated | not stated |
-
Differential expression was assessed using the DESeq package.↳ Could also: DESeq2 or edgeR — These later tools add shrinkage-based dispersion and fold-change estimation, which can also be used to improve stability of variance estimates when replicate numbers are small.
-
The RNA-seq comparison used two KO replicates and a single WT replicate.↳ Could also: Additional biological replicates per condition, or a paired/blocked design if feasible — More replicates per group would also allow variance to be estimated directly from each condition rather than relying on limited within-group information.
-
SNPs were filtered using SAMtools-based 'statistical support' without a named significance threshold in the text.↳ Could also: An explicit quality/probability cutoff or an FDR-controlled filtering step, e.g. via bcftools call or GATK — Stating a specific threshold or applying multiplicity control across all candidate positions would also make the confidence criteria for retained SNPs explicit to readers.
-
Genome-wide differential expression results are shown via an MA-plot without a stated p-value or fold-change cutoff for 'significant' genes.↳ Could also: Reporting adjusted p-values and a fold-change threshold alongside the plot (e.g., a volcano plot or a supplementary results table) — This would also let readers see the exact numerical criteria used to call a gene differentially expressed, complementing the visual MA-plot summary.
-
H3K4me3 ChIP-seq peaks were called with MACS in each tissue separately.↳ Could also: A differential ChIP-seq peak caller (e.g., diffReps, DiffBind) or a broad-peak caller (e.g., SICER) for marks like H3K27me3 — These tools could also directly quantify statistical differences in signal between tissues/conditions rather than comparing peak calls made independently per sample.
-
Long-range 4C-seq interactions were identified using an in-house domainogram algorithm.↳ Could also: Established 4C-seq analysis packages with built-in significance testing (e.g., fourSig, FourCSeq) — Such packages could also provide a formal statistical significance measure (e.g., an FDR-controlled score) for called interaction domains.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Data identity is unproblematic — GSE31570 is fully open and all 5 in-scope HTS samples (SRR333824-SRR333828) were pulled 1:1 from ENA with 85.13-96.05% alignment to mm9. The structural problem is that HTSstation is a software paper: it reports no dataset-specific quantitative value for GSE31570, so there is no reported endpoint to place our 31902/31674 H3K4me3 peak counts against (q2 red — an artefact of the paper type, not an authors' defect). The deviations that do exist are on our side: the Python2 bbcflib mapseq/chipseq code path was never executed (substituted with bowtie2 2.5.4 + MACS3 3.0.4 after bioconda MACS2 failed with ImportError: undefined symbol __log_finite), and the one precisely parameterised claim — the 100 bp extension / 75 bp shift / 29-fragment moving-average density track — was not attempted at all (source_file: none). What was reproducible reproduced well: 81 H3K4me3 peaks over HoxA+HoxD in anterior trunk vs 11 in forebrain, with H3K27me3 broad domains up to ~38 kb retained in both tissues, matching the described bimodal collinear Hox architecture. Overall: a solid, honestly disclosed partial reproduction with explainable, self-inflicted gaps — yellow, not red.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.