RAGER: A user-friendly computational platform for integrated analysis of RNA-Seq and ATAC-seq data.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
RAGER is described well enough to reproduce: a Snakemake wrapper chaining standard tools with explicit fixed parameters on public data (GSE85632, mouse RNA+ATAC; ENA PRJNA338980/PRJNA338975, 8 runs mapping cleanly to GFPpos=activated/GFPneg=control). Reproduced via P16 (run the named tools with the paper's exact params on the paper's data, mm10=GENCODE GRCm38/M25) on «our HPC» SLURM. OUTCOME = PARTIAL but with a STRONG core: C1 RNA-seq alignment rates reproduce tightly (UniqueAR 64-85% vs 60-84; MultiAR 11.27-29.29% vs 11.3-29.3 = near-exact; UnAR <=6.5% vs <6.5) -> within-tol. C2 ATAC fragment size reproduces qualitatively (canonical ATAC insert profile, 43% <=150bp, but modal peak ~84bp below the stated 100-150 band) -> partial. C3 ATAC peak annotation: the genomic-distribution SHAPE and the up->down promoter/intron shift reproduce, but absolute differential-peak counts are ~3x high because the paper's statistical differential-binding threshold is underspecified (we used presence/absence intersect of merged replicate MACS2 peaks) -> partial. NOT attempted (the hard ~20%, by design): C4 joint co-regulated genes (746/707), C5 correlation r=0.674, C6 TF/motif ID (KLF17, SNAI1) - need EnhancerAtlas enhancer-gene linking + genekitr + root MEME-AME + several underspecified thresholds. Two env gotchas fixed and carded: (1) compute-node read-only $HOME + inherited-invalid $TMPDIR; (2) MACS2 2.2.9.1 conda py311 build fails with undefined symbol __log_finite (old-glibc compile) -> pip-recompile MACS2 on the node. FABRICATION NOTE for the reviewer: Fig 2F p<1e-500 is below the double-precision floor and cannot be printed by a standard correlation test.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-16 ⛓ 99681b0068d8
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-24
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper addresses the problem that integrated analysis of RNA-seq and ATAC-seq data is hindered by low throughput, time-consuming manual processing, and reproducibility issues, and proposes that an automated, user-friendly computational platform (RAGER) can overcome these barriers to enable joint transcriptome-chromatin accessibility analysis and novel biological discovery.
- ★ RAGER integrates widely-used bioinformatics tools into an automated Snakemake-based pipeline for joint analysis of RNA-seq and ATAC-seq data resource
- ★ RAGER reduces processing time and minimizes the need for bioinformatics expertise compared to manual integration of RNA-seq/ATAC-seq analysis finding
- ★ RAGER identifies high-confidence regulatory gene circuits by intersecting differentially expressed genes with genes linked to differential chromatin accessibility at promoters and enhancers method
- ★ RAGER was demonstrated using RNA-seq/ATAC-seq datasets from murine embryonic stem cells (GSE85632) and human CD34+ cells (GSE261119) finding
- ★ RAGER constructs TF-gene and TF-enhancer regulatory networks using motif enrichment analysis (MEME AME) and Cytoscape visualization method
- RAGER includes an optional module to assemble and annotate unaligned RNA-seq and ATAC-seq reads via Trinity and BLAST, extending applicability to species with incomplete reference genomes method
- A Streamlit-based local GUI provides four ordered modules (Preprocess RNAseq, Preprocess ATACseq, Joint analysis, Custom analysis) for users without command-line expertise resource
- RAGER is extensible to user-provided gene sets and to plant species with complex genomes resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| RNA-seq | murine embryonic stem cells (GSE85632) | not stated | differentially expressed genes (FDR<0.05, |log2FC|>threshold) | HISAT2/StringTie/DESeq2 |
| ATAC-seq | murine embryonic stem cells (GSE85632) | not stated | differential chromatin accessibility peaks (q-value<0.05) at promoters/enhancers | Bowtie2/MACS2/ChIPseeker |
| RNA-seq | human CD34+ cells (GSE261119) | not stated | differentially expressed genes (FDR<0.05, |log2FC|>threshold) | HISAT2/StringTie/DESeq2 |
| ATAC-seq | human CD34+ cells (GSE261119) | not stated | differential chromatin accessibility peaks (q-value<0.05) at promoters/enhancers | Bowtie2/MACS2/ChIPseeker |
| GSEA (functional enrichment) | shared DEGs/accessible-chromatin genes from RNA-seq+ATAC-seq datasets | none | KEGG/GO pathway enrichment (|NES|>1, p<0.05) | clusterProfiler v4.14.0 |
| TF motif enrichment analysis | promoter sequences of shared genes and shared enhancer sequences | none | enriched TF binding motifs (FDR<0.05) | MEME Suite AME v5.5.7 with JASPAR2024 CORE database |
| De novo transcriptome assembly of unaligned reads | unaligned RNA-seq reads | none | assembled contigs annotated by sequence similarity | Trinity v2.15.2 / BLAST v2.16.0 |
| Realignment/classification of unaligned reads | unaligned ATAC-seq reads | none | read classification against organelle and repetitive/TE reference sequences | Bowtie2 (single-end mode) |
- other FDR < 0.05, |log2FC| > threshold (DESeq2 significance threshold for RNA-seq DEGs)
- pvalue q-value < 0.05 (MACS2 significance threshold for ATAC-seq peak calling)
- fold_change |log2FC| > 1 (threshold for differentially accessible enhancers (co-up/co-down classification))
- other normalized enrichment score |NES| > 1 and pvalue < 0.05 (GSEA significance criteria via clusterProfiler)
- pvalue FDR < 0.05 (AME motif enrichment significance (Fisher's exact test with Benjamini-Hochberg correction))
- other >70% mapped reads (quality control threshold for RNA-seq/ATAC-seq read alignment assessment)
- count 1,000 phenotype permutations (GSEA permutation testing parameter)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software/methods paper presenting RAGER, a Snakemake-based pipeline integrating RNA-seq and ATAC-seq analysis. The core statistical operations are: DESeq2 Wald testing for differential gene expression (FDR < 0.05, |log2FC| > user-defined threshold), MACS2 peak calling for chromatin accessibility (q-value < 0.05, |log2FC| > 1 for enhancers), GSEA for pathway enrichment, and Fisher's exact tests with Benjamini-Hochberg correction for transcription factor motif enrichment. The pipeline's utility is demonstrated on two publicly available datasets (GSE85632, GSE261119); results are presented as pipeline-generated figures rather than primary statistical tables.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| DESeq2 Wald test (negative binomial generalized linear model) | RNA-seq differential gene expression analysis across both demonstration datasets | — | not stated |
| MACS2 peak calling with q-value threshold; log2FC thresholding for differential accessibility | ATAC-seq differential chromatin accessibility at promoter and enhancer regions | — | not stated |
| GSEA with weighted scoring and phenotype permutation (1000 permutations) | Pathway enrichment analysis against KEGG and GO gene sets for differentially regulated genes | 1000 permutations stated | not stated |
| Fisher's exact test | Transcription factor binding motif enrichment via MEME-AME against JASPAR2024 CORE vertebrate database | — | not stated |
| Principal Component Analysis (PCA) | Quality control and reproducibility assessment of RNA-seq and ATAC-seq samples | — | na |
-
DESeq2 was chosen for RNA-seq differential expression analysis↳ Could also: edgeR (exactTest or glmQLFit) or limma-voom could also have been applied to the same read-count matrices — edgeR and limma-voom are established alternatives that differ in how they estimate dispersion and handle small sample sizes; offering multiple DE callers as configurable options would allow users to assess concordance and robustness of findings across methods
-
Differential ATAC-seq accessibility was defined by MACS2 peak calls filtered by a log2FC threshold (|log2FC| > 1) rather than a formal per-peak statistical test↳ Could also: DiffBind or DESeq2 applied to a consensus peak count matrix could also quantify differential accessibility with per-peak p-values and FDR estimates — Formal statistical testing on peak count matrices provides calibrated error rates across the peak set, complementing the fold-change-based filter and making the threshold choice more transparent
-
GSEA significance was assessed with p-value < 0.05 and |NES| > 1 without an explicit FDR correction across the full set of gene sets tested↳ Could also: Reporting adjusted p-values (e.g., Benjamini-Hochberg FDR on GSEA results) or complementing GSEA with over-representation analysis (ORA) using a hypergeometric test could also have been applied to the same gene lists — When many gene sets are tested simultaneously, an FDR adjustment reduces false positives; ORA additionally provides a membership-based view that is complementary to GSEA's rank-based approach
-
A fixed absolute log2FC threshold (user-defined for DEGs; |log2FC| > 1 for enhancers) was applied as the primary effect-size filter after DESeq2 testing↳ Could also: DESeq2's lfcShrink function (apeglm or ashr shrinkage estimator) could also have been used to stabilize log2FC estimates before thresholding — Shrinkage estimation reduces the instability of fold-change estimates for lowly expressed or low-count features, which can make the resulting gene list less sensitive to the chosen threshold, particularly in small-n experiments
-
PCA was used as the primary approach for assessing sample-level reproducibility and batch structure in both RNA-seq and ATAC-seq QC↳ Could also: A Spearman or Pearson correlation heatmap of sample-to-sample distances could also have been included alongside PCA — PCA captures dominant axes of global variance, while a pairwise distance heatmap reveals finer grouping structure; presenting both together is a common QC convention that helps identify outlier samples from complementary perspectives
-
Enriched TF expression was visualized as a heatmap of expression values without a stated measure of within-group spread↳ Could also: Reporting SD, SEM, or a 95% CI alongside group mean expression, or using a dot plot encoding both mean and fraction expressing, could also represent the same data — Displaying a measure of spread alongside the central tendency allows readers to judge biological variability across replicates, which is especially informative when sample sizes are small
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-42172220 (RAGER)
Paper: RAGER: A user-friendly computational platform for integrated analysis of RNA-Seq and ATAC-seq data. Liu et al., PLoS One 2026. PMID 42172220 / PMC13196991. Code: https://github.com/bioinfo202408/RAGER (Snakemake wrapper around standard tools). Data: GEO GSE85632 (mouse mESC, DUX4/Dux activation). ENA studies PRJNA338980 (RNA-seq), PRJNA338975 (ATAC-seq).
What RAGER is
A Snakemake pipeline that chains popular tools with fixed parameters:
- RNA-seq: Trim Galore
--phred33 -q 25 --length 35 --stringency 3→ HISAT2--dta→ StringTie quant → DESeq2 DE (FDR<0.05). - ATAC-seq: Bowtie2
-t -q -N 1 -L 25 --no-mixed --no-discordant→ Picard MarkDuplicates → MACS2--keep-dup all -q 0.05→ ChIPseeker annotation. - Joint: intersect RNA up-DEGs with genes near ATAC up-peaks; EnhancerAtlas 1MB windows; GSEA (clusterProfiler |NES|>1 p<0.05); MEME-AME motif (FDR<0.05); Cytoscape.
Sample map (GSE85632, the validation dataset in Fig 2)
RNA-seq (PRJNA338980, paired): SRR4032350 GFPpos-rep1, SRR4032351 GFPpos-rep2 (=RNA_A, DUX-activated); SRR4032352 GFPneg-rep1, SRR4032353 GFPneg-rep2 (=RNA_N, control). ATAC-seq (PRJNA338975, paired): SRR4032269/70 GFPpos-ATAC rep1/2; SRR4032271/72 GFPneg-ATAC rep1/2.
In scope (pipeline-derived, clearly specified — attempt)
- C1 RNA-seq alignment rates (Results: UniqueAR 60–84%, MultiAR 11.3–29.3%, UnAR <6.5%). Pipeline: Trim Galore + HISAT2 to mm10. Primary target — lowest-hanging.
- C2 ATAC fragment-size distribution ~100–150 bp for SRR4032269 (Fig 2C). Bowtie2 + samtools.
- C3 ATAC peak annotation distribution (Fig 2D up-peaks: 3511 intergenic, 2549 intronic, 910 promoter; down: 4242 intergenic, 4792 intronic). MACS2 + ChIPseeker. Heavier.
Out of scope / hard-20% (not attempted, with reason)
- 746/707 co-up/down-regulated genes (Fig 2E) and r=0.674 (Fig 2F): require the full joint analysis — DESeq2 on RNA + differential ATAC peaks + promoter-peak linkage + EnhancerAtlas enhancer-gene mapping. Many under-specified params (which |log2FC| threshold for GSE85632, how "promoter-open" is defined, enhancer linkage). Skipped as the fragile last 20%.
- TF motifs (KLF17, SNAI1): need MEME-AME (root install) + EnhancerAtlas. Skipped.
- Second dataset GSE261119 and unmapped-reads module (GSE214739): out of scope (one dataset suffices).
Approach
Run the named tools with the paper's exact parameters directly (brief P16 allows third-party / standard-pipeline reproduction) rather than installing the full fragile RAGER stack (genekitr, MEME-AME root, EnhancerAtlas). Same tools, same params, same data, on «our HPC» SLURM.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Input data (GSE85632) is fully public and 1:1 reproducible, and the QC endpoints (alignment rates, fragment size, peak counts) are directly comparable — but at the operator's early-finalize both jobs were still on sample 1 of 4, so no reproduced number was emitted for any in-scope claim (C1-C3). The harder integrative core (r=0.674, 746/707 co-regulated genes, TF validation) was deliberately not attempted, so the paper's central conclusion is untested rather than confirmed. The shortfall is overwhelmingly on our side (incomplete run + scoped-out joint analysis), not an authors' defect — with one genuine authors-side flag: the Fig 2F p<1e-500 is below the double-precision floor and cannot be printed directly from a standard correlation test. Overall this is an inconclusive/incomplete reproduction (yellow), not a substantiated discrepancy.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.