Fluent genomics with plyranges and tximeta.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓The central claim held under reproduction
- 🟡A deviation arose in the data or preprocessing
- 🟡The deviation was non-trivial in magnitude
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce 1:1 — F1000Research workflow paper (fluentGenomics), a fully reproducible Bioconductor vignette with all data shipped (RNA-seq salmon quants in the macrophage package; DA ATAC peaks pre-shipped as peaks.rda in the repo). Ran the full deterministic import->model->integrate pipeline on «our HPC» (tximeta + DESeq2 + plyranges, set.seed(2019-08-02) bootstrap). Result is PARTIAL but faithful: genes-tested count is EXACT (17806/17806); the paper's HEADLINE result — relative enrichment of DA peaks near DE vs non-DE genes — reproduces within ~5% (1.42->1.43, 2.90->2.97, 6.89->7.20), preserving the exact conclusion. All absolute counts run ~9% lower because a 4-years-newer DESeq2 (1.42 vs paper's 1.26) calls fewer DE genes (749 vs 822); this single factor propagates proportionally and the discrepancy is fully internally consistent. No fabrication signal: every reported number is regenerable from the shipped artifacts. NOT attempted (optional last 20%): re-deriving the DA peaks from the raw Zenodo ATAC matrix via limma (pre-shipped), and pinning the exact 2020 Bioc-3.10 stack to chase byte-exact counts.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 69assessed: 2026-06-15 ⛓ e0a7b96664ec
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusUsing published RNA-seq and ATAC-seq data from macrophage cell lines, the authors test whether differentially accessible (DA) ATAC-seq chromatin peaks are enriched in the vicinity of differentially expressed (DE) genes in response to IFNg stimulation, demonstrating a fluent import-model-integrate workflow.
- ★ A simple three-step workflow (import, model, integrate) using R/Bioconductor enables fluent, reproducible genomics data analysis integrating RNA-seq and ATAC-seq. method
- ★ tximeta imports RNA-seq transcript quantifications into a SummarizedExperiment with transcript ranges and provenance metadata automatically attached. method
- ★ plyranges fluently integrates assay results by genomic coordinates via overlap detection and aggregation to assess enrichment of DA peaks near DE genes. method
- ★ DESeq2 and limma are used to model differentially expressed genes and differentially accessible peaks within the SummarizedExperiment framework. method
- IFNg stimulation produces differentially expressed genes detectable at LFC>1 and 1% FDR. finding
- The fluentGenomics and macrophage packages provide the data resources for the workflow. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| bulk RNA-seq (transcript quantification, gene-level differential expression) | human iPSC-derived macrophage cell lines (24 RNA-seq samples) | IFNg stimulation vs naive | gene-level counts / log2 fold change of expression | Salmon quantification, Gencode 29 reference; imported via tximeta; modeled with DESeq2 |
| ATAC-seq (chromatin accessibility, differential abundance) | human iPSC-derived macrophage cell lines (145 ATAC-seq samples) | IFNg stimulation vs naive | cqn-normalized log2 read counts per peak / differential accessibility | cqn normalization; SummarizedExperiment; modeled with limma |
- ▲ Genes upregulated by IFNg stimulation (LFC > 1.00, adjusted p < 0.01) 502 genes (2.8%)
- ▼ Genes downregulated by IFNg stimulation (LFC < -1.00, adjusted p < 0.01) 247 genes (1.4%)
- – tximeta automatically attached transcript ranges and identified reference genome (hg38 / GENCODE release 29)
- – RNA-seq quantifications imported into RangedSummarizedExperiment of 205870 transcripts x 24 samples 205870 x 24
- count 502 (2.8%) up (DE genes LFC>1.00, adjusted p<0.01, IFNg vs naive)
- count 247 (1.4%) down (DE genes LFC<-1.00, adjusted p<0.01, IFNg vs naive)
- count 17806 (genes with nonzero total read count tested by DESeq2)
- other FDR alpha = 0.01, lfcThreshold = 1 (DESeq2 results thresholds for DE testing)
- count 24 (RNA-seq samples in macrophage package)
- count 145 (ATAC-seq samples across all conditions)
- count 205870 (transcripts in imported SummarizedExperiment)
- other at least 10 counts in at least 6 samples (gene filtering threshold before DESeq2)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This methods workflow article demonstrates a bioinformatics pipeline integrating RNA-seq and ATAC-seq data from macrophage iPSC lines stimulated with IFNg versus naive controls. RNA-seq quantifications were imported via tximeta, summarized to gene level, and tested for differential expression using DESeq2 with a log2 fold-change (LFC) threshold of 1 incorporated into the Wald test null hypothesis and a 1% FDR threshold. Pre-normalized ATAC-seq peaks (cqn-normalized by the source study) were modeled for differential accessibility using limma (referenced in the workflow overview figure; full code section truncated in the provided text). Results from both assays were integrated using plyranges overlap and aggregation operations to assess enrichment of differentially accessible peaks near differentially expressed genes.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| DESeq2 Wald test (null hypothesis: |LFC| ≤ 1; Benjamini-Hochberg adjusted p-values) | RNA-seq differential gene expression, IFNg vs. naive macrophages | 24 samples (6 donors × 4 conditions); 17806 genes with nonzero counts after low-count filtering (≥10 counts in ≥6 samples) | not stated |
| limma (exact variant not stated; full ATAC analysis code section truncated in provided text) | ATAC-seq differential chromatin accessibility, IFNg vs. naive (referenced in Figure 1 caption) | 145 ATAC-seq samples across all conditions; number of peaks not stated in provided text | not stated |
-
RNA-seq differential expression was performed with DESeq2 using a Wald test against a null hypothesis of |LFC| ≤ 1↳ Could also: edgeR (quasi-likelihood F-test or exact test) or limma-voom could also be applied to RNA-seq count data for differential expression — All three are well-established methods for count-based RNA-seq DE; limma-voom may perform comparably or better at larger sample sizes, while edgeR's quasi-likelihood framework explicitly models extra-Poisson dispersion; the choice among them is a common, well-documented analytical decision with broadly similar results
-
The LFC threshold of 1 was incorporated directly into DESeq2's Wald test as the null hypothesis (testing whether |LFC| > 1)↳ Could also: Posterior LFC shrinkage estimators such as apeglm or ashr (via DESeq2's lfcShrink) could also be applied to stabilize LFC estimates for low-count genes — Shrinkage reduces noise in LFC estimates for genes with few counts, which is useful for ranking genes or for visualization; the threshold-based Wald test and shrinkage address related but distinct goals and can be used together
-
Enrichment of DA peaks near DE genes was assessed descriptively by counting overlaps and aggregating over LFC thresholds using plyranges↳ Could also: A permutation-based enrichment test or a hypergeometric/Fisher's exact test (controlling for genomic features such as gene length or expression level) could also be used to assign a p-value to the observed enrichment — Formal enrichment tests yield a statistical measure of whether co-occurrence exceeds chance expectation after accounting for potential confounders; the overlap-and-aggregate approach used here is well-suited for exploratory visualization of the trend across LFC thresholds
-
ATAC-seq count data arrived pre-normalized using cqn (conditional quantile normalization, correcting for GC content and peak length) from the source study↳ Could also: DESeq2 or edgeR could also be applied directly to raw ATAC-seq read counts using their own normalization (size-factor or TMM), treating ATAC-seq peaks analogously to RNA-seq genes — Applying count-based DE tools directly to ATAC-seq counts is an established approach; cqn's GC-content and peak-length correction addresses ATAC-specific systematic biases, making it a common choice for that assay type
-
Donor (cell line) was modeled as a fixed-effect covariate in the DESeq2 design formula (~line + condition)↳ Could also: A mixed-effects model such as dream (from the variancePartition package) could also model donor as a random effect — Mixed-effects models may improve power when the number of donors is larger and can better partition inter-individual variance; fixed-effect blocking as used here is a simpler and well-established approach that is appropriate for the sample size and design
-
Results were summarized by the count and proportion of significantly DE genes at the chosen FDR and LFC thresholds, alongside an MA plot↳ Could also: Reporting shrinkage-based LFC estimates with 95% confidence intervals (e.g., from lfcShrink) could also accompany count summaries — Confidence intervals on LFC estimates convey the precision of effect size estimates alongside significance, which helps distinguish large-effect findings from borderline ones and supports downstream interpretation
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-32528659 "Fluent genomics with plyranges and tximeta"
F1000Research workflow/tutorial article (Lee, Lawrence, Love 2020). The entire paper IS a reproducible Bioconductor vignette; there is no wet-lab component. The analysis re-uses published RNA-seq + ATAC-seq from Alasoo et al. 2018 (macrophage cell lines, IFNg vs naive).
In scope (pipeline-derived, attempted)
Three-stage workflow, all deterministic:
- Import —
tximetareads salmon RNA-seq quants (24 samples) → SummarizedExperiment →summarizeToGene. - Model —
DESeq2DE genes (IFNg vs naive, lfcThreshold=1, FDR 1%);limmaDA peaks. - Integrate —
plyrangesoverlap join of DE genes (±10kb of TSS) with DA peaks; bootstrap of non-DE genes (set.seed(2019-08-02), 10 resamples); enrichment of DA peaks near DE vs non-DE genes across LFC thresholds.
Reproduced numeric claims: DESeq2 summary (C1), origin_peak_lfc table (C2), relative-enrichment table (C3, the headline result / Fig. line chart), reduce-summarize gene/peak counts (C4). Figures (MA plot, boxplot, two line charts) are visual renderings of these same numbers.
Data availability (all shipped — no external download needed)
- RNA-seq quants:
macrophageBioconductor ExperimentData package,inst/extdata/quants(salmon 0.12.0, Gencode v29). Installed via bioconda. - DA ATAC peaks: shipped
peaksGRanges in repodata/peaks.rda(limma DA results, cqn-normalized). The raw ATAC matrix on Zenodo 10.5281/zenodo.3633505 (~109 MB) was NOT downloaded — the limma result is pre-shipped, so re-running limma on raw counts is the optional last 20% and was skipped. - Code: github.com/sa-lee/fluentGenomics @ fc9849dc926e4666beb9a4924ed2788289e2415e (the authors' own workflow package — P16 not needed, but it's their own code anyway).
Out of scope / not attempted
- Re-deriving the DA
peaksfrom the raw Zenodo ATAC matrix via limma (pre-shipped; deterministic given the same input; skipped per 80/20). - Pinning the exact 2020 Bioconductor 3.10 stack (DESeq2 1.26 etc.) — we used current bioconda (DESeq2 1.42); this is the sole source of the ~9% count drift, documented honestly rather than chased.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Faithful 1:1-intent reproduction on fully shipped data: the genes-tested count is exact and the paper's headline relative-enrichment result (1.42/2.90/6.89 -> 1.43/2.97/7.20) reproduces within ~5%, preserving the exact conclusion. All absolute counts run a coherent ~9% lower, traced entirely to a newer DESeq2 (1.42 vs 1.26) calling 749 vs 822 DE genes — a technical version-drift effect on our side, not an authors' defect, and no fabrication signal. Largest deviations sit in the count outputs but stem from a single propagated gene-set-size factor; severity is moderate for absolute counts yet negligible for the central claim. Overall: solid reproduction with fully explainable, internally consistent deviations.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.