Data-driven projections of candidate enhancer-activating SNPs in immune regulation.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- 🟡Could not use the authors’ exact input data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH, and the headline number reproduces essentially 1:1. The paper's step-1 GAS-motif scan is, in the repo, a transparent Python regex (TTC...GAA|TTC....GAA) and the 'nearly-complete GAS' = exactly one mismatch at one of six fixed positions (variant types T1/T2/C/G/A1/A2_GAS in get_create_GAS_motif_SNPs.py). We reproduced that definition independently (no FIMO PWM) on the paper's own data (hg38 25 primary chromosomes from UCSC) on «our HPC» (SLURM 2175667, 3m). Result: ncGAS = 17,042,730 (forward strand, overlapping/FIMO-style) vs reported 17,039,967 -> diff +2,763 (+0.0162%), a 99.98% match -> within-tol. The non-overlapping count (16,992,903) and both-strand count (~34.1M) confirm the paper used a single-strand, overlapping scan. Strong evidence the headline value is genuine and pipeline-derived; no fabrication signal. NOT ATTEMPTED (hard 20%): the dbSNP155 SNP intersection (3,301,512) -- tens of GB, and the paper cites the hs1/T2T build for dbSNP while motifs are hg38 (a build inconsistency flagged for the human auditor); and the downstream funnel (50,265->...->6) which depends on many GEO ChIP/RNA-seq datasets, GO gene lists, bedtools windows, and explicit manual curation. Repo is a script-dump (no env/entrypoint, hardcoded local paths, two README-referenced FIMO scripts missing/404) but the core logic is transparent and checks out.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 57assessed: 2026-06-14 ⛓ 7285d39e64c8
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a data-driven, multi-layered computational protocol prioritize the small number of high-potential single nucleotide polymorphisms (SNPs) that create de novo gain-of-function STAT/GAS transcription factor binding motifs in cis-regulatory elements of the JAK-STAT immune pathway, narrowing millions of candidates down to a feasible set for experimental validation?
- ★ A data-driven computational protocol integrating genomic, epigenomic, and cross-species conservation data can narrow millions of SNPs to six high-potential gain-of-function candidates that create GAS motifs in JAK-STAT regulatory regions. method
- ★ Six candidate gain-of-function SNPs (rs560898780, rs1257658099, rs370669851, rs571421696, rs910130021, rs138606888) in enhancers upstream of IRF3, IL7R, JAK2, JAK3, SOCS1, and PTPN2 were prioritized as feasible gene-editing targets. finding
- ★ Scanning hg38 for sequences one nucleotide away from the canonical STAT GAS motif (TTCnnnGAA) identifies positions where a single SNP could create a de novo transcription factor binding site. method
- ★ Successive filtering by open chromatin (H3K27ac), proximity to immune genes, dbSNP recurrence, TSS distance, motif spacing, and mouse conservation progressively reduces candidate SNPs. method
- SNPs in cis-regulatory elements can create, disrupt, or supercharge GAS motifs, leading to gain-of-function, loss-of-function, or supercharged target gene scenarios in the JAK-STAT pathway. mechanism
- ★ JAK2, SOCS1, and PTPN2 show significant differential expression under cytokine stimulation, supporting potential enhancement of the JAK-STAT pathway via GAS motif modification. finding
- The protocol is adjustable and intended as a reusable resource to inspire collaboration between bioinformaticians and experimentalists for studying de novo TFBS-creating SNPs. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Genome-wide motif search (FIMO, MEME suite) | Human genome (hg38) | none (in silico single-nucleotide motif analysis) | Number of nearly complete GAS motifs (ncGAS) and overlapping SNPs | FIMO / MEME suite |
| H3K27ac ChIP-seq (publicly available, reanalyzed) | PBMC and T-cells | none | Open-chromatin regions overlapping candidate SNPs/motifs | — |
| ChIP-seq for acetylation (independent dataset, reanalyzed) | IL4-stimulated PBMC cells | IL-4 stimulation | Acetylation/accessible CREs at identified SNPs | — |
| Bulk RNA-seq (publicly available, reanalyzed) | Primary cells derived from human lungs | Cytokine stimulation (IFNα, IFNβ, IFNγ, IL-6, IL-7) | DESeq2-normalized gene counts for IRF3, IL7R, JAK2, JAK3, SOCS1, PTPN2 | DESeq2 (analysis) |
| Cross-species conservation analysis | Mouse genome (mm10) vs human | none | Presence of conserved GAS motif 10 kb upstream of target genes | — |
| Variant annotation / database filtering | Human SNP databases (dbSNP, ClinVar) | none | SNP recurrence (≥2 subjects), clinical reporting status | dbSNP, ClinVar |
- – FIMO identified nearly complete GAS motifs across the human genome 17,039,967 ncGAS motifs
- – SNPs identified within ncGAS motif locations per dbSNP 3,301,512 SNPs
- ▼ Filtering for H3K27ac open chromatin in PBMC and T-cells 50,265 SNPs
- ▼ Filtering for proximity to immune genes (GO database) 16,017 SNPs
- ▼ Filtering for motifs within 10 kb upstream of TSS 4,391 SNPs
- ▼ Filtering for dbSNP recurrence (≥2 subjects), spacing (200 bp), and mouse conservation 416 → 273 → 214 SNPs
- ▼ Final manual prioritization to six candidate gain-of-function SNPs in JAK-STAT enhancers 6 SNPs (from 30 JAK-STAT targets)
- – JAK2, SOCS1, and PTPN2 showed significant differential expression under cytokine stimulation
- count 17,039,967 (potential nearly complete GAS motifs found by FIMO across human genome)
- count 3,301,512 (SNPs identified in ncGAS motif locations per dbSNP)
- count 50,265 (SNPs remaining after H3K27ac open-chromatin filter (PBMC and T-cells))
- count 16,017 (SNPs remaining after immune-gene proximity filter)
- count 4,391 (SNPs remaining after 10 kb upstream TSS filter)
- count 416 (SNPs found in at least two independent human subjects)
- count 214 (SNPs remaining after mouse genome conservation filter)
- pvalue * p ≤ 0.05, ** p ≤ 0.01 (independent t-tests comparing cytokine-stimulated gene expression (DESeq2 normalized counts))
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This paper presents a sequential bioinformatic filtering pipeline to prioritize SNPs capable of creating STAT-binding GAS motifs, reducing ~3.3 million candidates to six for experimental follow-up. The core methodology is not inferential but rather rule-based attrition: genome-wide motif scanning (FIMO), histone-modification-defined open chromatin, gene-proximity annotation, dbSNP frequency filtering, spacing constraints, and cross-species conservation. As a secondary validation step, publicly available RNA-seq data (Lee et al.) were analyzed using DESeq2-normalized counts, and independent t-tests were applied to compare gene expression across cytokine stimulation conditions, with results reported as significance bands.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Independent t-test | Fig. 4 — pairwise comparisons of DESeq2-normalized gene counts for IRF3, IL7R, JAK2, JAK3, SOCS1, PTPN2 across IFNα, IFNβ, IFNγ, IL-6, IL-7 vs. control | — | not stated |
| FIMO motif scanning (position-weight matrix score with internal p-value threshold) | Genome-wide identification of nearly-complete GAS motifs (ncGAS) in hg38 | — | not stated |
-
Independent t-tests were applied to DESeq2-normalized RNA-seq counts to compare stimulation conditions↳ Could also: DESeq2's built-in Wald test (or likelihood ratio test) on raw count data, or a non-parametric rank-based test such as Mann-Whitney U on the normalized values — RNA-seq count data follow a negative-binomial distribution; DESeq2's native differential-expression test accounts for overdispersion and heteroscedasticity across conditions, which are assumptions the t-test does not model; a non-parametric alternative makes no distributional assumption and is robust to small n
-
Multiple pairwise t-tests were performed across six genes and five stimulation conditions with no stated correction↳ Could also: Benjamini-Hochberg false discovery rate (FDR) adjustment across the full family of gene × condition comparisons — Controlling the FDR across a joint family of tests reduces the expected proportion of false positives among declared significant results, which can matter when several comparisons are made simultaneously; this is standard practice in multi-condition transcriptomic analyses
-
Open-chromatin filtering used a binary presence/absence criterion based on H3K27ac ChIP-seq peaks↳ Could also: Quantitative ranking by ChIP-seq signal intensity (e.g., normalized read counts or fold-enrichment over input) to score candidates on a continuous scale — A continuous score would allow ranked prioritization rather than a hard binary cut, potentially retaining biologically relevant sites near the peak threshold and enabling sensitivity analysis of the cutoff choice
-
Cross-species conservation was applied as a binary filter (ncGAS motif present/absent in mm10 within 10 kb of the orthologous gene)↳ Could also: Quantitative phylogenetic conservation scores such as phyloP or phastCons from UCSC multi-alignment tracks — Continuous conservation scores capture the degree of constraint at each nucleotide position, allowing ranked filtering rather than an all-or-nothing comparison to a single species, and are routinely used in regulatory variant prioritization frameworks such as CADD
-
Proximity to gene regulatory regions was defined by a fixed 10 kb upstream window from the TSS↳ Could also: Integration of experimentally mapped enhancer–gene links from eQTL catalogs (e.g., GTEx), chromatin conformation data (Hi-C, HiChIP), or activity-by-contact scores — A fixed-distance window assumes all regulatory activity lies within 10 kb upstream, whereas long-range enhancers can act from tens to hundreds of kilobases away; activity-by-contact or eQTL evidence would anchor each candidate to a specific gene–SNP regulatory relationship rather than proximity alone
-
Significance was reported as categorical bands (* ≤ 0.05, ** ≤ 0.01) without effect sizes or dispersion measures↳ Could also: Reporting fold-change (or log2 fold-change) alongside 95% confidence intervals and the underlying n per group — Effect sizes communicate biological magnitude independently of sample size, and confidence intervals convey estimation uncertainty; these are complementary to p-values and are recommended by reporting guidelines (e.g., ARRIVE, Nature reporting checklist) to help readers assess practical significance
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-40011812 (GAS_motifs)
Paper: Hoffmann et al. 2025, BMC Genomics. "Data-driven projections of candidate enhancer-activating SNPs in immune regulation." DOI 10.1186/s12864-025-11374-7. Repo: https://github.com/Firestar93/GAS_motifs (Python/R scripts, no env file, hardcoded Windows/Mac paths, no single entrypoint). Data: hg38 (UCSC), dbSNP155, many GEO ChIP/RNA sets.
Pipeline funnel reported (Results & Discussion)
ncGAS motifs (hg38, FIMO almost-GAS) 17,039,967 <-- HEADLINE, step 1 SNPs in ncGAS (dbSNP155) 3,301,512 <-- HEADLINE, step 2 SNPs in open chromatin (H3K27ac>20) 50,265 SNPs near immune genes (GO) 16,017 SNPs within 10kb upstream TSS 4,391 SNPs in >=2 subjects 416 SNPs spaced >=200bp from existing GAS 273 SNPs conserved in mm10 214 SNPs in JAK-STAT pathway 30 Final candidates 6 (rs560898780, rs1257658099, rs370669851, rs571421696, rs910130021, rs138606888)
In scope (deterministic, public data, reproducible 1:1)
- C1 — genome-wide ncGAS motif count on hg38. The repo detects GAS motifs by a pure regex (main.py/motif.py: "TTC...GAA|TTC....GAA"). The "almost/nearly-complete GAS" = exactly one mismatch at a fixed position, enumerated as 6 variant types (T1/T2/C/G/A1/A2_GAS, per get_create_GAS_motif_SNPs.py). Fully deterministic given hg38; no PWM/threshold subtleties for the exact-one-mismatch definition. Compare to 17,039,967.
- C2 — canonical complete GAS motif count on hg38 (TTCnnnGAA 3-gapper, TTCnnnnGAA 4-gapper) via the repo's own regex. Context/clean repo reproduction (deterministic).
Out of scope / not attempted (the hard 20%, stated honestly)
- C2b SNP intersection (3,301,512): needs full dbSNP155 (~tens of GB; paper cites hs1/T2T build for dbSNP while motifs are hg38 — a build inconsistency). Skipped per 80/20.
- Downstream funnel (50,265 -> ... -> 6): depends on many GEO ChIP/RNA-seq datasets, GO term lists, bedtools windows, manual curation. Multi-dataset + manual steps; out of 80/20 scope.
- Exact FIMO PWM/threshold replication (--thresh 0.05 ...): the paper's 17M is a FIMO count; we reproduce the definition (exact one-mismatch enumeration) independently on the same hg38 data — an equally-valid independent-tool reproduction (P16), not the FIMO binary.
Method for C1/C2
Download hg38 chromosomes from the exact URL the paper cites (https://hgdownload2.soe.ucsc.edu/goldenPath/hg38/chromosomes/), scan each chromosome with the repo's regex (canonical) and the 6 one-mismatch variant regexes (ncGAS). Count both non-overlapping (repo re.findall semantics) and all-overlapping (FIMO semantics) on the forward strand; report both. Run on «our HPC» SLURM, data on «infra».
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The paper's foundational claim — 17,039,967 nearly-complete GAS motifs on hg38 — reproduces essentially 1:1 (we got 17,042,730, +0.0162%) from a transparent regex on public hg38, strong evidence the value is genuine and pipeline-derived (no fabrication). The deviation is rounding-level and sits on the input/tooling side (FIMO PWM vs our regex, contig set), i.e. technical/expected. However, the paper's actual central conclusion — the 3,301,512-SNP dbSNP155 intersection and the downstream funnel to 6 candidate enhancer-activating SNPs — was not attempted (data volume, manual curation, missing/404 FIMO scripts, plus a flagged hs1-vs-hg38 build inconsistency), so the headline conclusion is only partially verified. Net: a clean, reproducible foundation but partial coverage with explainable deviations → overall yellow.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.