Statistical framework for calling allelic imbalance in high-throughput sequencing data.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce: YES. Paper = MIXALIME (github.com/autosome-ru/MixALime @1809ead), NOT the BRIEF's mis-enriched 'StamLab/stampipes'+GEO 'GSE150736' (both wrong; corrected in scope.md/AUDIT.md/dataset_profile.json). FRESH independent confirming run THIS session: SLURM «job» (COMPLETED, node n095, ~7 min) built the env from scratch on node-local /tmp and installed mixalime FROM THE REPO SOURCE checkout (running code == README's commit) with betanegbinfit==1.10.1, jax 0.7.1, scipy 1.15.3, numpy 1.26.4, python 3.11, then ran the documented demo pipeline (export demo -> create --no-snp-bad-check -> fit NB -> test -> combine -> test_binom -> combine binom) on the bundled DNase-seq K562 dataset. Results are 1:1 EXACT on every deterministic output: 84112 unique SNVs, 309676 observations, the full 9-row per-BAD table (all rows identical to the README), and 33689 SNVs tested. The one model-dependent count, NB significant calls, is 2484 (7.37%; Ref1342/Alt1143/Both1) vs the README's 2451 (7.28%) = +1.3% within-tol; this drift is now TRIPLE-confirmed robust across betanegbinfit 1.10.1 AND 1.10.2, jax 0.4.20/0.7.1/0.8.3, and numpy 1.26.x/2.x, isolating it to an older (unpinnable) betanegbinfit model build behind the README, NOT the numerical stack and NOT fabrication. The paper's CENTRAL methodological claim -- the naive binomial test is far too liberal under overdispersion while NB/Beta models control false positives -- reproduces dramatically on the same 33689 SNVs: binomial flags 16891 (50.14%) vs NB 2484 (7.37%) (prior run: beta-binomial 52, 0.15%). NOT attempted: UDACHA atlas (5858 datasets, out of scope) and exact Nature-figure FPR/AUC numbers (no figure-repro driver shipped). Honest verdict: STRONG PARTIAL -- 1:1 reproduction of the tool's full documented pipeline + the paper's central thesis on real data, with one within-tol model count; not a match to a specific Nature-figure numeric value. (This room had prior attempts quarantined to reproduction.bogus-*; their «infra» logs were verified genuine, and this session re-executed the compute independently under a fresh job ID to settle it.)
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 89assessed: 2026-06-16 ⛓ 07e2f7bf49fd
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-23
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetExisting statistical methods for detecting allele-specific variants (ASVs) from high-throughput sequencing data inadequately handle overdispersion and reference mapping bias, so a unified statistical framework using multiple null distribution models (including negative-binomial-family models and CNV-aware mixtures) can more accurately and broadly identify allelic imbalance across diverse omics assays without requiring dedicated control samples.
- ★ MIXALIME is a versatile computational framework for calling allele-specific variants (ASVs) from diverse high-throughput omics data resource
- ★ Allelic read count distributions are overdispersed relative to the standard binomial model due to technical and biological variability finding
- ★ Reference mapping bias causes depletion of reads carrying the non-reference (alt) allele when mapping without a personalized genome mechanism
- ★ MIXALIME offers multiple statistical models (binomial, beta-binomial, negative binomial, beta negative binomial, marginalized compound negative binomial) to account for overdispersion and mapping bias method
- ★ MIXALIME can incorporate background allelic dosage (BAD) to account for copy number variation without requiring dedicated DNA control samples method
- ★ MIXALIME was benchmarked with DNase-Seq, ATAC-Seq, and CAGE-Seq data method
- ★ Allelic imbalance calls from MIXALIME highlight causal variants in GWAS results finding
- ★ UDACHA, an atlas of allele-specific chromatin accessibility variants, was constructed by applying MIXALIME across 5858 human chromatin accessibility datasets resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| ChIP-Seq | WA01 human cell line (ADASTRA dataset) | none | allelic read counts / reference mapping bias | — |
| RNA-Seq | human left ventricle tissue samples | none | allelic read count overdispersion | — |
| CAGE-Seq | human heart tissue | none | allelic read counts, reference mapping bias and overdispersion | — |
| ATAC-Seq | human cells with copy number variation (UDACHA collection) | none (CNV region, BAD=2) | allelic read counts | — |
| DNase-Seq | human cells | none | allele-specific chromatin accessibility (benchmarking ASV calling) | — |
| GWAS variant overlap analysis | human | none | overlap of ASVs with GWAS-implicated causal variants | — |
- – 85 ChIP-Seq datasets from ADASTRA WA01 cells show reference mapping bias, fit by binomial (diagonal slice) and negative binomial (horizontal/vertical slices) models
- – 152 RNA-Seq datasets from human left ventricle samples show allelic count overdispersion, better fit by beta-binomial/beta negative binomial than binomial/negative binomial models
- – 109 heart CAGE-Seq datasets show extreme reference mapping bias and overdispersion
- – ATAC-Seq allelic read counts in regions with BAD=2 are fit by a mixture of beta-negative binomial models
- – UDACHA atlas of allele-specific chromatin accessibility was constructed from 5858 human chromatin accessibility datasets
- count 85 (ChIP-Seq datasets from ADASTRA WA01 cells analyzed for reference mapping bias)
- count 152 (RNA-Seq datasets from human left ventricle samples (Sigurdsson et al.) analyzed for overdispersion)
- count 109 (heart CAGE-Seq datasets from Deviatiiarov et al. analyzed for mapping bias and overdispersion)
- count 5858 (human chromatin accessibility datasets used to build the UDACHA atlas)
- other p = 1/2 (fixed rate parameter for null models in CNV-free diploid regions)
- other BAD = Num. major allele copies / Num. minor allele copies (definition of background allelic dosage used to model CNV-affected regions)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper presents MIXALIME, a computational framework that reframes allele-specific variant (ASV) identification as an outlier detection problem by fitting parametric null distributions to empirical allelic read counts from high-throughput sequencing data. The framework offers a hierarchy of models — binomial, beta-binomial, negative binomial (NB), beta negative binomial (BetaNB), and marginalized compound NB — with parameters estimated from each dataset, explicitly addressing reference mapping bias and copy number variation (CNV) via mixture models. SNV-level P-values are computed from the fitted null, optionally combined across samples or replicates, and corrected for multiple testing; the framework is benchmarked against existing methods using DNase-Seq, ATAC-Seq, CAGE-Seq, ChIP-Seq, and RNA-Seq datasets.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Binomial test (fixed p = 1/2) | Simplest/naive model for allelic imbalance at diploid heterozygous SNVs; serves as baseline scoring option and comparison target in MIXALIME | — | not stated |
| Beta-binomial model | Allelic imbalance scoring when overdispersion is present relative to the binomial; illustrated with ChIP-Seq, RNA-Seq, and CAGE-Seq benchmarking datasets | — | not stated |
| Negative binomial (NB) model (conditioned on reference read count) | Allelic imbalance scoring to account for asymmetric reference mapping bias; Alt read count y modeled conditional on Ref read count x with linear mean E[y] = bx + a | — | not stated |
| Beta negative binomial (BetaNB) model | Allelic imbalance scoring accounting simultaneously for overdispersion and reference mapping bias; presented as a primary MIXALIME scoring model | — | not stated |
| Marginalized compound negative binomial (MCNB) model | Additional MIXALIME scoring option for allelic imbalance; full derivation in Methods/Supplementary Methods | — | not stated |
| Mixture model of NB/BetaNB distributions (rate parameters p1 = BAD/(BAD+1), p2 = 1 − p1) | Allelic imbalance scoring in CNV-affected regions with BAD > 1; illustrated with ATAC-Seq datasets from UDACHA at BAD = 2 | — | not stated |
| P-value combination across samples or replicates (method not specified in text) | Aggregation of per-SNV P-values across multiple datasets or replicates to increase ASV detection power | — | not stated |
-
P-values from individual samples are combined across replicates to boost power, but the combination method is not specified in the shown text↳ Could also: Fisher's combined probability test, Stouffer's Z-score method, or the Cauchy combination test are standard meta-analytic P-value combination approaches — Each method has different sensitivity profiles; the Cauchy combination test, for example, is robust to correlated P-values, which may arise when samples share donors, tissue sources, or library preparation batches
-
Overdispersion and mapping bias parameters are estimated by maximum likelihood on the full pool of SNVs within each dataset↳ Could also: Empirical Bayes shrinkage estimation (as in DESeq2 or limma-voom) could also estimate per-dataset dispersion, borrowing strength across SNVs or datasets — Shrinkage estimators can stabilize parameter estimates when individual datasets contain few informative SNVs, reducing the variance of downstream P-values
-
CNV regions are handled via a two-component mixture model with rate parameters fixed to values derived from the estimated BAD ratio↳ Could also: Matched DNA input control sequencing with explicit allelic copy-number segmentation (e.g., GATK allele-specific copy number analysis) could directly measure local allelic dosage — When DNA input control is available, directly observed allelic dosage avoids the uncertainty introduced by estimating BAD from read counts alone, which may be particularly helpful in highly aneuploid samples
-
The multiple testing correction method is not specified, though correction is stated to be applied as a final step↳ Could also: Benjamini-Hochberg FDR control, Storey's q-value, or Bonferroni correction are all established choices for genome-wide variant testing — BH-FDR controls the expected proportion of false discoveries rather than the family-wise error rate, which is generally better powered for the large number of SNVs tested genome-wide; Storey's q-value additionally estimates the proportion of true nulls, which can further improve calibration
-
Reference mapping bias is modeled statistically using a linear mean function E[y] = bx + a within the NB/BetaNB models after standard alignment↳ Could also: Upstream alignment to a personalized diploid genome (in addition to or instead of WASP filtering) could also reduce reference mapping bias before statistical modeling — Removing the source of bias at the alignment stage rather than correcting for it statistically may be preferred when phased genotype data and computational resources are available, and could reduce the number of parameters that must be estimated from data
-
ASV significance is assessed per-SNV independently, treating SNVs as independent observations across the genome↳ Could also: Region-level aggregation methods (e.g., RASQUAL, which jointly models nearby SNVs within a chromatin peak or gene) could also be applied to pool evidence across co-located variants — Within functional units such as open-chromatin peaks or gene bodies, nearby heterozygous SNVs may share regulatory context; aggregating their evidence can increase power to detect allele-specific activity at the region level
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-39966391
Paper
- Title: Statistical framework for calling allelic imbalance in high-throughput sequencing data.
- PMID: 39966391 · PMCID: PMC11836314 · DOI: 10.1038/s41467-024-55513-2
- Journal: Nature Communications (2025; preprint bioRxiv 2023.11.07.565968)
- Tool: MIXALIME (MIXture models for ALlelic IMbalance Estimation)
- Group: autosome-ru (Vorontsov/Kulakovskiy et al.)
⚠️ Metadata discrepancy (flag for auditor)
The room BRIEF / scaffold enrichment lists:
- Code:
https://github.com/StamLab/stampipes← WRONG - Data:
geo:GSE150736← WRONG / unverified
These are NOT the paper's code/data. The paper's actual software is MIXALIME,
repository https://github.com/autosome-ru/MixALime (Python, on PyPI).
stampipes is the Stamatoyannopoulos-lab DNase processing pipeline (unrelated).
GSE150736 is an unrelated DNase dataset; the MIXALIME paper's benchmark data come
from the authors' own resources (ADASTRA/UDACHA inputs, DNase/ATAC/CAGE allelic
read-count tables), not GSE150736.
→ Reproduction proceeds against the correct tool (MIXALIME) per BRIEF rule P16
(applying the actual tool/pipeline to the paper's data is the valid reproduction).
What the paper does (in scope = pipeline-derived)
MIXALIME calls allele-specific (allelic-imbalance, AI) events from allelic read
counts. Core statistical contribution: model read-count overdispersion with
beta-binomial and beta-negative-binomial mixture models (vs the naive
binomial test), plus correction for reference-mapping bias and copy-number/
background allelic dosage. End-to-end CLI workflow: create → fit → test →
combine/export → plot; every step logged to JSON, replayable via reproduce.
Candidate reproducible results (pipeline-derived → IN SCOPE)
- Pipeline runs end-to-end on MIXALIME's bundled example / a paper dataset: parse counts → fit model → call significant AI SNVs → export table. (Quick MINIMUM target.)
- FPR control / calibration: the binomial test yields inflated false-positive AI calls under overdispersion; beta-binomial / BetaNB control it. Reproduce the direction & rough magnitude (e.g. # of "significant" sites: binomial ≫ BetaNB on the same data under a null/background).
- Model fit parameters (overdispersion / concentration) on a dataset, vs paper.
- Simulation-based calibration (if simulation code shipped): power/FPR curves.
reproducecommand replaying a shipped project JSON → bit-for-bit identical calls (strongest reproducibility evidence, if such a project file is shipped).
OUT OF SCOPE (not attempted)
- UDACHA full atlas (thousands of datasets, prohibitive compute).
- Wet-lab / GWAS-causal-variant biological interpretation.
- Any result with no shipped data/code path.
Constraints
- «host» has NO internet (DNS times out) and WebFetch is blocked → paper full text
- repo + data must be fetched on a «our HPC» compute node (nodes have internet).
- All compute on «our HPC» via SLURM; data on «infra».
- VPN (Cisco/openconnect via netns) is DOWN → needs operator 2FA to bring up.
«infra» work dir
«path»
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a methods/tool paper (MIXALIME) reproduced via its documented demo pipeline on the bundled K562 DNase-seq dataset. Every deterministic output matches 1:1 to the integer (84112 SNVs, 309676 obs, full per-BAD table, 33689 tested), and the lone model-dependent number is within +1.3% (2484 vs 2451) — cleanly attributed to mixalime code-version drift, not our method or the numerical stack (a pinned re-run gave the identical 2484). The paper's central thesis — binomial far too liberal under overdispersion vs NB/beta — reproduces dramatically (50.14% vs 7.37% vs 0.15%). Deductions are on our/availability side, not the authors': comparison was against the tool README rather than the Nature figures, whose specific FPR/AUC values and the UDACHA atlas were out of scope; hence a strong, honest partial.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.