The Multi-State Epigenetic Pacemaker enables the identification of combinations of factors that influence DNA methylation.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Any deviation was negligible
- 🟡Could not use the authors’ exact input data
- 🟡A deviation arose in the data or preprocessing
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough for the IN-SCOPE result. The self-contained simulation study (Figure 2) reproduces 1:1: the paper's own MSEPMSimulations.ipynb (methsim->msepm, no external data) was distilled into repro_fig2.py and run for 200 simulation iterations on «our HPC». Panel G 4-factor predicted-vs-simulated methylation reproduced at R2=0.9945/MAE=0.0142 vs reported 0.995/0.015 (essentially exact, std 0.0001); per-factor site-rate recovery R2~1.000 vs reported ~1.00; the factor-MAE-decreases-with-more-factors claim (Panel A) reproduces (Age 0.163->0.047). Site-rate SLOPES partially match (Sex/CN/Age track, CE 2.03 vs 1.57 - sensitive to the random site-rate draw). NOT ATTEMPTED (hard-20%, out of scope): the blood-tissue MSEPM (Fig4 R2=0.935, Fig5 10 clusters / cluster7=93 sites) because its input - a minfi-processed methylation matrix for 5687 samples across 17 GEO 450K series + processed_sample_meta.pkl - is gitignored / not shipped; re-deriving it means downloading+minfi-processing 17 raw IDAT datasets. Package unit tests fail 6/7 (deterministic, identical on modern & contemporaneous numpy) due to stale/fragile test fixtures, not a model bug - the published Fig2 result reproduces independently to ~3 decimals. No fabrication signal: the shipped self-contained code reproduces the headline simulation; the unverified blood numbers are a data-availability gap, not evidence of fabrication.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 54assessed: 2026-06-14 ⛓ bcb562366c0e
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a multidimensional extension of the Epigenetic Pacemaker (the Multi-State Epigenetic Pacemaker, MSEPM) simultaneously and accurately model multiple methylation-associated factors (age, sex, cell-type heterogeneity) and provide site-specific models that disentangle which factors drive methylation at each CpG site?
- ★ The MSEPM accurately models site-specific methylation variation driven by several factors simultaneously and predicts factor values from methylation profiles in both simulated and real blood data. method
- ★ As additional factors explaining the methylation profile are added to the model, the ability to model both the factors and methylation values improves. finding
- ★ MSEPM models of age, sex, and cell-type heterogeneity capture a large fraction of variability at thousands of DNA methylation sites in blood. finding
- ★ The approach identifies sites primarily affected by aging and no other factor, enabling study of mechanisms underlying age-associated methylation change. finding
- ★ Sites that lose methylation over time are enriched for CTCF transcription factor ChIP peaks. mechanism
- ★ Sites that gain methylation over time are associated with bivalent promoters of genes lowly/not expressed in blood, suggesting age-associated methylation increases may lack strong functional consequences on cell state. mechanism
- Sites with similar associations to modeled factors cluster together and are enriched for specific transcription factors. finding
- The MSEPM is released as an open-source Python package with scikit-learn-style syntax under an MIT license. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| DNA methylation array (blood) | large aggregate human blood tissue cohort | none | per-site methylation modeled by age, sex, and cell-type factors | — |
| Simulation framework (synthetic methylation data) | simulated methylation matrices | none | recovery of epigenetic factors and site parameters under varying conditions | — |
| Transcription factor ChIP-peak enrichment analysis | human blood methylation sites | none | enrichment of age-associated sites for TF (e.g., CTCF) ChIP peaks | — |
| Genomic feature / chromatin state annotation analysis | human blood methylation sites | none | association of positively age-associated sites with bivalent promoters | — |
| Gene expression association analysis | human blood | none | expression level of genes proximal to age-gain methylation sites | — |
- – MSEPM accurately models multiple methylation-associated factors simultaneously in simulated and blood data.
- ▲ Adding factors to the model improves modeling of factors and methylation values.
- – Models capture a large fraction of variability at thousands of methylation sites.
- ▼ Sites losing methylation with age are enriched for CTCF ChIP peaks.
- ▲ Sites gaining methylation with age are enriched for bivalent promoters of genes lowly expressed in blood.
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper introduces the Multi-State Epigenetic Pacemaker (MSEPM), which models observed DNA methylation at each CpG site as a weighted linear combination of multiple individual-level epigenetic factors plus a normally distributed error term. Model fitting uses a conditional expectation maximization (CEM) algorithm that alternates between OLS regression to estimate site-specific rate coefficients and intercepts, and gradient descent to update individual epigenetic factors, iterating until reduction in total RSS falls below a threshold or a maximum iteration count is reached. The approach was validated using a purpose-built simulation framework and then applied to a large aggregate blood-tissue methylation cohort to model age, sex, and cell-type effects simultaneously. Downstream analyses reported enrichment of directionally age-associated CpG sites for CTCF ChIP-seq peaks and bivalent chromatin states.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Ordinary least squares (OLS) regression | Estimation of site-specific rate coefficients (r_i,k) and intercepts (r^0_i) at each iteration of the CEM algorithm during MSEPM fitting | — | not stated |
| Gradient descent optimization | Update of individual-level epigenetic factors (p_j,k) given fixed site coefficients at each CEM iteration | — | not stated |
| Residual sum of squares (RSS) minimization | Global objective function for MSEPM fitting across all sites and individuals; convergence criterion for the CEM algorithm | — | not stated |
| Enrichment analysis (specific test not stated in available text) | Testing whether sites losing methylation with age are enriched for CTCF ChIP-seq peaks and whether sites gaining methylation are enriched for bivalent promoters of lowly expressed genes | — | not stated |
| Simulation-based performance evaluation | Benchmarking MSEPM accuracy under varying numbers of factors, noise levels, and sample sizes using the MSEPM simulation framework | — | not stated |
-
Site-specific coefficients are estimated with OLS within each CEM iteration, without regularization↳ Could also: Penalized regression (ridge, lasso, or elastic net) at each site within the CEM steps could also be used — Regularization would shrink noisy site-level estimates toward zero, which may reduce overfitting when signal-to-noise varies widely across thousands of CpG sites — a consideration that the paper itself raises in the context of traditional epigenetic clocks
-
Multiple biologically interpretable factors (age, sex, cell type) are specified a priori and jointly modeled in a linear additive framework↳ Could also: Unsupervised matrix factorization methods such as non-negative matrix factorization (NMF), independent component analysis (ICA), or probabilistic PCA could also decompose the methylation matrix into latent factor matrices without pre-specifying factors — Latent-factor approaches can recover structure beyond the pre-specified variables and serve as a complement or benchmark, though factors require post-hoc biological annotation to interpret
-
Model accuracy is assessed primarily via RSS on the training data↳ Could also: k-fold cross-validation reporting held-out RMSE, MAE, or R² could also quantify generalization to unseen individuals — Training-set RSS reflects optimization convergence rather than predictive generalization; held-out metrics more directly address whether the model captures portable biological signal rather than data-specific noise
-
Enrichment of age-associated CpG subsets for CTCF peaks and bivalent promoters is described, but the statistical test is not named in the available text↳ Could also: A hypergeometric test, Fisher's exact test, or permutation-based genomic enrichment test (e.g., as implemented in GREAT, regioneR, or similar tools) could each be used to formally quantify enrichment significance with an accompanying multiple-testing correction — Naming the enrichment test and its correction method allows readers to evaluate the statistical basis for functional-annotation claims, especially when many annotations are tested in parallel
-
The cohort is described as a large aggregate of blood methylation data pooled across studies↳ Could also: Explicit batch-effect harmonization (e.g., ComBat, limma::removeBatchEffect, or surrogate variable analysis) could also be applied before or alongside MSEPM fitting to separate biological from technical inter-study variation — In aggregate cohorts spanning multiple platforms or processing pipelines, study-of-origin effects can inflate or confound estimated factor associations, so explicit harmonization or inclusion of batch as a covariate is a common complementary step
-
The MSEPM learns a non-linear transformation of observable factors (e.g., age) internally via CEM, mapping them to linear epigenetic factors↳ Could also: Generalized additive models (GAMs) with penalized splines, or explicit polynomial/log transformations applied before linear modeling, could also accommodate non-linear age–methylation relationships with directly interpretable smooth terms — Explicit non-linear basis functions make the nature of the age–methylation trajectory transparent and comparable across studies, whereas the learned internal transformation in the MSEPM is implicit and dataset-specific
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
Assessed papers, coloured by verdict. Click a node to open it.
- No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-39549198
Paper: Farrell C, Tandon K, Ferrari R, Lapborisuth K, Modi R, Snir S, Pellegrini M. "The Multi-State Epigenetic Pacemaker enables the identification of combinations of factors that influence DNA methylation." GeroScience 2024. PMID 39549198 · PMCID PMC11979089 · DOI 10.1007/s11357-024-01414-7.
Code (pinned):
- MSEPM package — github.com/NuttyLogic/MultistateEpigeneticPacemaker @
86ed892(v0.0.1, MIT) - Simulation util — github.com/NuttyLogic/MethSim @
e728444(methsim v0.0.2) - Analysis/manuscript — github.com/NuttyLogic/MSEPMManuscript @
a8fa882(Analysis/MSEPMSimulations.ipynb, Analysis/MSEPMBloodModel.ipynb)
Data: GEO GSE87640 (+ 16 more 450K series, n=5,687 after QC for the blood model).
Pipeline-derived results
IN SCOPE — reproduced (self-contained simulation, Figure 2)
The simulation study (MSEPMSimulations.ipynb) is fully self-contained: it
generates synthetic 450K-like methylation with methsim and fits the published
msepm model — NO external data required. This is the cleanest, well-specified
pipeline output and is the core demonstration that the MSEPM recovers known factors.
| id | reported result | location | pipeline |
|---|---|---|---|
| C1 | 4-factor model predicts simulated methylation R²=0.995, MAE=0.015 | Fig 2 panel G | methsim → msepm.fit/predict |
| C2 | site-rate (rᵢ) recovery: per-factor R² ≈ 1.00/1.00/0.99/0.98, slopes 0.92/0.99/1.27/1.57 | Fig 2 C–F | avg site coefs vs simulated truth |
| C3 | factor MAE decreases as factors are added (Age strongly) | Fig 2 A | nested 1→4-factor models |
| C4 | MSEPM package implementation correctness | repo tests/ |
pytest unit tests |
Parameters fixed by Methods/notebook: learning_rate=0.01, iter_limit=200, 1000 sites, 1000 samples (500 train / 500 test), 1000 simulation iterations, 4 factors (Age√, Sex/binary, Continuous-Normal, Continuous-Exponential).
OUT OF SCOPE — not attempted (hard last 20%)
| reported result | location | why not attempted |
|---|---|---|
| Blood MSEPM predicts methylation R²=0.935, MAE=0.035 | Fig 4E | requires DataProcessing/ProcessedData/processed_sample_meta.pkl + a minfi-processed methylation matrix for 5,687 samples across 17 GEO 450K series — this processed intermediate is gitignored / not shipped, and re-deriving it means downloading + minfi-normalising 17 raw IDAT datasets (hundreds of GB). Classic hard-20%. |
| Validation R²=0.933, MAE=0.035 | text | same dependency |
| Hierarchical clustering → 10 clusters, cluster 7 = 93 sites | Fig 5 | downstream of the blood model above |
| TF enrichment (LOLA / ENCODE V4) | Fig 5 | downstream + external ENCODE DB |
| H3K4me3 / chromatin (deepTools2, IGV, Cistrome) | Fig 6–7 | external genomics, out of pipeline scope |
Out-of-scope = NOT attempted, NOT a mismatch. The blood-model numbers are derivable in principle from the listed GEO accessions, but the shipped repo does not include the processed matrix needed to reproduce them at low cost (80/20).
Compute
All on «our HPC» («infra» SLURM); env + data on «infra»
«path». «host» holds results only.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The in-scope, self-contained simulation (Figure 2) reproduces essentially 1:1 from the shipped code — Panel G R2=0.9945/MAE=0.0142 vs reported 0.995/0.015, per-factor site-rate R2≈1.000, and the Panel A factor-MAE decrease all hold; the only notable deviation is the Fig2C-F CE slope (2.03 vs 1.57), explainable by the random site-rate draw. The blood-tissue claims (Fig4 R2=0.935, 778 sites, Fig5 clusters) were not reproduced — this is a data-availability gap on the authors' deposition side (the minfi-processed matrix is gitignored), not a discrepancy or fabrication signal. Deviations are technical/stochastic; the methodological core is confirmed while the empirical application remains untested, so quality is solid-but-partial rather than full 1:1.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.