The Multi-State Epigenetic Pacemaker enables the identification of combinations of factors that influence DNA methylation.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Any deviation was negligible
- 🟡Could not use the authors’ exact input data
- 🟡A deviation arose in the data or preprocessing
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough for the IN-SCOPE result. The self-contained simulation study (Figure 2) reproduces 1:1: the paper's own MSEPMSimulations.ipynb (methsim->msepm, no external data) was distilled into repro_fig2.py and run for 200 simulation iterations on «our HPC». Panel G 4-factor predicted-vs-simulated methylation reproduced at R2=0.9945/MAE=0.0142 vs reported 0.995/0.015 (essentially exact, std 0.0001); per-factor site-rate recovery R2~1.000 vs reported ~1.00; the factor-MAE-decreases-with-more-factors claim (Panel A) reproduces (Age 0.163->0.047). Site-rate SLOPES partially match (Sex/CN/Age track, CE 2.03 vs 1.57 - sensitive to the random site-rate draw). NOT ATTEMPTED (hard-20%, out of scope): the blood-tissue MSEPM (Fig4 R2=0.935, Fig5 10 clusters / cluster7=93 sites) because its input - a minfi-processed methylation matrix for 5687 samples across 17 GEO 450K series + processed_sample_meta.pkl - is gitignored / not shipped; re-deriving it means downloading+minfi-processing 17 raw IDAT datasets. Package unit tests fail 6/7 (deterministic, identical on modern & contemporaneous numpy) due to stale/fragile test fixtures, not a model bug - the published Fig2 result reproduces independently to ~3 decimals. No fabrication signal: the shipped self-contained code reproduces the headline simulation; the unverified blood numbers are a data-availability gap, not evidence of fabrication.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 54assessed: 2026-06-14 ⛓ bcb562366c0e
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-09-19
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetDNA methylation variability is shaped by multiple factors (e.g., age, sex, cell-type) simultaneously rather than by a single trait as assumed by standard epigenetic clocks, and a multidimensional extension of the Epigenetic Pacemaker (the Multi-State Epigenetic Pacemaker, MSEPM) can jointly model these factors and their site-specific effects on methylation.
- ★ The Multi-State Epigenetic Pacemaker (MSEPM) can accurately model multiple methylation-associated factors (e.g., age, sex, cell-type) simultaneously. method
- ★ MSEPM produces site-specific models describing the per-site relationship between methylation and each modeled factor. method
- ★ MSEPM models capture a large fraction of the variability at thousands of DNA methylation sites in a large aggregate blood methylation cohort. finding
- ★ MSEPM allows identification of sites primarily affected by aging and not by other modeled factors. finding
- ★ Sites that lose methylation with age are enriched for CTCF transcription factor ChIP peaks. finding
- ★ Sites that gain methylation with age are associated with bivalent promoters of genes not expressed in blood. finding
- ★ Age-associated increases in methylation may not have strong functional consequences on cell states. mechanism
- ★ As more explanatory factors are added to the MSEPM, the model's ability to fit both the factors and methylation values improves. finding
- MSEPM is released as an open-source Python package with scikit-learn style syntax under an MIT license. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| DNA methylation profiling (array-based, aggregate cohort) | human blood tissue (large aggregate cohort) | none/observational | site-specific methylation values modeled jointly against age, sex, and cell-type heterogeneity factors | — |
| simulation framework based on MSEPM formulation | simulated methylation datasets | none (parametric simulation of factor magnitude and non-linear transformation) | accuracy of recovered site parameters and individual factor estimates versus known simulated values | — |
- – MSEPM models capture a large fraction of the variability at thousands of DNA methylation sites in the blood cohort
- ▼ Sites losing methylation over time are enriched for CTCF transcription factor ChIP peaks
- ▲ Sites gaining methylation over time are associated with bivalent promoters of genes not expressed in blood
- ▲ Adding more explanatory factors to the MSEPM improves the ability to model both the factors and the methylation values
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper introduces the Multi-State Epigenetic Pacemaker (MSEPM), which models observed DNA methylation at each CpG site as a weighted linear combination of multiple individual-level epigenetic factors plus a normally distributed error term. Model fitting uses a conditional expectation maximization (CEM) algorithm that alternates between OLS regression to estimate site-specific rate coefficients and intercepts, and gradient descent to update individual epigenetic factors, iterating until reduction in total RSS falls below a threshold or a maximum iteration count is reached. The approach was validated using a purpose-built simulation framework and then applied to a large aggregate blood-tissue methylation cohort to model age, sex, and cell-type effects simultaneously. Downstream analyses reported enrichment of directionally age-associated CpG sites for CTCF ChIP-seq peaks and bivalent chromatin states.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Ordinary least squares (OLS) regression | Estimation of site-specific rate coefficients (r_i,k) and intercepts (r^0_i) at each iteration of the CEM algorithm during MSEPM fitting | — | not stated |
| Gradient descent optimization | Update of individual-level epigenetic factors (p_j,k) given fixed site coefficients at each CEM iteration | — | not stated |
| Residual sum of squares (RSS) minimization | Global objective function for MSEPM fitting across all sites and individuals; convergence criterion for the CEM algorithm | — | not stated |
| Enrichment analysis (specific test not stated in available text) | Testing whether sites losing methylation with age are enriched for CTCF ChIP-seq peaks and whether sites gaining methylation are enriched for bivalent promoters of lowly expressed genes | — | not stated |
| Simulation-based performance evaluation | Benchmarking MSEPM accuracy under varying numbers of factors, noise levels, and sample sizes using the MSEPM simulation framework | — | not stated |
-
Site-specific coefficients are estimated with OLS within each CEM iteration, without regularization↳ Could also: Penalized regression (ridge, lasso, or elastic net) at each site within the CEM steps could also be used — Regularization would shrink noisy site-level estimates toward zero, which may reduce overfitting when signal-to-noise varies widely across thousands of CpG sites — a consideration that the paper itself raises in the context of traditional epigenetic clocks
-
Multiple biologically interpretable factors (age, sex, cell type) are specified a priori and jointly modeled in a linear additive framework↳ Could also: Unsupervised matrix factorization methods such as non-negative matrix factorization (NMF), independent component analysis (ICA), or probabilistic PCA could also decompose the methylation matrix into latent factor matrices without pre-specifying factors — Latent-factor approaches can recover structure beyond the pre-specified variables and serve as a complement or benchmark, though factors require post-hoc biological annotation to interpret
-
Model accuracy is assessed primarily via RSS on the training data↳ Could also: k-fold cross-validation reporting held-out RMSE, MAE, or R² could also quantify generalization to unseen individuals — Training-set RSS reflects optimization convergence rather than predictive generalization; held-out metrics more directly address whether the model captures portable biological signal rather than data-specific noise
-
Enrichment of age-associated CpG subsets for CTCF peaks and bivalent promoters is described, but the statistical test is not named in the available text↳ Could also: A hypergeometric test, Fisher's exact test, or permutation-based genomic enrichment test (e.g., as implemented in GREAT, regioneR, or similar tools) could each be used to formally quantify enrichment significance with an accompanying multiple-testing correction — Naming the enrichment test and its correction method allows readers to evaluate the statistical basis for functional-annotation claims, especially when many annotations are tested in parallel
-
The cohort is described as a large aggregate of blood methylation data pooled across studies↳ Could also: Explicit batch-effect harmonization (e.g., ComBat, limma::removeBatchEffect, or surrogate variable analysis) could also be applied before or alongside MSEPM fitting to separate biological from technical inter-study variation — In aggregate cohorts spanning multiple platforms or processing pipelines, study-of-origin effects can inflate or confound estimated factor associations, so explicit harmonization or inclusion of batch as a covariate is a common complementary step
-
The MSEPM learns a non-linear transformation of observable factors (e.g., age) internally via CEM, mapping them to linear epigenetic factors↳ Could also: Generalized additive models (GAMs) with penalized splines, or explicit polynomial/log transformations applied before linear modeling, could also accommodate non-linear age–methylation relationships with directly interpretable smooth terms — Explicit non-linear basis functions make the nature of the age–methylation trajectory transparent and comparable across studies, whereas the learned internal transformation in the MSEPM is implicit and dataset-specific
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
Assessed papers, coloured by verdict. Click a node to open it.
- No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-39549198
Paper: Farrell C, Tandon K, Ferrari R, Lapborisuth K, Modi R, Snir S, Pellegrini M. "The Multi-State Epigenetic Pacemaker enables the identification of combinations of factors that influence DNA methylation." GeroScience 2024. PMID 39549198 · PMCID PMC11979089 · DOI 10.1007/s11357-024-01414-7.
Code (pinned):
- MSEPM package — github.com/NuttyLogic/MultistateEpigeneticPacemaker @
86ed892(v0.0.1, MIT) - Simulation util — github.com/NuttyLogic/MethSim @
e728444(methsim v0.0.2) - Analysis/manuscript — github.com/NuttyLogic/MSEPMManuscript @
a8fa882(Analysis/MSEPMSimulations.ipynb, Analysis/MSEPMBloodModel.ipynb)
Data: GEO GSE87640 (+ 16 more 450K series, n=5,687 after QC for the blood model).
Pipeline-derived results
IN SCOPE — reproduced (self-contained simulation, Figure 2)
The simulation study (MSEPMSimulations.ipynb) is fully self-contained: it
generates synthetic 450K-like methylation with methsim and fits the published
msepm model — NO external data required. This is the cleanest, well-specified
pipeline output and is the core demonstration that the MSEPM recovers known factors.
| id | reported result | location | pipeline |
|---|---|---|---|
| C1 | 4-factor model predicts simulated methylation R²=0.995, MAE=0.015 | Fig 2 panel G | methsim → msepm.fit/predict |
| C2 | site-rate (rᵢ) recovery: per-factor R² ≈ 1.00/1.00/0.99/0.98, slopes 0.92/0.99/1.27/1.57 | Fig 2 C–F | avg site coefs vs simulated truth |
| C3 | factor MAE decreases as factors are added (Age strongly) | Fig 2 A | nested 1→4-factor models |
| C4 | MSEPM package implementation correctness | repo tests/ |
pytest unit tests |
Parameters fixed by Methods/notebook: learning_rate=0.01, iter_limit=200, 1000 sites, 1000 samples (500 train / 500 test), 1000 simulation iterations, 4 factors (Age√, Sex/binary, Continuous-Normal, Continuous-Exponential).
OUT OF SCOPE — not attempted (hard last 20%)
| reported result | location | why not attempted |
|---|---|---|
| Blood MSEPM predicts methylation R²=0.935, MAE=0.035 | Fig 4E | requires DataProcessing/ProcessedData/processed_sample_meta.pkl + a minfi-processed methylation matrix for 5,687 samples across 17 GEO 450K series — this processed intermediate is gitignored / not shipped, and re-deriving it means downloading + minfi-normalising 17 raw IDAT datasets (hundreds of GB). Classic hard-20%. |
| Validation R²=0.933, MAE=0.035 | text | same dependency |
| Hierarchical clustering → 10 clusters, cluster 7 = 93 sites | Fig 5 | downstream of the blood model above |
| TF enrichment (LOLA / ENCODE V4) | Fig 5 | downstream + external ENCODE DB |
| H3K4me3 / chromatin (deepTools2, IGV, Cistrome) | Fig 6–7 | external genomics, out of pipeline scope |
Out-of-scope = NOT attempted, NOT a mismatch. The blood-model numbers are derivable in principle from the listed GEO accessions, but the shipped repo does not include the processed matrix needed to reproduce them at low cost (80/20).
Compute
All on «our HPC» («infra» SLURM); env + data on «infra»
«path». «host» holds results only.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The in-scope, self-contained simulation (Figure 2) reproduces essentially 1:1 from the shipped code — Panel G R2=0.9945/MAE=0.0142 vs reported 0.995/0.015, per-factor site-rate R2≈1.000, and the Panel A factor-MAE decrease all hold; the only notable deviation is the Fig2C-F CE slope (2.03 vs 1.57), explainable by the random site-rate draw. The blood-tissue claims (Fig4 R2=0.935, 778 sites, Fig5 clusters) were not reproduced — this is a data-availability gap on the authors' deposition side (the minfi-processed matrix is gitignored), not a discrepancy or fabrication signal. Deviations are technical/stochastic; the methodological core is confirmed while the empirical application remains untested, so quality is solid-but-partial rather than full 1:1.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at [email protected].
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.