ResnetAge: A Resnet-Based DNA Methylation Age Prediction Method.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- 🟡Reported values were only indirectly comparable
- 🔴A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🔴Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🔴The central claim did not (fully) hold under reproduction
- 🔴Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce the BASELINE, not the OWN method. GSE57484 (22 male preadolescents, 27K, per-sample ages 9.84-11.79 yr present in GEO) was downloaded on «infra» and the Horvath-clock baseline that the paper reports in Table 1 (MAE 8.63) was reproduced EXACTLY with methylclock 1.16.0 (MAE 8.633) -> the baseline column is faithful, and the authors almost certainly used methylclock. A second independent Horvath tool (pyaging) gave MAE 4.09, a documented cross-implementation sensitivity caused by different missing-CpG imputation on 27K data (not a fabrication). The paper's OWN ResnetAge number (6.93) is NOT reproducible: the repo (github.com/52hai/ResnetAge @3603ecc) ships only a training script that reads the authors' private local CSVs («path») for 32 training datasets - no trained weights, no inference/test code, no Table-1 evaluation code, and GSE57484 is not referenced in the code at all. NOT ATTEMPTED (hard-20%, brief rule 3): retraining ResnetAge from scratch and re-implementing the never-shipped test pipeline; this is also why 6.93 is flagged as a possible-fabrication SURFACE (value not derivable from shipped data/code) for human audit, not asserted as fabricated. Pearson r is uninformative for this dataset (age range <2 yr). Grades provisional; a human reviewer decides.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 75assessed: 2026-06-14 ⛓ 618fce4cacad
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a ResNet-based deep learning model using 22,278 CpG sites common to Illumina 27K and 450K arrays predict DNA methylation (biological) age more accurately and with better generalization than existing linear-regression and deep-learning epigenetic clocks?
- ★ ResnetAge, a ResNet-based deep neural network, predicts DNA methylation age from 22,278 CpG sites common to Illumina 27K and 450K arrays method
- ★ ResnetAge achieves higher age-prediction accuracy than other methylation-based methods (Horvath, Hannum, ZhangAge clocks) finding
- ★ The deep learning ResNet model captures nonlinear relationships between DNA methylation and age that linear regression clocks cannot mechanism
- ★ ResnetAge shows more stable predictions with smaller, near-zero-centered errors across diverse tissues than linear-regression clocks finding
- ResnetAge provides a trained model supporting both Illumina 27K and 450K identification frameworks, trained on 32 public datasets resource
- Prediction accuracy degrades on datasets containing diseased samples or tissue types not represented in training (e.g., GSE53740, GSE30758) finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| DNA methylation array (Illumina Infinium HumanMethylation27) | human tissues (whole blood, saliva, mouth, etc.) | none | beta methylation values at 22,278 CpG sites used to predict age | Illumina Infinium HumanMethylation27 (27K) |
| DNA methylation array (Illumina Infinium HumanMethylation450) | human tissues (whole blood, serum, saliva, buccal, uterine cervix, T cell) | none | beta methylation values at 22,278 CpG sites used to predict age | Illumina Infinium HumanMethylation450 (450K) |
| ResNet deep neural network training (14 convolutional layers, 5 residual blocks) | 32 GEO training datasets, 11,933 samples total (age 0.17–114 years) | none | predicted age (MAE, MAD) | Resnet model; methyLImp imputation; ELU activation; batch normalization; dropout; batchsize 80 |
| Independent test-set comparison vs Horvath/Hannum/ZhangAge clocks | 8 independent GEO datasets (three 27K, five 450K) | none | age prediction error per dataset and per tissue | — |
- – Training set DNA methylation age prediction error MAE 1.29 years; MAD 0.98 years
- – Validation set DNA methylation age prediction error MAE 3.24 years; MAD 2.3 years
- ▲ ResnetAge outperforms Horvath, Hannum, and ZhangAge clocks on most independent datasets
- – ResnetAge prediction errors are densely distributed around 0 with small span across six tissues, more stable than linear clocks
- – Poorer performance on GSE53740 (whole blood with Alzheimer's/disease samples) and GSE30758 (uterine cervix not in training)
- count 22,278 CpG sites (CpG sites overlapping between Illumina 27K and 450K arrays used as model input)
- count 11,933 samples (total samples across 40 datasets, age range 0.17 to 114 years)
- mean MAE 1.29 years (Mean Absolute Error on training set)
- other MAD 0.98 years (Median Absolute Deviation on training set)
- mean MAE 3.24 years (Mean Absolute Error on validation set)
- other MAD 2.3 years (Median Absolute Deviation on validation set)
- correlation R2 = 0.96 (Horvath clock DNAm age vs actual age (cited prior work, 8000 samples, 51 tissues))
- other RMSE 2.04 years (ZhangAge predictor reported error (cited prior work))
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
ResnetAge is a deep-learning regression model (14-layer ResNet) trained on 32 public GEO DNA methylation datasets (11,933 total samples, 22,278 CpG sites) to predict chronological age from beta-value inputs. Model fit is quantified as MAE and MAD on a single random internal 70/30 train-validation split. Generalization is assessed descriptively by comparing per-dataset MAE and MAD on eight held-out independent datasets against three published epigenetic clocks (Horvath, Hannum, ZhangAge), and prediction-error distributions across tissue types are visualized as violin plots with no accompanying formal inference.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Mean Absolute Error (MAE) — regression prediction error metric | Internal training set and internal validation set (70/30 random split of the 32 training datasets) | Total training pool ~11,933 samples across 32 datasets; exact per-split n not stated | not stated |
| Median Absolute Deviation (MAD) — robust regression prediction error metric | Internal training set and internal validation set | — | not stated |
| Descriptive numeric comparison of MAE and MAD across four methods — no formal hypothesis test | Table 1: eight independent external test datasets (3 Illumina 27K, 5 Illumina 450K) compared across ResnetAge, Horvath, Hannum, and ZhangAge | Eight datasets; per-dataset sample sizes not stated | na |
| Violin plot visualization of prediction error (predicted minus actual age) by tissue | Figure 5: six tissue categories (whole blood, serum, saliva, buccal, uterine cervix, T cell) across four methods | — | na |
-
Performance across 8 independent datasets was compared between methods by inspecting MAE/MAD point estimates in a table, without any formal test for whether differences were consistent or could reflect sampling variability.↳ Could also: Bootstrap confidence intervals around MAE (resampling per-sample absolute errors within each dataset) or a Wilcoxon signed-rank test on paired absolute errors across datasets could also quantify whether one method's advantage is statistically consistent. — Point-estimate MAE differences can reflect dataset-specific composition (tissue type, disease status, sample size) rather than a systematic method advantage; an interval or non-parametric paired test would help distinguish the two.
-
Internal performance was estimated from a single random 70/30 split of the pooled training data.↳ Could also: Leave-one-dataset-out cross-validation (LODOCV) or k-fold cross-validation stratified at the dataset level could also be used to estimate generalization error. — Because samples are drawn from heterogeneous GEO datasets with potential batch effects, a dataset-level holdout preserves batch structure and more closely mirrors the independent-test-set scenario, reducing optimistic bias from within-batch correlations.
-
Prediction accuracy was summarized with MAE and MAD only; correlation-based metrics were not reported for ResnetAge.↳ Could also: Pearson r (or R²) between predicted and actual age, and RMSE, are also standard metrics used by the Horvath (R²=0.96) and Hannum (RMSE=3.9 yr) papers and would enable direct numeric comparison on a shared scale. — Reporting r or R² alongside MAE separates systematic linear agreement from average absolute deviation, and allows like-for-like benchmarking against the published baseline clocks that report R² or RMSE rather than MAD.
-
Tissue-level prediction error distributions were compared across four methods using violin plots alone, with no formal statistical comparison.↳ Could also: A linear mixed-effects model with method and tissue type as fixed effects and dataset as a random effect could also formally estimate tissue-specific performance differences and their uncertainty. — A mixed model accounts for the nested data structure (samples within datasets, datasets within tissue categories) and provides estimates with confidence intervals, separating tissue effects from dataset-level variability.
-
Missing CpG beta values were imputed with methyLImp and no sensitivity analysis across imputation strategies was reported.↳ Could also: k-nearest-neighbor imputation or mean imputation, with a comparison of downstream MAE under each strategy, could also be reported. — Different imputation methods can shift the beta-value distribution systematically; reporting performance under alternative strategies would help readers assess how sensitive age predictions are to this preprocessing choice.
-
No uncertainty (SE, CI) is quantified around the reported MAE and MAD values themselves.↳ Could also: Bootstrap resampling of test-set predictions could also be used to construct 95% CIs around MAE and MAD for each method on each dataset. — Confidence intervals on error metrics make it possible to judge whether an observed difference in MAE (e.g., 2.153 yr vs. 3.24 yr) is likely to replicate or falls within sampling uncertainty, which varies with dataset size.
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
Assessed papers, coloured by verdict. Click a node to open it.
- No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
scope.md — pmid-38247911 (ResnetAge)
Publication
ResnetAge: A Resnet-Based DNA Methylation Age Prediction Method. Shi L, Hai B, Kuang Z, Wang H, Zhao J. Bioengineering 2023. PMID 38247911 / PMC10813502 / DOI 10.3390/bioengineering11010034.
Repo inspection (github.com/52hai/ResnetAge, cloned to «infra»)
- Repo ships exactly ONE file:
ResnetAge.py(332 lines) + minimal README. ResnetAge.pyis a training-only script:- Reads
*_beta.csv+*_pheno.csvfrom a hardcoded private path«path»for 32 GEO TRAINING datasets (geo_checklist). - Trains ResNet (Adam, lr=0.1, 250 epochs, batch=100) on a 62.5/37.5 split.
- Prints train/test MAE/MSE/MAD on that split.
- Reads
- No trained model weights are shipped (no torch.save / checkpoint).
- No independent-test / inference code: GSE57484 (and the other 7 test sets) and the Horvath/Hannum/ZhangAge baseline comparison from Table 1 are NOT in the repo.
- Note: shipped code uses batch=100 (paper says 80) and ResNet num_blocks=[2,1,1,1]; minor discrepancies vs the paper's "14 conv layers + 5 residual blocks" description.
Reported results for OUR assigned dataset (GSE57484), Table 1
- ResnetAge MAE = 6.93 (27K array, GSE57484)
- Horvath MAE = 8.63
- Hannum / ZhangAge = "—" (not applied to 27K test sets)
Dataset (GSE57484) — confirmed obtainable & has ground truth
- 22 whole-blood samples, all male Greek preadolescents.
- Per-sample chronological age IS present in the series matrix (range 9.84–11.79 yr, mean ≈10.5). Platform GPL8490 (Illumina HumanMethylation27, 27K).
- Series matrix (beta values) + suppl matrix downloaded to «infra».
IN SCOPE (pipeline-derived, reproducible)
- C1 — Horvath clock MAE on GSE57484 (reported 8.63). Pipeline: published deterministic Horvath-2013 multi-tissue clock (353 CpGs) applied to GSE57484 beta values vs real per-sample ages. Third-party tool (pyaging / methylclock). This is a baseline the AUTHORS computed and reported; it is the clean 1:1 check.
OUT OF SCOPE / NOT ATTEMPTED (the hard 20%, documented honestly)
- ResnetAge MAE = 6.93 on GSE57484 — NOT reproducible from shipped artifacts: no trained weights, no inference/test-set code, training inputs are private local CSVs. Retraining from the 32 GEO sets + re-deriving the never-shipped test pipeline is the hard ~20% and cannot reproduce a single specific trained-model number exactly anyway. Recorded as a possible-fabrication-surface note: the 6.93 is not derivable from the shipped data/code (a human auditor must judge).
- Training-set/validation MAE (1.29 / 3.24) — would require retraining on private data; not attempted.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The paper's Horvath baseline (MAE 8.63) reproduces exactly with methylclock 1.16.0 (8.633), so the baseline column is faithful. However, the paper's own headline result, ResnetAge MAE 6.93, is not derivable from the shipped artifacts: the repo contains only a training script reading private local CSVs, with no weights, no inference/test code, and no reference to GSE57484 — so the central claim that ResnetAge beats Horvath cannot be confirmed. The defect is on the authors' side (incomplete code/data deposition), and the unverifiable headline value is flagged as a possible-fabrication surface for human audit. A documented cross-tool caveat (pyaging Horvath = 4.09) is an implementation-sensitivity artifact on 27K data, not evidence against the paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.