ResnetAge: A Resnet-Based DNA Methylation Age Prediction Method.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- 🟡Reported values were only indirectly comparable
- 🔴A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🔴Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🔴The central claim did not (fully) hold under reproduction
- 🔴Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce the BASELINE, not the OWN method. GSE57484 (22 male preadolescents, 27K, per-sample ages 9.84-11.79 yr present in GEO) was downloaded on «infra» and the Horvath-clock baseline that the paper reports in Table 1 (MAE 8.63) was reproduced EXACTLY with methylclock 1.16.0 (MAE 8.633) -> the baseline column is faithful, and the authors almost certainly used methylclock. A second independent Horvath tool (pyaging) gave MAE 4.09, a documented cross-implementation sensitivity caused by different missing-CpG imputation on 27K data (not a fabrication). The paper's OWN ResnetAge number (6.93) is NOT reproducible: the repo (github.com/52hai/ResnetAge @3603ecc) ships only a training script that reads the authors' private local CSVs («path») for 32 training datasets - no trained weights, no inference/test code, no Table-1 evaluation code, and GSE57484 is not referenced in the code at all. NOT ATTEMPTED (hard-20%, brief rule 3): retraining ResnetAge from scratch and re-implementing the never-shipped test pipeline; this is also why 6.93 is flagged as a possible-fabrication SURFACE (value not derivable from shipped data/code) for human audit, not asserted as fabricated. Pearson r is uninformative for this dataset (age range <2 yr). Grades provisional; a human reviewer decides.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 75assessed: 2026-06-14 ⛓ 618fce4cacad
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-09-19
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper tests whether a ResNet-based deep learning model trained on CpG methylation beta values can predict DNA methylation (epigenetic) age more accurately than existing linear-regression-based epigenetic clocks (Horvath, Hannum, ZhangAge).
- ★ ResnetAge, a ResNet-based neural network using 22,278 shared Illumina 27K/450K CpG sites, predicts DNA methylation age from beta values. method
- ★ ResnetAge achieves higher age-prediction accuracy than the Horvath, Hannum, and ZhangAge epigenetic clocks on most independent test datasets. finding
- ★ DNA methylation-age relationships are non-linear, so a deep residual-learning model can extract higher-level features that linear regression clocks cannot. mechanism
- ResnetAge was trained on 32 public GEO datasets (multiple tissues: whole blood, saliva, mouth, etc.) and evaluated on 8 independent test datasets. method
- ★ ResnetAge performs worse on datasets containing disease samples (e.g., Alzheimer's, GSE53740) and on tissues absent from its training set (e.g., uterine cervix, GSE30758). finding
- Missing methylation values in downloaded datasets were imputed using the methyLImp method prior to model training. method
- Residual blocks in the ResNet architecture improve learning of residual information and overall model performance/interpretability via convolution. mechanism
- ★ ResnetAge shows more stable, tightly distributed prediction errors across six different tissues compared to linear-regression-based clocks. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| DNA methylation microarray profiling (Illumina Infinium HumanMethylation27, 27K) | Human tissue samples (whole blood, saliva, mouth, etc.) from GEO datasets | none | CpG site beta values used as input for age prediction | Illumina Infinium HumanMethylation27 (27K) |
| DNA methylation microarray profiling (Illumina Infinium HumanMethylation450, 450K) | Human tissue samples (whole blood, saliva, mouth, etc.) from GEO datasets | none | CpG site beta values used as input for age prediction | Illumina Infinium HumanMethylation450 (450K) |
| ResNet deep neural network training/validation for age regression | 22,278 shared CpG sites from 11,933 samples across 32 training datasets (70:30 train/validation split) | none | Predicted age vs. actual chronological age (MAE, MAD) | Custom ResNet model, 14 convolutional layers with 5 residual blocks, ELU activation, batch normalization, batch size 80 |
| Comparative age prediction benchmarking across tissue types | 8 independent GEO test datasets (3x 27K, 5x 450K) covering whole blood, serum, saliva, buccal, uterine cervix, and T cell tissue | disease state (e.g., Alzheimer's in GSE53740) vs. healthy samples | Prediction error (predicted age − actual age) compared across ResnetAge, Horvath, Hannum, and ZhangAge clocks | — |
- ▼ ResnetAge training set accuracy: MAE 1.29 years, MAD 0.98 years MAE=1.29 years
- ▼ ResnetAge validation set accuracy: MAE 3.24 years, MAD 2.3 years MAE=3.24 years
- ▼ ResnetAge outperformed the Horvath epigenetic clock on most of the 3 (27K) and 5 (450K) independent test datasets
- ▼ ResnetAge outperformed the Hannum epigenetic clock on most 450K array datasets
- – ResnetAge outperformed the ZhangAge predictor on most datasets, except slightly worse on saliva (GSE111223) and whole blood (GSE53740) samples
- ▲ ResnetAge performed worse on GSE53740 (diseased/Alzheimer's whole blood) and GSE30758 (uterine cervix, untrained tissue)
- – ResnetAge prediction errors densely distributed around zero across six tissues in violin plot analysis, more stable than linear clocks
- mean MAE = 1.29 years (ResnetAge training set absolute error)
- other MAD = 0.98 years (ResnetAge training set median absolute deviation)
- mean MAE = 3.24 years (ResnetAge validation set absolute error)
- other MAD = 2.3 years (ResnetAge validation set median absolute deviation)
- count 22,278 CpG sites (Overlapping CpG sites between Illumina 27K and 450K arrays used as model input)
- count 11,933 samples (Total samples across 40 GEO datasets, age range 0.17–114 years)
- count 32 training datasets / 8 independent test datasets (Dataset split for model training vs. comparative testing)
- correlation R2 = 0.96, MAD = 3.6 years (Horvath epigenetic clock (reference/prior work, 51 tissues))
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
ResnetAge is a deep-learning regression model (14-layer ResNet) trained on 32 public GEO DNA methylation datasets (11,933 total samples, 22,278 CpG sites) to predict chronological age from beta-value inputs. Model fit is quantified as MAE and MAD on a single random internal 70/30 train-validation split. Generalization is assessed descriptively by comparing per-dataset MAE and MAD on eight held-out independent datasets against three published epigenetic clocks (Horvath, Hannum, ZhangAge), and prediction-error distributions across tissue types are visualized as violin plots with no accompanying formal inference.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Mean Absolute Error (MAE) — regression prediction error metric | Internal training set and internal validation set (70/30 random split of the 32 training datasets) | Total training pool ~11,933 samples across 32 datasets; exact per-split n not stated | not stated |
| Median Absolute Deviation (MAD) — robust regression prediction error metric | Internal training set and internal validation set | — | not stated |
| Descriptive numeric comparison of MAE and MAD across four methods — no formal hypothesis test | Table 1: eight independent external test datasets (3 Illumina 27K, 5 Illumina 450K) compared across ResnetAge, Horvath, Hannum, and ZhangAge | Eight datasets; per-dataset sample sizes not stated | na |
| Violin plot visualization of prediction error (predicted minus actual age) by tissue | Figure 5: six tissue categories (whole blood, serum, saliva, buccal, uterine cervix, T cell) across four methods | — | na |
-
Performance across 8 independent datasets was compared between methods by inspecting MAE/MAD point estimates in a table, without any formal test for whether differences were consistent or could reflect sampling variability.↳ Could also: Bootstrap confidence intervals around MAE (resampling per-sample absolute errors within each dataset) or a Wilcoxon signed-rank test on paired absolute errors across datasets could also quantify whether one method's advantage is statistically consistent. — Point-estimate MAE differences can reflect dataset-specific composition (tissue type, disease status, sample size) rather than a systematic method advantage; an interval or non-parametric paired test would help distinguish the two.
-
Internal performance was estimated from a single random 70/30 split of the pooled training data.↳ Could also: Leave-one-dataset-out cross-validation (LODOCV) or k-fold cross-validation stratified at the dataset level could also be used to estimate generalization error. — Because samples are drawn from heterogeneous GEO datasets with potential batch effects, a dataset-level holdout preserves batch structure and more closely mirrors the independent-test-set scenario, reducing optimistic bias from within-batch correlations.
-
Prediction accuracy was summarized with MAE and MAD only; correlation-based metrics were not reported for ResnetAge.↳ Could also: Pearson r (or R²) between predicted and actual age, and RMSE, are also standard metrics used by the Horvath (R²=0.96) and Hannum (RMSE=3.9 yr) papers and would enable direct numeric comparison on a shared scale. — Reporting r or R² alongside MAE separates systematic linear agreement from average absolute deviation, and allows like-for-like benchmarking against the published baseline clocks that report R² or RMSE rather than MAD.
-
Tissue-level prediction error distributions were compared across four methods using violin plots alone, with no formal statistical comparison.↳ Could also: A linear mixed-effects model with method and tissue type as fixed effects and dataset as a random effect could also formally estimate tissue-specific performance differences and their uncertainty. — A mixed model accounts for the nested data structure (samples within datasets, datasets within tissue categories) and provides estimates with confidence intervals, separating tissue effects from dataset-level variability.
-
Missing CpG beta values were imputed with methyLImp and no sensitivity analysis across imputation strategies was reported.↳ Could also: k-nearest-neighbor imputation or mean imputation, with a comparison of downstream MAE under each strategy, could also be reported. — Different imputation methods can shift the beta-value distribution systematically; reporting performance under alternative strategies would help readers assess how sensitive age predictions are to this preprocessing choice.
-
No uncertainty (SE, CI) is quantified around the reported MAE and MAD values themselves.↳ Could also: Bootstrap resampling of test-set predictions could also be used to construct 95% CIs around MAE and MAD for each method on each dataset. — Confidence intervals on error metrics make it possible to judge whether an observed difference in MAE (e.g., 2.153 yr vs. 3.24 yr) is likely to replicate or falls within sampling uncertainty, which varies with dataset size.
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
Assessed papers, coloured by verdict. Click a node to open it.
- No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
scope.md — pmid-38247911 (ResnetAge)
Publication
ResnetAge: A Resnet-Based DNA Methylation Age Prediction Method. Shi L, Hai B, Kuang Z, Wang H, Zhao J. Bioengineering 2023. PMID 38247911 / PMC10813502 / DOI 10.3390/bioengineering11010034.
Repo inspection (github.com/52hai/ResnetAge, cloned to «infra»)
- Repo ships exactly ONE file:
ResnetAge.py(332 lines) + minimal README. ResnetAge.pyis a training-only script:- Reads
*_beta.csv+*_pheno.csvfrom a hardcoded private path«path»for 32 GEO TRAINING datasets (geo_checklist). - Trains ResNet (Adam, lr=0.1, 250 epochs, batch=100) on a 62.5/37.5 split.
- Prints train/test MAE/MSE/MAD on that split.
- Reads
- No trained model weights are shipped (no torch.save / checkpoint).
- No independent-test / inference code: GSE57484 (and the other 7 test sets) and the Horvath/Hannum/ZhangAge baseline comparison from Table 1 are NOT in the repo.
- Note: shipped code uses batch=100 (paper says 80) and ResNet num_blocks=[2,1,1,1]; minor discrepancies vs the paper's "14 conv layers + 5 residual blocks" description.
Reported results for OUR assigned dataset (GSE57484), Table 1
- ResnetAge MAE = 6.93 (27K array, GSE57484)
- Horvath MAE = 8.63
- Hannum / ZhangAge = "—" (not applied to 27K test sets)
Dataset (GSE57484) — confirmed obtainable & has ground truth
- 22 whole-blood samples, all male Greek preadolescents.
- Per-sample chronological age IS present in the series matrix (range 9.84–11.79 yr, mean ≈10.5). Platform GPL8490 (Illumina HumanMethylation27, 27K).
- Series matrix (beta values) + suppl matrix downloaded to «infra».
IN SCOPE (pipeline-derived, reproducible)
- C1 — Horvath clock MAE on GSE57484 (reported 8.63). Pipeline: published deterministic Horvath-2013 multi-tissue clock (353 CpGs) applied to GSE57484 beta values vs real per-sample ages. Third-party tool (pyaging / methylclock). This is a baseline the AUTHORS computed and reported; it is the clean 1:1 check.
OUT OF SCOPE / NOT ATTEMPTED (the hard 20%, documented honestly)
- ResnetAge MAE = 6.93 on GSE57484 — NOT reproducible from shipped artifacts: no trained weights, no inference/test-set code, training inputs are private local CSVs. Retraining from the 32 GEO sets + re-deriving the never-shipped test pipeline is the hard ~20% and cannot reproduce a single specific trained-model number exactly anyway. Recorded as a possible-fabrication-surface note: the 6.93 is not derivable from the shipped data/code (a human auditor must judge).
- Training-set/validation MAE (1.29 / 3.24) — would require retraining on private data; not attempted.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The paper's Horvath baseline (MAE 8.63) reproduces exactly with methylclock 1.16.0 (8.633), so the baseline column is faithful. However, the paper's own headline result, ResnetAge MAE 6.93, is not derivable from the shipped artifacts: the repo contains only a training script reading private local CSVs, with no weights, no inference/test code, and no reference to GSE57484 — so the central claim that ResnetAge beats Horvath cannot be confirmed. The defect is on the authors' side (incomplete code/data deposition), and the unverifiable headline value is flagged as a possible-fabrication surface for human audit. A documented cross-tool caveat (pyaging Horvath = 4.09) is an implementation-sensitivity artifact on 27K data, not evidence against the paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at [email protected].
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.