Systems Analysis Reveals Ageing-Related Perturbations in Retinoids and Sex Hormones in Alzheimer's and Parkinson's Diseases.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Any deviation was negligible
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Reproduction of Lam et al. 2021 (Biomedicines, retinoids/sex-hormones in AD/PD). The repo is the authors' own code. Most headline numbers are NOT independently reproducible from shipped artifacts: the human AD/PD branch depends on access-restricted ROSMAP data (Synapse DUA), and all GEM / reporter-metabolite results (incl. zebrafish Table 5, 9-cis-RA p=0.044567) require a proprietary MATLAB toolchain (COBRA/RAVEN/tINIT) — both recorded as controlled drops, no fabrication asserted. The one fully-open, public-data branch — zebrafish brain RNA-seq DE (GSE102426 = SRP115132, 13 single-end runs) via the authors' kallisto+DESeq2 — was reproduced on «our HPC»: the sample design n=5/5/3 matches the paper EXACTLY (C1, clean 1:1 from GEO/ENA metadata), and the DESeq2 pipeline executes cleanly (C2), graded partial because the paper prints no zebrafish DE count to match 1:1. Net verdict: PARTIAL — the reproducible portion is reproducible (and exact where a published value exists), while the bulk is under-reproducible due to restricted data and proprietary tooling rather than evident error. Technical divergence noted (DESeq2 version; biomaRt-join count inflation in the original script). Provisional grade; human reviewer decides via AUDIT.md.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 75assessed: 2026-06-14 ⛓ b4cf8d3b98d6
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetGiven clinical, pathological, and ageing-related commonalities between Alzheimer's disease (AD) and Parkinson's disease (PD), the paper tests whether integrating transcriptomic and genome-scale metabolic modelling data can stratify AD/PD patients into molecularly distinct subclasses and reveal ageing-related metabolic perturbations (particularly in retinoids and sex hormones) underlying disease heterogeneity.
- ★ AD and PD patients can be stratified by transcriptomic clustering into three subclasses with distinct gene expression and metabolic profiles finding
- ★ Retinoids are a key ageing-related feature dysregulated across all three subclasses of AD and PD finding
- ★ Dysregulation of androgen biosynthesis/metabolism occurs via three different independent mechanisms and is a source of heterogeneity between AD/PD subclasses finding
- A brain-specific genome-scale metabolic model (iBrain2845) was constructed from the adipocyte GEM iAdipocytes1850 and the generic human GEM HMR3/HMR2.0 resource
- Cluster-specific metabolic models (iADPD1, iADPD2, iADPD3, iADPDControl) were reconstructed via tINIT using cluster consensus expression data resource
- A curated zebrafish reference GEM (ZebraGEM2.1) was generated from ZebraGEM2 for tert mutant analysis resource
- ★ Retinoid metabolism/function GO term is commonly altered in all subclasses: upregulated in cluster 1 but downregulated in clusters 2 and 3 finding
- ★ Each subclass shows a distinct perturbation within the androgen biosynthesis/metabolism pathway: oestradiol metabolism (cluster 1), cholesterol biosynthesis (cluster 2), and testosterone metabolism (cluster 3) finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| bulk RNA-seq / transcriptomics | human post-mortem brain tissue (ROSMAP, Rajkumar, Zhang/Zheng datasets; AD, PD, control) | disease (AD/PD) vs control | differential gene expression, gene set enrichment | kallisto; DESeq2; piano |
| single-cell RNA-seq (pseudo-bulk) | human brain samples (ROSMAP) | disease (AD/PD) vs control | pseudo-bulk expression profiles per sample | Cell Ranger |
| unsupervised clustering / gene set enrichment analysis | human brain transcriptomic data (AD/PD/control) | none (stratification analysis) | patient subclasses; enriched GO terms (up/downregulated) | ConsensusClusterPlus; piano |
| genome-scale metabolic modelling / flux balance analysis | human brain (cluster-specific GEMs iADPD1, iADPD2, iADPD3, iADPDControl) | disease subclass vs control | reaction flux changes; ATP synthesis-maximising FBA; reporter metabolites | RAVEN Toolbox 2.0 (tINIT algorithm) |
| gene co-expression network analysis | human brain (AD and PD non-blood, non-control samples) | disease (AD/PD) | co-expression modules, centrality distributions, community structure | Spearman correlation; Leiden algorithm; Erdős–Rényi null models; Enrichr/Revigo |
| RNA-seq / differential expression / gene set enrichment | zebrafish (Danio rerio), tert+/+, tert+/-, tert-/- (whole animal and dissected tissues) | tert (telomerase) nonsense mutation, genetic (heterozygous/homozygous) | differential gene expression; enriched GO terms | kallisto; DESeq2; piano |
| genome-scale metabolic modelling / reporter metabolite analysis | zebrafish tert mutants (ZebraGEM2.1) | tert mutation (heterozygous/homozygous) vs wild-type | reporter metabolites (FBA results not presented) | RAVEN Toolbox 2.0 |
- – 1572 samples (629 AD, 54 PD, 889 control) passed QC and were used for clustering into 3 disease subclasses plus a control cluster
- – Cluster 1 showed mixed up/downregulation vs control, cluster 2 showed more downregulation, cluster 3 showed vast downregulation of genes
- – Retinoid metabolism/function GO term enrichment was upregulated in cluster 1 but downregulated in clusters 2 and 3, appearing as a common feature across all subclasses
- – iADPD1 and iADPD2 both showed upregulated flux in cholesterol biosynthesis and downregulated flux in O-glycan metabolism, more pronounced in iADPD2
- ▲ iADPD1 uniquely showed upregulated flux in oestrogen metabolism and the Kandutsch–Russell pathway, including reactions HMR_2055 and HMR_2059 (oestrone to 2-hydroxyoestrone to 2-methoxyoestrone), which carried zero flux in iADPDControl
- ▲ iADPD2 uniquely showed upregulated cholesterol metabolism flux, including increased flux through HMR_1457 and HMR_1533 producing geranyl pyrophosphate and lathosterol (cholesterol precursors)
- – iADPD3 showed roughly equal up- and downregulation across several pathways including androgen metabolism, with decreased production of testosterone from 4-androstene-3,17-dione via HMR_1974 despite increased production of 4-androstene-3,17-dione via HMR_1971
- – DEGs across clusters 1-3 were globally enriched for upregulated immune response, olfaction, retinoid function, and apoptosis GO terms, and downregulated copper ion transport and telomere organisation GO terms compared to control
- count 629 AD samples, 54 PD samples, 889 control samples (final sample set after QC and normalisation used for clustering/DEG analysis)
- count 64,794 genes and 2055 samples after initial QC/normalisation, reduced to 1572 after removing non-AD/PD/control samples (data processing pipeline sample/gene filtering)
- pvalue Benjamini–Hochberg adjusted p-value ≤ 1 × 10^-10 (DESeq2 threshold for calling significantly differentially expressed genes)
- pvalue adjusted p-value ≤ 0.05 (distinct-directional and/or mixed-directional) (piano GSE analysis threshold for statistically significant GO terms)
- count n = 5 wild-type (tert+/+), n = 5 heterozygous mutant (tert+/-), n = 3 homozygous mutant (tert-/-) (zebrafish RNA-seq sample sizes for tert mutant ageing model)
- other resolution scan of 10,000 points between 10^-3 and 10; global maxima at resolutions 0.077526 (AD) and 0.089074 (PD) (Leiden algorithm CPMVertexPartition optimisation for co-expression network community detection)
- count Cluster 1: 127 female, 73 male; Cluster 2: 186 female, 95 male, 14 sex not recorded; Cluster 3: 114 female, 74 male (sex distribution of samples across the three disease subclasses)
- count Control cluster: 495 female, 262 male, 13 sex not recorded, 119 aggregate-source samples (composition of the artificially added control cluster)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This systems biology study integrated multi-dataset bulk and single-cell RNA-seq from human AD/PD post-mortem brain tissue and a zebrafish tert-mutant ageing model. Patients were stratified by unsupervised consensus clustering, followed by differential gene expression analysis (DESeq2 Wald test) comparing each cluster to a control cluster, gene set enrichment analysis (piano), genome-scale metabolic modelling with flux balance analysis, and Spearman-correlation-based co-expression network construction with Leiden community detection. Results were reported as enriched GO terms and qualitative flux comparisons rather than formal effect sizes or confidence intervals.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| DESeq2 Wald test (negative binomial GLM) | Differential gene expression: each disease cluster (1–3) vs. control cluster (human); tert−/− and tert+/− vs. tert+/+ (zebrafish) | 629 AD + 54 PD + 889 controls = 1572 human samples; n=5 tert+/+, n=5 tert+/−, n=3 tert−/− zebrafish | not stated |
| piano gene set enrichment analysis (multiple gene-level statistic aggregation methods) | GO term enrichment of DESeq2-identified DEGs per cluster (human and zebrafish); distinct-directional and mixed-directional methods | null | not stated |
| Spearman rank correlation | Pairwise gene co-expression network construction (AD network and PD network separately) | null | not stated |
| ConsensusClusterPlus consensus clustering | Unsupervised stratification of AD/PD samples into k=3 clusters | 683 AD/PD samples | not stated |
| Leiden algorithm (CPMVertexPartition optimisation) | Community detection in co-expression networks; resolution selected via 10,000-point scan | null | not stated |
| Reporter metabolite analysis (z-score aggregation) | Identification of significantly perturbed metabolites in each cluster-specific GEM and zebrafish GEM | null | not stated |
| Enrichr (Fisher exact / odds ratio-based) | Enrichment of co-expression network modules >30 nodes against GO Biological Process, KEGG, and OMIM libraries | null | not stated |
-
Cluster-specific consensus expression values for metabolic modelling were computed as arithmetic means across all samples in each cluster, with no measure of within-cluster variability reported↳ Could also: Median expression or a variance-weighted summary could also be used; reporting spread (SD or IQR) alongside the mean would characterise within-cluster heterogeneity — Because cluster membership spans multiple datasets and tissue regions, quantifying within-cluster dispersion would help assess whether mean-derived flux solutions are representative of the range of patient states captured by each cluster
-
Differential expression was performed with a single very stringent BH-adjusted p-value threshold (1×10−10) without reporting fold-change magnitudes or effect sizes↳ Could also: A combined threshold (e.g., |log2 fold-change| ≥ 1 AND BH-FDR ≤ 0.05) is also widely used; reporting shrinkage-estimated log2 fold-changes (available natively in DESeq2 via lfcShrink) would convey effect magnitude — With n=1572 samples, even biologically trivial differences can reach extreme statistical significance; pairing a fold-change filter with the p-value threshold helps distinguish statistically from practically meaningful changes
-
Co-expression networks were built from the top 1% of significant Spearman correlations without a formal multiple-testing correction for the pairwise correlation step↳ Could also: Weighted Gene Co-expression Network Analysis (WGCNA) with soft-thresholding, or applying FDR correction to all pairwise correlations before edge selection, are also standard approaches — A percentile cutoff retains a fixed edge density regardless of effect magnitude; WGCNA's soft threshold preserves continuous correlation strength and has established scale-free topology diagnostics for threshold selection
-
Patient stratification used ConsensusClusterPlus with k=3 selected for downstream analysis; the basis for choosing k=3 over other values is described only by reference to Figure S1↳ Could also: Cluster number could also be selected using the gap statistic, silhouette width, or the proportion of ambiguous clustering (PAC) score derived from the consensus CDF, with the chosen criterion stated explicitly — Documenting the quantitative criterion used to select k makes the subclass number reproducible and allows readers to assess how stable the three-cluster solution is relative to neighbouring values of k
-
Zebrafish differential expression used n=3 homozygous mutants (tert−/−), the smallest group in the design↳ Could also: With such small n, a quasi-likelihood F-test in edgeR or a voom-limma approach with empirical Bayes moderation are also used and may provide better type-I error control under very small group sizes — DESeq2's Wald test relies on asymptotic approximations that can be less well-calibrated with n=3 in one group; edgeR QLF and limma-voom both use approaches specifically designed to stabilise variance estimates for small biological replicates
-
Gene set enrichment was performed by testing DESeq2-called DEG lists as discrete input to piano, rather than using a ranked continuous statistic across all genes↳ Could also: Pre-ranked GSEA (e.g., fgsea) using the full ranked list of Wald statistics or log2 fold-changes for all expressed genes is also a standard approach — List-based enrichment depends on the discrete DEG threshold and can miss genes near the cutoff; pre-ranked methods use the full ordering of effect sizes and are generally considered less sensitive to the threshold choice
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
Androgen biosynthesis and metabolism fluxes were broadly increased in the iADPD2 brain metabolic model, alongside elevated production of cholesterol precursors geranyl pyrophosphate and lathosterol.other human brain up 2021×1papers★ This paper is the founder (earliest)
-
In iADPD1 and iADPD2 brain metabolic models, cholesterol biosynthesis fluxes were upregulated and O-glycan metabolism was downregulated, with effects more pronounced in iADPD2.other human brain mixed 2021×1papers★ This paper is the founder (earliest)
-
Estrogen metabolism was uniquely upregulated in the iADPD1 brain metabolic model, with increased flux converting estrone to 2-hydroxyestrone and 2-methoxyestrone, reactions carrying zero flux in controls.other human brain up 2021×1papers★ This paper is the founder (earliest)
-
Sex hormone biosynthesis and metabolism was dysregulated across all three AD/PD transcriptomic subgroups in genome-scale metabolic modelling, most pronounced in cluster 2.other human brain mixed 2021×1papers★ This paper is the founder (earliest)
-
In the iADPD3 brain metabolic model, androstenedione production was increased but its conversion to testosterone was decreased, indicating a block in the final androgen biosynthesis step.other human brain mixed 2021×1papers★ This paper is the founder (earliest)
-
Retinoid metabolism was dysregulated across all AD/PD transcriptomic subgroups, upregulated in cluster 1 and downregulated in clusters 2 and 3.RNA-seq human brain mixed 2021×1papers★ This paper is the founder (earliest)
-
AD and PD post-mortem brain samples stratified into three transcriptomic subgroups: mixed up/downregulation (cluster 1), predominant downregulation (cluster 2), and vast downregulation (cluster 3) relative to controls.RNA-seq human brain mixed 2021×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
scope.md — pmid-34680427
Paper: Lam S et al. (2021) Systems Analysis Reveals Ageing-Related Perturbations in Retinoids and Sex Hormones in Alzheimer's and Parkinson's Diseases. Biomedicines 9(10):1310. PMID 34680427 / PMC8533098. Repo: https://github.com/SimonLammmm/ad-pd-retinoid (HEAD 907baad, 2022-10-31).
Rule 2 — code provenance
The repo is the authors' own analysis code: author of all scripts is
Simon Lam, King's College London («email»), who is the
paper's first author ("Lam S"). The network-analysis notebook is by co-author
Rui Benfeitas. So this is the authors' own pipeline, not merely a cited tool.
(Operator clarification: applying a third-party tool to the paper's data would
have been equally valid; here both conditions hold.)
Pipeline inventory (what produces the reported numbers)
The study is a large multi-omics + genome-scale metabolic-modelling (GEM) work:
- Human AD/PD transcriptomics → QC/normalisation → consensus clustering (k=3 subclasses) → DESeq2 → GSEA → GEM (tINIT) → reporter metabolites (Tables 1–3). Inputs: ROSMAP (Synapse), GTEx, FANTOM5, HuRI, HPA, Rajkumar, Zhang/Zheng (GSE20295).
- Zebrafish tert mutant ageing model → kallisto → DESeq2 → GSEA → GEM (tINIT) → reporter metabolites (Table 5). Input: GEO GSE102426/29/31/34.
- GEM reconstruction (iBrain2845, iADPD1-3, ZebraGEM, context-specific GEMs; Supp Files 1–4) in MATLAB with COBRA Toolbox + RAVEN + CellFie + libSBML.
IN SCOPE (public data, open-source toolchain, tractable on «our HPC»)
- Zebrafish brain RNA-seq differential expression from GSE102426
(= SRA SRP115132, 13 single-end runs). Pipeline:
zeb_kallisto.sh(kallisto quant, GRCz11 cdna Ensembl r96,--single -l 200 -s 20) →zeb_cts.R(count = round(est_counts·eff_length/length)) →zeb_DESeq2.R(DESeq2, design ~tert1, significance padj ≤ 0.1). Fully public data + open tools. - Zebrafish sample design n = 5 wt (tert+/+), 5 het (tert+/−), 3 dko (tert−/−) — verifiable directly from GEO/ENA metadata (Sec 2.5).
OUT OF SCOPE (recorded, not attempted — with reason)
- Human clustering / DE / reporter metabolites (Tables 1–3; "2055 samples",
"1572 accepted", "64,794 genes", k=3): primary expression input is ROSMAP,
access-restricted via Synapse (data-use agreement / application required).
→ drop_reason =
data-access-restricted. - All GEM reconstruction + reporter-metabolite results (Supp Files 1–4;
Tables 3 & 5, incl. zebrafish 9-cis-retinoic acid p=0.044567): require
MATLAB R2020b + COBRA/RAVEN/CellFie/libSBML (proprietary, licensed
toolchain; long tINIT chain). → drop_reason =
proprietary-toolchain. - Other zebrafish tissues (Liver/Muscle/Skin: GSE102429/31/34) and the human GSEA: deliberately not chased (80/20) — same pipeline as the brain branch we do run; adds cost without a distinct reported number to check.
Honest note on anchor numbers
The paper text prints no explicit zebrafish DE-gene count and no zebrafish
figure value that is purely DESeq2-derived (the headline zebrafish number,
Table 5's reporter metabolite, is a GEM/MATLAB product → out of scope). So the
clean 1:1 anchor is the sample design (n=5/5/3); the DESeq2 step is
reproduced as a pipeline-runs / internally-consistent artifact (graded
partial, no printed reference value), which is the honest outcome.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
What deviates: nothing measurable — where a published value exists and was in scope (C1 zebrafish sample design n=5/5/3) the reproduction is exact, and the open kallisto+DESeq2 pipeline (C2) runs cleanly with only a technical DESeq2 version/biomaRt divergence. The gaps (C3 human ROSMAP totals, C4 9-cis-RA p=0.044567) are on the data-availability/tooling side — restricted ROSMAP (legitimate Synapse DUA) and a proprietary MATLAB GEM chain — not the authors' demonstrable defect, and no fabrication is asserted. Severity is negligible for what was comparable, but the central conclusion is untested because the headline branches are controlled drops. Overall: a solid partial reproduction whose limits stem from restricted data and proprietary tooling, warranting yellow rather than green or red.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.