randPedPCA: rapid approximation of principal components from large pedigrees.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓The central claim held under reproduction
- 🟡A deviation arose in the data or preprocessing
- 🟡The deviation was non-trivial in magnitude
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce 1:1 for the core method, using ONLY data shipped in the CRAN package (pedMeta2/pedLInv2 = the 2pop scenario, n=2650). The paper's central claim -- that randPedPCA's randomised, never-form-A approximation equals exact pedigree PCA -- reproduces EXACTLY: |correlation| with the exact eigenvectors of A is 1.000 for PC1/PC2 under both the randSVD and RSpectra methods, centred and non-centred; reproduced variance proportions match the exact eigendecomposition of the same matrix to 6 digits; Hutch++ trace estimates are accurate to <0.1%; the generation-aligned PC reproduces (centred PC2 vs generation rho=-0.982 vs paper -0.97). Two honest minor gaps, both explained and not fabrication: (1) Fig.1's annotated percentages are ~1-2.5 pp lower than what the shipped data actually yields (27.4/21.6 non-centred, 28.4/6.4 centred) -- most likely a Hutch++-estimated denominator and/or loose figure rounding; (2) the headline >10,000x speed-up is directionally confirmed (we measured 186-1125x) but the literal factor is hardware/baseline-dependent and did not transfer. NOT attempted: the Labrador Retriever real pedigree (1.49M records, Fig.3/Table.1) -- proprietary UK Kennel Club data, not publicly downloadable (data-unavailable); the 4pop scenario (Fig.2) and the SNP/genotype PCA columns of Fig.1 -- deprioritised under 80/20, same machinery as the reproduced 2pop case. A transparency note: an intermediate centred-validity number of 0.23 in results.json was a wrong-reference-space artifact (identical eigenvalues, wrong singular-vector space); the corrected double-centred reference gives 1.000 (results_centred.json).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 84assessed: 2026-06-14 ⛓ 77e8ecf0d611
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan the leading principal components of large pedigree relationship matrices be computed rapidly and memory-efficiently by exploiting the sparse Cholesky factor of the pedigree precision matrix, thereby enabling PCA-based visualisation of pedigrees with millions of individuals that is infeasible with naive dense PCA?
- ★ The randPedPCA R package computes leading principal components of the additive relationship matrix A by implicitly operating on the sparse inverse Cholesky factor L^-1 rather than the dense matrix A. method
- ★ Matrix-vector products with A can be performed implicitly via forward and backward substitution using the sparse triangular L^-1, each costing O(n) operations, scaling to millions of individuals. method
- ★ Two decomposition methods are implemented: randomised SVD (faster for very few PCs) and Eigen decomposition via RSpectra/Spectra Lanczos-based algorithm (faster for more PCs). method
- ★ On simulated data the package delivers a speed-up greater than 10,000 times compared to naive PCA and enables analyses impossible with naive PCA. finding
- ★ When only two principal components are desired, the randomised PCA method can halve the running time required compared to RSpectra. finding
- ★ randPedPCA is an open-access R package providing a tool to visualise large pedigrees via scatter plots of PC scores. resource
- The number of non-zero elements in the precision matrix and its Cholesky factor is proportional to the number of individuals, enabling efficient linear algebra on large pedigrees. mechanism
- Visualising a pedigree via PCA scatter plots is considerably faster than rendering plots of a pedigree graph for large pedigrees. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Pedigree PCA via randomised SVD (benchmarking of speed/accuracy) | Simulated pedigree data | none | Running time and PCA scores/speed-up vs naive PCA | randPedPCA R package (with spam, pedigreeTools R packages) |
| Pedigree PCA via Eigen decomposition (Lanczos-based) | Simulated pedigree data | none | Running time as a function of number of principal components requested | RSpectra R package / Spectra C++ library |
| Pedigree PCA (randomised SVD vs RSpectra, two PCs) | UK Kennel Club registered Labrador Retriever population pedigree (almost 1.5 million individuals) | none | Running time and PC score scatter plot | randPedPCA R package |
- ▲ Rapid pedigree PCA achieved a speed-up greater than 10,000 times compared to naive PCA on simulated data. >10,000-fold
- ▼ For two principal components, the randomised PCA method halved the running time compared to RSpectra. ~2-fold reduction
- – Randomised SVD tends to be faster when very few principal components are requested, while Eigen decomposition via RSpectra tends to be faster when more principal components are of interest.
- – The method successfully analysed the Labrador Retriever pedigree of almost 1.5 million individuals, an analysis impossible with naive PCA. ~1,500,000 individuals
- fold_change >10,000 times (Speed-up of rapid pedigree PCA vs naive PCA on simulated data)
- count almost 1.5 million (Individuals in UK Kennel Club Labrador Retriever pedigree analysed)
- other O(n) (Cost of each forward/backward substitution for implicit A-vector multiplication)
- other O(min(nk^2, n^2k)) (Complexity of SVD of a dense n x k matrix)
- other O(nkr) (Complexity of truncated SVD of a dense matrix to obtain first r components)
- other O(zk) (Complexity of sparse SVD algorithms for z non-zero elements)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a computational methods paper introducing the randPedPCA R package for efficient principal component analysis (PCA) of large pedigrees. The core approach avoids materialising the dense n×n additive relationship matrix A by operating instead on the sparse Cholesky factor of its inverse (L⁻¹), enabling implicit matrix–vector products used by either randomised singular value decomposition (rSVD) or a Lanczos-based eigendecomposition via RSpectra. Performance was evaluated through computational benchmarking on simulated pedigrees of varying sizes and on an empirical pedigree of approximately 1.5 million UK Kennel Club Labrador Retrievers, with results reported as wall-clock timing comparisons and speedup ratios relative to naive (dense) PCA.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Randomised singular value decomposition (rSVD) for truncated PCA | Computation of leading principal components of the additive relationship matrix A across all simulated and empirical pedigree benchmarks | — | not stated |
| Lanczos-based eigendecomposition via RSpectra (Spectra C++ library) | Alternative method for extracting leading principal components, benchmarked against rSVD on simulated pedigrees and the Labrador Retriever pedigree | — | not stated |
| Stochastic trace estimation (Girard–Hutchinson-type estimator) | Estimation of trace(A) (total variance) without materialising A, used to compute the proportion of variance explained by each PC | — | not stated |
-
Timing benchmarks are reported as single point-estimate speedup ratios without measures of run-to-run variability↳ Could also: Benchmarks could be repeated multiple times and summarised with a distributional measure (e.g., median ± IQR or mean ± SD across replicate runs), as is standard in systems and algorithmic benchmarking literature — Repeated timing measurements would quantify variability due to system load, CPU cache state, or stochastic initialisation in rSVD, making speedup claims more reproducible and enabling formal comparison of methods' distributions rather than point estimates
-
Accuracy of the approximate rSVD-based PCA is presented conceptually (citing theoretical guarantees) rather than via an empirical numerical accuracy metric comparing approximate to exact PC scores↳ Could also: A quantitative accuracy metric such as the relative Frobenius-norm error ‖A − Â‖_F / ‖A‖_F, the subspace principal angle, or the correlation between approximate and exact PC scores could also be reported as a function of tuning parameters — Explicit empirical accuracy quantification would let practitioners assess the speed–accuracy trade-off for their own pedigree sizes and eigenspectrum shapes, supporting principled selection of oversampling and power-iteration parameters
-
The two implemented methods (rSVD and RSpectra/Lanczos) are compared on wall-clock time, with breakeven behaviour described qualitatively as a function of the number of PCs requested↳ Could also: Empirical runtime-vs-n scaling curves fitted across several orders of magnitude of pedigree size, with estimated scaling exponents, could also be reported alongside the point comparisons — Scaling curves would make the claimed O(n) behaviour explicit and allow users to extrapolate expected runtimes to pedigrees larger than those benchmarked, aiding planning for production-scale analyses
-
PCA of the additive relationship matrix A (capturing expected identity-by-descent) is used as the sole dimensionality reduction approach for pedigree visualisation↳ Could also: A genomic relationship matrix (GRM) built from SNP marker data could serve as input to the same rSVD pipeline when genotype data are available, enabling genomic PCA on the same individuals — Genomic PCA reflects realised rather than expected relatedness and can capture finer population structure; running both pedigree PCA and genomic PCA on matched individuals is an increasingly common validation strategy in animal breeding and conservation genetics
-
Sensitivity of rSVD output to its tuning parameters (oversampling parameter p, number of power iterations q) is described algorithmically but no systematic empirical sensitivity analysis is presented in the available text↳ Could also: A factorial parameter sweep reporting PC-score accuracy as a function of p and q across pedigrees with different eigenspectrum decay rates could also be included — Such a sensitivity analysis would provide data-driven default recommendations and characterise the parameter regime under which the randomised approximation is reliable, which is especially useful for pedigrees with slower spectral decay
-
Scatter plots of the top two or three PC scores are presented as the visualisation output for large pedigrees↳ Could also: Non-linear dimensionality reduction methods such as t-SNE or UMAP, applied to the leading PC scores as input, could also be offered as a visualisation layer — Non-linear methods can resolve cluster structure that linear PCA may compress onto a single axis, and have been applied for population-structure and pedigree visualisation in genomics; they would complement rather than replace PCA scatter plots and could be a natural extension of the package
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
Assessed papers, coloured by verdict. Click a node to open it.
- No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-40877802 (randPedPCA)
Paper: Lee H, Craddock RF, Gorjanc G, Becher H. randPedPCA: rapid approximation of principal components from large pedigrees. Genet Sel Evol 2025. PMID 40877802 · PMCID PMC12392600 · DOI 10.1186/s12711-025-00994-y
Code: https://github.com/HighlanderLab/RandPedPCA — pinned commit
1ab53f6c10164309ed7b50ede5cfcac9cd4c1f41 (2025-09-16). CRAN package
randPedPCA v1.1.4. The R package ships the 2pop simulated dataset as
built-in objects pedMeta2 (pedigree metadata) and pedLInv2 (the sparse L⁻¹
spam matrix). Simulation source: randPedPCA/data-raw/pedXXX2.R, which sets
set.seed(987765543) then builds the pedigree with AlphaSimR → fully
deterministic given the AlphaSimR version.
Data: zenodo:10.5281/zenodo.10876220 (same simulated data + matrices, also
mirrored under datasets/ in the repo). No download required for the in-scope
work — the 2pop dataset is shipped inside the CRAN package.
Method (what the tool computes)
randPedPCA obtains the leading PCs of the additive relationship matrix A of a
pedigree without ever forming A. It operates implicitly on L⁻¹, the Cholesky
factor of A⁻¹ (the precision matrix), which is extremely sparse. PCs are obtained
by randomised SVD (method="randSVD", default) or Lanczos eigendecomposition
(method="rspec", RSpectra/Spectra). The total variance (= trace of A) needed for
variance proportions is estimated with the Hutch++ stochastic trace estimator
(hutchpp). Centring is optional.
In scope (pipeline-derived, reproducible from shipped CRAN data)
Pipeline = randPedPCA::rppca (+ pedigreeTools for A/L/L⁻¹) on the 2pop
simulated pedigree (pedMeta2/pedLInv2, n = 2,650 individuals, 3 populations
A/B/AB over 20 generations).
| # | Result | Paper location | How reproduced |
|---|---|---|---|
| C1 | Validity: approximate (rppca) PC1/PC2 ≈ exact PCA of A — | correlation | near 1 |
| C2 | 2pop non-centred variance proportions: PC1 ≈ 25%, PC2 ≈ 20% | Fig. 1 | summary(rppca(ped, center=FALSE)) |
| C3 | 2pop centred variance proportions: PC1 ≈ 26%, PC2 ≈ 5.7% | Fig. 1 | summary(rppca(ped, center=TRUE)) w/ Hutch++ totVar |
| C4 | Speed-up > 10,000× vs naive PCA on 2pop (paper: prcomp 35,051 ms vs rppca rSVD 2.8 ms) | Table 1 / abstract | wall-clock prcomp(A) vs rppca (hardware-dependent → magnitude only) |
| C5 | Spearman PC-vs-generation: non-centred PC1 ρ ≈ 0.88; centred PC2 ρ ≈ −0.97 (pedigree) | Fig. 1 | cor(PC, pedMeta2$generation, method="spearman") |
| C6 | Hutch++ total variance estimate ≈ exact trace(A) = Σ(inbreeding+1) | vignette / Methods | compare hutchpp vs sum(inbreeding(ped)+1) |
Out of scope (not attempted — and why)
- Labrador Retriever real pedigree (1,486,764 records, UK Kennel Club; Fig. 3, Table 1 Labrador row, PC1 4.1% / PC2 0.70% / PC3 0.33%, ρ values vs year/gen). The pedigree is proprietary UK Kennel Club data, not publicly downloadable → data-unavailable (restricted). The method on this data is identical to the 2pop case, so reproducing the 2pop validity is representative of the algorithm.
- 4pop scenario (Fig. 2): lower-priority; shipped as
*50/*50opvariants. Same machinery as 2pop; attempt only if time permits (80/20 → skipped initially). - SNP/genotype PCA comparison columns in Fig. 1 (the "SNP" percentages) come
from genotype data (
pedGeno2, not exported by default) via standardprcomp, not from randPedPCA — secondary, skipped.
Possible-fabrication watch
All in-scope numbers are derivable from the shipped pedMeta2/pedLInv2 objects
- the documented
rppca/hutchppcalls, so they are checkable. Hutch++ and randomised SVD use RNG — exact variance-proportion digits depend on the seed; graded within-tol where the method is stochastic.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The paper's central scientific claim — that randPedPCA's randomised, never-form-A approximation recovers exact pedigree PCs — reproduces exactly on the package-shipped 2pop data (|cor|=1.000 for PC1/PC2, Hutch++ relerr <0.1%, centred PC2 vs generation -0.982 vs paper -0.97). The deviations are on the technical/figure-annotation side, not the authors' core logic or fabrication: Fig.1 variance labels run ~1-2.5pp below the shipped data (rounding/Hutch++ denominator), and the >10,000x speed-up reproduced only as 186-1125x (hardware/baseline-dependent). The proprietary Labrador pedigree (Fig.3) is restricted and out of scope. Overall: solid, with moderate but explainable deviations — yellow.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.