Fractional ridge regression: a fast, interpretable reparameterization of ridge regression.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- ✓Overall, the reproduction was clean
- 🟡A deviation arose in the data or preprocessing
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough and reproduced 1:1 on the algorithmic core. fracridge is the authors' own open-source tool; its computational claims rest on synthetic, self-contained data, so no external download was needed. On «our HPC» (SLURM «job», sklearn 1.8.0) the authors' own pytest suite passes (326 tests) and a purpose-written script reproduces the central claims essentially exactly: fractional L2-norm accuracy to 0.76% (paper: '~1% or less'), gamma=1 == OLS to 1e-16, FRR == standard ridge at the matched alpha to ~1e-14 (vs sklearn), and monotone alpha(gamma). The Fig 3 timing LAW reproduces (FRR linear in f, R^2=0.996; naive cheaper at small f then much costlier) but the exact crossover is ~f=52 here vs the paper's ~20 — a hardware/MATLAB-vs-Python difference the paper itself anticipates (SVD is the fixed bottleneck), hence graded partial. The Fig 2 simulations reproduce qualitatively (low-dim needs ~no regularization, high-dim peaks at intermediate gamma; smooth coefficient paths). NOT attempted: Fig 4 fMRI on the Natural Scenes Dataset (data_restricted — signed Data Access Agreement required; and it is a visualization, not a tabulated number). Provenance note: the brief's data accession zenodo:10.5281/zenodo.3741781 is a mis-harvest (it is sphinx-gallery v0.6.1, unrelated). All grades provisional, human-checkable.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 84assessed: 2026-06-18 ⛓ cd7af9423844
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-18
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-18no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan ridge regression be reparameterized in terms of the ratio (γ) between the L2-norms of the regularized and unregularized coefficients to make hyperparameter selection faster, automatically range-spanning, and more interpretable than the standard practice of searching over α values?
- ★ Ridge regression can be reparameterized in terms of the fraction γ between the L2-norms of the regularized and unregularized coefficient solutions (fractional ridge regression, FRR). method
- ★ Specifying γ values spanning 0 to 1 guarantees that candidate solutions differ from one another and span the full range from under- to over-regularization, so the best solution lies within the explored range. finding
- ★ FRR is computationally efficient and scalable for large-scale data problems. finding
- ★ FRR delivers results that are straightforward to interpret and compare across models and datasets, demonstrated in brain imaging data. finding
- ★ Open-source FRR software is provided in Python and MATLAB (https://github.com/nrdg/fracridge). resource
- Using SVD, ridge regression coefficients are scaled-down versions of OLS coefficients, with larger singular values shrunk less than smaller ones. mechanism
- When the singular value spectrum of X is flat, there is an analytic solution α = λ²(1/γ − 1) relating α and γ, recapitulating Hoerl and Kennard's result. mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| benchmarking simulations / computational performance evaluation | synthetic large-scale data (varying numbers of data points d, predictors p, targets t) | none | computational efficiency / scalability (runtime) | — |
| general linear model (ridge/fractional ridge regression) analysis | brain imaging data | none | interpretability and comparability of regularized solutions across models and datasets | — |
- – FRR is fast and scalable for large-scale data problems in benchmarking simulations.
- – FRR applied to real-world brain imaging data yields clear, interpretable results comparable across models and datasets.
- – Solutions for different γ are guaranteed to differ, avoiding redundant/wasted calculations.
- count >25,000 citations (classic ridge regression references, per Google Scholar) (popularity of ridge regression method)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a mathematical methods/software paper proposing fractional ridge regression (FRR), a reparameterization of standard ridge regression in which the hyperparameter is the ratio γ of the L2-norm of the regularized to unregularized coefficient vector rather than the penalty scalar α. The paper derives the approach analytically via singular value decomposition (SVD), provides open-source Python and MATLAB implementations, benchmarks computational performance on simulated large-scale data, and demonstrates interpretability on brain imaging data. No classical inferential hypothesis tests (e.g., t-tests, ANOVA) are reported in the provided text; the evaluations are computational benchmarks and regression-fit comparisons.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Cross-validation (fold count and variant not stated in provided text) | Described as the standard method for selecting the ridge hyperparameter α; used as the reference framework against which FRR's automation is compared | — | not stated |
| Benchmarking simulation (computational timing/scalability; exact metric not specified in provided excerpt) | Evaluation of FRR computational efficiency for large-scale data problems | — | not stated |
| Regression coefficient comparison (FRR vs OLS vs standard ridge regression on brain imaging data; specific statistic not specified in provided excerpt) | Brain imaging application demonstrating interpretability across models and datasets | — | not stated |
-
The paper reparameterizes the ridge hyperparameter as γ, the ratio of L2-norms of regularized to unregularized coefficients (ranging 0–1)↳ Could also: Alternative scale-free reparameterizations include the effective degrees of freedom (df) of the ridge estimator, defined as tr(X(X'X+αI)⁻¹X'), which maps α to an interpretable count of effective predictors — Degrees-of-freedom parameterization is also scale-invariant and has a direct model-complexity interpretation; it is widely used in smoothing spline and GAM literature, so researchers familiar with those frameworks may find it more intuitive
-
Cross-validation over a candidate set is described as the standard method for selecting the optimal α (or γ)↳ Could also: Generalized cross-validation (GCV) or leave-one-out cross-validation computed analytically via the hat matrix provides an exact or near-exact CV score in O(p²d) without explicitly holding out data — Analytic GCV avoids the need to partition data and is computationally cheaper for moderate d, making it a natural pairing with the SVD already computed for FRR
-
The paper uses L2 (ridge) regularization exclusively, penalizing the squared norm of coefficients↳ Could also: L1 (lasso) regularization or elastic net (L1+L2 combined) could also be used for regression with high-dimensional predictors — Lasso and elastic net induce sparsity in the coefficient vector, which may be preferable when only a subset of predictors is expected to be non-zero; elastic net additionally handles correlated predictors, an overlap with the motivating use case described for ridge
-
The algorithm uses SVD of the design matrix X as its core computational step↳ Could also: A Cholesky decomposition of X'X or iterative conjugate-gradient methods could also solve the ridge system, particularly when p is large but X'X is sparse — For very high-dimensional sparse design matrices, forming and decomposing X'X via Cholesky or solving iteratively can be substantially cheaper than a full SVD, though the SVD approach offers additional geometric insight exploited by FRR
-
Model performance on brain imaging data appears to be evaluated by comparing coefficient patterns across γ values↳ Could also: Prediction accuracy on a held-out test set (R² or RMSE) stratified by γ, together with a bias-variance decomposition, could also be reported — Explicit held-out test-set metrics and a bias-variance breakdown would allow readers to quantify how prediction performance changes across the regularization path, complementing the interpretability argument made for FRR
-
The benchmarking evaluation compares computational performance of FRR against standard ridge regression↳ Could also: A formal scaling analysis reporting wall-clock time as a function of d, p, and t with confidence intervals across multiple random seeds could also be provided — Replicated timing experiments with dispersion measures allow readers to assess variability in runtime and verify the claimed scalability properties under different hardware and data configurations
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Reproduction scope — pmid-33252656
Paper: Rokem A, Kay K. Fractional ridge regression: a fast, interpretable reparameterization of ridge regression. GigaScience 9(12):giaa133, 2020. PMID 33252656 · PMCID PMC7702219 · DOI 10.1093/gigascience/giaa133
Code: https://github.com/nrdg/fracridge (own code, P16 = authors' own tool)
Cloned to «infra»: …/reproductions/pmid-33252656/fracridge (master tarball,
2026-02-11 snapshot — git clone blocked by a broken git-template config on «our HPC»,
so the GitHub master tarball was used instead).
Nature of the paper
A methods/software paper. It proposes fractional ridge regression (FRR), a
reparameterization of ridge regression by the fraction
gamma = ||b_frr||_2 / ||b_ols||_2 (1 = no regularization, 0 = full). It ships
Python + MATLAB implementations and demonstrates the method on (a) synthetic
simulations and (b) one fMRI dataset.
The reproducible computational results are almost entirely simulation-based and self-contained — the "data" for Fig 2 and Fig 3 is synthetic and generated in-code. No external download is needed for the core claims.
In scope (pipeline-derived, attempted)
Pipeline = the FRR algorithm (fracridge) and its benchmark/simulation scripts.
| ID | Result | Paper location | Pipeline |
|---|---|---|---|
| C1 | Fraction accuracy: actual ` | b_frr | |
| C2 | gamma=1 recovers the OLS solution | Methods/Fig 2 | fracridge() |
| C3 | FRR solution equals standard ridge at the matched alpha | Methods, docstring | fracridge() vs sklearn Ridge/normal eq. |
| C4 | alpha decreases monotonically as gamma increases ("solutions guaranteed to vary") | Abstract/Results | fracridge() |
| C5 | Fig 3 timing/scaling: FRR ~linear in f; naive matrix-inversion cheaper for f<~20 then crosses over | Fig 3 | timing harness, base d=5000,p=5000,t=1000 |
| C6 | Fig 2 simulations: two scenarios (A d=100,p=5; B d=100,p=100), 50/50 split, f=21; CV R² + coefficient paths + gammas | Fig 2 | FracRidgeRegressor |
Also run: the package's own pytest suite (validates sklearn-Ridge equivalence, OLS limit, fraction accuracy <1%) as an independent implementation check.
Out of scope / not attempted (with reason)
- Fig 4 — fMRI application on the Natural Scenes Dataset (NSD). Single
participant, design matrix 9,841 images × 625 predictors, response 9,841 ×
783,432 voxels. The NSD (naturalscenesdataset.org) is access-restricted:
it requires a signed NSD Data Access Agreement (Google Form) before any
download; it is not openly downloadable. Drop-class for this result:
data_restricted. The figure is a demonstration/visualization of FRR on real data (brain maps + two example voxels with gamma=0.15 and gamma=0.35), not a numeric claim with a tabulated value, so even with access it offers little quantitatively checkable beyond what C1–C6 already establish. - Absolute timings (Fig 3) are hardware-dependent; only the scaling shape
and the crossover are treated as reproducible. The reported Fig 3 timings were
produced on the authors' machine via MATLAB (
Fig3.m), not on «our HPC».
Data accession note (IMPORTANT — profiling finding)
The accession recorded in the room brief, zenodo:10.5281/zenodo.3741781, does NOT point to fracridge data — it resolves to sphinx-gallery v0.6.1 (a doc tool), a mis-harvested DOI. The paper's actual supporting deposits are:
- GigaScience GigaDB 10.5524/100816 (code/figures snapshot), and
- the Natural Scenes Dataset (external, access-restricted) for Fig 4. The core simulation results need neither — they are synthetic.
Compute plan
- Env built with conda on «our HPC» front1 (internet there); compute on SLURM.
- C1–C4, C6 are tiny (seconds); C5 is a moderate CPU/SVD benchmark → SLURM job.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a near 1:1 reproduction of the authors' own open-source tool on self-contained synthetic data: the algorithmic core (gamma=1==OLS, FRR==ridge at matched alpha, monotone alpha, <1% fraction accuracy) reproduces to machine precision and the authors' 326-test pytest suite passes. The only non-trivial deviation is the Fig 3 timing crossover (~52 vs the paper's ~20), which is a hardware/MATLAB-vs-Python artifact the paper itself anticipates (SVD bottleneck), and Fig 2 reproduces qualitatively given an unspecified seed. No authors'-side or derivability problem; the restricted NSD/Fig 4 and a mis-harvested Zenodo accession are availability/provenance notes, not defects.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.