Does It Reproduce?

We automatically re-run the analyses behind computational papers and check whether — and how far — their results reproduce from the deposited data and code. Every record is readable on its own, transparently justified, and version-sealed.

How it works

1. 1

   An agent reads each paper, fetches its data and code, and re-runs the analysis autonomously on the brainbox compute.

2. 2

   Every reported claim is graded against the re-run; the grades roll up into one 0–100 reproducibility score.

3. 3

   Each record carries a level: L1 (AI) → L2 (a scientist plausibility-checks the grades) → L3/L4 (a scientist re-runs it with documented methods and results).

4. 4

   Every verdict is version-sealed and contestable — a failed reproduction is a finding, never an accusation.

Progress

How far we are toward all reproducible literature. The middle stages are extrapolated from a broad random sample.

How much code the literature discloses

The “with code/pipeline” stage above is an estimate. This curve is counted.

How this is counted: the base set is the open PMC full-text archive — every paper of a given year that can be read in full. Within it we count those naming a GitHub repository in their methods section (Europe PMC section search). Two limits to the figure: a mention in the methods may be the authors’ own repository or a third-party tool they used, and we cannot separate the two — so as a measure of “discloses its own code” the first curve is an upper bound. And whatever sits behind a paywall is not included at all: we can only check what is open.