GenomeChronicler: The Personal Genome Project UK Genomic Report Generator Pipeline.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓The central claim held under reproduction
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce. GenomeChronicler (authors' own pipeline) was run on its documented NA12878 demonstration data (= the sample of the repo's bundled exemplar_report.pdf) on «our HPC», in a native conda env replicating the Dockerfile (no container runtime available) with the bundled release-0.91 reference + software (GATK 3.4-46 and PLINK v1.90b3.35 — exact matches to the paper's version table). The genotype->phenotype report reproduced the bundled exemplar EXACTLY: 57 beneficial / 181 harmful / 25 genoset rows (C3/C4/C5), including matching genoset identities; ancestry placed NA12878 squarely in the European/CEU(=CEPH) cluster (C2a) and genoset gs145 confirmed female (C2b) — 1:1 with the paper. The Excel output and LaTeX report source were produced (C1 partial: final PDF render skipped — pdflatex not installed; cosmetic). One shipped-code bug was found and patched: genoTables crashes under a UTF-8 locale on non-UTF-8 text ('Sjögren') in the bundled SQLite DBs, truncating the tables (27/88 before the fix); the one-line text_factory patch changes only text decoding, not genotype logic, after which the counts matched exactly. The paper's headline 99.97% NA12878-vs-GIAB concordance was attempted best-effort (C6) and reproduced only as 94.18% on the documented low-coverage WGS input — essentially all mismatches are GC hom-ref vs GIAB-variant low-coverage false negatives; the paper's exact concordance recipe (sites/no-call handling, likely higher-coverage input) is not in the repo. NOT attempted: the 123 individual PGP-UK reports behind PRJEB24961 (pipeline product, no single checkable value), the Nextflow/Sarek wrappers, and the R ancestry-plot image. All grades provisional; human-auditable artifacts in datasets/pmid-33193602/.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 80assessed: 2026-06-18 ⛓ 1f442c68da7c
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-18
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-18no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusThere is a lack of free, open tools for generating easy-to-understand reports interpreting personal whole genome/exome sequencing data; GenomeChronicler was developed to leverage existing public resources to find known phenotypes associated with detected genotypes and produce personal genomic reports that can be run offline or in the cloud.
- ★ GenomeChronicler is, to the authors' knowledge, the first pipeline that can be run offline or in the cloud to generate non-disease-limited personal genomics reports from whole genome or whole exome sequencing data. resource
- ★ The pipeline infers known phenotypes associated with detected genotypes by comparing them to local versions of public resources (SNPedia, ClinVar, GETevidence, gnomAD), with SNPedia as the core resource. method
- ★ GenomeChronicler infers ancestry via Principal Components Analysis against 1000 Genomes Project reference genotypes. method
- ★ The pipeline can run in a self-contained environment without internet access, making it suitable where data privacy is essential. method
- ★ GenomeChronicler can be run for one sample at a time or in parallel using the Nextflow workflow manager, and end-to-end from raw FASTQ via integration with the Sarek variant calling pipeline. method
- ★ The pipeline outputs a PDF report (variant statistics, ancestry, beneficial/harmful phenotype traits, genosets) plus an Excel file of genotype-phenotype links. resource
- ★ GenomeChronicler genotype calls showed high concordance with Genome-in-a-Bottle high-confidence calls for sample NA12878. finding
- More than one hundred reports have been produced and made openly available as part of PGP-UK. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Whole genome/exome germline variant calling and genotype assessment (gVCF generation at positions of interest) | Human individuals (PGP-UK participants); GRCh38 reference | none | Genotypes at ancestry/phenotype-informative positions | GATK best practices via Sarek pipeline / Nextflow; BAM/CRAM input |
| Ancestry inference via Principal Components Analysis | Sample genotypes intersected with 1000 Genomes Project (26 populations) | none | Projection onto first three principal components to identify population clusters | PLINK; R Statistical Computing platform |
| Variant phenotype annotation | Human personal genome genotypes (rs identifiers) | none | Genotype-to-phenotype links, allele frequencies, clinical relevance | SQLite databases of SNPedia, ClinVar, GETevidence, gnomAD |
| Pipeline validation by genotype concordance | 1000 Genomes sample NA12878 (female, CEPH ancestry, pedigree 1463), GRCh38 | none | Genotype concordance vs Genome-in-a-Bottle high-confidence calls; ancestry/genoset correctness | GATK calls; BCFtools; SAMtools; Genome-in-a-Bottle reference |
| Validation against direct-to-consumer genotyping | PGP-UK participant PGP-UK1 | none | Comparison of GenomeChronicler results to 23andMe results and pilot participant phenotype feedback | 23andMe |
- – GenomeChronicler GATK genotype calls matched Genome-in-a-Bottle high-confidence calls for NA12878 at the genotype level, with no resulting phenotype changes. 99.97% concordance
- – NA12878's known female sex and CEPH ancestry were correctly reflected in the ancestry and genoset sections of the report.
- ▲ More than one hundred reports were produced and made publicly available via the PGP-UK open access data page. >100 reports
- – 67% of participants chose the 'release immediately' option for their report and data. 67%
- other 99.97% (Genotype-level concordance of GenomeChronicler calls vs Genome-in-a-Bottle high-confidence calls for NA12878)
- count more than one hundred (Reports produced and made available as part of PGP-UK)
- other 67% (Participants who selected the 'release immediately' option)
- count 26 (Worldwide populations in the 1000 Genomes reference dataset used for PCA)
- other MAF < 0.05 (Rare variant filtering threshold for ancestry reference SNPs)
- pvalue pval < 0.0001 (Hardy-Weinberg equilibrium filter threshold for ancestry SNPs)
- other r2 > 0.1 (LD pruning threshold (100-SNP windows, 5-SNP intervals) for selecting unlinked SNPs)
- count twenty (Number of principal components extracted in PCA)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software methods paper describing the GenomeChronicler pipeline for personal genome reporting; it does not present a clinical or experimental study. The primary statistical component is a Principal Components Analysis (PCA) applied to genotype data intersected with 1000 Genomes Project reference samples for ancestry inference. Pipeline validation was performed on one benchmark sample (NA12878) by comparing GATK genotype calls against Genome-in-a-Bottle high-confidence calls using BCFtools, with results reported as a concordance rate. Several QC filters (Hardy-Weinberg equilibrium, LD pruning, MAF, missingness) were applied to the reference SNP set prior to PCA.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Principal Components Analysis (PCA) on variance-standardized relationship matrix | Ancestry inference for each submitted sample against 1000 Genomes Project reference panel | 26 worldwide populations from 1000 Genomes Project (exact N of individuals not stated); one sample of unknown ancestry per run | not stated |
| Hardy-Weinberg equilibrium test (p-value threshold < 0.0001 used as QC filter) | Pre-PCA SNP filtering of 1000 Genomes reference genotypes | — | not stated |
| Genotype concordance rate (proportion of matching calls, computed via BCFtools) | Pipeline validation: GATK calls from GenomeChronicler on NA12878 vs. Genome-in-a-Bottle high-confidence genotypes in high-confidence regions | Single sample (NA12878); number of sites compared not stated | na |
-
Ancestry was inferred using PCA projecting the query sample onto axes derived from 1000 Genomes reference genotypes↳ Could also: Model-based clustering methods such as ADMIXTURE or STRUCTURE could also be used to estimate admixture proportions from the same SNP set — PCA is computationally efficient and widely interpretable as a visualization; ADMIXTURE additionally provides quantitative per-population ancestry fractions, which can be informative for admixed individuals who sit between PCA clusters
-
Validation was summarized as a single overall concordance rate (99.97%) with no uncertainty estimate↳ Could also: A 95% confidence interval around the concordance proportion (e.g., Wilson or Clopper-Pearson interval) or Cohen's kappa statistic could also accompany the point estimate — A CI conveys the precision of the concordance estimate given the number of sites compared; kappa adjusts for chance agreement, which is useful when one genotype class (homozygous reference) is highly prevalent
-
Validation was performed on a single benchmark sample (NA12878)↳ Could also: Validation across a panel of multiple samples with diverse known ancestries and varying coverage depths could also be reported — A single sample provides a proof-of-concept; multiple samples with known ground truth would allow estimation of variance in concordance across sequencing conditions, coverage levels, and population backgrounds
-
LD pruning used a fixed r² threshold of 0.1 with 100-SNP windows shifted at 5-SNP intervals↳ Could also: Alternative pruning parameters (e.g., r² = 0.2–0.5, or stepwise clumping) or explicitly reporting the number of SNPs retained after each QC step could also be described — The choice of r² threshold affects the number and independence of SNPs entering PCA; reporting the final SNP count and sensitivity of results to threshold choice aids reproducibility and interpretation
-
Ancestry visualization used projection onto the first three PCs only↳ Could also: Reporting variance explained by each PC, or examining additional PCs, could also be included — The proportion of variance explained by each PC contextualizes how much ancestry information the plot captures; additional PCs sometimes resolve sub-continental structure not visible in PC1–PC3
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-33193602 (GenomeChronicler)
Paper: Guerra-Assunção JA et al. (2020) GenomeChronicler: The Personal Genome Project UK Genomic Report Generator Pipeline. Front Genet 11:518644. DOI 10.3389/fgene.2020.518644 · PMID 33193602 · PMCID PMC7541957.
Code: https://github.com/PGP-UK/GenomeChronicler (own tool, branch main,
snapshot Oct 2023; reference/software bundles from release 0.91, Feb 2023).
This is the authors' own pipeline → P16 not invoked, but the reproduction strategy
is identical to the "third-party tool on the paper's data" model: run the published
tool on the documented demonstration data with the documented parameters.
Data: Demonstration/validation sample is NA12878 (1000 Genomes CEU,
GRCh38 low-coverage WGS CRAM, the exact file the README documents as test data).
The paper's umbrella deposit PRJEB24961 (ENA) is the 100+ PGP-UK participant
genomes; those reports are the product of the pipeline, not its validation input.
What the pipeline computes (all pipeline-derived → in scope)
GenomeChronicler takes a pre-processed BAM/CRAM (or gVCF) on GRCh38 and produces a
personal genomic report. Stages (from main.py + scripts/):
- Ancestry — GATK3 genotyping at ~862k 1000G exome SNPs → PLINK merge with
1000G reference (2504 individuals) → LD-prune (
--indep-pairwise 100 5 0.1, r²>0.1) → PCA (plink --pca, 20 PCs) → R/py PCA plot. - Genotyping of trait SNPs — GATK3 HaplotypeCaller+GenotypeGVCFs restricted to
the 104,743-position trait BED (
snps.19-114.unique.nochr.bed) →afogeno38genotype table. - Phenotype tables — query bundled SQLite DBs (SNPedia, ClinVar, GETevidence, gnomAD) → "possibly beneficial" / "possibly harmful" genotype tables + genosets (multi-variant phenotypes, 260 defined).
- Report assembly — XLSX of all genotype-phenotype links + LaTeX→PDF report.
In-scope reproduction targets (this room)
| # | Target | Source of truth | Difficulty |
|---|---|---|---|
| C1 | Pipeline runs end-to-end on NA12878, emits PDF + XLSX | Paper "Output Formats"; README test-data workflow | low (binary) |
| C2 | Ancestry: NA12878 → European/CEU (CEPH) cluster; sex = female (genoset gs145) | Paper Results ("female individual of CEPH ancestry") | low (qualitative) |
| C3 | # beneficial-trait genotype rows = 57 | exemplar_report.pdf §3.1 (v0.91, Feb 2023, same DBs) | medium (exact count) |
| C4 | # harmful-trait genotype rows = 181 | exemplar_report.pdf §3.2 | medium (exact count) |
| C5 | # genosets reported = 25 | exemplar_report.pdf §3.3 | medium (exact count) |
| C6 | Genotype-level concordance with GIAB high-confidence = 99.97% | Paper Results | HARD — GIAB truth + exact comparison method not shipped |
C1–C5 reproduce the v0.91 pipeline against its own bundled exemplar report (the strongest available 1:1, since exemplar_report.pdf is itself the NA12878wxs report). C6 is the paper's headline validation number; the comparison recipe is not in the repo, so it is attempted on a best-effort basis and graded honestly.
Out of scope (not attempted)
- The 100+ individual PGP-UK reports behind PRJEB24961 (product, not a checkable single reported value; would require per-sample restricted handling).
- Wet-lab steps (sequencing, library prep) — none claimed as computational.
- CloudOS/Lifebit deployment, Nextflow wrappers (GenomeChronicler-nf, -Sarek-nf) — same core results, redundant.
Environment / compute
- «our HPC» («infra»), «infra» work dir
«path». - No container runtime on «our HPC» → native conda env replicating the Dockerfile
(openjdk 8, samtools/bcftools/htslib 1.16, python 3.11 + pinned deps, R) + the
bundled
software/(plink v1.90b3.35, GATK 3.4-46 jar) +reference/bundle. - Heavy step (GATK genotyping of low-cov WGS) → SLURM (
--partition=std).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a strong, mostly-1:1 reproduction of the authors' own pipeline: the genotype→phenotype→report outputs matched the bundled exemplar exactly (57 beneficial / 181 harmful / 25 genosets with matching identities), plus correct female + CEU/CEPH ancestry, so the central claim that GenomeChronicler produces the documented report fully holds. The only substantive deviation is the headline 99.97% → 94.18% GIAB concordance (C6), which sits on the input/preprocessing side (low-coverage WGS false negatives) and reflects an authors-side underspecification of the exact validation recipe rather than a core-logic defect. A shipped UTF-8/SQLite bug was found and patched (text-decoding only), and the PDF render was skipped as cosmetic. Net: solid reproduction with explainable, moderate deviations — not a fabrication or significance concern.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.