Genetic architecture of natural variation of cardiac performance from flies to humans.
The main results reproduced, with only marginal, non-material deviations.
- ✓Same input data as the authors
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH + REPRODUCES DIRECTIONALLY (partial 1:1). In-scope pipeline = the human iPSC-CM siRNA-KD bulk RNA-seq DEG analysis (Fig 5J/K) — the only part of Saha et al. (eLife 2022) with code+data shipped (repo gvogler/eLife-2022-Saha-et-al @ df4b14d; nf-core/rnaseq v3.0 STAR+salmon GRCh37 -> DESeq2). The merged salmon.merged.gene_counts.rds the scripts read is NOT deposited; we reconstructed the 9-sample matrix (57773 genes) from the deposited per-sample salmon TSVs in GSE217655_RAW.tar (GSM6724017-25) and ran the authors' DESeq2 (~condition; EGR2=cols1:3+4:6, PAX9=cols1:3+7:9; DEG padj<0.05 & |log2FC|>=1) on «our HPC» («job»). RESULT: every checkable falsifiable marker-direction claim reproduces (6/6 genes present: SCN5A/KCNH2/KCNQ1/NPR1 up directionally, NPPA up significantly +1.10 padj 1.6e-38, CASQ2 down significantly -1.20 padj 2e-29); ZERO directional mismatches; GSE N=9 exact. Pipeline-output DEG sets: EGR2 29 DEGs, PAX9 544 DEGs (paper prints heatmaps, no N to compare 1:1). HONEST CAVEATS: (1) NPPB, named in the same Fig 5J sentence, is absent from the deposited salmon matrix -> uncheckable from the deposit; (2) siRNA targets PAX9/EGR2 are lowly expressed (baseMean 57 / 1.5) so knockdown is only weakly reflected at mRNA level; (3) non-NPPA marker up-regulations are directionally correct but individually sub-threshold, consistent with a described gene-set/heatmap trend. No fabrication signal: all reported directions are derivable from the shipped data+code. NOT ATTEMPTED (out of scope, no code/reproducible data): fly DGRP GWAS (FastLMM/FastEpistasis variant/gene counts) and SOHA heart-imaging functional screen; and (stretch, not run) raw-FASTQ nf-core/rnaseq re-run from PRJNA899898.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-19 ⛓ b9c623b341b9
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-29
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe genetic architecture underlying natural variation in cardiac performance can be dissected using the Drosophila Genetic Reference Panel (DGRP), and the genes/pathways identified are conserved between flies and humans such that fly findings can accelerate discovery of human cardiac disease genes.
- ★ Natural genetic variation significantly influences cardiac performance traits (rhythmicity and contractility) across 167 DGRP lines finding
- ★ GWAS across seven cardiac traits identified candidate variants/genes enriched for transcription factors, signaling receptors, and cell adhesion molecules finding
- ★ Epistatic interactions among SNPs (via FastEpistasis) extend and significantly overlap the gene network identified by single-marker GWAS mechanism
- ★ Non-coding variants were used to predict regulatory regions and transcription factor binding sites; candidate TFs were validated in vivo by heart-specific RNAi knockdown method
- ★ Phenotypic variability (coefficient of variation) in cardiac traits is itself heritable and associated with genes overlapping trait-mean-associated genes, but via different variants finding
- ★ The genetic architecture regulating cardiac performance is conserved from Drosophila to humans, based on overlap of orthologous genes identified in fly and human GWAS finding
- ★ dmPox-meso/PAX9 and dmStripe/EGR2 have conserved roles in regulating cardiac rhythm, validated in both fly hearts and human iPSC-derived cardiomyocytes finding
- The DGRP serves as a resource enabling quantitative genetic dissection of cardiac traits under controlled genetic and environmental conditions resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| High-speed video recording of semi-intact heart preparations (SOHA) | Drosophila, 167 DGRP inbred lines, 1-week-old females | none (natural genetic variation) | cardiac traits: systolic interval, diastolic interval, heart period, arrhythmia index, end diastolic/systolic diameter, fractional shortening | SOHA (Semi-automated Heartbeat Analysis) software |
| Genome-wide association study (single-marker linear mixed model) | DGRP lines (sequenced genomes) | none | SNP-phenotype association p-values for seven cardiac traits | FaST-LMM |
| Epistasis analysis | DGRP SNPs (focal SNPs vs all other SNPs) | none | SNP-SNP interaction statistics | FastEpistasis |
| Heart-specific RNAi knockdown with cardiac phenotyping (SOHA) | Drosophila adult heart in vivo | RNAi-mediated knockdown of candidate transcription factors | cardiac function parameters | — |
| Gene Ontology / network enrichment analysis | gene lists from GWAS and epistasis (in silico) | none | enrichment of functional categories (TFs, signaling receptors, cell adhesion molecules) | — |
| siRNA-mediated gene knockdown | human induced pluripotent stem cell-derived cardiomyocytes (hiPSC-CMs) | siRNA knockdown of PAX9 and EGR2 orthologs | cardiomyocyte function | — |
| Comparative genomics / GWAS cross-species comparison | human GWAS datasets of cardiac disorders vs Drosophila GWAS results | none | overlap of orthologous genes associated with cardiac traits | — |
- – Significant genetic variation and heritability found for all seven cardiac traits H2 = 0.258-0.566
- – GWAS identified unique variants mapped to genes associated with cardiac traits 530 unique variants; 417 mapped to 332 genes
- ▲ Epistatic SNP network identified and significantly overlaps single-marker GWAS gene set 288 SNPs mapped to 261 genes; 31-gene overlap, FC=6, p=6.8e-16
- ▲ GO enrichment shows over-representation of TFs, signaling receptors, and cell adhesion molecules among cardiac-trait-associated genes FC=2.9 (p=1.4e-9), FC=2 (p=5e-4), FC=4.6 (p=3e-3)
- – Combined GWAS and epistasis analyses yield a compendium of genes associated with natural variation in heart performance 562 genes
- – PAX9 and EGR2 orthologs regulate cardiac rhythm in both flies and human iPSC-CMs
- ▲ Cardiac-trait-associated variants are non-randomly distributed across genomic regions relative to gene TSS/TES chi-square p=2.778e-13
- other H2 = 0.258-0.566 (broad-sense heritability of the seven cardiac traits)
- count 530 unique variants; 417 mapped to 332 genes (top-ranked GWAS variants across 7 traits, MAF>4%)
- count 288 unique SNPs mapped to 261 genes (FastEpistasis-identified interacting SNPs)
- other FC=6, hypergeometric p=6.8e-16 (overlap between GWAS-associated and epistasis-associated gene sets)
- pvalue p=2.778e-13 (chi-square test of genomic location bias of associated variants)
- fold_change FC=2.9, p=1.4e-9 (GO enrichment of signaling receptor genes)
- fold_change FC=4.6, p=3e-3 (GO enrichment of cell adhesion molecule genes)
- count 562 genes (total compendium of genes associated with natural variation in heart performance)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper used a quantitative-genetics framework in the Drosophila Genetic Reference Panel (167 inbred lines, ~12 females/line) to estimate heritability and variance components for seven cardiac traits, then performed genome-wide association (single-marker linear mixed model via FastLMM) and epistasis screening (FastEpistasis) rather than relying solely on a strict genome-wide significance cutoff. Supporting analyses included a chi-squared test for genomic-location enrichment of associated variants, a hypergeometric test for gene-set overlap between LMM and epistasis results, Levene's test for among-line variance heterogeneity, and Spearman correlations between traits. Results were reported with exact p-values, fold-change effect sizes, standard deviations/coefficients of variation, and heritability estimates, without confidence intervals.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Linear mixed model single-marker GWAS (FastLMM, Lippert et al., 2011) | association of line means with common variants for each of 7 cardiac traits | 167 DGRP lines, ~1956 individuals total | not stated |
| FastEpistasis (Schüpbach et al., 2010) | pairwise SNP-SNP interaction testing using top GWAS SNPs as focal markers | 167 DGRP lines | not stated |
| Chi-squared test (Pearson residuals) | comparison of SNP genomic-location categories between cardiac-trait-associated variants and all DGRP variants (Figure 1C) | 417 variants mapped to 332 genes vs. genome-wide DGRP variant set (MAF>4%) | not stated |
| Hypergeometric test | overlap between genes from single-marker GWAS (332 genes) and epistasis (261 genes) (Figure 1D) | 332 and 261 genes, overlap n=31 | not stated |
| Levene test | heterogeneity of within-line variance for each cardiac trait (Table 1) | 167 lines, per-trait individual counts (e.g., 1914 for DI) | not stated |
| Spearman correlation | pairwise correlations among the seven cardiac phenotypes (Figure 1—figure supplement 1B) | line means across 167 DGRP lines | na |
| GO/functional enrichment test (hypergeometric/Fisher-type, exact test not named) | over-representation of signaling receptors, TFs, and cell adhesion genes among GWAS+epistasis gene sets | 562 combined genes | not stated |
-
No variant met the strict Bonferroni genome-wide threshold, so the top 100 ranked variants per trait were used for downstream analysis instead of a corrected significance cutoff.↳ Could also: A false discovery rate (Benjamini-Hochberg) approach or permutation-based empirical significance threshold — These approaches are commonly used in modestly powered GWAS panels like the DGRP to balance discovery power against false positives, and could complement a fixed top-N variant selection strategy.
-
Gene-set overlaps (e.g., LMM vs. epistasis gene lists, GO enrichment) were assessed with a hypergeometric test.↳ Could also: Fisher's exact test or a permutation/resampling-based enrichment test — Fisher's exact test is mathematically equivalent for 2x2 overlap tables, and permutation approaches can be useful when the appropriate background gene universe is itself uncertain.
-
Among-line variance heterogeneity was tested with Levene's test.↳ Could also: Brown-Forsythe test (Levene's test centered on medians) — The Brown-Forsythe variant is often preferred when within-line trait distributions are not expected to be symmetric, as it is more robust to non-normality.
-
Pairwise relationships among the seven cardiac traits were summarized using Spearman correlations without a stated multiple-comparison adjustment.↳ Could also: An FDR correction (e.g., Benjamini-Hochberg) applied across the full trait-correlation matrix — Since many pairwise correlations are examined simultaneously, an FDR adjustment could help contextualize which correlations are more likely to reflect a shared genetic basis versus chance.
-
Broad-sense heritability (H2) was reported as a point estimate derived from variance components.↳ Could also: Bootstrap or REML-based confidence intervals around heritability estimates — Reporting an interval alongside the point estimate would convey the precision of heritability estimates, which can be useful for cross-study comparison.
-
Trait dispersion in Table 1 was reported as standard deviation and coefficient of variation.↳ Could also: 95% confidence intervals for line means alongside SD — CIs directly communicate the uncertainty around estimated means and can be a useful complement to SD/CV, particularly for lines with fewer surviving individuals after outlier removal.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-36383075 (Saha et al., eLife 2022)
Paper: Genetic architecture of natural variation of cardiac performance from
flies to humans. eLife 2022;11:e82459. PMID 36383075 / PMC9668334.
Repo: https://github.com/gvogler/eLife-2022-Saha-et-al @ commit
df4b14dab23a061c5148e4d702488548c08da9bc (main, 2023-02-10).
Data: GEO GSE217655 (human iPSC-CM RNA-seq; SRA BioProject PRJNA899898).
What the paper contains (overview)
A large multi-part study spanning Drosophila genetics and human validation:
- Fly DGRP GWAS of cardiac performance traits (FastLMM / FastEpistasis) — "530 unique variants across seven traits; 417 mapped to 332 genes; 288 SNPs (FastEpistasis) → 261 genes".
- Fly functional screen (RNAi knockdown, semi-automated heart imaging — SOHA).
- Human cross-species network / GO analyses.
- Human iPSC-CM siRNA-knockdown RNA-seq of EGR2 and PAX9 (Figure 5) — the only part with deposited code + data.
In scope (pipeline-derived, code+data shipped) — ATTEMPTED
The repository ships ONLY the human iPSC-CM RNA-seq differential-expression pipeline (Figure 5J,K and associated text). This is what we reproduce.
| Result | Pipeline | Inputs shipped? | In scope |
|---|---|---|---|
| Upstream gene quantification (salmon counts) | nf-core/rnaseq v3.0 (commit 3643a94), STAR+salmon, GRCh37 Ensembl | raw FASTQ via SRA PRJNA899898; processed per-sample salmon gene counts in GSE217655 | YES (counts deposited → DESeq2 directly; FASTQ→counts as stretch) |
| siEGR2-vs-siCTRL DEGs | DESeq2 (DESeq2 Perrin VCM EGR2.R) padj<0.05 & |log2FC|≥1 |
yes (counts) | YES |
| siPAX9-vs-siCTRL DEGs | DESeq2 (DESeq2 Perrin VCM PAX9.R) padj<0.05 & |log2FC|≥1 |
yes (counts) | YES |
| Direction of named marker genes (PAX9 KD): SCN5A, KCNH2, KCNQ1 ↑; CASQ2 ↓; NPPA, NPPB, NPR1 ↑ | DESeq2 | yes | YES (falsifiable qualitative claims) |
Primary reproduction route (the paper's own data + own code)
The DESeq2 scripts read RNAseq VCM Perrin Colas/salmon.merged.gene_counts.rds
(the nf-core merged matrix, NOT deposited). GEO instead deposits the same counts
split per sample as 9 *.salmon.gene_counts.tsv.gz files
(GSM6724017–GSM6724025). We reconstruct the merged count matrix from these 9
files (column order CTRL_R1-3, EGR2_R1-3, PAX9_R1-3) and run the authors'
DESeq2 pipeline verbatim (same column subsetting 1:3+4:6 for EGR2, 1:3+7:9 for
PAX9; same round(); same ~condition design; same DEG threshold).
Stretch route (harder, full 1:1 from raw)
Download SRA FASTQ (PRJNA899898) and re-run nf-core/rnaseq v3.0 salmon on «our HPC» to regenerate the counts ourselves, then DESeq2 — verifying the deposited counts are reproducible from raw reads.
Out of scope (no code and/or no reproducible data shipped) — NOT attempted
- Fly DGRP GWAS / epistasis (FastLMM, FastEpistasis): no code in repo; DGRP phenotype/genotype handling is wet-lab + external. Reported variant/gene counts are not regenerable from this repo.
- SOHA heart-imaging functional screen: wet-lab + proprietary imaging.
- Cross-species network construction & GO/KEGG enrichment narrative: the DESeq2
scripts include GOstats/pathview/gprofiler2 calls but require the
geneset_ensembl.Rdataannotation (regenerable via biomaRt) and are visualization/enrichment, not the core DEG claim; treated as secondary.
Honesty notes
- The paper's MAIN TEXT does not print explicit DEG counts for EGR2/PAX9; the DEG sets appear as heatmaps (Fig 5J,K). Our DEG counts are therefore a pipeline-output reproduction (we report the numbers the shipped code yields), while the falsifiable published claims are the named marker-gene directions, which we check 1:1.
- The merged
.rdsthe scripts expect is not deposited; reconstructing it from the per-sample TSVs is faithful but introduces a small reconstruction step (column ordering / round) that we document.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.