Conservation and losses of non-coding RNAs in avian genomes.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- ✓Overall, the reproduction was clean
- Every checked point held up.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce 1:1 EXACT. Target: Table 1 (the paper's central ncRNA-count summary across human, chicken, and 48 birds). Ran the authors' own scripts/makeTable.pl at pinned commit 9b3ebd8 on the repo's shipped intermediate annotation+expression data on «our HPC» («job», ~13 s, pure-Perl, deterministic). All 68 numeric cells reproduced exactly, including the Total row (7340 / 1080.0 / 1194 / 865 = 72.4%) and the RNA-seq false-positive-rate note (865/123/1194, FPR 10.3%). No fabrication concern: every published value is deterministically derivable from the deposited data via the deposited script. NOT attempted (the hard ~20%, by design): re-running the upstream Infernal/Rfam-11.0 + tRNAscan-SE + miRBase scan of 48 genome assemblies and the compete_clans merge — those heavy steps' outputs are deposited in the repo as data/{rfam,trnascan,mirbase,merged-annotations}/*.gff, so the downstream summary is reproducible without re-deriving them; and the heatmaps.R figures (visual, not numeric). This is a P16-style reproduction using authors' own code + shipped intermediates with an exact pinned commit.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 100assessed: 2026-06-16 ⛓ 35a684bb4bc5
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-16
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-09-19
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetUsing homology-based covariance model methods on 48 avian genomes, the paper investigates the extent to which non-coding RNA loci (classical ncRNAs, snoRNAs, miRNAs, and mammalian lncRNAs) are conserved in birds, and examines apparent losses to determine whether they reflect genuine gene loss, sequence divergence, or missing assembly data (particularly related to microchromosomes).
- ★ Homology-based covariance model (Rfam CM) annotation, combined with tRNAscan-SE and miRBase-derived models, identifies ncRNA loci across 48 avian genomes method
- ★ 66,879 ncRNA-associated loci were identified across the 48 avian genomes, classified into 626 families finding
- ★ 34 lncRNA-associated loci are conserved between birds and mammals, with 12 validated by RNA-seq expression evidence in chicken finding
- ★ Several mammalian lncRNAs are conserved in birds even though their known mammalian function may not be finding
- ★ Classical ncRNAs (e.g. tRNAs) and more recently discovered ncRNAs (snoRNAs, miRNAs) show extensive conservation in birds finding
- ★ Numerous apparent losses of ncRNA families in birds are attributable to genuine loss, sequence/structural divergence, or missing genome assembly data finding
- ★ Many apparent ncRNA losses stem from difficulties in assembling avian microchromosomes mechanism
- SNORD93 shows an unusual copy-number expansion (92 copies) in the tinamou genome relative to other vertebrates (1-2 copies) finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Covariance model homology search (cmsearch) | 48 avian genomes | none | ncRNA loci matching Rfam families above GA threshold and E-value < 5x10^-4 | INFERNAL 1.1, Rfam database v11.0 |
| tRNA gene annotation | 48 avian genomes | none | tRNA gene predictions, isoacceptor type, functional vs pseudogene status | tRNAscan-SE v1.3.1 |
| microRNA covariance model homology search | 48 bird genomes plus American alligator and green turtle outgroups | none | conserved miRNA seed sequence and hairpin secondary structure | INFERNAL v1.1rc3, miRBase v19, RALEE alignment editor |
| snoRNA homology search | 48 avian genomes (queries from human, platypus, chicken snoRNA families) | none | snoRNA family annotation | snoStrip |
| Small RNA-seq | chicken, 14 tissues (27 samples) | none | expression-based validation of ncRNA predictions | Illumina HiSeq2000; mapped with SEGEMEHL v0.1.9 to galGal4 |
| Strand-specific (dUTP) RNA-seq | chicken whole embryo, 7 developmental stages | none | expression-based validation of ncRNA predictions | Illumina HiSeq; mapped with SEGEMEHL v0.1.9 to galGal4 |
- – 66,879 loci identified in 48 avian genomes, classified into 626 families, mostly miRNAs and snoRNAs 66,879 loci; 626 families
- – 34 lncRNA-associated loci conserved between birds and mammals; 12 validated in chicken via RNA-seq 34 loci; 12 validated (35.3%)
- – HOX cluster lncRNAs HOTAIRM1, HOXA11-AS1 and HOTTIP show conserved RNA-domains and preserved gene order (synteny) across mammalian and avian lineages 15 RNA-domains total (5, 6, 4 per lncRNA)
- ▼ tRNA and tRNA-pseudogene copy numbers are reduced in birds relative to human/turtle/alligator ~900 and ~580 copies (human/turtle/alligator) drop to ~280 and ~100 copies (birds)
- – mir-17/mir-92 cluster II (mir-106b, mir-93, mir-25) absent in turtles, crocodiles and birds
- – let-7 miRNA cluster A completely lost in the avian lineage; cluster F (two let-7 paralogs) also lost
- – Y RNA paralog Y5 is absent from all bird genomes but present in alligator and turtle
- ▲ SNORD93 copy number greatly expanded in tinamou relative to other vertebrates 92 copies vs 1-2 copies
- count 66,879 loci; 626 families (Total ncRNA loci identified across avian genomes)
- count Long non-coding RNA: human 62, median(48 birds) 25.0, chicken 34, confirmed 12 (35.3%) (Table 1 lncRNA counts)
- count microRNA: human 356, median(48 birds) 499.5, chicken 427, confirmed 280 (65.6%) (Table 1 miRNA counts)
- count Total ncRNAs: human 7340, median(48 birds) 1080.0, chicken 1194, confirmed 865 (72.4%) (Table 1 totals across all ncRNA types)
- count 971 million reads, 27 samples, 14 tissues (Small RNA-seq validation dataset (Bioproject PRJNA204941))
- count 1.46 billion reads, 7 developmental stages (Strand-specific RNA-seq validation dataset (SRP041863))
- pvalue E-value < 5 x 10^-4 (Threshold for retaining Rfam CM matches)
- count 92 copies (tinamou) vs 1-2 copies (other vertebrates) (SNORD93 copy number expansion)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a large-scale comparative bioinformatics study annotating ncRNA loci across 48 avian genomes using covariance model (CM)-based homology search (INFERNAL 1.1 with Rfam v11.0 CMs, tRNAscan-SE, and miRBase-derived CMs). Results are reported descriptively as counts, medians, presence/absence heatmaps, and validation percentages; no formal inferential statistical tests (e.g., t-tests, regression) are applied. Significance is operationalized algorithmically via E-value thresholds and Rfam gathering (GA) thresholds rather than through classical hypothesis testing. RNA-seq read mapping in chicken is used to validate predicted loci, with expression thresholds set at a declared false-positive rate of less than 10%.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| E-value threshold (≤ 5×10⁻⁴) applied to cmsearch hits above Rfam GA threshold | All ncRNA family annotations across 48 avian genomes | 48 avian genomes; 66,879 loci retained | not stated |
| 10% prevalence conservation filter (family retained only if found in ≥10% of avian genomes) | Post-annotation filtering of Rfam, miRNA, and snoRNA hits | 48 avian genomes | not stated |
| False-positive rate threshold (<10%) for RNA-seq expression confirmation | Chicken ncRNA expression validation using two RNA-seq datasets | 971 million reads (small RNA-seq, 27 samples, 14 tissues); 1.46 billion reads (strand-specific, 7 embryonic stages) | not stated |
| Clan competition (best-hit retention for overlapping Rfam family hits within a clan) | Resolution of overlapping annotations across all avian genomes | — | na |
-
Conservation of ncRNA families was assessed by a fixed 10% prevalence threshold across 48 genomes↳ Could also: Phylogenetic ancestral-state reconstruction methods (e.g., Dollo parsimony, maximum-likelihood models of gene gain/loss such as those in CAFE or BayesTraits) could also have been applied to the presence/absence matrix — Model-based approaches would also provide probabilistic estimates of gain and loss rates along specific lineages, allowing one to distinguish stochastic absence from lineage-specific loss with an associated confidence measure, rather than relying on a single prevalence cutoff
-
Copy-number summary across 48 birds was reported as the median (Table 1) with no measure of spread↳ Could also: The interquartile range (IQR) or a bootstrapped 95% confidence interval around the median could also have been reported alongside the median — Adding a dispersion statistic would allow readers to assess how variable copy-numbers are across the avian clade, distinguishing families with uniformly low counts from those with a few extreme outliers like SNORD93 in tinamou
-
E-value and GA thresholds from INFERNAL/Rfam were used as the sole criteria for calling a hit 'significant'↳ Could also: A precision-recall analysis or receiver-operating-characteristic (ROC) evaluation against a curated gold-standard set could also have been used to empirically benchmark threshold choice for this particular dataset — An empirical threshold evaluation would also quantify the sensitivity/specificity trade-off for these specific avian genomes, since model-specific GA thresholds were derived from diverse training data that may differ from avian sequence composition
-
RNA-seq expression validation used a single FPR cutoff (<10%) without a formal statistical model for read counts↳ Could also: Negative-binomial count models (e.g., DESeq2 or edgeR) applied to per-locus read counts across the 27 RNA-seq samples could also have provided per-locus statistical evidence for expression above background — Model-based differential expression testing would also yield adjusted p-values and fold-changes over background regions, enabling a more granular ranking of validated loci and an explicit correction for the large number of loci tested simultaneously
-
Syntenic conservation of lncRNA domains was described qualitatively by visual inspection of genome browsers and gene-order diagrams↳ Could also: A synteny-block scoring method (e.g., MCScanX or i-ADHoRe) could also have been applied to quantify the probability of observed gene-order conservation under a null model of random chromosomal rearrangement — A quantitative synteny score would also provide a statistical basis for distinguishing coincidental co-localization from genuinely conserved chromosomal neighborhoods, particularly useful when the number of co-occurring RNA domains is small
-
Losses and absences were partitioned into three qualitative categories (genuine loss, divergence, missing data) based on expert interpretation of search results↳ Could also: A genome-completeness correction approach (e.g., using BUSCO scores or regression of detection rate on assembly N50/coverage) could also have been applied to estimate the expected detection rate given assembly quality, separating assembly-driven absences from biological losses more formally — Correcting for assembly completeness would also allow quantitative partitioning of absences into assembly-attributable versus biology-attributable fractions, reducing reliance on qualitative expert judgment for the 'missing data' category
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
The MIR106B/MIR93/MIR25 miRNA cluster is absent from bird, turtle, and crocodile genomesother bird down 2015×1papers★ This paper is the founder (earliest)
-
66,879 ncRNA loci across 626 families are conserved in more than 10% of 48 avian genomesother bird 2015×1papers★ This paper is the founder (earliest)
-
RNY5 (Y5 RNA) is absent from all bird genomes but present in alligator and turtle outgroupsother bird down 2015×1papers★ This paper is the founder (earliest)
-
tRNA gene copy number is reduced in birds (~280 copies) relative to human, turtle, and alligator (~900 copies)other bird down 2015×1papers★ This paper is the founder (earliest)
-
34 chicken lncRNA loci are conserved with mammals, of which 12 are validated by RNA-seq expressionother chicken 2015×1papers★ This paper is the founder (earliest)
-
SNORD93 is expanded to 92 copies in the tinamou genome compared to 1-2 copies in other vertebratesother tinamou up 2015×1papers★ This paper is the founder (earliest)
-
865 of 1194 (72.4%) annotated chicken ncRNA loci are confirmed by RNA-seq expressionrna-seq chicken 2015×1papers★ This paper is the founder (earliest)
-
278 of 300 (92.7%) predicted chicken tRNA genes are supported by RNA-seq expression evidencerna-seq chicken 2015×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-25822729
Paper: Gardner PP et al. (2015) "Conservation and losses of non-coding RNAs in avian genomes." PLoS One 10(3):e0121797. PMCID PMC4378963.
Code: https://github.com/ppgardne/bird-genomes (authors' own; commit
9b3ebd87f678c3f908aab17758c0e568e3a94c1d, 2014-08-07; default branch master).
Data: SRA PRJNA204941 (RNA-seq) + the Avian Phylogenomics genome assemblies.
The pipeline (what produced the paper's numbers)
- Annotation of 48 bird genomes (+ human + 3 reptile outgroups) for ncRNAs:
- Rfam 11.0 covariance models scanned with Infernal (
cmsearch, E ≤ 0.0005) →data/rfam/*.gff - tRNAscan-SE →
data/trnascan/*.gff - miRBase homology →
data/mirbase/*.gff - Stadler-group tools →
data/stadler-annotations/*.gff
- Rfam 11.0 covariance models scanned with Infernal (
- Merge / overlap resolution across methods+clans (
scripts/compete_clans.plusingdata/clan_info.txt) →data/merged-annotations/*.gff, and the10%-conserved subset →
data/conserved-merged-annotations/*.gff. - Count-matrix build (
scripts/gffs2heatmaps.pl) →data/R/*.dat(e.g.allRNA.dat: per-family copy number in every species). - Summary table (
scripts/makeTable.pl) → Table 1 of the manuscript: per-RNA-type ncRNA counts in human / median-of-48-birds / chicken, plus the number of chicken ncRNAs confirmed expressed by RNA-seq (max RNA_i > 13.0, thresholds {5,12} on McCarthy+Ulitsky tissue data, with shipped randomized negative controls giving the false-positive rate). - Figures (
scripts/heatmaps.R) → heatmaps/plots fromdata/R/*.dat.
In scope (attempted — clearly specified, low-compute, deterministic)
- Table 1 in full — regenerate via
scripts/makeTable.plfrom the shippeddata/R/allRNA.dat,data/rfam2type.txt,data/RNA-seq/*.dat, anddata/conserved-merged-annotations/Gallus_gallus.gff. Pure-Perl, no external modules, fully deterministic (negative controls are precomputed and shipped). This is step 4 above and reproduces the paper's central quantitative summary.
Out of scope / not attempted (the hard ~20%, by design — 80/20 rule)
- Steps 1–2 (Infernal/tRNAscan/miRBase scan + merge of 48 genomes). The heavy
upstream compute. Its outputs are shipped in the repo (
data/rfam/*.gff,data/trnascan/*.gff, …), so the downstream summary is reproducible without re-running it. Re-running cmsearch over 48 genome assemblies with the exact Rfam 11.0 / Infernal version is the expensive last 20%; not attempted. - Figures (
heatmaps.R) — visual, not numeric claims; skipped (Table 1 is the clearer, machine-checkable target). - Wet-lab / external — none material to the pipeline numbers.
Reproduction strategy: authors' own code + authors' shipped intermediate data, exact pinned commit, run on «our HPC». Equivalent to verifying that the published Table 1 is faithfully and deterministically derivable from the deposited data.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Exact, clean reproduction. All 68 numeric cells of Table 1 plus the Total row (7340/1080.0/1194/865 = 72.4%) and the FPR note (123/1194 = 10.3%) match the published values bit-for-bit, produced by the authors' own makeTable.pl at pinned commit 9b3ebd8 on the repo's deposited data. Every published value is deterministically derivable from the deposited inputs — no fabrication concern. The only nuance, not a defect, is that this verifies the downstream summary from shipped intermediates (the upstream genome scan was not re-run), so it is a code-reuse/downstream reproduction rather than a full independent pipeline rebuild.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at [email protected].
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.