Bayesian prediction of RNA translation from ribosome profiling.
Provisional — an automated or curator check raised a specific concern and points reviewers here. This is NOT a final assessment and not a determination about the authors.
The main results reproduced, with only marginal, non-material deviations.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
REPRODUCED (partial, well-described, 1:1 attempt on the paper's own data). Rp-Bp 4.x (authors' own actively-maintained tool; method unchanged since v1.x) run on HEK293 sample SRR2433794 with GENCODE v19/GRCh37.p13. Table 3 HEADLINE reproduces strongly: total predicted ORFs 16,733 vs reported 17,198 (97.3%); dominant canonical class 11,574 vs 11,056 (within-tol); uORF 2,597 vs 2,244 and dORF 319 vs 383 also within tolerance. ncRNA (1,744 vs 2,201) and canonical_variant (490 vs 1,097) are PARTIAL and 'other' (9 vs 217) is a MISMATCH -- all explained by the documented v1.x->v4 ORF-type taxonomy change (v4 split 'internal'->intORF, renamed labels, reclassifies borderline ncRNA/variant) plus MCMC non-determinism, NOT a pipeline failure. Table 2 read-filtering also reproduces: raw reads 31,967,082 = reported 3E7 (exact), ribosomal-removed 6.6% vs 7% (within-tol); usable/periodic 48.7% vs 59% and non-periodic differ because v4 selected 5 periodic read lengths (25,26,28,29,30; offset -12) and rejected the abundant 27-mer on Bayes-factor grounds (genuine v1->v4 method drift). COMPUTE NOTE: estimate-orf-bayes-factors (per-ORF Stan MCMC over 560,473 candidates) exceeded a single 12h «our HPC» node both oversubscribed (2212967) and exclusive (2214712); solved by splitting into 8 «infra» chunk jobs (2218900/2218937/2219404, ~2h each on dedicated nodes), merging the Bayes-factor BEDs, and running select-final-prediction-set (2219410) + orf_type grouping (2219617). Dataset SRR2433794 profiled grade A (N matches SRA exactly, strong 3-nt periodicity). NOT ATTEMPTED: exact counts (impossible); other 4 Table-1 datasets (separate RUs); downstream figures + wet-lab MS validation (out of scope). All grades PROVISIONAL pending human audit.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-23
- Rubric version
- not recorded
- Assessed by
- —
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper tests whether an unsupervised Bayesian approach that models the 3-nt periodicity of ribosome profiling (ribo-seq) signal can accurately predict translated ORFs, including automatically selecting which read lengths and P-site offsets carry periodic signal, without requiring manually labeled training ORFs.
- ★ Rp-Bp is an unsupervised Bayesian approach that uses a two-component 'high-low-low' mixture model to predict translated ORFs from ribosome profiles method
- ★ Rp-Bp uses Bayesian model selection (Bayes factors via MCMC/Stan) to incorporate and propagate uncertainty when labeling ORFs as translated or untranslated method
- ★ BPPS (Bayesian Periodic fragment length and P-site offset Selection) automatically selects periodic read lengths and their P-site offsets, allowing distinct offsets per read length without manual intervention method
- ★ Automated BPPS read length selection modestly improves sensitivity by identifying more canonical and non-canonical ORFs than manual expert selection finding
- ★ Proteomics- and QTI-seq-based validation verifies the high quality of Rp-Bp's ORF translation predictions finding
- ★ Rp-Bp produces more peptide identifications and more proteomics-validated ORF predictions than RiboTaper finding
- Unlike ORF-RATER and riboHMM, Rp-Bp is unsupervised and thus not biased toward ORFs resembling a priori labeled training examples mechanism
- Rp-Bp naturally identifies all translated ORFs exhibiting periodicity on a transcript, regardless of how many occur on the same transcript mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| ribosome profiling (ribo-seq) metagene profiling | HEK293 cells | none | 5' read-end counts around annotated translation start sites, used to assess 3-nt periodicity per read length | — |
| proteomics | — | none | peptide identifications used to validate predicted translated ORFs | — |
| quantitative translation initiation sequencing (QTI-seq) | — | none | independent sequencing-based validation of predicted translation/ORFs | — |
| computational benchmark comparison (Rp-Bp vs RiboTaper) | ribo-seq datasets | none | number of peptide identifications and proteomics-validated ORF predictions | — |
- – Reads of length 21 bp show clear 3-nt periodicity in HEK293 metagene profiles while 20 bp reads do not
- ▲ Automated BPPS selection identifies more canonical and non-canonical ORFs than manual selection by an expert modest (not numerically specified in this excerpt)
- – Numbers of variant and out-of-frame predicted ORFs remain similar between automated and manual read length/offset selection
- ▲ Rp-Bp yields more peptide identifications than RiboTaper
- ▲ Rp-Bp yields more proteomics-validated ORF predictions than RiboTaper
- other log BF > 5 (threshold considered 'very strong' evidence favoring the periodic model H_p over the non-periodic model H_n)
- other k = 0.5 (probability threshold constant: H_p is selected when P(log BF_Y > 5) > k)
- count 200 MCMC iterations (first 100 discarded as burn-in, final 100 used to fit a normal approximation) (Stan NUTS sampler settings used to approximate marginal likelihoods for the Bayes factor)
- count metagene window spanning 50 bp upstream to 20 bp downstream of annotated TSS (window used to construct the metagene profile for each read length)
- count read lengths of 20 bp and 21 bp compared (example read lengths shown in HEK293 metagene profiles (Figure 1A) illustrating periodic vs non-periodic behavior)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper introduces Rp-Bp, an unsupervised Bayesian computational pipeline for predicting translated ORFs from ribosome profiling data. The core statistical framework uses two-component mixture models and Bayesian model selection via Bayes factors to distinguish periodic (translationally active) from non-periodic read-length metagene profiles, and to label each ORF as translated or untranslated. Posterior distributions of model log-likelihoods are estimated with Hamiltonian MCMC (Stan/NUTS), explicitly propagating inferential uncertainty through a final probability statement on the Bayes factor. Validation is performed by comparing counts of predicted ORFs and proteomics-identified peptides against those produced by RiboTaper, and by cross-referencing with QTI-seq data.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Bayesian model selection via Bayes factors (log BF, threshold > 5 with posterior probability k > 0.5) between periodic two-component mixture model (H_p) and best-fitting non-periodic alternative (H_n) | BPPS phase: classifying each ribo-seq read length as periodic or non-periodic from its metagene profile | Seven codons (21 positions) of metagene profile per read length; 200 MCMC iterations per fit (100 burn-in, 100 retained); dataset-level n not stated in provided text | not stated |
| Hamiltonian MCMC (No-U-Turn Sampler via Stan) to approximate marginal log-likelihoods and derive posterior Bayes factor distribution modeled as N(mu_p - mu_n, sigma_p + sigma_n) | Both BPPS (read length periodicity selection) and ORF translation prediction phases | 200 MCMC iterations per model (100 burn-in, 100 used for normal fit); ORF-count basis not stated in provided text | not stated |
| Bayesian model selection via posterior Bayes factor distribution for per-ORF translation labeling | Translation prediction phase: each ORF labeled translated or untranslated based on P(log BF_Y > 5) > 0.5 | — | not stated |
| Empirical count comparison of predicted ORFs and proteomics-validated peptide identifications | Benchmarking Rp-Bp vs RiboTaper across canonical and non-canonical ORF categories | — | na |
-
Metagene profile periodicity is assessed via Bayesian model selection between a two-component mixture model and non-periodic alternatives, yielding a Bayes factor↳ Could also: A frequentist chi-squared goodness-of-fit test or discrete Fourier transform power at the 1/3 frequency could also quantify 3-nt periodicity — Spectral or chi-squared approaches are computationally lighter and their p-values are familiar to broad audiences; the ORFscore and RiboTaper (multitaper) methods use analogous ideas, making them natural reference points for readers comparing frameworks
-
Posterior marginal log-likelihoods are approximated using 200 Hamiltonian MCMC iterations (100 burn-in, 100 retained) per metagene profile↳ Could also: Variational inference (e.g., ADVI available within Stan) or a Laplace approximation could also approximate posterior distributions — Variational methods generally converge faster and scale better when many thousands of ORFs must be evaluated, at the cost of potentially underestimating posterior variance relative to full MCMC
-
Each ORF is classified translated/untranslated using a fixed posterior-probability threshold k = 0.5 on log BF > 5, applied independently per ORF↳ Could also: A Bayesian FDR analogous to the Storey q-value (computed on posterior inclusion probabilities across all ORFs) could also control the global false-discovery rate — When thousands of ORFs are evaluated simultaneously, a genome-wide FDR criterion explicitly characterizes the expected proportion of false-positive translated calls, which can aid interpretation of the full prediction set
-
Benchmark comparison between Rp-Bp and RiboTaper is based on counts of predicted and proteomics-validated ORFs at each tool's default operating threshold↳ Could also: Precision-recall curves or ROC analysis sweeping across Bayes factor thresholds could also characterize performance — Threshold-free metrics give a fuller picture of each tool's sensitivity/specificity trade-off and allow comparison at operating points other than those chosen by each tool's defaults
-
The unsupervised two-component mixture model treats aggregated in-frame vs out-of-frame read counts as the periodicity signal within each ORF↳ Could also: A hidden Markov model (as in riboHMM) could also capture the sequential codon-by-codon structure along the ORF — HMMs explicitly represent positional dependencies, which may better accommodate variable ribosome density across an ORF; the paper acknowledges riboHMM as a related approach, so the trade-off between the two is directly relevant context for readers
-
MCMC convergence is assessed implicitly (Stan/NUTS is stated to guarantee convergence in long runs); specific convergence diagnostics (R-hat, effective sample size) are not mentioned in the provided text↳ Could also: Reporting R-hat statistics and effective sample sizes per model fit could also characterize MCMC mixing quality — With only 100 post-burn-in samples, convergence diagnostics provide readers with direct evidence that the normal approximation to the log-likelihood distribution is reliable, particularly for ORFs with sparse read coverage
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — PMID 28126919 (Rp-Bp)
Paper: Malone, Atanassov, Aeschimann, Li, Großhans, Dieterich. Bayesian prediction of RNA translation from ribosome profiling. Nucleic Acids Research 45(6):2960–2972, 2017. DOI 10.1093/nar/gkw1350. PMCID PMC5389577.
Tool / code: https://github.com/dieterich-lab/rp-bp (authors' own tool, P16 N/A
— it IS the authors' code). Rp-Bp = unsupervised Bayesian prediction of translated
ORFs from ribosome profiling. Latest release 4.0.1 (bioconda rpbp); paper
described v1.x. We use the bioconda release on the paper's own data.
Data (this RU): sra:SRR2433794 = the paper's Human HEK293 Ribo-seq sample
(Table 1). GSM1887643 / GSE73136 / SRP063852, Illumina HiSeq 2000, single-end 47 nt,
31,967,082 reads. (Other Table-1 datasets — HEK293-Gao SRR1630831, mouse liver/MEF
SRR1630812/16, C. elegans SRR50263xx — are OUT OF SCOPE for this RU; only SRR2433794
is tagged here.)
In scope (pipeline-derived results we attempt)
The Rp-Bp pipeline applied to SRR2433794 produces a set of predicted translated Ribo-seq ORFs, classified by biotype. Table 1 of the paper reports, for HEK293:
| metric | reported |
|---|---|
| total predicted ORFs | 17,408 |
| canonical | 11,056 |
| uORF | 2,244 |
| dORF | 383 |
| ncRNA | 2,201 |
These are the comparison targets. Pipeline: Flexbar (adapter trim) → Bowtie2 (in-silico rRNA removal) → STAR (splice-aware genome alignment, GENCODE v19 / GRCh37) → Rp-Bp metagene periodicity / P-site offset estimation (Stan MCMC) → ORF Bayes-factor prediction → biotype labelling. Reference: GENCODE v19 annotation + GRCh37.p13 genome (paper Methods).
Commands (Rp-Bp 4.x):
prepare-rpbp-genome config.yaml— build STAR + Bowtie2 indices, extract ORFs, label them.run-all-rpbp-instances config.yaml— Flexbar/Bowtie2/STAR + periodicity + prediction.
Out of scope / not attempted (the hard ~20%)
- The other 4 organisms/datasets in Table 1 (separate RUs / accessions).
- Exact bit-for-bit match: the Bayesian step uses MCMC (cmdstanpy) and is not
fully deterministic; the authors' own regression test compares predictions only
on the intersection with tolerance on Bayes factors. Annotation/version drift
(GENCODE v19 release vintage, Rp-Bp v1 → v4) further shifts exact counts. We
therefore expect within-tolerance / partial agreement on counts, not
exact. - Downstream figures (Fig 2–6: periodicity plots, comparison vs ORF-RATER/ribotaper, micropeptide validation by MS) — manual/external, not attempted.
Validation step (cheap, de-risks the toolchain)
Before the heavy human run, run the repo's shipped C. elegans chrI example (deterministic reference output bundled for regression tests) end-to-end to confirm the conda install + cmdstanpy model compilation + full pipeline work.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.