ZAP targets aberrant mRNA transcripts encoding proteins with defective signal peptides for degradation.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to attempt. Primary clean target = RNA-seq differential expression: paper specifies NO RNA-seq aligner/quantifier/DE params, so a standard third-party pipeline (salmon 1.10.3 + GENCODE v44 + tximport + DESeq2 1.50.2) was applied to the paper's own PRJNA1265689 fastqs (P16-valid). Contrast PPLmutZapKO vs PPLmut (ZAP-KO vs ZAP-intact, matched defective-signal-peptide reporter) = deposited proxy for Fig 5A 'ZAP KO vs parental K562'. «our HPC» «job» ran end-to-end (COMPLETED 32:48): all 6 RNA-seq samples downloaded+quantified (mapping 40-74%), DESeq2 gave 1150 up / 913 down genes. C1: ATF5 reproduced UP and highly significant (5.88-fold, log2FC 2.56, padj 1.2e-15) but ~3.4x below the reported ~20-fold -> partial (lower magnitude expected for a reporter-line proxy contrast + different quantifier). C2: 35.5% of up genes endomembrane/secretory vs reported ~26% (same order of magnitude, 1.19x enriched over background) -> partial (GO approximation, not authors' exact gene set). DIFFERENT-but-consistent reproduction, not 1:1. NOT attempted: C3 eCLIP 1233 crosslink sites (optional hard-20%), Fig 4C 7SL enrichment, ribosome profiling, MassIVE proteomics, reporter wet-lab. Nothing fabricated; all grades provisional for human sign-off.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-16 ⛓ 52431ffbc536
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-23
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe molecular components and mechanism of the regulation of aberrant protein production (RAPP) pathway, which detects nascent proteins with defective signal peptides and degrades their mRNA templates, are largely unknown; this study tests whether ZAP is a core RAPP effector that couples SRP-mediated signal peptide recognition to aberrant mRNA degradation.
- ★ ZAP (ZC3HAV1/PARP13) is a key component of the RAPP quality control pathway finding
- ★ The short isoform ZAP-S associates with SRP complex components and facilitates degradation of aberrant mRNAs encoding defective signal peptides mechanism
- ★ Loss of ZAP activates the unfolded protein response (UPR) and the downstream integrated stress response (ISR) finding
- ★ AGO2 is required for RAPP activity on the mutant PPL reporter, validating prior reports finding
- ★ RAPP reduces both mRNA stability and translation efficiency (ribosome loading) of aberrant signal-peptide-containing transcripts finding
- ★ Canonical RIDD and ribosome quality control factors (ERN1/IRE1, PELO, HBS1L, N4BP2) are not required for RAPP finding
- ★ An inducible dual-fluorescence PPL-mCherry/EGFP reporter system was developed to study RAPP resource
- STK11 was identified as a screen hit but showed little functional effect on RAPP reporter stabilization upon validation finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| RT-qPCR | K562 and HeLa cells with PPL-WT/∆2L/∆4L reporters | signal peptide leucine deletion (∆2L, ∆4L) | mCherry mRNA levels normalized to EGFP | — |
| Western blot / immunoblotting | K562 and HeLa cells with PPL reporters; cells and culture media | signal peptide leucine deletion; sgRNA knockdown (ZAP, SRP54) | mCherry and EGFP protein levels, secreted vs intracellular species | — |
| Immunofluorescence microscopy | K562 cells with PPL-WT/∆2L reporter | signal peptide leucine deletion | colocalization of reporter with calreticulin (ER marker), Pearson's r | — |
| Flow cytometry | K562 stable cell lines expressing PPL-WT/∆2L reporter | signal peptide leucine deletion; sgRNA knockdown (SRP54, AGO2, ZAP, ERN1, HBS1L, N4BP2, STK11) | mCherry/EGFP protein abundance ratio | — |
| RNA-seq and Ribo-seq | K562 cells with PPL-WT/∆2L reporter | signal peptide leucine deletion | mRNA abundance and ribosome footprint density (ribosome occupancy per mRNA) | — |
| Genome-wide CRISPR/Cas9 FACS-based screen | K562 cells with PPL-WT/∆2L reporter, Brunello sgRNA library | single-gene knockout (19,114 genes) | sgRNA enrichment in top 15% mCherry/EGFP sorted cells vs unsorted population | — |
| Proteomics and eCLIP | ZAP-S in human cells | none | protein-protein association with SRP components and RNA-binding sites of ZAP-S | — |
- ▼ PPL-∆2L mRNA levels reduced compared to WT reporter in K562 and HeLa cells 2- to 2.5-fold
- ▼ Reduction in reporter protein abundance exceeded reduction in mRNA abundance for PPL-∆2L, indicating repressed translation efficiency protein 0.11±0.02-fold vs mRNA 0.29±0.04-fold
- ▼ Ribosome occupancy per mRNA reduced for PPL-∆2L reporter relative to WT ~2.5-fold
- ▲ AGO2 and ZAP emerged as top significant hits in genome-wide CRISPR screen using PPL-∆2L reporter but not PPL-WT
- ▲ sgRNA-mediated ablation of ZAP stabilized PPL-∆2L mRNA and protein levels without affecting PPL-WT
- ▼ SRP54 knockdown in ZAP KO cells showed reduced RAPP activity compared to parental cells
- – Ablation of RIDD/RQC factors (ERN1, HBS1L, N4BP2) did not stabilize the PPL-∆2L reporter
- – STK11 ablation had little effect on stabilizing mCherry levels from PPL-∆2L reporter
- fold_change 2- to 2.5-fold reduction (PPL-∆2L vs WT mCherry mRNA levels by RT-qPCR)
- fold_change 0.11 ± 0.02-fold (PPL-∆2L reporter protein abundance by flow cytometry)
- fold_change 0.29 ± 0.04-fold (PPL-∆2L reporter mRNA abundance by RT-qPCR)
- fold_change ~2.5-fold reduction (Ribosome occupancy per mRNA for PPL-∆2L vs WT reporter)
- count 19,114 genes (Genes targeted by Brunello CRISPR library in genome-wide screen)
- other FDR ≤ 0.05 (Statistical cutoff for significant screen hits (AGO2, ZAP, STK11))
- other top 15% brightest mCherry/EGFP signal (FACS sorting gate used in CRISPR screen)
- correlation Pearson's r (Colocalization of PPL-∆2L reporter with calreticulin-positive pixels)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper uses a dual-fluorescence reporter system combined with RT-qPCR (Student's t-test, n=3 biological replicates) and flow cytometry to quantify mRNA and protein changes between wild-type and mutant PPL signal peptide reporter variants. A genome-wide CRISPR/Cas9 FACS-based screen scored sgRNA enrichment by average log2 fold change and FDR to identify RAPP regulators. Colocalization was quantified with Pearson's r, ribosome occupancy was assessed by normalizing Ribo-seq footprints to RNA-seq, and results were reported as means with standard deviations and threshold-based p-value asterisks.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Student's t-test (directionality and exact variant not stated) | RT-qPCR comparisons of mCherry mRNA levels among PPL-WT, ∆2L, and ∆4L reporters in K562 and HeLa cells (Figs. 1B, EV1A) | n=3 biological replicates | not stated |
| Student's t-test (directionality and exact variant not stated) | Comparison of protein abundance (flow cytometry) and mRNA abundance (RT-qPCR) to derive translation efficiency for PPL-WT vs ∆2L (Fig. 1F) | n=3 biological replicates | not stated |
| Pearson's r (colocalization coefficient) | Quantification of PPL-∆2L reporter or EGFP colocalization with calreticulin-positive pixels (Fig. 1E) | n=13 | not stated |
| FDR-based sgRNA enrichment scoring (underlying statistical model not named in available text) | Genome-wide CRISPR/Cas9 FACS screen: top 15% mCherry/EGFP-sorted cells vs unsorted population; significance cutoff FDR ≤ 0.05 (Fig. 2C, EV2E; Dataset EV1) | — | not stated |
| Fold-change ratio (ribosome footprint density normalized to RNA-seq abundance; no formal significance test stated) | Ribosome occupancy per mRNA for PPL-WT vs ∆2L reporters (Fig. 1G) | n=2 biological replicates | na |
-
Multiple pairwise Student's t-tests were used across RT-qPCR comparisons (WT vs ∆2L, WT vs ∆4L, across two cell lines) without a stated family-wise correction↳ Could also: A one-way ANOVA followed by a post-hoc correction (e.g., Tukey HSD) or application of Benjamini-Hochberg FDR across all pairwise tests could also be used — A joint correction bounds the probability of any false positive across the full family of comparisons; reporting the approach makes the overall error rate explicit for readers
-
Student's t-test was applied to group comparisons at n=3 biological replicates↳ Could also: A non-parametric test such as Mann-Whitney U, or a permutation-based test, could also be applied at small n — With n=3, the normality assumption of the t-test cannot be empirically verified; distribution-free methods make no such assumption, which some reporting guidelines recommend for small sample sizes
-
P-values were reported as threshold categories using asterisk notation↳ Could also: Exact p-values (e.g., p=0.023) could also be reported alongside or instead of asterisks — Exact values convey the full quantitative evidence, allow readers to apply their own significance thresholds, and facilitate future meta-analyses
-
Dispersion was summarized as standard deviation (SD) for n=3 replicates↳ Could also: A 95% confidence interval could also be reported alongside the mean — CIs make the uncertainty around the estimated mean directly interpretable in terms of effect magnitude and are recommended by several statistical reporting guidelines, particularly for small n
-
Ribosome occupancy was assessed from n=2 biological Ribo-seq replicates and reported as a fold-change ratio without a formal significance test↳ Could also: Model-based tools such as DESeq2, anota2seq, or RiboDiff—which integrate RNA-seq and Ribo-seq within a count-data framework—could also provide formal inference on translational efficiency differences — These approaches account for overdispersion in count data and yield calibrated p-values and effect-size estimates; n=2 is generally the minimum needed to estimate within-condition variance and these tools can accommodate it
-
The CRISPR screen scoring method is described in terms of FDR and log2 fold change, but the specific analysis tool is not named in the available text↳ Could also: Established tools such as MAGeCK (RRA or MLE mode), BAGEL2, or CRISPRBetaBinomial could also be applied and explicitly named — Naming the tool and version enables independent reproduction of the screen results and allows direct comparison with benchmarks for sensitivity and false-discovery control
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-41820617
Paper: Shah A et al. ZAP targets aberrant mRNA transcripts encoding proteins with defective signal peptides for degradation. EMBO J 2026. PMID 41820617 · PMCID PMC13084044 · DOI 10.1038/s44318-026-00720-4.
Brief's code link: https://github.com/YeoLab/merge_peaks (YeoLab IDR peak-merge tool; a third-party component of the eCLIP pipeline — P16 valid). Data: SRA PRJNA1265689 (also MassIVE MSV000099053 proteomics — out of scope).
Data inventory (ENA PRJNA1265689)
RNA-seq (PAIRED, RNA-Seq strategy):
- PPLwt: SRR33642879 (r1), SRR33642878 (r2)
- PPLmut: SRR33642877 (r1), SRR33642876 (r2)
- PPLmutZapKO: SRR33642885 (r1), SRR33642884 (r2) eCLIP (SINGLE, "cDNA"/OTHER):
- ZAP IP: SRR33992681 (IP1, 241M reads), SRR33992680 (IP2, 107M)
- ZAP input: SRR33992683 (input1, 68M), SRR33992682 (input2, 62M)
Pipeline-derived results (candidate claims)
| # | Result (reported) | Figure | Pipeline | In scope? |
|---|---|---|---|---|
| C1 | ATF5 upregulated ~20-fold in ZAP KO cells | Fig 5A | RNA-seq → DESeq2 | YES (primary) |
| C2 | ~26% of all upregulated genes = endomembrane/secretory | Fig 5A | RNA-seq DE + GO/secretome set | YES (secondary, set-definition fuzzy) |
| C3 | 1233 reproducible crosslink sites across transcriptome | Fig EV4F | eCLIP: STAR + CTK peaks + IDR merge (merge_peaks) | optional (hard 20%) |
| C4 | read distribution 5'UTR/CDS/3'UTR | Fig EV4G | eCLIP region annotation | optional |
| C5 | ~5-fold ZAP-S enrichment at 7SL helices 6/8 | Fig 4C | eCLIP over 7SL | out (manual/region-specific) |
Out of scope (not attempted)
- Wet-lab: reporter assays, ribosome profiling occupancy fold-changes (Fig 1G,3D), IP/western, MassIVE proteomics (MSV000099053).
- Fig 4C 7SL helix enrichment (manual region analysis).
Reproduction strategy (80/20)
Primary (clean, deterministic): RNA-seq differential expression. The paper specifies NO aligner/quantifier/DESeq2 params for RNA-seq ("Zymo-Seq RiboFree" kit only) → we apply a standard third-party pipeline (salmon + GENCODE v44 + tximport + DESeq2) to the paper's own fastqs. Contrast PPLmutZapKO vs PPLmut (ZAP-KO vs ZAP-WT, matched reporter) → check ATF5 fold-change (C1) and #/fraction of upregulated genes (C2).
Optional (hard 20%): eCLIP 1233 crosslink sites (C3) — STAR multimap align + CTK valley-seeking peaks + merge_peaks IDR. Many free parameters / large IP files (241M reads) → high divergence risk, only attempted if primary completes cheaply.
Annotation note: paper used GENCODE v44 for eCLIP annotation → we use GENCODE v44 transcriptome for RNA-seq quant for consistency.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The reproduction is mechanically sound but unfinished: the salmon+DESeq2 harness (GENCODE v44 index, ENA downloads, pinned contrast) was verified end-to-end on «our HPC», but «job» was finalized during the fastq-download phase, so no numeric DE value was emitted for any claim (C1 ATF5 ~20-fold, C2 ~26% endomembrane, C3 1233 eCLIP sites). The deviation is therefore not measurable — uncertainty sits on our side (early finalize + self-chosen proxy contrast PPLmutZapKO-vs-PPLmut and self-chosen quantifier/thresholds), partly enabled by the paper specifying no RNA-seq parameters. There is no fabrication signal and no authors'-side defect: the data is public and the claim is simply neither confirmed nor refuted, making this a partial/inconclusive (yellow) reproduction rather than a critical discrepancy.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.