Discovery and functional interrogation of SARS-CoV-2 RNA-host protein interactions.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🔴A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described-well-enough and largely 1:1 for the PIPELINE-DERIVED results whose code is shipped. The repo (juliabelk/sarscov2_chirp_ms) ships both the processed data tables and the R analysis code; I re-ran the exact code paths headlessly on «our HPC» (env repro-pmid33, R 4.3.3, SLURM «job»). EXACT reproductions: Huh7.5 and Vero ChIRP-MS PCA variance (35.5/12.9 and 31.2/24.2, identical to shipped eps). WITHIN-TOL reproductions: CRISPR mini-pool SARS-CoV-2 hit counts 182 proviral / 347 antiviral vs reported 179 / 343 (antiviral % 25.8% matches exactly; off by 2-4 genes, attributable to p.adjust/data-freeze), and mitochondrial pool 59/177 vs 57/175. PARTIAL: the headline ChIRP-MS interactome (309 = 229 Huh + 163 Vero - 83 core; FDR<=0.05, log2FC>0) is NOT reproducible from the repo because the MS significance-test code was NOT shipped (only the figure-generation code for the enrich>=1 'expanded' sets is). My independent reconstruction of the described t-test yields a union of ~264-301 proteins (right magnitude) but with an inverted cell-line split, so the exact 229/163/83 depend on unshipped Perseus parameters (S0, permutation FDR). NOT ATTEMPTED (out of scope, data not in repo): genome-wide CRISPR screen (131 factors = 33 core + 98 expanded), all wet-lab/imaging/mitochondrial functional assays. Data-availability gap recorded: raw mass-spec proteomics has no public ProteomeXchange/PRIDE/MassIVE accession — only processed spreadsheets on GitHub.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 84assessed: 2026-06-18 ⛓ 80e4a4fab499
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-18
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusHow does the SARS-CoV-2 viral RNA physically interact with host proteins during active infection, and which of these RNA-binding host factors functionally influence infection outcome? The paper tests whether an RNA-centric map of vRNA-host protein interactions can reveal shared and SARS-specific host pathways and functional antiviral factors.
- ★ ChIRP-MS identifies 309 host proteins that bind SARS-CoV-2 RNA during active infection across Huh7.5 and Vero E6 cells. finding
- ★ ChIRP-MS is an RNA-centric method using formaldehyde crosslinking to recover protein complexes associated with viral RNA during infection. method
- ★ Comparative ChIRP-MS across SARS-CoV-2, Zika, Dengue, and rhinovirus defines viral specificity of RNA-host protein interactions, with a shared core of ~425 proteins. finding
- ★ Targeted/CRISPR screens reveal that the majority of functional SARS-CoV-2 RNA-binding proteins protect the host from virus-induced cell death. finding
- ★ Host mitochondria serve as a general organelle platform for antiviral activity against SARS-CoV-2. mechanism
- ★ SARS-CoV-2 RNA and viral proteins largely interact with distinct host protein complexes, demonstrating orthogonality of RNA-centric vs PPI approaches. finding
- ★ The catalog provides a comprehensive resource of functional SARS-CoV-2 RNA-host protein interactions to nominate therapeutic host pathways. resource
- The vast majority of core ChIRP-MS factors are expressed in primary human lung SARS-CoV-2 target cell types. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| ChIRP-MS (comprehensive identification of RNA-binding proteins by mass spectrometry) | Huh7.5 human hepatocyte cells and Vero E6 monkey kidney cells | SARS-CoV-2 infection (24 and 48 h.p.i.) vs mock | host and viral proteins enriched on viral RNA | 108 biotinylated oligonucleotide probes tiling vRNA; mass spectrometry |
| ChIRP-RNA-seq | Huh7.5 and Vero E6 cells | SARS-CoV-2 infection vs mock | viral and host RNA enrichment / % reads mapping to viral genome | — |
| SDS-PAGE protein staining | Huh7.5 and Vero E6 cells | SARS-CoV-2 infection (24/48 h) vs mock | total enriched protein recovered after ChIRP pull-down | — |
| CRISPR screens (genome-wide and focused mini-pool) | infected host cells | gene knockout | effect on virus-induced cell death / antiviral function | — |
| Comparative ChIRP-MS (inter-virus) | Huh7.5 cells (SARS-CoV-2, ZIKV, DENV); HeLa cells (rhinovirus) | viral infection | expanded interactome host factors; GO term/PCA comparison | — |
| single-cell RNA-seq re-analysis | primary human lung tissue (30,700 non-immune cells, 17 cell types) | none | expression of core interactome factors, ACE2, TMPRSS2 | — |
- – 309 host factors bound SARS-CoV-2 RNA aggregated across two cell lines (163 in Vero E6, 229 in Huh7.5) 309 proteins
- – Core set of 83 host factors co-bound in both cell lines 83 proteins
- ▲ vRNA enrichment after ChIRP pull-down increased reads mapping to viral genome from 2.7% to 60% (Huh7.5) and 14.4% to 68% (Vero E6) at 48 h.p.i. 2.7%→60%; 14.4%→68%
- – 13 of 26 viral proteins reproducibly enriched by ChIRP-MS 13/26
- – 425 proteins shared across all ChIRP-MS viral datasets (~1,000 total in expanded interactomes) 425 of ~1,000
- – Only 11/332 host factors (3.3%) from the PPI study overlapped with the ChIRP-MS network 11/332, 3.3%
- – Majority of RAP-MS factors (30/47, 64%) also enriched in ChIRP-MS; ChIRP-MS enriched 199 additional proteins 30/47, 64%; +199
- – 219/229 (95.6%) of core human ChIRP-MS factors expressed in SARS-CoV-2 lung target cell types; 215/219 at level ≥ ACE2 219/229, 95.6%
- count 309 host proteins (total host factors binding SARS-CoV-2 RNA across both cell lines)
- count 163 (Vero E6) and 229 (Huh7.5) (host factors bound to vRNA per cell line)
- count 83 factors (core set co-bound in both cell lines)
- other FDR ≤ 0.05, LFC > 0 (threshold defining high-confidence interactomes)
- count 30/47 (64%) (RAP-MS factors also enriched in ChIRP-MS)
- count 11/332 (3.3%) (PPI host factors overlapping ChIRP-MS network)
- count 219/229 (95.6%) (core ChIRP-MS factors expressed in lung target cells)
- count 425 proteins (shared across all viral ChIRP-MS datasets in expanded interactome)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper uses ChIRP-MS (comprehensive identification of RNA-binding proteins by mass spectrometry) to pull down host proteins bound to the SARS-CoV-2 genomic RNA in two cell lines (Huh7.5 and Vero E6) at 24 and 48 h post-infection, identifying high-confidence interactors by FDR and log fold-change (LFC) thresholds relative to mock-infected controls. Cross-virus comparative analysis of four positive-strand RNA viruses used PCA and UpSet plots on expanded interactomes, while GO term analysis characterised pathway representation per virus. Integration with a published human lung scRNA-seq dataset used Louvain clustering and dimensionality reduction to assess expression of ChIRP-MS hits in disease-relevant cell types. The paper text provided is truncated before the CRISPR screen statistical methods are described.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| FDR-based enrichment threshold (FDR ≤ 0.05, LFC > 0) for mass-spectrometry protein enrichment | High-confidence host-protein interactomes in Vero E6 and Huh7.5 cells versus mock (Figures 2A, 2B, S3D) | — | not stated |
| LFC-only threshold (average LFC ≥ 0 or ≥ 1, reproducible across replicates) for expanded interactome | Cross-virus expanded interactome comparisons and RAP-MS vs ChIRP-MS enrichment correlations (Figure S3D, Table S3) | — | not stated |
| Principal component analysis (PCA) | Inter-virus comparison of ChIRP-MS enrichment profiles across SARS-CoV-2, ZIKV, DENV, and RV (Figure 3D) | — | na |
| Gene ontology (GO) term analysis | Characterisation of expanded interactomes for each virus (Figure 3F); specific GO tool and correction method not stated in provided text | — | not stated |
| Louvain graph-based clustering | Clustering of single-cell RNA-seq profiles from primary human lung tissue (Figures S4A, S4B) | 30,700 cells after doublet (score > 0.15) and CD45-positive cell removal | not stated |
-
The primary high-confidence interactome was defined by FDR ≤ 0.05 combined with LFC > 0 (any positive enrichment), while a secondary 'expanded' interactome applied a more stringent LFC ≥ 1 filter without a formal FDR gate↳ Could also: A single volcano-plot threshold requiring both a minimum LFC (e.g., ≥ 1) and FDR ≤ 0.05 simultaneously could serve as the primary filter, with the looser set as a sensitivity analysis — Applying a minimum effect-size cutoff alongside statistical significance for the primary set distinguishes biologically meaningful enrichment from statistically significant but small-magnitude associations, which is especially relevant in high-depth proteomics where very small differences can reach significance
-
Dataset overlaps (ChIRP-MS vs RAP-MS; ChIRP-MS vs PPI) were reported descriptively as counts and percentages↳ Could also: A hypergeometric test or Fisher's exact test against a defined proteome background could also quantify whether each overlap exceeds chance expectation — Formal significance testing of set overlaps provides a probability that the co-identification rate is greater than background, complementing descriptive percentages and enabling comparison across overlap pairs
-
PCA was used to visualise and interpret inter-virus differences in ChIRP-MS enrichment profiles↳ Could also: Hierarchical clustering with a heatmap, or UMAP dimensionality reduction, could also represent inter-virus and inter-time-point relationships in the same enrichment matrix — Hierarchical clustering makes pairwise distances explicit and can reveal sub-groupings not apparent in two PCs; UMAP can preserve local non-linear structure; both are common alternatives in multi-condition proteomics comparisons
-
GO term analysis was applied to the expanded interactome of each virus to characterise pathway representation↳ Could also: Gene Set Enrichment Analysis (GSEA) on continuously ranked LFC values, or over-representation analysis (ORA) with explicit Benjamini-Hochberg correction and reporting of the background gene set, could also be used — GSEA avoids the binary membership cut-off required by ORA and uses the full enrichment gradient; reporting the correction method and background set for GO analysis aids reproducibility regardless of the tool chosen
-
scRNA-seq clustering used the Louvain algorithm at an unstated resolution parameter, with a fixed doublet-score threshold of 0.15 for quality filtering↳ Could also: The Leiden algorithm (which avoids the poorly connected communities that can arise with Louvain) could also be used; sensitivity analyses across resolution values, and doublet detection with tools such as DoubletFinder or Scrublet with data-driven thresholds, are also standard — Leiden clustering is increasingly preferred in single-cell workflows for producing better-connected partitions; reporting the resolution parameter and its sensitivity ensures the cluster structure is interpretable and reproducible
-
The number of independent biological replicates for ChIRP-MS experiments is not stated in the provided text, though 'reproducible enrichment' is used as a filter criterion↳ Could also: Explicit reporting of biological replicate counts (e.g., n = 3 independent infection experiments) and the statistical model used to aggregate replicates (e.g., mixed-effects model or moderated t-statistic in limma) could also accompany the FDR threshold — Minimum proteomics reporting standards (e.g., MIAPE) recommend stating biological replicate numbers; knowing the replicate structure allows readers to assess the power behind the FDR estimates and the reliability of the enrichment calls
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-33743211 (Flynn et al. 2021, Cell)
Paper: "Discovery and functional interrogation of SARS-CoV-2 RNA-host protein interactions." Code: github.com/juliabelk/sarscov2_chirp_ms @ commit 7484226c6fbd83de8aa09917317eff5a95b0c69a Data: GEO GSE167341 (sequencing) + processed MS/CRISPR spreadsheets shipped in the repo.
In scope (pipeline-derived, code + data shipped in repo) — ATTEMPTED
- ChIRP-MS PCA (Huh7.5 + Vero) —
pca_plt()in crispr_utils.R, on the log2-imputed LFQ enrichment tables. Reference = variance % printed in shipped example_outputs/*-pca.eps. - CRISPR survival screen hit-calling — crispr_utils.R pipeline
(normalize_counts -> create_condition_list -> guide_zscores -> gene_zscores), gene-level
z-scores, proviral (z>0) / antiviral (z<0) at FDR<=0.001. Two shipped pools:
exp(RNA-interactome mini-pool) andmito(mitochondrial pool), 7 viruses each. Reference = the per-virus counts the authors' owncrispr_volcano()prints in figure titles and the numbers stated in the paper text (SARS-CoV-2: 179/343 exp, 57/175 mito). - ChIRP-MS "expanded" sets + UpSet/heatmap inputs — chirp_utils.R, enrich>=1 over mock. Reproduced as derived values (no single clean paper number to grade against).
Partially in scope — hit-calling test NOT shipped
- Headline 309-protein SARS-CoV-2 RNA interactome (229 Huh + 163 Vero - 83 core; FDR<=0.05 & log2FC>0, infected-vs-mock triplicates). The repo ships ONLY the figure code operating on enrich>=1 sets; the per-protein significance test (done in Perseus) that produced 229/163/83/309 is NOT in the repo. Attempted by independent reconstruction of the described t-test (see AUDIT.md) -> magnitude-consistent (~264-301) but not bit-identical.
Out of scope — NOT attempted
- Genome-wide CRISPR screen ("131 factors = 33 core + 98 expanded", FDR<=0.05): genome-wide count table NOT in repo (only the two targeted mini-pools are shipped).
- All wet-lab / imaging / functional-validation results (knockdowns, viral titers, microscopy, mitochondrial functional assays): manual/experimental, not pipeline-derived.
- Raw FASTQ-level reprocessing of GSE167334/GSE167336: not needed — repo ships the processed count/LFQ tables that are the direct inputs to the published figures.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
For every result whose analysis code was actually shipped, the reproduction is 1:1 (PCA 35.5/12.9, 31.2/24.2) or within-tol (CRISPR 182/347, 59/177 vs 179/343, 57/175) — clean, with deviations explained by p.adjust/data-freeze. The one substantive gap is the headline 309-protein interactome: the MS significance-test code and its Perseus parameters (S0, permutation FDR) were never deposited, so an independent t-test reconstruction yields a magnitude-consistent ~264-301 but an inverted cell-line split. This sits on the authors' side (incomplete code/params plus undeposited raw MS), not a methodological error of ours, and is not fabrication-suspect since the magnitude reconstructs. Net: solid, reproducible study with one under-specified headline number → overall yellow.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.