Harnessing secretory pathway differences between HEK293 and CHO to rescue production of difficult to express proteins.
The main results reproduced, with only marginal, non-material deviations.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- 🟡A deviation arose in the data or preprocessing
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH and reproduced 1:1 (independent re-run after the room was requeued and its «infra» workdir reclaimed). The Lewis-Lab CHO_HEK repo ships the per-sample Salmon quant.sf for all 78 GEO samples plus all reference DBs and the figures.Rmd analysis code; the only missing inputs were the two genome GFFs (public NCBI RefSeq), re-obtained release-matched (CHO CriGri_1.0 rel103 2018 GCF_000223135.1, human GRCh38.p13 109.20200815 GCF_000001405.39) and used to build symbol-level tx2gene via the authors' makeTxDbFromGFF->select(GENEID,TXNAME) (with TXNAME version-stripped to match the version-less salmon Names). Running the authors' exact driver (tximport -> CHO->human ortholog collapse -> DESeq2 -> fgsea) reproduced: (A) '>80% of genes DE between the two organisms' = 87.4% of tested genes; (B) Table 2 log2FC for 4/5 named genes to ~0.02-0.08 (HSPA1B 12.89->12.81, SRP9 2.85->2.83, ATF4 1.71->1.69, HSPA8 -2.42->-2.44), the 5th (EIF2AK2) lost only to ortholog-symbol namespace drift; (C) the secretory pathway is significantly enriched among differentially-activated genes (fgsea all.secM padj=1.99e-4 < 5e-4) with Fig5C subsystem directions consistent (glycosylation/trafficking HEK-high, folding/UPR CHO-high). One sub-claim (exact top-20 hypergeometric) did not reproduce. Grades are identical to the 2026-06-20 archived run, confirming stability across two independent rebuilds. No fabrication concern: every reproduced number is derivable from the shipped data+code. Infra note: this pass was repeatedly blocked by an account-wide «infra»+HOME quota wall on shared user «user»; I worked around it by running compute node-locally on /tmp and did NOT delete other rooms' data. NOT attempted: wet-lab/experimental results and chunks depending on files outside the repo (correctly out of scope).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 69assessed: 2026-06-20 ⛓ bd0182bc4fde
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-22
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper tests whether switching the expression host from CHO to HEK293 can rescue secreted production of difficult-to-express human proteins, and whether transcriptomic differences in secretory pathway gene expression between the two cell lines can identify metabolic engineering targets to improve productivity.
- ★ Swapping expression host from CHO to HEK293 improves secreted titers for roughly one third of difficult-to-express human proteins finding
- ★ Neither ExpiCHO nor QMCF CHO platforms is universally superior; each favors expression in a protein-feature-specific manner (size, glycosylation) finding
- ★ Transcriptome comparison reveals secretory pathway utilization differs between CHO and HEK293, driven largely by a limited set of extreme outlier genes rather than broad systemic differences finding
- ★ Co-expressing secretory pathway genes ATF4, SRP9, JUN, PDIA3 and HSPA8 (identified as HEK293-enriched outliers) boosts recombinant protein productivity when added to CHO cells finding
- ★ More heavily glycosylated recombinant proteins benefit more from HEK293's elevated N- and O-glycosyltransferase activity finding
- Transgene mRNA abundance does not correlate with secreted protein titer finding
- ★ A methodology combining transcriptomics and co-expression screening can identify secretory pathway metabolic engineering targets for CHO and HEK293 method
- Some HEK293-enriched secretory pathway genes (e.g. EIF2AK2, RAB11FIP1, MGAT3, DERL3, SVIP1, GALNT18) sharply decrease titer when overexpressed in CHO, showing host-context-dependent effects finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| transient recombinant protein expression and titer quantification | ExpiCHO cells vs CHOEBNALT85 (QMCF) | transgene expression of 22 difficult-to-express human proteins | secreted protein titer | — |
| semi-stable episomal (QMCF) recombinant expression | CHOEBNALT85-1E9 vs 293ALL | transgene expression of 24 difficult-to-express human proteins | secreted protein titer (western blot) | — |
| transient recombinant expression | Freestyle 293-F / 293-F vs Freestyle CHO-S | transgene expression of 24 difficult-to-express human proteins | secreted and lysate protein titer (western blot) | — |
| targeted protein quantification (SIS PrEST) | culture supernatants from CHO and HEK293 | none (validation of titer measurements) | relative protein titer | LC-MS/MS |
| transcriptome profiling | CHO and HEK293 cell lines, expressing and non-expressing (empty plasmid) cells | recombinant transgene expression vs empty plasmid | mRNA expression levels across molecular process/secretory pathway gene sets, transgene transcript levels | — |
| co-expression screening of secretory pathway genes | CHO and HEK293 cells | co-expression of 21 secretory pathway helper genes with model protein THBS4 at varying plasmid ratios | secreted THBS4 titer | — |
| co-expression validation | CHO cells | co-expression of selected secretory pathway genes with ARTN | secreted ARTN titer, viable cell density/viability at harvest | — |
- – Titer fold change between ExpiCHO and QMCF correlated with protein size R=-0.47, p=0.028
- ▲ Titer fold change between ExpiCHO and QMCF correlated with glycosylation R=0.4, p=0.078
- ▲ 9 of 24 genes showed >2-fold improved expression in HEK293 (293ALL) vs CHO in semi-stable QMCF system >2-fold
- ▲ 15 of 24 proteins showed >2-fold higher secreted titers in HEK293 vs CHO in transient expression >2-fold
- ▲ 8 of 24 genes consistently expressed better in HEK293 in both semi-stable and transient setups; only PLG consistently better in CHO 8/24 genes
- ▲ ATF4 or SRP9 co-expression improved secreted THBS4 titer in CHO >2-fold
- ▲ JUN co-expression increased secreted THBS4 titer in CHO 1.5-fold
- ▲ ATF4, PDIA3 and HSPA8 co-expression significantly increased ARTN secretion in CHO
- correlation R = -0.47, p = 0.028 (titer fold change (ExpiCHO vs QMCF) vs protein size)
- correlation R = 0.4, p = 0.078 (titer fold change (ExpiCHO vs QMCF) vs glycosylation)
- count 84% (recently approved monoclonal antibodies produced in CHO cell lines)
- count ~35% (proportion of human secretome project proteins challenging to express in CHO)
- count 9/24 (genes with >2-fold improved expression in HEK293 vs CHO, semi-stable QMCF comparison)
- count 15/24 (proteins with >2-fold higher secreted titer in HEK293 vs CHO, transient comparison)
- fold_change >2-fold (THBS4 titer increase in CHO upon ATF4 or SRP9 co-expression)
- fold_change 1.5-fold (THBS4 titer increase in CHO upon JUN co-expression)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The study used a multi-platform comparative design to evaluate secreted recombinant protein titers of 22–24 difficult-to-express human proteins across CHO and HEK293 cell-line variants. Primary expression comparisons relied on a >2-fold threshold applied to point-estimate titers (with a pseudocount of +1) rather than formal inferential tests, while Pearson correlations linked protein features to titer fold-changes. Co-expression experiments with 21 secretory pathway helper genes were evaluated for statistical significance, though the specific tests were not named in the provided text. Transcriptome differences between cell lines were characterised through RNA-seq-based gene expression profiling with differential gene expression analysis (named as a keyword but method not specified in the provided excerpt).
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Pearson correlation (R) | Titer fold-change (ExpiCHO vs QMCF) correlated with protein size (R=−0.47, p=0.028) and glycosylation (R=0.4, p=0.078) — Fig. 1B | 22 proteins | not stated |
| Unnamed significance test (result described as 'significant') | Co-expression of individual helper genes (HSPA1B, ATF4, AGAP2, SRP9, JUN, etc.) with THBS4 in HEK293 and CHO at plasmid ratios 1:2, 1:10, 1:1 — Fig. 4B, 4C | — | not stated |
| Unnamed significance test (result described as 'significant') | Co-expression of ATF4, PDIA3, HSPA8, HSPA1B, SRP9 with ARTN in CHO — Fig. 4D | — | not stated |
| Unnamed correlation (described as 'significant negative correlation') | R-protein titer vs secretory pathway gene expression in protein folding and ER glycosylation subgroups — Supplemental Fig. S3H | — | not stated |
| Differential gene expression analysis (method unspecified in provided text) | Transcriptome comparison between CHO and HEK293 cell lines; identification of secretory pathway gene outliers — Fig. 3, Fig. 4A | — | not stated |
| Fold-change threshold (>2-fold, with pseudocount +1) as binary classification criterion | Classifying expression improvement from CHO to HEK293 in semi-stable (Fig. 2B) and transient (Fig. 2C) expression systems | 24 proteins | na |
-
Expression improvement between cell lines was classified using a >2-fold threshold on point-estimate titers (with pseudocount) rather than by formal statistical testing↳ Could also: A linear mixed-effects model or per-protein t-test/Mann-Whitney U with Benjamini-Hochberg FDR correction across the 24 proteins could also have been applied — Formal tests would quantify the probability that each observed fold-change arose by chance given within-condition variability, and FDR correction would account for the family of 24 simultaneous protein comparisons, allowing readers to evaluate which individual improvements are statistically supported
-
Co-expression helper-gene experiments used an unnamed significance test, with results reported only as 'significant' or 'not significant' without test statistics or p-values↳ Could also: A one-way ANOVA followed by Dunnett's post-hoc test (each helper gene vs. a shared no-helper control) could also have been applied and reported with exact p-values — Dunnett's test is designed for many-to-one comparisons and controls the family-wise error rate across all 21 helper genes tested against a common control, while naming the test and reporting exact values allows independent reproducibility assessment
-
Pearson correlation (R) was used to relate protein features (size, glycosylation) to titer fold-changes↳ Could also: Spearman rank correlation could also have been used — Protein titers span orders of magnitude and may not meet Pearson's normality and homoscedasticity assumptions; Spearman is robust to outliers and monotonic non-linear relationships, which is relevant when fold-change distributions are heavily skewed
-
Secretory pathway gene outliers between cell lines were identified by visual inspection of expression-level plots rather than a formal statistical criterion↳ Could also: Formal differential expression analysis with DESeq2 or edgeR (Wald or likelihood-ratio test with Benjamini-Hochberg FDR) could also have been used to rank and threshold outlier genes — Formal DEG frameworks account for sequencing depth, biological variability, and multiple testing across the transcriptome, providing confidence estimates for each gene's differential status and reducing the risk of selecting noise-driven outliers for the co-expression screen
-
Protein titers were reported as point estimates (fold-changes or concentrations) with no stated measure of dispersion↳ Could also: Reporting SD or 95% CI alongside each point estimate would also convey within-condition variability — Dispersion metrics help readers judge whether titer differences between conditions exceed within-group noise, and are especially informative when the number of replicates is small or not stated
-
The number of biological replicates underlying the protein titer comparisons and co-expression experiments was not stated in the provided text↳ Could also: Explicit reporting of the number of independent biological replicates per condition, and ideally a power or sample-size justification, could also have been included — Knowing the replication structure allows readers to evaluate reliability of effect estimates and significance claims; a power calculation would contextualise the sensitivity to detect biologically meaningful titer differences
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Reproduction scope — PMID 35301123
Paper: Harnessing secretory pathway differences between HEK293 and CHO to rescue production of difficult-to-express proteins. Metab Eng 2022. PMCID PMC9189052, DOI 10.1016/j.ymben.2022.03.009. Code: https://github.com/LewisLabUCSD/CHO_HEK (cloned @ commit 1b581ffd, last commit 2020-09-05). Data: GEO GSE157729 / SRA SRP281874 (RNA-seq, HEK293 vs CHO producer & non-producer cells).
What the repo ships (key finding)
The repo is self-contained for the RNA-seq pipeline: it ships the per-sample
Salmon quant.sf outputs (not raw FASTQ) under CHO/salmon_{TG,noTG}/ and
HEK/salmon_{TG,noTG}/, plus all reference DBs (databases/: CHO↔human ortholog
table, MSigDB v6.2 GMTs, UniProt PTM), the sample/titer sheet
(Final concentrations per sample update_V4.xlsx), and the analysis code
(figures.Rmd, ppi_assist_func.R, figures.R).
- Salmon dirs: CHO 42 TG + 42 noTG; HEK 81 TG + 81 noTG. quant.sf transcripts: CHO RefSeq (Cricetulus griseus, XM_/NM_/XR_, ~40.8k tx), HEK RefSeq (human GRCh38, ~158.7k tx).
- The only inputs NOT shipped: the two genome GFF annotations used to build the
transcript→gene (tx2gene) maps —
«path»and«path». These are public NCBI RefSeq and were re-obtained (release-matched): CHO = CriGri_1.0 annotation release 103 (2018) (GCF_000223135.1; 70% of quant tx → 18.4k genes), human = GRCh38.p13 / release 109.20200815 (GCF_000001405.39; 94% of quant tx → 37.8k genes). GENEID is taken as gene SYMBOL, matching the authors'makeTxDbFromGFFoutput (the ortholog table keys on symbols).
Pipeline (per Methods + figures.Rmd)
Salmon quasi-mapping (shipped) → tximport (tx2gene, gene-level) → cross-species
ortholog collapse (CHO symbol → human symbol via shipped table) → DESeq2 →
results + fgsea GSEA against MSigDB v6.2 + manually-curated secretory-pathway
gene set (secM).
IN SCOPE (pipeline-derived, attempted)
| id | claim (paper) | reported | pipeline | repro object |
|---|---|---|---|---|
| A | >80% of genes significantly DE between the two organisms (CHO vs HEK293) | ">80%" of genes, padj<0.05 | tximport→DESeq2 ~producer+cellLine.binary, betaPrior=T |
results(dds.cellLine), fraction padj<0.05 |
| B | Table 2 top differentially-activated (cell-line) genes, log2FC HEK-vs-CHO | HSPA1B 12.89; EIF2AK2 12.71; SRP9 2.85; ATF4 1.71; HSPA8 −2.42 | DESeq2 ~cellLine.binary, lfcShrink(type='normal') |
res_cellLine.binary |
| C (stretch) | Top differential genes enriched for secretory pathway (Fig 5C / hypergeom p<0.0005); GSEA secretory NES | enrichment of secM | fgsea(secM, MSigDB) on ranked LFC | gsea.cellLineBinary |
OUT OF SCOPE (not pipeline / not reproducible from shipped data)
- Wet-lab titers (ELISA protein concentrations,
titercolumn) — experimental. - Per-transgene transient/QMCF expression-improvement counts ("9/24", "15/24
2-fold") — derived from wet-lab titers, not the pipeline.
- Bayesian regression (
rethinking/Stan) of titer vs PTM-enzyme expression (Fig 5D) — depends on wet-lab titers + heavy MCMC; out of the core RNA-seq scope. - Chunks referencing files outside the repo (
../GSEA.R,../Austin_glyco/…,../../Sequence/2020_ExpiCHO/…for the "DE2020" Expi/Icosagen comparison) — those inputs are not in the repo; not attempted.
Approach note (P16)
Authors' own repo + own data. tx2gene rebuilt from release-matched public RefSeq
GFFs (the only missing input); dropInfReps=TRUE (we don't use bootstraps);
DESeq2 1.50.2 vs authors' ~2020 version (exact LFC magnitudes may drift slightly;
the >80%-DE fraction and ranking are robust). All else runs the authors' exact
figures.Rmd chunk code via ppi_assist_func.R helpers.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.