spliceJAC: transition genes and state-specific gene regulation from single-cell transcriptome data.
Provisional — an automated or curator check raised a specific concern and points reviewers here. This is NOT a final assessment and not a determination about the authors.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓Reported values are derivable from the shared data
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce 1:1. The brief's harvested code link (BoolODE) is only the third-party synthetic-data simulator; the paper's actual tool is spliceJAC (first author Bocci), pip splicejac==0.0.1, repo federicobocci/spliceJAC @4939ca9 (MIT). It ships the input (scVelo pancreas) AND the author reference output (results/export_output/transition_score.csv), making the pancreas transition-gene result (Fig 3) a clean target. On «our HPC» we built the pinned env (numpy1.23.2/scanpy1.9.1/scvelo0.2.4/anndata0.8.0/sklearn1.1.2), replicated the repo's Transitions.ipynb pipeline, and regenerated the 7-transition per-gene transition-score table. RESULT: the spliceJAC computation reproduces essentially EXACTLY (per-transition Pearson 0.998-1.000, Spearman 0.985; deterministic seed=100). The endocrine biology reproduces: Iapp->Beta, Gcg->Alpha, Ghrl->Epsilon are exact top-1 matches and Sst->Delta is top-5. The ONLY divergence is the upstream 50-highly-variable-gene SELECTION: 39/50 overlap, and 4 of the reference's top genes (Spp1, Neurog3, Chga, Chgb) fell outside our HVG set, flipping 4/7 top-1 calls; the shipped CSV was most likely generated by the repo's now-removed Pancreas.ipynb (referenced in .ipynb_checkpoints) rather than the current Transitions.ipynb. No fabrication signal: near-perfect numeric agreement on every commonly-selected gene confirms the reported values are genuine pipeline outputs. NOT attempted (the hard 20%): BoolODE synthetic-circuit GRN-inference benchmarking (AUPRC/AUROC, Fig 2), the A549 EMT/GSE147405 analysis, and grn_statistics network stats (its nx.from_numpy_matrix is gone in networkx>=3 and is not needed for transition scores).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 53assessed: 2026-06-15 ⛓ b1e4bb75b1c5
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a bottom-up, theory-oriented framework that models multivariate mRNA splicing from scRNA-seq infer cell state-specific gene-gene regulatory interactions and predict driver genes that govern transitions between cell states?
- ★ spliceJAC quantifies multivariate mRNA splicing from unspliced/spliced count matrices to construct cell state-specific gene-gene (Jacobian) interaction matrices. method
- ★ Spectral (linear stability) analysis of inferred Jacobian matrices predicts the most unstable genes that drive transitions between cell states. method
- ★ spliceJAC accurately reconstructs ground-truth state-specific interaction matrices from in silico multistable circuits. finding
- ★ spliceJAC outperforms existing GRN inference methods (Beeline pipeline) on synthetic circuit data. finding
- ★ Applied to mouse pancreas endothelium development and A549 EMT, spliceJAC predicts state-specific signaling roles and new transition genes. finding
- ★ Linearizing nonlinear splicing dynamics around each attractor enables inference of attractor-specific gene regulation (Eqs. 5a-5b). mechanism
- spliceJAC is provided as an easy-to-use Python package with downstream analysis options. resource
- spliceJAC correctly distinguishes stable from unstable cell states, predicting positive eigenvalues at unstable fixed points. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| In silico simulation of bistable toggle switch circuit | Two-gene (X,Y) mutual-inhibition synthetic circuit | Small stochastic perturbations around fixed points | Inferred state-specific gene-gene interaction matrices vs. analytic ground truth | — |
| In silico simulation of three-gene mutual-activation loop | Three-gene monostable synthetic circuit | Small perturbations around fixed point | Inferred interaction matrix vs. ground truth | — |
| In silico simulation of EMT regulatory circuit | EMT circuit (miR-34, miR-200, ZEB, SNAIL, TGF-beta, E-cadherin, N-cadherin) tuned to tristability | TGF-beta inducer level set for coexistence of epithelial/hybrid/mesenchymal states | Inferred interaction matrices and eigenvalue spectra per state | — |
| GRN inference benchmarking (Beeline pipeline comparison) | In silico circuit states (epithelial, hybrid E/M, mesenchymal) | none | Matrix element absolute difference, fraction of correct signs, AUPRC ratio | Beeline pipeline |
| scRNA-seq splicing analysis (RNA velocity-style unspliced/spliced) | Mouse pancreas endothelium development | none | State-specific gene regulation, signaling hubs, transition driver genes | — |
| scRNA-seq splicing analysis (RNA velocity-style unspliced/spliced) | A549 lung carcinoma cells undergoing EMT | TGF-beta-induced EMT (cell line context) | State-specific gene regulation, transition genes, DEGs | — |
- – spliceJAC reconstructs toggle switch interaction matrices for both stable fixed points with high precision, capturing reversed interaction amplitudes between states.
- – spliceJAC correctly recovers the interaction matrix of the three-gene activation loop.
- – spliceJAC robustly recovers epithelial, hybrid E/M, and mesenchymal interaction matrices from the tristable EMT circuit.
- ▲ spliceJAC predicts a positive eigenvalue at the unstable fixed point of the toggle switch, indicating instability.
- ▲ For mixtures of cells from multiple EMT states, spliceJAC predicts the largest eigenvalue to be positive or near zero, flagging unstable mixing.
- ▲ spliceJAC outperforms existing GRN inference methods on absolute matrix difference, correct sign fraction, and AUPRC ratio.
- – Inference remains robust when genes are removed one at a time from the input count matrix.
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a computational methods paper presenting spliceJAC, a tool that infers cell state-specific gene–gene (Jacobian) interaction matrices from unspliced/spliced scRNA-seq counts and uses linear stability (eigenvalue) analysis to predict transition driver genes. Rather than classical hypothesis testing, the statistical approach centers on regression-based parameter inference and benchmarking: performance is evaluated against analytically derived ground-truth circuits using metrics such as absolute matrix-element difference, fraction of correctly signed matrix elements, and AUPRC (via the Beeline pipeline), and applied to real pancreas-endothelium and A549 EMT datasets. Results are reported largely as inferred matrices, eigenvalue spectra, instability/signaling scores, and comparisons with prior differentially expressed gene analyses.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Linear regression to infer attractor-specific gene–gene interaction (Jacobian) matrices from steady-state assumption | Inference step within each cell state (Eqs. 5a–5b; Fig 1B–C, Fig 2) | number of cells within a given cell state (stated as upper limit on number of inferable interactions); exact n not given in visible text | stated |
| Eigenvalue (linear stability / spectral) analysis of the inferred Jacobian | Stability of cell states and identification of transition driver genes (Fig 1E, Fig EV1) | — | stated |
| Benchmark metric: absolute difference between ground-truth and inferred matrices | In silico EMT circuit comparison across epithelial/hybrid/mesenchymal states (Fig 2I) | — | na |
| Benchmark metric: fraction of correctly signed matrix elements | In silico circuit comparison (Fig 2J) | — | na |
| AUPRC (area under precision–recall curve) ratio, normalized per state | Comparison against GRN inference methods in the Beeline pipeline (Fig 2K) | — | na |
-
Method performance was summarized with point metrics (absolute matrix difference, fraction of correctly signed elements, AUPRC ratio) across in silico circuits.↳ Could also: Reporting these metrics with dispersion across multiple simulation replicates (e.g., mean ± SD/95% CI or full distributions via boxplots) and bootstrap confidence intervals. — Adding variability estimates would convey how stable the benchmark rankings are across stochastic simulations and would complement the central-tendency comparison.
-
spliceJAC was compared against other GRN inference methods using normalized AUPRC ratios.↳ Could also: Complementing AUPRC with AUROC, early-precision, and a paired statistical comparison (e.g., Wilcoxon signed-rank across circuits) of per-circuit scores. — Multiple complementary metrics and a paired test across shared circuits would give a more complete picture of relative performance under class imbalance.
-
Cell annotations (clusters) were assumed to correspond to stable cell states, with stability analysis offered as self-consistent validation.↳ Could also: Sensitivity analyses across alternative clustering resolutions or annotation schemes, reported as agreement statistics. — Quantifying how inference outputs vary with upstream clustering choices would characterize robustness to the state-definition step.
-
Interaction matrices were inferred via linear regression under a steady-state, near-attractor linearization with a unique-solution constraint (genes ≤ cells).↳ Could also: Regularized regression (e.g., ridge/LASSO/elastic-net) with cross-validated penalty selection. — Regularization would allow stable estimation when gene count approaches or exceeds cell number and could improve generalization in high-dimensional, noisy settings.
-
Robustness was assessed by removing genes one at a time and re-running inference.↳ Could also: Bootstrap or subsampling of cells and repeated resampling, reporting the distribution of inferred edges/eigenvalues. — Resampling-based stability assessment would quantify uncertainty in individual inferred interactions and driver-gene rankings.
-
Transition driver genes were prioritized by an instability score derived from the unstable manifold projection.↳ Could also: Attaching empirical significance or confidence to driver-gene scores via permutation/null-model comparison. — A null distribution would help distinguish genuinely high-instability genes from those expected under randomized interaction structure.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
spliceJAC robustly recovers distinct gene-gene interaction matrices for epithelial, hybrid, and mesenchymal states from a tristable EMT regulatory circuit.other in-silico emt-circuit 2022×1papers★ This paper is the founder (earliest)
-
spliceJAC outperforms existing GRN inference methods on absolute matrix element difference, fraction of correctly signed interactions, and AUPRC ratio across epithelial, hybrid, and mesenchymal EMT states.other in-silico emt-circuit up 2022×1papers★ This paper is the founder (earliest)
-
spliceJAC GRN inference is robust to single-gene removal from the input count matrix, with no significant degradation in inferred interactions.other in-silico emt-circuit none 2022×1papers★ This paper is the founder (earliest)
-
spliceJAC predicts a positive or near-zero largest eigenvalue for mixed-state EMT cell populations, flagging state instability from co-mingled epithelial and mesenchymal cells.other in-silico emt-circuit up 2022×1papers★ This paper is the founder (earliest)
-
spliceJAC correctly recovers the interaction matrix of a three-gene mutual-activation circuit at its monostable fixed point.other in-silico three-gene-loop 2022×1papers★ This paper is the founder (earliest)
-
spliceJAC predicts a positive leading eigenvalue at the unstable fixed point of a bistable toggle switch, correctly indicating state instability.other in-silico toggle-switch up 2022×1papers★ This paper is the founder (earliest)
-
spliceJAC reconstructs state-specific gene-gene interaction matrices for both stable fixed points of a bistable toggle switch with high precision, capturing reversed interaction amplitudes between states.other in-silico toggle-switch 2022×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-36321549 (spliceJAC)
Paper: Bocci F, Zhou P, Nie Q. spliceJAC: transition genes and state-specific gene regulation from single-cell transcriptome data. Mol Syst Biol 2022. PMID 36321549 · PMCID PMC9627675 · DOI 10.15252/msb.202211176.
What spliceJAC is
A Python tool that uses spliced + unspliced scRNA-seq counts (RNA velocity inputs) to fit a cell-state-specific linear Jacobian by regression, then does stability analysis: the leading eigenvector of the Jacobian at an unstable steady state ranks transition (driver) genes for each cell-state transition.
Code situation (P16 note)
- The link harvested into the brief (
github.com/Murali-group/BoolODE) is not the paper's analysis code — BoolODE is the third-party synthetic-data simulator the paper used only for its benchmarking circuits (toggle switch, EMT circuit…). - The paper's actual analysis tool spliceJAC is released by the first author:
github.com/federicobocci/spliceJAC(MIT, default branchmain, pipsplicejac==0.0.1). This is what we reproduce — running the paper's own tool on the paper's own bundled data is the most faithful reproduction.
In scope (pipeline-derived, attempted)
| result | pipeline | data | reproducible? |
|---|---|---|---|
| Pancreas transition genes / per-gene transition scores (Fig 3) | spliceJAC estimate_jacobian → transition_genes |
bundled datasets/Pancreas/pancreas.h5ad (scVelo pancreas endocrinogenesis) |
YES — repo ships both the input pancreas.h5ad AND the reference output results/export_output/transition_score.csv (7 transitions × ~50 genes). We regenerate the table and compare 1:1. |
The shipped transition_score.csv is the concrete, machine-checkable reference.
Top-scoring genes per transition are the canonical lineage markers (Ins1/Ins2/Iapp
→ Beta, Gcg → Alpha, Sst → Delta, Ghrl → Epsilon), i.e. the biology the paper
reports for the pancreas in Fig 3.
Out of scope (not attempted — 80/20)
- BoolODE synthetic-circuit benchmarking (AUPRC/AUROC vs other GRN methods,
Fig 2 / EV figs): would require regenerating BoolODE simulations + the separate
jacobian-inference-benchmarkingrepo. Larger effort, deferred (the hard 20%). - A549 EMT / TGFβ (GSE147405) analysis: not bundled; would require fetching + full RNA-velocity preprocessing. Deferred.
- All wet-lab / manual / schematic figure content — out of scope by definition.
Reproduction strategy
- «our HPC» SLURM job, conda env pinned to spliceJAC
requirements.txt(numpy 1.23.2, scanpy 1.9.1, scvelo 0.2.4, anndata 0.8.0, sklearn 1.1.2 …). - Load bundled
pancreas.h5ad, follow the repo's Pancreas/Transitions notebook steps, runestimate_jacobian+transition_genes. - Export our
transition_scoretable; compare to repotransition_score.csv(correlation + top-gene overlap per transition).
All heavy compute on «our HPC»/«infra»; «host» holds only small result artifacts.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The spliceJAC algorithm reproduces essentially exactly — per-transition Pearson 0.998–1.000 and mean Pearson 0.9997 across the 39/50 shared genes, with no fabrication signal. The sole deviation is upstream HVG selection: 4 of the reference's top genes (Spp1, Neurog3, Chga, Chgb) fell outside our 50-gene set, flipping 4/7 top-1 calls (C1–C3, C6), most plausibly because the shipped CSV came from the repo's now-removed Pancreas.ipynb rather than the current Transitions.ipynb. This is a preprocessing/code-version issue on a mix of our pipeline choice and an author-side code mismatch — explainable and non-severe; the well-separated endocrine branches (Iapp→Beta, Gcg→Alpha, Ghrl→Epsilon, Sst→Delta top-5) confirm the central biology.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.