Single-Cell Differential Network Analysis with Sparse Bayesian Factor Models.
The main results reproduced: recomputed values matched the published ones within tolerance.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
PARTIAL (described well enough; mostly 1:1 where reproducible). Built scSFMnet 0.2.0 fresh on «our HPC» (rstan 2.32.7; «job») -- repo ships STALE pre-built src/*.o that break linking, fixed by removing them before R CMD INSTALL. (1) PRIMARY main vignette (deterministic, shipped diff.gene.mat 50x200, fixed seeds): all 13 scalar corrD/corr0/corr1 reproduce WITHIN-TOL (|delta|<=0.041), top differential pair Gene22-Gene28 + 9/10 Top-10 ranking EXACT; not bit-identical because Stan optimizing()/L-BFGS is only deterministic within a fixed rstan/compiler/BLAS stack (paper pinned none). (2) SIMULATION Table 1 (Sim1 Network A, N=1000): SFM-SHS F8 TPR 0.834 vs 0.782, FDR 0.000 EXACT, AUROC 0.989 vs 0.980; SFM-DHS F10 TPR 0.852 vs 0.809, AUROC 0.964 vs 0.966 (~exact). NOTE the model-fit + metric code is NOT shipped (rendered sim vignette stops at count generation) -- reconstructed from the package's published CI.eval.d (diff edge = corrD 95% bootstrap CI excludes 0) + p.approx vs the 325-pair true differential net; close agreement validates the reconstruction. (3) REAL-DATA GSE162086 608-edge network NOT reproduced -- the 1,833-cell memory-T subset + 130-gene MAST selection are not recoverable from the deposit (104,417 cells deposited; hospitalization split absent); documented + dataset fully profiled. Both datasets (GSE162086 human scRNA-seq+TCR; GSE90975 mouse-microglia mmc.gene.mat source) profiled in data/dataset_profile.json. NOT ATTEMPTED: DAVID GO enrichment (web tool), comparator-tool absolute edge counts (scdNet/DGCA), figure aesthetics.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 81assessed: 2026-06-21 ⛓ 16ce2460b35a
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-25
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper tests whether a sparse hierarchical Bayesian factor model can identify differences in gene-gene co-expression network structures across biological conditions (e.g., treatment vs. control) from scRNA-seq data while accounting for zero-inflation, overdispersion, and cell-to-cell variability inherent to single-cell data.
- ★ A hierarchical Bayesian factor model using treatment-dependent latent factor loadings can construct gene co-expression networks from scRNA-seq data and identify differences in network structure between two (or more) biological conditions. method
- ★ Treatment-dependent parameters (alpha_gf;t) determine which latent factors are activated in a gene, enabling calculation of both within-group co-expression and between-group co-expression differences. mechanism
- ★ Horseshoe priors placed on alpha_gf;t (Single HorseShoe, SFM-SHS) or additionally on the shared parameter alpha-tilde_gf (Double HorseShoe, SFM-DHS) induce sparsity in treatment differences and/or the underlying common network. method
- ★ The model's latent factor structure with a squared-term adjustment accounts for zero-inflation and overdispersion typical of scRNA-seq data while preserving the intended marginal mean per treatment group. mechanism
- ★ A nonparametric bootstrap procedure (resampling cells, obtaining posterior mode estimates via Stan's optimizing function) is the preferred estimation strategy compared to full HMC sampling or single posterior optimization. finding
- Under the conditional Poisson model, the marginal variance of gene expression exceeds the marginal mean, demonstrating the model produces overdispersion. finding
- The methodology's overall differential network results remain relatively consistent across different choices of the tuning parameter F (number of latent factors). finding
- ★ The methodology is demonstrated on simulated datasets and a real SARS-CoV-2 case study dataset. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| scRNA-seq (simulated count data) | simulated data based on mouse microglia cell gene expression parameters | simulated treatment vs. control group assignment with fixed differential correlation structure | gene-gene correlation/network differences (estimated vs. true differential edges) | NORTA algorithm via R package SimCorMultRes (rnorta); ZIM package for zero-inflated negative binomial parameter estimation; Stan for model fitting |
| scRNA-seq | SARS-CoV-2 case study dataset | biological condition (case study group comparison, not further specified in provided text) | differential gene-gene co-expression network structure | Stan (HMC, posterior optimization, bootstrap) |
- – Simulation schemes generated datasets with G=50 genes (Sim 1-3) or G=100 genes (Sim 4), and N=500, 1,000, or 2,000 total cells, split equally into control and treatment groups.
- – True differential network structure was fixed at 325 differing edges for Sim 1-3 (G=50) and 1,300 differing edges for Sim 4 (G=100). 325 edges (Sim1-3); 1,300 edges (Sim4)
- ▲ Marginal variance of Y_gi under the model exceeds the marginal mean, confirming overdispersion relative to a standard Poisson.
- count 325 differential edges (true differential network structure in Sim 1, Sim 2, Sim 3 (G=50 genes))
- count 1,300 differential edges (true differential network structure in Sim 4 (G=100 genes))
- count N = 500, 1,000, or 2,000 cells (total simulated cell counts across four simulation schemes, split equally between control and treatment groups)
- other half-Cauchy(0,1) priors on tau_f, kappa_gf;t, zeta, omega_gf, sigma_beta, sigma_delta (hyperprior specification for horseshoe shrinkage parameters in SFM-SHS and SFM-DHS models)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper proposes two sparse hierarchical Bayesian factor models (SFM-SHS and SFM-DHS) for differential gene co-expression network analysis in scRNA-seq data, using horseshoe priors to induce sparsity in treatment-dependent factor loadings and a conditionally Poisson likelihood to handle zero-inflation and overdispersion. Inference is performed via Hamiltonian Monte Carlo in Stan, or alternatively via posterior mode (MAP) optimization paired with a nonparametric bootstrap to generate confidence intervals for pairwise gene-gene correlation differences between two groups. Performance is evaluated on four simulation schemes (varying gene count G and cell count N) using data generated from zero-inflated negative binomial distributions, and on a SARS-CoV-2 real-data case study, with results reported as ranked differential network edges identified by credible interval or bootstrap CI exclusion of zero and approximate posterior p-values.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Bayesian posterior credible interval (CrI) for gene-gene correlation and differential correlation (approximate posterior p-value = proportion of posterior outside the smallest CrI including zero) | Determination of significant within-group gene-gene edges and differential edges between treatment groups under full HMC inference | N=500, 1,000, or 2,000 total cells (split equally between groups) in simulations; real-data n not stated in provided text | not stated |
| Nonparametric bootstrap confidence interval for gene-gene correlation difference θ_gg' (cells resampled with replacement within each group; preferred inference method) | Quantification of variability around MAP estimates of differential network edges; applied to both simulated and real data | N=500, 1,000, or 2,000 total cells in simulations; real-data n not stated in provided text | not stated |
| Posterior mode (MAP) optimization via Stan 'optimizing' function | Computationally lighter alternative to full HMC; used as the point estimate underpinning the bootstrap procedure | — | not stated |
| Hamiltonian Monte Carlo (HMC) MCMC sampling via Stan | Full Bayesian posterior inference for factor loadings, gene-gene correlations, and differential network structure | — | not stated |
-
Significant gene-gene edges are declared when zero is excluded from the posterior CrI (or bootstrap CI), with the approximate posterior p-value used only for ranking↳ Could also: Local false discovery rates (local FDR) or Bayesian FDR control (e.g., via posterior inclusion probabilities) could also be applied across the full family of G(G-1)/2 tested pairs — With hundreds to thousands of gene-pair comparisons, an explicit multiplicity-aware decision rule directly controls the expected proportion of false discoveries among declared edges, providing a complementary interpretation to CrI exclusion of zero
-
The observation model is conditionally Poisson, with overdispersion arising marginally through integration over the latent factors↳ Could also: A negative binomial or zero-inflated negative binomial likelihood could also be specified directly at the observation level — Specifying overdispersion directly in the likelihood yields an explicit, interpretable dispersion parameter and can more flexibly capture mean-variance relationships that may not be fully accounted for by the factor structure alone
-
Sparsity in factor loadings is induced via continuous horseshoe priors (half-Cauchy global and local shrinkage parameters)↳ Could also: Spike-and-slab priors (a point mass at zero mixed with a diffuse slab) could also induce sparsity in factor-gene relationships — Spike-and-slab priors produce explicit binary inclusion indicators for each factor-gene link, enabling direct inference on inclusion probabilities and potentially cleaner separation between active and inactive loadings, at the cost of more complex MCMC samplers
-
Uncertainty in MAP estimates is quantified via a nonparametric bootstrap (cells resampled with replacement within groups), which the authors describe as their preferred approach↳ Could also: Full MCMC sampling via HMC (also implemented and described in the paper) could also be used as the primary inference strategy — Full MCMC provides exact posterior uncertainty quantification under the specified generative model without the approximation introduced by bootstrapping around a point estimate; the trade-off is higher computational cost, which the authors acknowledge as the motivation for the bootstrap alternative
-
Simulation datasets are generated from zero-inflated negative binomial distributions via the NORTA algorithm, with parameters estimated from an independent mouse microglia dataset — a generating model that deliberately differs from the Poisson factor model used for estimation↳ Could also: Dedicated scRNA-seq simulation frameworks (e.g., Splatter, scDesign2/3) could also be used to generate benchmark data — Purpose-built scRNA-seq simulators can reproduce a broader range of data characteristics (batch effects, multi-modal expression, cell-type heterogeneity) and offer systematic parameter sweeps, which can provide additional evidence about a method's robustness across diverse realistic scenarios
-
Gene-gene networks are characterized using marginal correlations of log-means derived from factor loadings, summed across all active factors↳ Could also: Partial correlations or precision-matrix-based approaches (e.g., graphical LASSO, or a Bayesian Gaussian graphical model) could also define and compare conditional gene-gene associations — Partial correlations condition on all other genes simultaneously and thus distinguish direct associations from indirect ones mediated by a third gene; comparing sparse precision matrices across conditions is a well-established alternative paradigm in differential network analysis that may reduce spurious indirect edges in dense networks
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-35186014 (scSFMnet)
Paper: Sekula, Gaskins, Datta (2022). Single-Cell Differential Network
Analysis with Sparse Bayesian Factor Models. Front Genet 12:810816.
PMCID PMC8855158 · DOI 10.3389/fgene.2021.810816.
Repo: https://github.com/mnsekula/scSFMnet @ 065237fafee2c5a4835013dd41f2906acbfbfec0 (v0.2.0, 2021-12-22).
Data accession: GEO GSE162086 (Bacher et al. 2020, SARS-CoV-2-reactive memory T cells).
Method (what the pipeline does)
scSFMnet = sparse hierarchical Bayesian factor model for differential co-expression
network analysis of scRNA-seq. Latent factors with treatment-dependent loadings;
horseshoe priors give sparsity. Two variants: SFM-SHS (single horseshoe) and
SFM-DHS (double horseshoe). Inference: HMC (rstan), single-mode optimization,
or bootstrap optimization (recommended) via Stan's optimizing(). R package
scSFMnet (R + Stan via rstan). Key functions: sfm(), sfm.results(),
summary(), controlCorr(), treatCorr(), sfm.hmc().
IN SCOPE (pipeline-derived, reproducible)
A. Main usage vignette — DETERMINISTIC, shipped data + fixed seeds [PRIMARY 1:1 TARGET]
Uses shipped data(diff.gene.mat) (50 genes × 200 cells; 100 control C + 100
treatment T), t_i = rep(0:1,each=100), and sfm() with fixed seeds. Every
printed number in scSFMnet.html is a regenerable target:
res1 = sfm(..., Fac=7, mod.type="SHS", boot.iter=10, ori.data=TRUE, seed=123)→summary(sfm.results(list(res1)))point estimates corrD.controlCorr/treatCorrpoint estimates (corr0, corr1).resO = sfm(..., boot.iter=1, ori.data=TRUE, seed=12345)single optimization → Top-10 ordered corrD (Gene22–Gene28 = −1.1949 …). These are the cleanest auditable claims: deterministic given Stan optimizer + seed.
B. Simulation study (Tables 1–3) — reproducible, heavier [SECONDARY]
scSFMnet-Simulation.html: estimate ZINB params from mmc.gene.mat (ZIM::zim,
set.seed(600)), build Network A/B correlation structure, simulate counts
(SimCorMultRes::rnorta, set.seed(12345), G=50, N=500/1000/2000), fit SFM-SHS/
SFM-DHS bootstrap, compute TPR/FDR/AUROC vs the known true differential network.
Targets: Table 1 metrics (e.g. Sim 1 Net A SFM-SHS F=8 TPR=0.782 FDR=0.000
AUROC=0.980). Reproducible but the exact per-cell numbers depend on the full
(partly truncated in the rendered vignette) simulation code; we reconstruct from
the vignette source in the repo on «our HPC».
C. Real-data application (GSE162086) — CONDITIONAL [SECONDARY / may be PARTIAL]
Paper: G=130 DE genes (MAST, log2FC ≥ log2(1.2)) over 1,833 memory-T cells
(462 non-hosp + 1,371 hosp); SFM-SHS = 608 differential edges; scdNet = 6,690;
388 edges common to all 4 methods; shared DCGs RPS26/RGCC/RPL3; CD200/TNFSF8 up
in hospitalized, IFI44L/XAF1 up in non-hospitalized. Whether this is fully
reproducible depends on whether the shipped mmc.gene.mat IS the 130-gene memory-T
matrix or only a parameter-estimation subset; the 130-gene DE selection step may
require GEO raw counts (MAST on full GSE162086). To be checked on «our HPC».
OUT OF SCOPE (not pipeline / not attempted)
- DAVID GO-enrichment web-tool results (manual web submission, not in repo).
- Cytoscape/igraph figure rendering aesthetics (Figs 6/7 layout).
- Comparator tools DGCA / scdNet absolute counts (separate third-party packages; reproduced only if time permits as cross-checks, not primary).
- Wet-lab / clinical metadata (hospitalization status assignment from Bacher et al.).
Pipelines per result
| Result | Pipeline | Scope |
|---|---|---|
| Main vignette corrD/corr0/corr1, Top-10 | scSFMnet sfm()+sfm.results() on shipped diff.gene.mat, fixed seeds |
IN (primary) |
| Table 1–3 sim metrics | ZIM+SimCorMultRes sim → scSFMnet bootstrap → TPR/FDR/AUROC | IN (secondary) |
| Real-data 608 edges, DCGs | MAST DE on GSE162086 → scSFMnet | conditional |
| GO enrichment, comparator counts | DAVID web / DGCA / scdNet | OUT |
Compute plan
All c
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.