NETISCE: a network-based tool for cell fate reprogramming.
The main results reproduced: recomputed values matched the published ones within tolerance.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
REPRODUCED (primary iPSC case study). Ran the repo's shipped ipsc_validation NETISCE.nf pipeline AS-IS on «our HPC» via SLURM (Nextflow local executor, faithful env pins). Two real environment bugs fixed and documented as faithful adaptations: (1) the bundled ./nextflow launcher regex accepts Java 8..15 only -> pinned openjdk=11; (2) NumPy>=1.24/2.0 removed np.int/np.mat that the 2022-era code uses -> pinned numpy 1.23 / pandas 1.5 / scipy 1.9 / sklearn 1.1 + alias shim. RESULTS: deterministic backbone reproduces EXACTLY (36-node network, 6 FVS control nodes with identical identities, 729=3^6 perturbations). The paper's HEADLINE result reproduces EXACTLY: 15 successful reprogramming perturbations (expanded markers), and all 15 over-express Nanog == the paper's Nanog-OE target. Original-marker filter gave 129 vs reported 132 (within-tol). Only the intermediate stochastic ML-pass count differs (252-273 vs 375) because the pipeline fixes no RNG seed (100k/10k random SFA states, k-means, RF/SVM/NB split) - documented, not a fabrication concern. NOT attempted: the 3 stretch case studies (CRC/ascidian/noise) - outside the primary-target scope. Data note: the in-scope iPSC expression actually derives from GSE62155 (per repo README), not the registry's GSE88928; profiled both. Verdict is provisional - a human signs off.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-26
- Rubric version
- not recorded
- Assessed by
- —
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetCell fate reprogramming targets can be identified in large biological systems using only static network topology (feedback vertex set control combined with signal flow analysis and machine learning), without requiring full dynamical model parameterization or kinetic data.
- ★ NETISCE predicts cell fate reprogramming targets in static (GRN/signaling) networks without needing full kinetic parameterization of a dynamical model. method
- ★ Appropriate concerted perturbations applied to a network's minimal feedback vertex set (FVS) can drive a system from any initial state to any of its attractors. mechanism
- ★ Signal flow analysis (SFA) estimates steady-state node values from network topology and initial states without complete dynamical information. method
- ★ Machine learning clustering of SFA-estimated attractors (from expression data and random initial states) partitions them into phenotype-associated regions to build an attractor landscape. method
- ★ NETISCE reproduces previously experimentally validated cell fate reprogramming targets across developmental, stem cell, and cancer biology systems. finding
- ★ NETISCE can identify potentially novel combinations of reprogramming targets beyond those previously validated experimentally. finding
- NETISCE is implemented as a user-friendly command-line Nextflow pipeline and Galaxy Project workflow usable by non-experts. resource
- The prior tool OCSANA+ implements FVS-finding and SFA for perturbation analysis but cannot incorporate expression data into SFA, estimate an attractor landscape, or automate perturbation configuration, motivating NETISCE. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Signal flow analysis (SFA) simulation | Ascidian embryo (Ciona intestinalis) gene regulatory network, in silico | Combinatorial overexpression/knockout of 6 experimentally identified FVS control nodes (Foxa.A, Foxd, Erk Signaling, Neurog, Tbx6-r.b, Zic-r.b) | Expression values of 7 tissue-specific internal-marker nodes in resulting attractors | — |
| FVS-finding algorithm on network topology | Ascidian embryo GRN (92 nodes, 329 edges) | none | Enumeration of minimal feedback vertex sets (FVSes) | — |
| SFA + k-means clustering attractor landscape estimation and ML classification | Pluripotent stem cell signaling network (36 nodes, 143 edges); EpiSC vs ESC cell states | 729 combinations of perturbations on 6 FVS control nodes (Nanog, Oct4, Klf4, Sox2, Gata6, Tbx3) | Attractor expression values of internal-marker nodes (Oct4, Sox2, Nanog, EpiTFs, plus 3 additional markers) and ML classification of phenotype shift | — |
| Comparative Boolean vs. ODE model simulation | Drosophila melanogaster patterning specification system, in silico | — | — | — |
- ▲ SFA-based simulation of experimentally verified FVS control node perturbations reprogrammed the ascidian embryo model to 6 of 7 tissue fates 85% accuracy (6/7)
- – Pan-neural tissue fate reprogramming was not reproduced via its designated perturbation, but the pan-neural marker (Celf3.a) was upregulated under the brain+pan-neural perturbation combination
- – NETISCE identified 26 total minimal FVSes in the ascidian embryo network, including the exact 6-node FVS experimentally tested by Kobayashi et al. 6 of 26 FVSes matched
- ▲ Of 729 simulated FVS perturbation combinations in the pluripotency network, 375 passed ML classification filtering as shifting attractors toward the ESC-associated cluster 375/729 (~51%)
- – k-means clustering of 100,006 SFA-estimated attractors from the pluripotency network identified an optimal k=2, separating EpiSC- and ESC-initialized attractors into distinct clusters k=2
- – Only one minimal FVS, comprising 6 nodes (Nanog, Oct4, Klf4, Sox2, Gata6, Tbx3), was identified in the pluripotency signaling network
- other 85% accuracy (6/7 tissue fates reproduced) (SFA reprogramming accuracy in ascidian embryo GRN validation)
- count 92 nodes, 329 edges (Size of ascidian embryo GRN)
- count 26 FVSes (Number of minimal feedback vertex sets identified in ascidian embryo GRN)
- count 36 nodes, 143 edges (Size of pluripotency signaling network)
- count 100,006 attractors (Total attractors estimated for pluripotency network (6 from expression data + 100,000 random initial states))
- count 729 combinations tested; 375 passed filter (FVS control node perturbation combinatorics in pluripotency network reprogramming)
- other k=2 (Optimal number of k-means clusters for pluripotency attractor landscape)
- other up to 80% accuracy (Prior validation (ref. 47) of SFA reproducing ODE model steady-states and perturbation biology expression changes)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The paper validates a computational method (NETISCE) that combines signal flow analysis, feedback vertex set control, k-means clustering, and machine learning classification to predict cell fate reprogramming targets in static gene regulatory/signaling networks. Method performance is assessed mainly by comparing in silico predictions to previously published experimental outcomes and reporting overall percentage accuracy (e.g., 6/7 tissue fates reproduced, 85% accuracy in the ascidian embryo case; a separate 80% accuracy figure cited from prior validation of the underlying SFA algorithm), rather than through classical inferential hypothesis testing. Candidate attractors and perturbation combinations are filtered/ranked using unsupervised clustering and supervised machine-learning classifiers with feature-importance analysis. No p-values, dispersion measures, effect sizes, or multiplicity corrections are reported in the provided text.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| k-means clustering, with the number of clusters (k) chosen via elbow and silhouette methods | partitioning SFA-estimated attractors into phenotype-associated clusters for the pluripotent stem cell (EpiSC/ESC) reprogramming network | 100,006 attractors (6 from EpiSC/ESC gene expression initial states plus 100,000 randomly generated initial states) | not stated |
| machine learning classification algorithms (including SVM) with feature importance analysis (top 10 percent ranked features) | filtering the 729 combinations of perturbations on FVS control nodes to identify those shifting simulated attractors from the undesired to the desired phenotype cluster | 729 candidate perturbation combinations; 375 passed the classification filtering criterion | not stated |
| overall percentage accuracy against experimentally reported outcomes | reproducing Kobayashi et al.'s ascidian embryo FVS-perturbation-induced tissue fates (6/7 reproduced, 85% accuracy) and comparison to prior SFA validation against ODE-model steady states (80% accuracy) | 7 experimentally verified tissue-fate perturbation combinations (ascidian embryo case); not stated for the 80% figure | na |
| directional comparison of attractor expression values (described as analogous to log2 fold-change) | evaluating whether internal-marker node expression increased/decreased between unperturbed and perturbed attractors across both validation networks | not stated (based on single deterministic SFA attractor per initial state/perturbation) | na |
-
Model validation was reported primarily as overall percentage accuracy against experimentally known outcomes (e.g., 6/7 tissue fates, 85% accuracy; 80% accuracy vs. ODE steady states).↳ Could also: Reporting additional classification metrics (precision, recall, F1-score, confusion matrix) alongside a confidence interval for the accuracy estimate (e.g., via bootstrap resampling) — These would convey how errors are distributed and quantify uncertainty around the point estimate, which can be especially informative when the total number of tested cases is small (e.g., 7 tissue fates).
-
The optimal number of k-means clusters was selected using the elbow and silhouette methods.↳ Could also: Additional cluster validity indices such as the gap statistic, Davies-Bouldin index, or a Gaussian mixture model with BIC/AIC-based selection — These provide complementary criteria for choosing cluster count and can help confirm that the chosen k is robust across different validity measures for high-dimensional attractor data.
-
Feature importance was assessed by taking the top 10 percent ranked features from each of several classifiers (including SVM) independently.↳ Could also: A model-agnostic attribution method such as SHAP values, or a consensus ranking across classifiers with a stated stability threshold — This would let feature rankings be compared on a common scale across classifiers and support assessment of ranking stability across resampling or cross-validation folds.
-
729 perturbation combinations were filtered via machine-learning classification into a pass/fail criterion, without a stated correction for evaluating many combinations simultaneously.↳ Could also: A multiple-comparison correction (e.g., Benjamini-Hochberg FDR) applied to any per-combination scores, or a permutation-based null distribution for the classification outputs — This would help control the expected rate of false-positive perturbation combinations when many candidates are screened at once.
-
Differences in attractor expression values between perturbed and unperturbed simulations were interpreted directionally (analogous to log2 fold-change) without a reported significance threshold or variance estimate.↳ Could also: Pairing directional comparisons with a formal test (e.g., a permutation test across repeated stochastic simulation runs) or reporting variance across multiple random-initial-state replicates — Since SFA attractors can depend on initial conditions, quantifying variability across independent runs would convey how consistent a directional call is.
-
100,000 randomly generated initial states were used to help estimate the attractor landscape, without a stated random seed or repetition of the sampling procedure.↳ Could also: Reporting the random seed(s) and/or repeating the random sampling procedure multiple times to assess sensitivity of clustering results to sampling variability — This would allow the reproducibility and stability of the attractor landscape estimate to be assessed given the stochastic nature of the sampling.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-35725577 (NETISCE)
Paper: Marazzi, Shah, Balakrishnan, Patil, Vera-Licona (2022). NETISCE: a network-based tool for cell fate reprogramming. NPJ Syst Biol Appl. PMID 35725577 / PMC9209484 / DOI 10.1038/s41540-022-00231-y.
Code: https://github.com/VeraLiconaResearchGroup/Netisce (Nextflow pipeline; Python + Component Pascal [SFA / FVS simulated-annealing] + Shell + R). The repo is the authors' own tool and ships the validation case studies as self-contained sub-pipelines.
What NETISCE does (pipeline stages)
- Attractor-landscape estimation — Signal Flow Analysis (SFA) on normalized expression + 100,000 random initial states → k-means clustering (Elbow + Silhouette) → estimated attractors / phenotype clusters.
- FVS identification — simulated-annealing minimal Feedback Vertex Set over the network's directed cycles.
- Perturbation screening + dual filtering — simulate 3^n perturbation combinations on FVS nodes; filter by (a) ML classifiers (Random Forest / SVM / Naive Bayes) into the target phenotype, then (b) internal-marker-node expression validation (≥90% threshold).
In scope (pipeline-derived, attemptable on «our HPC»)
The repo ships four self-contained validation pipelines, each with its own NETISCE.nf,
nextflow.config, run.sh, input_data/, and results/ (= authors' reference outputs to
compare against):
| RU result | folder | pipeline | reported numbers to reproduce |
|---|---|---|---|
| Toy example | toy_example_results/ (toy_example_1, _2) |
NETISCE.nf | sanity/smoke; exact counts TBD from repo |
| iPSC (GSE88928) | ipsc_validation/ |
NETISCE.nf | 729 perturbations screened → 375 pass ML → 132 pass marker → 15 expanded (Fig 6, Supp Table 5); 36-node/143-edge net; Nanog OE target |
| Ascidian embryo | ascidian_embryo_validation/ |
NETISCE.nf | 6/7 tissue fates reprogrammed (~85%); 92-node/329-edge net (Fig 4, Supp Table 4) |
| Colorectal cancer | colorectal_cancer_validation/ |
NETISCE.nf | 68 FVSes; 1,594,323 (=3^13) combos → 232,114 ML → 52,703 marker; 95-node/337-edge (Fig 8) |
| Noise robustness | noise_studies/ |
(Drosophila ODE / pancreatic SDE) | >92% @30% noise, >75% @50% (Droso); 89% @20%, 84.1% @50% (panc) (Supp T8-9) |
Primary target (paper's named dataset): iPSC / GSE88928 — smallest network (36 nodes), deterministic combination count 3^6 = 729, clearest filtering cascade. Toy example first as smoke test. CRC and ascidian are stretch targets (CRC 3^13 ≈ 1.6M combos = heavy).
Out of scope (not pipeline-derived / not attempted)
- Wet-lab / literature-curated network construction (manual; networks are taken as given inputs).
- Experimental validation of reprogramming targets (Yachie-Kinoshita et al. wet-lab) — comparison only, not reproduced.
- Galaxy web-server deployment.
Reproduction strategy
Run each shipped validation sub-pipeline as-is on «our HPC» via SLURM, comparing reproduced
counts to (a) the paper's reported numbers and (b) the repo's own results/ reference files.
Stochastic stages (SFA random initial states, simulated-annealing FVS, ML classifiers) may not be
bit-identical; grade with tolerance and note seed availability.
Known adaptation needed
The shipped nextflow.config uses executor='slurm' with --qos=general --mem=4G and
--partition=general. «infra» forbids --mem and uses --partition=std. The config must be
patched (remove --mem, set partition=std, set valid QOS) before submission — recorded as a
faithful environment adaptation, not a method change.
Datasets the paper relies on (profiled in same pass)
- GSE88928 — pluripotent stem cell (EpiSC/ESC) expression (primary, named in brief).
- GSE62155 — referenced in iPSC README as source of normalized expression for initial activities (to confirm at the clone).
- CCLE (HT29) — colorectal case expression (external resource, not a GEO deposit of this paper).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.