Artificial Intelligence Approach in Machine Learning-Based Modeling and Networking of the Coronavirus Pathogenesis Pathway.
The main result did not reproduce in this reproduction attempt. Where our recomputation produced values that differ from the published ones, those discrepancies are listed below. This is a single automated attempt — not peer review and not a finding of error or misconduct — and differences can also arise from data access, undocumented parameters or the computing environment. The verdict can be contested via “report an error”.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🔴Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🔴A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🔴Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🔴Overall, the reproduction showed a material discrepancy
This paper has a computational component, but its primary data is legally or ethically access-restricted — identifiable patient cohorts, rare-disease genomes, or controlled-access biobanks that cannot be openly shared. The reproduction therefore could not be attempted. That is a neutral verdict: it does not mean the result is wrong or that the authors fell short — only that, for legitimate privacy reasons, it cannot be independently checked from public data. We deliberately do NOT assign a 0–100 score here, because a low number would wrongly read as a failed reproduction.
▸Reproduction agent’s raw note
DROP / data_unavailable, with a strong no-fabrication PROVENANCE finding. Paper: Tanabe et al. 2025, 'AI Approach in ML-Based Modeling and Networking of the Coronavirus Pathogenesis Pathway' (Curr Issues Mol Biol; DOI 10.3390/cimb47060466). The three headline ML results — CNN pre-transfer acc 0.30 (R1), VGG16 transfer-learning acc 0.65 (R2), GNN acc 0.40 (R3) — CANNOT be independently re-executed: both notebooks load their 100 input pathway-diagram images (50 activated / 50 inactivated) at runtime from the authors' PRIVATE Google Drive (drive.mount -> /content/drive/MyDrive/CNN-corona2/data/ and .../GNN-corona2/). Those images are screenshots of commercial IPA (Qiagen) pathway diagrams, hand-labelled, NEVER deposited: not in the GitHub repo (git ls-files at HEAD a97cd30 -> 0 image/data files; only notebooks + Japanese coursework PDFs), not in GEO, not in any public store. The cited GEO accessions (GSE154613/156754/147507/17400) are upstream expression data fed to IPA, not the model input. WHAT WE ESTABLISHED THIS PASS (auditable): (a) PROVENANCE re-verified on a fresh «infra» clone — every reported number is exactly the value saved in the cited notebooks' own committed cell outputs at the cited commits (R1=cell26 'acc= 0.3' with sensitivity 0.0; R2=cell42 'Accuracy： 65.0 %'; R3=cell3 'Final Test Accuracy: 0.4000'), so the numbers are genuine outputs of the shipped code, NOT fabricated; (b) a metric-labelling DISCREPANCY — the paper presents 0.3 and 0.65 as a before/after pair of test accuracies, but 0.3 is a true 10-image test-set accuracy while 0.65 is the final-epoch (epoch49) VALIDATION accuracy (val_acc 0.6500); the VGG model is never scored on the test set; (c) model-quality caveats — R1 predicts a single class (sens 0.0), R3 is flat 0.40 across all 100 epochs (no learning) on an n=100 hand-built image set; (d) the blocker is DATA, not env/docs — «our HPC» «job» rebuilt the exact env (torch 2.12.1 / torchvision 0.27.1 / torch-geometric 2.8.0) and ran the verbatim CNN/VGG16/GNN code on synthetic images, COMPLETED 0:0. NOT attempted: IPA Table 1 z-scores (commercial tool, out of scope); independent re-run of R1/R2/R3 (impossible without the private input images). All grades are PROVISIONAL and must be checked by a human; no fabrication detected.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessmentassessed: 2026-06-16 ⛓ 8c75d19040d5
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-22
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe study aims to develop an artificial intelligence/machine learning model, based on images of the coronavirus pathogenesis pathway generated by pathway analysis, that can predict the activation state of this pathway from gene expression data.
- ★ The coronavirus pathogenesis pathway is activated in SARS-CoV-2-infected iPSC-derived cardiac cells and in SARS-CoV/SARS-CoV-2-infected LUAD cells finding
- ★ A CNN-based prediction model (with VGG16 transfer learning) can classify coronavirus pathogenesis pathway images as activated or inactivated method
- ★ Transfer learning improved the CNN model's classification accuracy compared to the model before transfer learning finding
- ★ The CNN image-based model outperformed a graph neural network model built on the same pathway data finding
- ★ SARS-CoV-2 infection analyses show correlation with diffuse-type gastric cancer networks, sharing upstream regulators such as TP53 and let-7 finding
- Grad-CAM can be used to interpret which regions of pathway images drive the AI's activation-state classification method
- Several drugs (e.g., telmisartan, acetaminophen, arsenic trioxide) target nodes within the coronavirus pathogenesis pathway resource
- TGFβ1-SMAD3, NFκB/FOS/JUN, and interferon type I signaling pathways are activated during coronavirus infection, contributing to lung fibrosis, hypercytokinemia, and adaptive immunity mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| IPA network/pathway analysis (activation z-score) | ACE2-transfected A549 lung adenocarcinoma cells | SARS-CoV-2 infection (0.2 or 2 MOI) | activation z-score of coronavirus pathogenesis pathway and canonical pathways | Ingenuity Pathway Analysis (IPA), dataset GSE147507 |
| IPA network/pathway analysis (activation z-score) | Calu-3 bronchial epithelial LUAD cell line | SARS-CoV Urbani strain infection (0.1 MOI) vs mock | activation z-score of coronavirus pathogenesis pathway | IPA, dataset GSE17400 |
| IPA network/pathway analysis | LUAD cells (amlodipine series) | SARS-CoV-2 infection | activation z-score / pathway analysis | IPA, dataset GSE154613 |
| IPA Analysis Match (comparative network analysis) | iPSC-derived cardiomyocytes and cardiac fibroblasts vs diffuse-type gastric cancer tissue | SARS-CoV-2 infection (0.001-0.1 MOI) vs mock | shared upstream regulators, causal network regulators, and downstream disease/function entities | IPA, dataset GSE156754 |
| RNA sequencing / pathway overlay | Skin-derived induced pluripotent stem cells (iPSCs) | SARS-CoV-2 infection (0.006 MOI) vs mock | gene expression changes (e.g., TP53, STAT3, JAK1, STAT2, EP300, ATP6AP1, DPP9, ZC3HAV1, NUP98) and pathway activation | IPA, dataset GSE156754 |
| Convolutional neural network image classification (with Grad-CAM and VGG16 transfer learning) | Coronavirus pathogenesis pathway images (100 images: 50 activated, 50 inactivated) derived from IPA analyses | none (binary classification training) | model classification accuracy on test dataset | Python 3.11, Google Colaboratory |
| Graph neural network modeling | Coronavirus pathogenesis pathway network data | none | model classification accuracy, compared to CNN model | Python code generated by Google Gemini 2.5 Flash |
| IPA drug-target network analysis | Coronavirus pathogenesis pathway nodes | none/other (in silico drug interaction mapping) | drugs targeting pathway nodes and their predicted activation state effect | IPA |
- ▲ CNN model accuracy on test dataset was 0.3 before transfer learning and increased to 0.65 after VGG16 transfer learning 0.3 to 0.65 (accuracy)
- ▼ Graph neural network model accuracy was 0.4, lower than the CNN transfer learning model's 0.65 0.4 vs 0.65
- ▲ Coronavirus pathogenesis pathway activation z-score was elevated in SARS-CoV-2-infected A549 cells across 8 analyses (GSE147507)
- ▲ TP53, STAT3, JAK1, STAT2, EP300, ATP6AP1, DPP9, ZC3HAV1, and NUP98 were upregulated in SARS-CoV-2-infected iPSCs compared to mock
- – Analysis Match identified shared upstream regulators (TP53, let-7, CDKN2A, calcitriol, NUPR1, SMARCB1, MEF2D, decitabine, SPARC, RB1) between diffuse-type gastric cancer and SARS-CoV-2 analyses
- – let-7 activation z-score was similar when comparing diffuse-type gastric cancer data to SARS-CoV-2 analyses
- other accuracy 0.65 (65.0%) (CNN/VGG16 transfer learning model accuracy on test dataset after transfer learning)
- other accuracy 0.3 (CNN model accuracy on test dataset before transfer learning)
- other accuracy 0.4 (Graph neural network model accuracy)
- count 100 images (50 activated, 50 inactivated) (Training data for machine learning classification of pathway activation images)
- count 70 training / 20 validation / 10 test images (Dataset split for CNN model)
- count 49 analyses total (22 LUAD, 27 non-LUAD) related to human and SARS coronavirus 2 (IPA database search results filtered from 106 SARS-CoV-related analyses)
- count 13 analyses (GSE154613), 8 analyses (GSE147507), 9 analyses (GSE156754) (Breakdown of dataset sources analyzed in IPA)
- count more than 100,000 analyses/datasets screened; >10,000 analyses in activity plot (Scale of IPA database searched for SARS-CoV-2-related activation z-score data)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a primarily computational study combining IPA-based pathway enrichment scoring with CNN image classification. Public GEO gene expression datasets were analyzed with IPA to compute activation z-scores for the coronavirus pathogenesis pathway, and colored pathway images were used as model inputs. A CNN with VGG16 transfer learning was trained on 100 balanced pathway images in a fixed 70/20/10 train/validation/test split, with classification accuracy on the 10-image test set as the sole performance metric. A graph neural network was briefly evaluated for comparison using accuracy alone.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| IPA activation z-score (weighted directional enrichment score, formula provided in paper) | Quantification of coronavirus pathogenesis pathway activation across all SARS-CoV-2 and LUAD analyses (Table 1, Figures 1–4) | Varies by analysis; drawn from 49 IPA analyses of publicly available GEO datasets | not stated |
| CNN binary classification accuracy (VGG16 transfer learning) | Prediction of activated vs inactivated pathway image states (Figure 5, Section 3.6) | 10 test images | not stated |
| IPA Analysis Match (qualitative entity-overlap comparison) | Comparison of diffuse-type gastric cancer dataset with five SARS-CoV-2 iPSC analyses (Table 2, Section 3.3) | 5 SARS-CoV-2 analyses vs 1 gastric cancer analysis | not stated |
| GNN classification accuracy (single accuracy value) | Informal comparison against CNN model performance (Section 3.6) | not stated | not stated |
-
Model accuracy was evaluated on a single fixed split of 10 test images↳ Could also: k-fold cross-validation (e.g., 5- or 10-fold) or repeated random train/test splits could also be used to estimate generalization performance — With n=10 test images a single accuracy value carries high variance; cross-validation averages estimates over multiple partitions and yields a standard deviation across folds, giving a more stable picture of likely out-of-sample performance
-
Classification performance was reported solely as accuracy on a balanced 50/50 dataset↳ Could also: Precision, recall, F1-score, and area under the ROC curve (AUC-ROC) could also be reported for binary classifiers — These metrics characterize how errors are distributed between the two classes; AUC-ROC in particular is threshold-independent and allows fairer comparison across models with different decision boundaries
-
No uncertainty estimate was provided for the 65% accuracy figure↳ Could also: A bootstrap confidence interval or Wilson score interval for a proportion could also be computed around the point estimate — With n=10 test samples the 95% CI for 65% accuracy spans roughly 35–87%; reporting this interval contextualizes the precision of the estimate and is standard practice in small-sample ML evaluations
-
The paper used binary classification (activated vs inactivated) as the modeling target↳ Could also: A regression model predicting the continuous IPA activation z-score directly could also be used — The paper itself notes this as a candidate refinement; regression on the continuous z-score preserves ordinal information lost by dichotomization and avoids the need to choose a threshold for the activated/inactivated label
-
The CNN and GNN were compared by reporting a single accuracy figure for each with no formal test↳ Could also: McNemar's test or a permutation test on the difference in classification accuracy could also be applied — A formal test would indicate whether the observed gap (65% CNN vs 40% GNN) exceeds what might arise by chance given the small shared test set, complementing the descriptive comparison
-
SARS-CoV-2 datasets were matched to gastric cancer data via IPA Analysis Match (qualitative entity overlap)↳ Could also: A Fisher's exact test or hypergeometric test on the overlap of upstream regulators or pathway entities could also be applied — A formal overlap test would supply a p-value and odds ratio quantifying whether shared entities exceed chance expectation given the sizes of both entity lists, adding a quantitative dimension to the descriptive list in Table 2
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
Downstream reach in the literature
9 downstream papers · 1 datasetsHow widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.
- SARS-CoV-2 infection of human iPSC-derived cardiac c... 2021 · 196 cites
- Reverse-transcribed SARS-CoV-2 RNA can integrate int... 2021 · 183 cites
- Highly Efficient SARS-CoV-2 Infection of Human Cardi... 2021 · 37 cites
- PPMS: A framework to Profile Primary MicroRNAs from... 2022 · 8 cites
- Unraveling the relevance of SARS-Cov-2 infection and... 2024 · 5 cites
- Genetic screening for hypertension and COVID-19 reve... 2022 · 4 cites
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-40699865
Paper: Tanabe S, Quader S, Ono R, et al. "Artificial Intelligence Approach in Machine Learning-Based Modeling and Networking of the Coronavirus Pathogenesis Pathway." Curr Issues Mol Biol 2025. PMID 40699865 · PMCID PMC12191508 · DOI 10.3390/cimb47060466.
Declared code: https://github.com/Shihori/AI (Jupyter notebooks; cited at
fixed commits — CNN @ 68bbeaf7944ff8d1c3bbcb40fbf011c853cf4383, GNN @
018d5cfa90265cf00c6b5cf2a66cd8a2a8009b50).
Declared data: GEO GSE154613, GSE156754, GSE147507, GSE17400.
Pipeline-derived results and their pipelines
| # | Reported result | Pipeline | In scope? |
|---|---|---|---|
| R1 | CNN (from scratch) test accuracy 0.30 | PyTorch custom 4-conv CNN, 300 epochs (notebook cell 16/26) | code present, data unavailable |
| R2 | VGG16 transfer-learning accuracy 0.65 (65.0%) | torchvision VGG16(pretrained), retrain classifier[6], 50 epochs (cells 37–42) | code present, data unavailable |
| R3 | GNN accuracy 0.40 | torch_geometric GCN on image-as-32×32-pixel-graph (GNN notebook) | code present, data unavailable |
| T1 | IPA activation z-scores (Table 1, e.g. −1.706, 3.464, 1.147, 3.742, −0.2…) | Ingenuity Pathway Analysis (Qiagen, commercial license) | OUT of scope (proprietary tool, not a public pipeline) |
| — | Pathway network figures / Grad-CAM maps | IPA + manual interpretation; Grad-CAM on the unavailable model | out of scope / not re-runnable |
The reproduction blocker (central finding)
The CNN and GNN do not consume the GEO expression data directly. Both notebooks
mount the authors' private Google Drive and read 100 pre-made pathway-diagram
images (50 activated/, 50 inactivated/) from
/content/drive/MyDrive/CNN-corona2/data/ (CNN) and .../GNN-corona2/ (GNN):
# CNN_corona2_GitHub.ipynb cells 1,4,5,10
drive.mount('/content/drive'); %cd /content/drive/MyDrive/CNN-corona2/
path="/content/drive/MyDrive/CNN-corona2"; im_fd="«path»"
folder=['inactivated','activated']; usenumber=50
file0 = glob.glob(path+im_fd+folder[0]+"/*.jpg")[0] # e.g. .../inactivated/120.jpg
These images are the model's actual input. They are:
- not in the GitHub repo (
git ls-files→ zero image/data files; repo holds only notebooks + Japanese coursework PDFs); - not in GEO — they are screenshots of IPA-generated pathway diagrams, manually labelled activated/inactivated, then exported to the private Drive;
- not deposited in any public store referenced by the paper.
The GEO accessions are the upstream expression data fed into IPA (commercial) to build the pathways that were screenshotted. The deep-learning step that produces the headline accuracies (R1–R3) therefore cannot be re-executed, because its input data was never made public.
Additional non-determinism even if data existed: the train/val/test random_split
(cell 14) and the per-class 50-image random_split (cell 10) are not seeded with
the fixed generator, so the 70/20/10 partition — and hence the exact accuracy —
is not deterministic across runs.
What we attempt
- Provenance verification (not re-execution): confirm whether the paper's reported R1/R2/R3 numbers equal the values saved in the committed notebook outputs at the cited commits.
- Executability smoke test («our HPC» «job»): rebuild the env and run the exact CNN/VGG16/GNN code on synthetic placeholder images, to prove the blocker is data, not env/docs. This does not reproduce any accuracy.
Out of scope / not attempted
- IPA activation z-scores (Table 1) — proprietary Qiagen tool, no license, not a public computational pipeline (P-class: non-reproducible third-party commercial).
- Independent re-run of R1/R2/R3 — impossible without the private input images.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Drop / data_unavailable, no fabrication. The three headline accuracies (CNN 0.30, VGG16 0.65, GNN 0.40) exactly match the authors' own committed notebook outputs but cannot be independently re-run because the n=100 input images exist only on a private Google Drive and were never deposited (the cited GEO accessions are upstream IPA inputs, not model data). Beyond unavailability there is a genuine authors-side flaw: the paper sells 0.3→0.65 as a before/after test accuracy improvement, but in code 0.65 is final-epoch validation accuracy and VGG16 is never scored on the test set — and both surviving models are degenerate (single-class / flat across epochs). Severity is moderate factually (numbers provenance-match) but the central ML narrative is compromised and unverifiable, so overall is red.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.