Representing and querying disease networks using graph databases.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🔴Could not use the authors’ exact input data
- 🔴Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Application/methods paper; headline computational result is Table 1 (node/edge occurrence counts of an integrated Neo4j graph database). The repo ibalaur/ProteinFramework @ d87cd9c (GPL-3.0, Java7/Maven, Neo4j 2.3.0) ships ONLY the graph-construction code -- the files/ directory of ~19 version-pinned input datasets it reads is NOT in the repo (0 releases, 42KB, no files/ dir). Those inputs (DisGeNET 2014 curated, Reactome 2014, IntAct snapshot, HPA v13, DrugBank licensed link CSVs, UniProt 2015-04-07 all-vs-all BLAST, and limma-derived DEG files for the 3 GEO studies) are not shipped and several are not obtainable in those exact versions today; current versions would mismatch by construction (2014->2026 DB growth). Therefore Table 1's data-derived counts are NOT independently regenerable -> docs_insufficient. The single named accession GSE27876 is only one of three GEO inputs; it is public (GPL6480, 19 GSM) and I verified 1:1 that the paper's factual description (5 NC + 5 mild + 5 severe in-vivo blood samples) matches the GEO record exactly (the other 4 GSM are in-vitro THP1 controls the paper did not use). Structural Table 1 rows (GEO Study=3, GEO Comparison=9, PART_OF=9) match by definition. The aggregate DEG_RELATED_TO count (87849, over 3 studies x 9 comparisons, limma thresholds unstated, no per-study value) is not 1:1 pinnable; I deliberately did NOT run an orphan limma job to manufacture an unmatchable number (HARD RULE 6). NOT attempted: rebuilding the full graph (inputs unavailable), Table 2 / shortest-path Cypher queries (require the built DB or the now-offline diseaseknowledgebase.etriks.org server), and the Girvan-Newman clustering script (points at that same live server). No fabrication asserted -- the counts are plausibly genuine but not publicly verifiable. No «our HPC» heavy-compute was warranted; only control-plane GEO metadata was needed.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 77assessed: 2026-06-14 ⛓ f6cca8c0ca59
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusThe paper investigates whether graph databases (specifically Neo4j) provide a flexible and powerful framework for integrating, storing, and querying heterogeneous, highly connected biological data to provide biological context to disease-related genes and support hypothesis generation, using asthma as an application case.
- ★ Graph databases are well suited for representing biological information that is highly connected, semi-structured, and unpredictable. finding
- ★ A Neo4j-based prototype disease network can integrate multi-omics data (PPI, pathways, sequence similarity, disease-gene, gene-tissue, protein-drug associations) from a protein-centric perspective. resource
- ★ Graph databases excel at traversal-type queries that reveal unexpected paths/relationships between entities, supporting 'link discovery' and hypothesis generation. method
- ★ Gene expression study results (DEGs) can be easily added to the database and contextualized within the disease network. method
- Relational databases are poorly suited to highly connected, sparse, semi-structured biological data because traversal joins become computationally expensive and schema changes are costly. finding
- ★ The Neo4j Cypher query language enables complex queries to contextualise disease-associated genes and explore their network neighbourhood. method
- Curated respiratory disease gene sets (asthma, COPD, TB, E-HTN) can be incorporated as starting points to explore disease-associated pathways and processes. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| All-versus-all sequence similarity search (Tera-BlastP) | UniProt Human reference proteome | none | bidirectional sequence similarity hits scored as -log10 of average e-value | Timelogic DeCypher system (TeraBlastP) |
| Graph database integration/storage and querying | Human proteome (UniProt/Swissprot human subset) | none | nodes and edges representing proteins, diseases, pathways, tissues, drugs, GEO studies and their relationships | Neo4j v2.3.1 graph database (Cypher query language) |
| Protein-protein interaction data integration | Human | none | interaction edges annotated with experimental methods and confidence score | EBI IntAct database (PSI-MI tab format) |
| Gene expression microarray differential expression analysis | Human asthma patient samples (bronchial), GEO series GSE27876, GSE43696, GSE63142 | disease subtype vs control (NC vs MiA/MMA/SA) | differentially expressed genes (DEGs) with adjusted p-values | Agilent-014850 Whole Human Genome Microarray 4x44K G4112F (GPL6480); limma Bioconductor package |
| Tissue-enhanced gene expression integration | Human tissues | none | tissue-enhanced expression edges with RNA-TS-FPKM values | Human Protein Atlas (HPA) version 13 |
| Disease-gene association integration | Human | none | disease nodes linked to protein nodes | DisGeNET (2014 release, curated subset) |
| Metabolic pathway membership integration | Human | none | pathway nodes linked to proteins by membership edges | Reactome database |
| Drug-target association integration | Human | drug | drug compounds linked to target proteins via UniProt references | DrugBank (csv files, downloaded 27/03/2015) |
- – Integrated graph database contained 20762 Protein nodes, 4745 Disease, 1288 Pathway, 32 Tissue, 1602 Drug, 9 GEO Comparison, and 3 GEO Study nodes 20762 proteins
- – Sequence similarity relationships (SEQ_SIM edges) were the most abundant edge type in the database 92089 edges
- – Protein-protein interaction association edges (PPI_ASSOCIATION) integrated from IntAct 66678 edges
- – Differentially-expressed-gene relationships (DEG_RELATED_TO) linking proteins to GEO comparisons 87849 edges
- – Protein-pathway membership edges (IN_PATHWAY) from Reactome 26085 edges
- – Protein-disease BIOMARKER edges integrated from DisGeNET 17216 edges
- – Curated respiratory disease gene set comprised 219 genes across four overlapping categories (104 BA, 58 COPD, 35 TB, 54 E-HTN) 219 genes
- – Drug-target association edges (DRUG_TARGET) from DrugBank 6429 edges
- count 20762 (Protein node occurrences in the graph database)
- count 92089 (SEQ_SIM (sequence similarity) edges)
- count 66678 (PPI_ASSOCIATION edges)
- count 87849 (DEG_RELATED_TO edges (protein to GEO comparison))
- pvalue 1e-05 (significance threshold for Tera-BlastP sequence similarity search; effective database size 16.9 bln bases)
- count 219 (curated respiratory-disease genes (104 BA, 58 COPD, 35 TB, 54 E-HTN))
- other 0.60 (minimum ratio of alignment length to shortest sequence required for sequence similarity hits)
- count 4745 (Disease node occurrences in the graph database)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This paper is primarily a computational methods study demonstrating Neo4j graph database construction and querying for heterogeneous biomedical data integration, with asthma as an application case. The only inferential statistical component is differential gene expression (DEG) analysis applied to three GEO microarray datasets using the limma Bioconductor package; adjusted p-values from those analyses are stored as edge properties in the graph. The paper does not report group-level comparisons in a traditional inferential framework; results are presented as database query outputs and network structures.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| limma moderated linear model / empirical-Bayes moderated t-statistic | DEG identification between asthma subtypes and normal controls in GSE27876, GSE43696, GSE63142 (all on GPL6480 Agilent microarray platform) | GSE27876: 5 NC, 5 MiA, 5 SA; GSE43696: 20 NC, 50 MMA, 38 SA; GSE63142: 27 NC, 72 MMA, 56 SA | not stated |
| Tera-BlastP e-value threshold (1e-05) with bidirectional hit filtering and alignment-length ratio filter (>0.60) | All-vs-all sequence similarity computation for human UniProt reference proteome | effective database size adjusted to 16.9 billion bases | not stated |
-
DEG analysis was performed with limma, which applies empirical-Bayes moderated t-statistics suited to microarray data with small n↳ Could also: For the smallest dataset (GSE27876, n=5 per group), a non-parametric approach such as the SAM (Significance Analysis of Microarrays) permutation method or a rank-based test could also be applied — Permutation-based methods make no distributional assumptions and can be more conservative with very small group sizes, providing an alternative perspective on significance especially when normality of residuals cannot be verified
-
Three GEO studies were analyzed independently, with DEG results from each stored separately in the graph database↳ Could also: A cross-study meta-analysis (e.g., using the R package metaMA or Fisher's combined probability method across the three studies) could also be applied before or alongside graph integration — Combining evidence across studies increases statistical power and identifies genes consistently dysregulated across independent cohorts, which may be a more robust signal than any single study; this is especially relevant given the small n in GSE27876
-
Adjusted p-values are stored as the sole quantitative edge property for DEG relationships↳ Could also: Log2 fold-change (and its standard error from limma) could also be stored as additional edge properties alongside the adjusted p-value — Effect size (log2FC) conveys the magnitude and direction of expression change independently of sample size; storing it alongside the adjusted p-value would allow graph queries to filter or rank DEGs by both statistical significance and biological relevance
-
Sequence similarity edges were created using a fixed e-value threshold (1e-05) and a minimum alignment-length ratio (0.60)↳ Could also: Percent sequence identity or bit-score thresholds, or a continuous similarity score on every edge, could also serve as the inclusion/weighting criterion — E-value is database-size dependent; percent identity and bit-score are more comparable across databases of different sizes and are widely used in orthology inference pipelines such as OrthoFinder or OrthoMCL
-
The multiplicity correction method is not named; it is implied by the term 'adjusted p-value' produced by limma↳ Could also: Explicitly reporting and storing the FDR method (e.g., Benjamini-Hochberg) and the unadjusted p-value alongside the adjusted p-value as separate edge properties could also be done — Storing both values in the graph would allow downstream queries to apply alternative thresholds or recompute corrections across combined gene sets, increasing the analytical flexibility of the database
-
All three GEO datasets use the same microarray platform (GPL6480 Agilent 44K) and were analyzed with the same pipeline, but no cross-study concordance metric is reported↳ Could also: A rank-correlation or overlap analysis (e.g., Jaccard index of significant DEG lists across studies) could also be reported to characterize reproducibility across datasets before graph integration — Quantifying inter-study concordance would contextualize the reliability of the DEG edge set in the graph and help users interpret query results that span multiple GEO studies
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
Downstream reach in the literature
77 downstream papers · 3 datasetsHow widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.
- Moderate-to-severe asthma in individuals of European... 2019 · 189 cites
- Gene expression in relation to exhaled nitric oxide... 2014 · 157 cites
- An airway epithelial iNOS-DUOX2-thyroid peroxidase m... 2014 · 101 cites
- Genetic analyses identify GSDMB associated with asth... 2021 · 85 cites
- Androgen regulation of pulmonary AR, TMPRSS2 and ACE... 2021 · 79 cites
- HSP90 inhibitor geldanamycin reverts IL-13- and IL-1... 2019 · 62 cites
- Gene expression in relation to exhaled nitric oxide... 2014 · 157 cites
- A network medicine approach to investigation and pop... 2020 · 147 cites
- Gene Expression Correlated with Severe Asthma Charac... 2017 · 121 cites
- Genetic analyses identify GSDMB associated with asth... 2021 · 85 cites
- Meta-analysis of airway epithelium gene expression i... 2018 · 43 cites
- Benefits of Airway Androgen Receptor Expression in H... 2021 · 43 cites
- RNA m<sup>6</sup>A methylation modulates airway infl... 2023 · 65 cites
- Integrated Transcriptomics Establish Macrophage Pola... 2015 · 49 cites
- Long non-coding RNA HOTTIP induces inflammation in a... 2022 · 6 cites
- Gene expression data analysis identifies multiple de... 2018 · 5 cites
- Anti-asthmatic fraction screening and mechanisms pre... 2022 · 1 cites
- Prognostic value of the relative neutrophil-monocyte... 2025 · 0 cites
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-27462371
Paper: Lysenko, Roznovăţ, Saqi, Mazein, Rawlings, Auffray (2016).
"Representing and querying disease networks using graph databases." BioData Mining 9:23.
DOI 10.1186/s13040-016-0102-8 · PMC4960687.
Article type: review/application case (per JATS article-type="review-article").
Repo: https://github.com/ibalaur/ProteinFramework @ d87cd9cab9db92ce8401f8fe2fe68a3df58a3798
(GPL-3.0, Java 7 + Maven, Neo4j 2.3.0 embedded).
What this paper's computational artifact is
An integrated Neo4j property-graph database ("the Protein Framework") that fuses ~10 heterogeneous biological sources into one network, then is queried with Cypher. The reported, pipeline-derived quantitative results are:
| Result | Where | In scope? | Pipeline |
|---|---|---|---|
| Table 1 — node-type occurrences (7 types) and edge-type occurrences (16 types) of the assembled graph | Table 1 | IN scope (headline) | App.java/ProteinNetwork.java parse the input files and write the Neo4j DB; counts are MATCH (n) … count. |
| Table 2 — common protein sets Asthma↔{COPD,TB,EHTN} (Kaneko & DisGeNET) | Table 2 | in scope (secondary) | Cypher Listing 1/2 over the built DB. |
| Shortest-path examples between asthma & core-clock proteins | Figs 8–9, Listings 5–6 | in scope (secondary) | Cypher allShortestPaths over the built DB. |
| GSE27876/43696/63142 DEG identification (limma) | Methods | in scope (feeds DEG_RELATED_TO edges) | scripts + limma; only aggregate count reported. |
| Girvan–Newman clustering of asthma biomarkers | scripts/clustering_protein_network.R | in scope but depends on the live public server (diseaseknowledgebase.etriks.org) |
RNeo4j + igraph. |
| Biological interpretation / wet-lab references | throughout | OUT of scope (manual/literature) | — |
Inputs required by the code (the blocker)
ProteinNetwork.java reads ~19 fixed paths under files/ — none shipped in the repo
(repo = 42 KB, 0 releases, no files/ dir). They are version-pinned to 2014–2015:
- DisGeNET 2014 curated gene-disease assoc. + gene→UniProt map
UniProt2Reactome_HomoSapiens.txt(Reactome 2014)- IntAct PPI snapshot (PSI-MI tab) —
intact_output_protein_interactions.txt - Human Protein Atlas v13 tissue file +
Ensembl_Uniprot.txt - DrugBank target/enzyme/transporter/carrier link CSVs (licensed, account-gated)
uniprot_07_04_2015_blast.tab.txt(all-vs-all UniProt BLAST, dated 2015-04-07)- limma-derived
diff_expr_GSE{27876,43696,63142}.txt(+ uniprot_all.xls, uniprot_GSE55962.xls)
Only one of these (GSE27876, the named accession) is in scope as a public dataset, and
it is just one of three GEO inputs. The DEG files themselves (diff_expr_*) are derived,
not shipped, and the paper reports only an aggregate edge count (87 849) across all 3
studies × 9 comparisons with unstated limma thresholds → no per-study value to match 1:1.
Decision
- Headline (Table 1): not regenerable from shipped artifacts. Code present, but the
assembled, version-pinned inputs are absent and several (DrugBank old, UniProt-2015 BLAST,
DisGeNET-2014, HPA-v13) are not obtainable in those exact versions today. Even a best-effort
re-assembly with current Reactome/IntAct/DrugBank/UniProt would mismatch by construction
(database growth 2014→2026). →
docs_insufficient. - One clean 1:1 we CAN do: verify the paper's factual description of the named accession GSE27876 against the live GEO record (control-plane metadata, no heavy compute needed).
- We deliberately did not run an orphan limma job to manufacture a DEG number with no reported counterpart (would be a fabricated-to-avoid-drop result; HARD RULE 6).
No «our HPC» heavy-compute job was warranted: the headline is blocked by missing inputs, and the only reproducible check is a tiny metadata verification.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
A graph-database methods/application paper whose headline computational result (Table 1 node/edge occurrence counts) is not independently regenerable: the repo ships only the Java/Neo4j construction code, while the ~19 version-pinned 2014-2015 input datasets (DisGeNET, Reactome, IntAct, licensed DrugBank, UniProt BLAST, limma DEG files) were never deposited and are unobtainable in those versions today. The gap is on the authors'/data-deposit side (value not derivable from shared data), but with no fabrication signal — the only checkable items (structural rows 3/9/9; GSE27876's 5NC+5mild+5severe composition vs the live GEO record) matched 1:1. The agent correctly declined to manufacture an unmatchable DEG_RELATED_TO count. Overall: reproducibility-limited, not a substantive discrepancy.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.