Machine learning algorithm predicts fibrosis-related blood diagnosis markers of intervertebral disc degeneration.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🔴A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
PARTIAL reproduction. The paper is a re-analysis (no authors' own pipeline code; the only cited repo is the third-party tool MCPcounter). We reproduced the in-scope pipeline steps on the training set GSE150408 (Agilent microarray) on «our HPC»: (1) limma DEGs, (2) MCPcounter immune deconvolution on the paper's own data, (3) direction of the two diagnostic genes. Key findings: the DEG step is only reproducible in MAGNITUDE, not 1:1 - the paper states NO p/logFC threshold, the reported 1832 lies between our p<0.01 (694) and p<0.05 (2900) unadjusted results, and CRITICALLY 0 genes survive FDR correction, meaning the entire downstream chain (336 common DEGs -> 29 DE-FIGs -> 8-model ML -> AUC=1, nomogram) rests on uncorrected p-values. MCPcounter ran cleanly on the data (NK cells down, neutrophils up in IDD). The two headline diagnostic genes CEP120 and SPDL1 DO reproduce in direction (both down-regulated in IDD, nominally p<0.05), consistent with the paper, though not FDR-significant. NOT attempted (described well enough but out of 80/20 scope): GeneCards 2539-FIG list (external, time-varying), the full 8-algorithm ML panel + nomogram C-index 0.7681661 (depends on the FIG intersection and unspecified CV - the hard 20%), qRT-PCR (wet-lab), and miRNA/TF/ceRNA/drug networks (external DBs). Honest verdict: described well enough to reproduce the methods QUALITATIVELY, but the exact headline numbers are not deterministically recoverable due to an unstated DEG threshold and absent multiple-testing correction - flagged for the human auditor.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 68assessed: 2026-06-15 ⛓ d3ad42964584
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusBecause intervertebral disc cell fibrosis contributes to intervertebral disc degeneration (IDD), the study tests whether fibrosis-related genes can serve as blood-based diagnostic markers for IDD and aims to identify them using machine learning.
- ★ CEP120 and SPDL1 are fibrosis-related diagnostic genes for IDD, identified via a random forest model from 29 differentially expressed fibrosis-related genes finding
- ★ Both CEP120 and SPDL1 are down-regulated in IDD and were validated by qRT-PCR in patient blood samples finding
- ★ Random forest at 3-fold was the most suitable machine learning model, achieving AUC of 1 with the two diagnostic genes method
- ★ Immune infiltration differs between IDD and controls: natural killer cells lower, neutrophils and myeloid-derived suppressor cells higher in IDD finding
- ★ lncRNA AC144548.1 may regulate SPDL1 and CEP120 via hsa-miR-5195-3p and hsa-miR-455-3p respectively, and TFs FOXM1, PPARG, ATF3 regulate their transcription mechanism
- 56 drugs were predicted to target the diagnostic genes via the Comparative Toxicogenomics Database resource
- A nomogram combining CEP120 and SPDL1 provides accurate diagnostic prediction (C-index 0.768) exceeding single-gene performance resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| bulk RNA-sequencing (microarray/expression dataset, reanalyzed) | whole blood from IDD patients and controls (GSE150408: 17 IDD vs 17 controls; GSE124272: 8 lumbar disc prolapse vs 8 healthy) | none (disease vs control comparison) | differentially expressed genes between IDD and controls | GEO datasets GSE150408, GSE124272; limma v3.44.3 |
| machine learning diagnostic gene selection (random forest among 8 algorithms) | GSE150408 dataset (training/validation split by n-fold) | none | diagnostic gene selection by %IncMSE importance and AUC | randomForest v4.6-14, xgboost, e1071, neuralnet, Adaboost, etc. |
| ROC curve / PCA diagnostic effectiveness analysis | GSE150408 (training) and GSE124272 (validation) | none | AUC values, sample discrimination | — |
| ssGSEA functional enrichment (GO/KEGG) | GSE150408 dataset | none | enriched GO terms and KEGG pathways for CEP120 and SPDL1 | clusterProfiler v3.16.1 |
| immune infiltration analysis (MCP-counter and ssGSEA) | GSE150408 dataset, IDD vs control | none | immune cell type proportions (8 cells via MCP-counter; 28 cell types via ssGSEA) | MCP-counter |
| regulatory network construction (miRNA-mRNA-TF, ceRNA, drug-mRNA) | in silico prediction for diagnostic genes | none | predicted miRNAs, TFs, lncRNAs, drugs | miRWalk, ChEA3, StarBase, CTD; Cytoscape v3.7.2 |
| quantitative real-time PCR (qRT-PCR) | PBMCs from 24 whole blood samples (12 IDD vs 12 controls) | none | CEP120 and SPDL1 expression (2^-ΔΔCt, GAPDH reference) | BIO-RAD CFX96 Touch PCR detection system; Universal Blue SYBR Green qPCR Master Mix |
- – 336 common DEGs identified between GSE124272 and GSE150408 (230 up-regulated, 106 down-regulated) 336 genes
- – 29 DE-FIGs obtained from overlap of 336 DEGs with 2,539 fibrosis-related genes 29 genes
- – Random forest model reached AUC of 1 when number of variables reached 2 (CEP120 and SPDL1) AUC=1
- – ROC AUC for each diagnostic gene and the combination greater than 0.75 in both datasets AUC>0.75
- – Natural killer cells significantly lower and neutrophils higher in IDD samples (MCP-counter)
- ▲ MDSCs and neutrophils more abundant in IDD samples (ssGSEA)
- ▼ Down-regulation of SPDL1 and CEP120 validated by qRT-PCR
- – miRNA-mRNA-TF network with 139 nodes (2 genes, 17 miRNAs, 120 TFs) and 156 edges constructed 139 nodes, 156 edges
- count 336 common DEGs (230 up, 106 down) (DEGs shared between the two GEO datasets)
- count 29 DE-FIGs (differentially expressed fibrosis-related genes)
- other AUC = 1 (random forest model with 2 variables (CEP120, SPDL1))
- other C-index = 0.7681661 (corrected 0.7326992) (nomogram prediction accuracy)
- other AUC > 0.75 (ROC for diagnostic genes in GSE150408 and GSE124272)
- count 2,539 FIGs (fibrosis-associated genes from GeneCards after relevance score > 1)
- count 56 drugs predicted (drugs targeting CEP120 and SPDL1 via CTD)
- count 4,489 DEGs1 and 1,832 DEGs2 (DEGs from GSE124272 and GSE150408 respectively)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This bioinformatics study downloaded whole-blood RNA-sequencing data from two GEO datasets (GSE150408: 17 IDD vs. 17 controls; GSE124272: 8 lumbar disc prolapse vs. 8 controls) and applied limma to identify differentially expressed genes in each dataset separately, then intersected the two gene lists and further intersected with 2,539 curated fibrosis-related genes to yield 29 DE-FIGs. Eight machine learning classifiers were compared across n-fold values (n=1–7) using accuracy on held-out folds; random forest at n=3 was selected, and %IncMSE variable importance with AUC-guided variable selection identified two diagnostic genes (CEP120 and SPDL1). Diagnostic performance was assessed via ROC curves, a nomogram with C-index and calibration curve, ssGSEA pathway enrichment, and MCP-counter immune cell deconvolution, with experimental validation by qRT-PCR (2^(−ΔΔCt)) in 24 PBMC samples (12 IDD vs. 12 controls).
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| limma linear model (differential expression) | DEG identification in GSE124272 and GSE150408 separately; common DEGs obtained by intersection of both gene lists | GSE124272: 8 IDD + 8 controls; GSE150408: 17 IDD + 17 controls | not stated |
| Random forest with n-fold cross-validation (n=1–7 evaluated; n=3 selected by average accuracy across eight classifiers) | Diagnostic gene selection and model selection from 29 DE-FIGs in GSE150408 | 34 samples (17 IDD + 17 controls) | not stated |
| ROC curve analysis (AUC) for each gene individually and combined via logistic regression | Evaluation of CEP120 and SPDL1 in GSE150408 (training) and GSE124272 (validation) | GSE150408: 34 samples; GSE124272: 16 samples | na |
| Nomogram with C-index, calibration curve slope, and decision curve analysis (DCA) | Validation of combined CEP120 + SPDL1 nomogram in GSE150408 | 34 samples (GSE150408) | not stated |
| PCA (principal component analysis) | Visualization of IDD vs. control separation using two diagnostic genes in GSE150408 and GSE124272 | GSE150408: 34 samples; GSE124272: 16 samples | na |
| GO and KEGG functional enrichment (clusterProfiler) | Enrichment of 336 common DEGs; ssGSEA-ranked pathway enrichment for CEP120 and SPDL1 (|NES|>1; adjusted p<0.05; q<0.25) | 336 common DEGs; 34 samples for ssGSEA | not stated |
| ssGSEA (single-sample gene set enrichment analysis) | Pathway enrichment per diagnostic gene (Pearson correlation ranking) and immune cell proportion estimation across 28 cell types in GSE150408 | 34 samples (GSE150408) | not stated |
| MCP-counter algorithm | Immune cell content estimation for 8 immune cell types plus fibroblast and epithelial cell in IDD vs. control (GSE150408); between-group statistical test for box plot comparisons not named | 34 samples (GSE150408) | na |
| 2^(−ΔΔCt) quantification (qRT-PCR); statistical test for between-group comparison not stated in text | Validation of CEP120 and SPDL1 expression in PBMCs | 12 IDD vs. 12 controls | not stated |
-
DEGs were identified independently in each dataset with limma and then intersected by overlap; genes missing the threshold in one dataset were excluded even if consistently directional across both↳ Could also: A cross-study meta-analysis of effect sizes (e.g., fixed- or random-effects meta-analysis of log2 fold-changes, or Fisher's combined p-value method) could also integrate the two datasets — Intersection of independently thresholded lists discards genes that narrowly miss the cutoff in one dataset; meta-analysis of effect sizes preserves quantitative consistency information and is less sensitive to the choice of per-dataset significance threshold
-
Model selection and performance estimation both used the same n-fold splits within GSE150408 (the training dataset), with n selected by comparing average accuracy across folds↳ Could also: Nested cross-validation (outer loop for unbiased performance estimation, inner loop for model/hyperparameter selection) could also be applied — When the same data partition is used for both selection and evaluation, the estimated performance can be optimistic; nested CV explicitly separates these steps and is particularly relevant for small samples (n=34 in the training set)
-
Diagnostic performance was summarized as AUC point estimates (>0.75 described qualitatively); no confidence intervals were reported around AUC values↳ Could also: Bootstrap resampling or the DeLong method could also provide 95% confidence intervals around each AUC estimate — With n=34 and n=16 in the two datasets, uncertainty around AUC estimates can be substantial; CIs allow readers to assess precision and overlap between individual genes and their combination
-
Immune cell content comparisons between IDD and controls were displayed as box plots; the statistical test used for between-group inference is not named in the text↳ Could also: A Wilcoxon rank-sum test or t-test per cell type with Benjamini-Hochberg FDR correction across the multiple cell types compared could also be explicitly applied and reported — Naming the test, reporting test statistics, and applying multiplicity correction across the set of cell-type comparisons gives readers a basis for assessing the strength and reliability of each individual comparison
-
qRT-PCR validation quantified gene expression by 2^(−ΔΔCt) in 12 IDD vs. 12 controls, but no statistical test for comparing the two groups is named in the text↳ Could also: A Mann-Whitney U test or two-sample t-test on ΔCt values (which are approximately normally distributed on the log scale), with the test statistic and exact p-value reported, could also accompany the fold-change results — Reporting the inferential test for the experimental validation step allows readers to evaluate the statistical evidence from the wet-lab component independently of the computational discovery phase
-
Random forest %IncMSE variable importance ranked 29 DE-FIGs, and an AUC=1 at 2 variables served as the stopping criterion for gene selection↳ Could also: LASSO or elastic-net penalized logistic regression could also perform simultaneous feature selection and coefficient shrinkage on the same 29-gene candidate set — Regularization-based selection explicitly penalizes model complexity and produces a continuous importance gradient across all variables; when the ratio of candidate features to training samples is high (29 features, 34 samples), penalized regression can offer a well-characterized bias-variance tradeoff
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-37915003
Paper: Zhao W et al. "Machine learning algorithm predicts fibrosis-related blood diagnosis markers of intervertebral disc degeneration." BMC Med Genomics 2023. PMID 37915003 / PMC10619283 / doi:10.1186/s12920-023-01705-6.
Cited code: https://github.com/ebecht/MCPcounter (third-party immune-cell deconvolution tool — not the authors' own pipeline code). Per brief rule P16, running this third-party tool on the paper's own data is a fully valid reproduction.
Data: GEO GSE150408 (training; Agilent SurePrint G3 microarray GPL21185, whole blood, 17 IDD vs 17 control) and GSE124272 (validation, 8+8). We focus on GSE150408 (the primary training set, GEO-resolvable).
Pipeline-derived results — IN SCOPE (attempted)
| # | Result | Reported | Pipeline | Feasible? |
|---|---|---|---|---|
| C1 | DEG count in GSE150408 | "1,832 DEGs2" | limma 3.44.3, IDD vs control | YES — but DEG cutoffs not stated in paper (transparency gap); we sweep standard thresholds |
| C2 | Immune infiltration | MCP-counter cell populations (Fig.) | MCPcounter (the cited repo) on GSE150408 expr | YES — run cited tool on paper data |
| C3 | Diagnostic-gene direction | CEP120 & SPDL1 down-regulated in IDD (qRT-PCR p<0.0058 / p<0.0073) | check microarray expression direction & limma stats | YES — direction check (the qRT-PCR itself is wet-lab, out of scope) |
OUT OF SCOPE (not attempted — reason)
- GeneCards 2,539 FIGs (relevance >1) → 29 DE-FIGs. External, time-varying GeneCards query; the list is not shipped. Not reproducible 1:1.
- 8-ML-algorithm panel → CEP120+SPDL1 selection, AUC=1, nomogram C-index 0.7681661. Depends on the FIG intersection above and unspecified hyperparameters / CV seeds; this is the hard last ~20%. Not attempted as a numeric 1:1 (we only check the two genes' expression direction, C3).
- qRT-PCR validation (CEP120/SPDL1 p-values) — wet-lab.
- miRNA (miRWalk), TF (ChEA3), ceRNA (StarBase), drug (DGIdb) networks — external databases, not a numeric pipeline result.
- GSE124272 "4,489 DEGs1" — secondary dataset; out of focus (same method as C1, would add little).
Key reproducibility risk noted up front
The paper gives no limma cutoff (no p / adj.p / logFC threshold) for the DEG step, so the reported "1,832 DEGs" is not deterministically reproducible. We report DEG counts across a standard threshold grid and let the human auditor judge which (if any) the authors used. This is itself an auditable finding.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
GSE150408 is public so the input is reconstructible, but the paper specifies no DEG threshold and no multiple-testing correction, leaving the headline 1832 DEGs only bracketable (694@p<0.01 to 2900@p<0.05) and, critically, 0 genes survive FDR — the entire downstream chain (336 DEGs→29 DE-FIGs→ML AUC=1, nomogram) rests on uncorrected p-values. This is primarily an authors'-side transparency/rigor defect (underspecified method + missing FDR + AUC=1 overfit signature), not a data-availability problem. Mitigating it, the two diagnostic genes CEP120 and SPDL1 reproduce in direction (both down in IDD, nominally p<0.05) and MCPcounter ran cleanly, so magnitude/direction hold — overall a solid-but-concerning partial reproduction flagged for human audit rather than confirmed fabrication.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.