Discovery and validation of molecular patterns and immune characteristics in the peripheral blood of ischemic stroke patients.
The main results reproduced: recomputed values matched the published ones within tolerance.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH: YES. The repo (donkeycong/Ella @ 5202b4b) is the authors' own R/perl code and SHIPS the raw training data (GSE16561_series_matrix.txt + GPL6883.annot), so the GSE16561 training pipeline is fully self-contained. Ran the authors' scripts VERBATIM on «our HPC» («job», COMPLETED): 01.pl->02.R->07->08->09 + a read-only summary replicating 14.R's ROC. RESULT = strong PARTIAL reproduction. EXACT: down-DEGs (85), LASSO count (12, and it contains all 6 reported hub genes), and all four reported per-gene training AUCs (BAZ2B .892 / C5AR1 .966 / PDK4 .891 / STK3 .966). WITHIN-TOL: total DEGs 193 vs 188 and up-DEGs 108 vs 103 (5-gene gap from reconstructed s1/s2 group lists, which are not shipped), model AUC 1.000 vs 0.999. MISMATCH: SVM-RFE count 10 vs 34 -> ROOT CAUSE FOUND: 08.SVM-RFE.R has a typo 'methods=svmRadial' (should be method=), so caret::rfe silently runs randomForest-RFE, not SVM; the published 'SVM-RFE' set was in fact RF-RFE, and 34 is not reproducible from the script as written. Hub count is consequently 3 vs 6 (PARTIAL) -- our 3 are a correct subset, and the 3 'missing' hubs (BAZ2B,PDK4,PPIH) are present in our LASSO set and their AUCs reproduce exactly, corroborating the biomarker panel. Further audit finding: DEG threshold mismatch -- code filters RAW P<0.05 while Methods say adjusted P<0.05; reproduced count matches the code, so the paper misdescribes its own method. NOT ATTEMPTED (harder 20%): CIBERSORT (needs LM22 ref.txt, not shipped), GO/KEGG/GSEA (qualitative), GSE58294 external validation (AUC 0.940), qPCR (wet-lab). All grades provisional; a human auditor signs off.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-15 ⛓ c0abb66e6b29
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-23
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetImmune factors play a crucial but mechanistically unclear role in ischemic stroke (IS), so this study aims to identify immune-related biomarkers and characterize immune cell infiltration patterns in peripheral blood to elucidate IS immunological mechanisms and enable diagnosis.
- ★ 188 differentially expressed genes (DEGs) between IS and control blood samples were identified and enriched in immune-related biological pathways finding
- ★ Six hub genes (ANTXR2, BAZ2B, C5AR1, PDK4, PPIH, STK3) were identified as IS biomarkers using LASSO and SVM-RFE machine learning finding
- ★ Hub genes show distinct correlation patterns with immune cell subsets: ANTXR2, BAZ2B, C5AR1, PDK4, STK3 positively correlate with neutrophils and gamma delta T cells and negatively with T follicular helper cells and CD8 T cells, while PPIH shows the opposite pattern finding
- ★ IS patients show increased infiltration of monocytes, macrophages M0, neutrophils, and mast cells, and decreased infiltration of T follicular helper cells and CD8 T cells compared to controls finding
- ★ A ceRNA regulatory network of hub genes was constructed comprising miRNA-mRNA and miRNA-lncRNA interaction pairs resource
- ★ RT-qPCR validation confirmed significantly increased expression of BAZ2B, C5AR1, PDK4, and STK3 in IS patients finding
- ★ A logistic regression diagnostic model based on four hub genes (BAZ2B, C5AR1, PDK4, STK3) accurately classifies IS finding
- Combining LASSO and SVM-RFE machine learning algorithms provides a rigorous approach to hub gene screening method
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| microarray gene expression profiling / DEG analysis (limma) | whole blood, IS patients vs healthy controls (GSE16561, training set) | none (disease vs control) | differentially expressed genes | GPL6883 |
| microarray gene expression profiling (validation) | whole blood, IS patients vs healthy controls (GSE58294, validation set) | none (disease vs control) | hub gene expression levels, diagnostic model AUC | GPL570 |
| GO/KEGG functional enrichment and GSEA | DEGs from whole blood IS/CTL dataset (GSE16561) | none | enriched biological pathways and processes | clusterProfiler; GSEA software |
| CIBERSORT immune cell deconvolution | whole blood, IS vs CTL (GSE16561) | none | proportions of 22 immune cell types and correlation with hub genes | CIBERSORT; corrplot |
| LASSO regression | whole blood gene expression data (GSE16561) | none | selected candidate hub genes | glmnet package |
| SVM-RFE | whole blood gene expression data (GSE16561) | none | selected candidate hub genes | e1071 package |
| RT-qPCR | peripheral whole blood, clinical samples (5 IS patients vs 5 healthy controls) | none (disease vs control) | relative expression (2^-ΔΔCt) of six hub genes | ChamQ SYBR Color qPCR Master Mix (Vazyme) |
| ceRNA network prediction / logistic regression diagnostic modeling with ROC analysis | hub genes; whole blood datasets GSE16561 (training) and GSE58294 (validation) | none | miRNA-mRNA/miRNA-lncRNA interaction pairs; AUC of diagnostic model | miRanda, MicroRNADB, TargetScan, spongeScan, Cytoscape; Proc R package |
- – 188 DEGs identified between IS and CTL (85 down-regulated, 103 up-regulated) FoldChange > 1.5, adj. P < 0.05
- – DEGs enriched in immune response pathways, MAPK signaling, cell adhesion molecules, complement and coagulation cascades
- – Six hub genes identified via intersection of LASSO and SVM-RFE
- – Immune infiltration in IS group showed increased monocytes, macrophages M0, neutrophils, mast cells and decreased T follicular helper cells and CD8 T cells
- – ceRNA network built from miRNA-mRNA and miRNA-lncRNA interaction pairs 306 miRNA-mRNA pairs, 285 miRNA-lncRNA pairs
- ▲ RT-qPCR showed significantly increased expression of BAZ2B, C5AR1, PDK4, and STK3 in IS patients
- – Diagnostic model based on four hub genes achieved high classification accuracy AUC = 0.999 (training set), AUC = 0.940 (validation set)
- fold_change FoldChange > 1.5, adjusted P-value < 0.05 (criteria for defining DEGs)
- count 188 DEGs (85 down-regulated, 103 up-regulated) (differential expression analysis results)
- count 39 IS vs 24 CTL (GSE16561 training dataset composition)
- count 69 IS vs 23 CTL (GSE58294 validation dataset composition)
- other AUC = 0.999 (diagnostic model performance in training set (GSE16561))
- other AUC = 0.940 (diagnostic model performance in validation set (GSE58294))
- count 306 miRNA-mRNA interacting pairs (ceRNA network construction)
- count 285 miRNA-lncRNA interacting pairs (ceRNA network construction)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This bioinformatics study applied limma-based differential expression analysis to a GEO microarray training dataset (GSE16561: 39 IS vs 24 CTL), then used LASSO regression and SVM-RFE machine learning algorithms to select hub genes, with immune cell infiltration estimated by CIBERSORT and correlations assessed between hub genes and immune cell fractions. Six hub genes were validated by RT-qPCR in a small clinical cohort (n=5 IS, n=5 CTL) using normality-contingent tests (independent t-test or Wilcoxon–Mann–Whitney). A logistic regression diagnostic model built on four hub genes was evaluated by ROC/AUC on both the training set and an independent GEO validation set (GSE58294: 69 IS vs 23 CTL), yielding AUC values of 0.999 and 0.940 respectively.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Limma moderated t-test (empirical Bayes) | Differential expression analysis, IS vs CTL, training set GSE16561 | 39 IS, 24 CTL | not stated |
| Shapiro–Wilk test | Normality assessment of RT-qPCR continuous data prior to group comparison | 5 IS, 5 CTL | na |
| Independent samples t-test | RT-qPCR hub gene expression comparisons: normally distributed data | 5 IS, 5 CTL | stated |
| Wilcoxon–Mann–Whitney test | RT-qPCR hub gene expression comparisons: non-normally distributed data | 5 IS, 5 CTL | stated |
| Logistic regression | Diagnostic model construction for IS classification; trained on 70% of GSE16561, tested on 30% | 39 IS, 24 CTL (GSE16561), 7:3 internal split | not stated |
| Spearman correlation and Pearson correlation (both mentioned for hub-gene/immune-cell associations) | Association between CIBERSORT immune cell fractions and hub genes | null | not stated |
| GSEA (gene set enrichment analysis) | Pathway enrichment in IS vs CTL using C2/KEGG reference gene sets | 39 IS, 24 CTL | not stated |
| GO/KEGG over-representation test (Fisher's exact via clusterProfiler) | Functional annotation of 188 DEGs | 188 DEGs against genome background | not stated |
-
RT-qPCR clinical validation was performed in a cohort of 5 IS patients and 5 healthy controls↳ Could also: A larger independent clinical cohort (e.g., n ≥ 20–30 per group) could also have been used for the wet-lab validation step — Larger sample sizes increase statistical power to detect true expression differences and reduce the influence of individual outliers; small-n cohorts are common in pilot studies but are sometimes supplemented with a powered replication cohort to strengthen confidence in findings
-
Both Spearman and Pearson correlations are mentioned for the association between hub genes and CIBERSORT immune cell fractions (described in separate sentences for what appears to be the same analysis)↳ Could also: A single, pre-specified correlation method — Spearman for ranked/non-normal data, Pearson for continuous normally distributed data — could also be applied and stated consistently — Explicitly choosing and declaring one method in advance avoids ambiguity about which coefficients are displayed and ensures the analysis is fully reproducible; choosing between them based on the distribution of CIBERSORT-estimated proportions is a standard practice
-
Six hub genes were each tested by RT-qPCR as separate comparisons at the unadjusted P < 0.05 threshold↳ Could also: A family-wise or FDR correction (e.g., Bonferroni, Benjamini–Hochberg) across the six gene tests could also have been applied — When multiple comparisons are conducted simultaneously, a multiplicity correction limits the expected number of false-positive discoveries across the family; this is consistent with the FDR approach already applied in the upstream bioinformatics analyses
-
Immune cell infiltration was estimated using CIBERSORT's 22-cell-type LM22 signature matrix↳ Could also: Alternative deconvolution tools such as xCELL, TIMER2.0, or EPIC could also have been applied, or results compared across methods — Different deconvolution algorithms use different reference matrices and mathematical assumptions; comparing concordance across two or more tools is an approach used to assess whether immune infiltration patterns are robust to method choice
-
Diagnostic model AUC values (0.999 training, 0.940 validation) were reported as point estimates↳ Could also: Bootstrap-derived or DeLong-method 95% confidence intervals around each AUC could also have been reported — AUC confidence intervals convey the precision of the estimate and are standard in diagnostic accuracy reporting (e.g., STARD guidelines); they also allow formal statistical comparison between AUCs from different models or datasets
-
Hub genes were defined as the intersection of LASSO regression and SVM-RFE outputs↳ Could also: A third independent selection method (e.g., random forest variable importance, elastic net) could also have been included to further validate the intersection set — A three-way intersection reduces the risk that selected genes reflect the idiosyncrasies of any single algorithm's assumptions; reporting gene-level selection frequency across methods is an approach some studies use to rank candidate biomarkers by algorithmic consensus
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
- CoINcIDE: A framework for discovery of patient... L1 87/100
- A curated collection of transcriptome datasets... L1 62/100
- An NMF-Based Methodology for Selecting Biomark... L1 84/100
- Unveiling prognostics biomarkers of tyrosine m...⚑ L1 51/100 ⚑
- Meta-analysis of gene expression profiles of l... L1 78/100
- Colorectal Cancer Prediction Based on Weighted...⚑ L1 80/100 ⚑
- Curation of over 10 000 transcriptomic studies... L1 80/100
- Construction and Validation of an Immune Infil...⚑ L1 51/100 ⚑
- Identification of a novel 10 immune-related ge...
- Exploration of the shared diagnostic genes and... L1 76/100
- IRSN-23 gene diagnosis enhances breast cancer... L1 71/100
- Molecular Classification Models for Triple Neg... L1 86/100
- Predicting Bone Metastasis Using Gene Expressi... L1 62/100
- Autoencoder Networks Decipher the Association... L1 74/100
- Comprehensive analysis of a novel RNA modifica... L1 71/100
- Comparative profiling of skeletal muscle model... L1 64/100
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-38650649
Paper: Cong et al. 2024, PeerJ 12:e17208. "Discovery and validation of molecular
patterns and immune characteristics in the peripheral blood of ischemic stroke patients."
Code: https://github.com/donkeycong/Ella (R, 16 numbered scripts; default branch main,
last push 2023-01-13). Data: GEO GSE16561 (Illumina HumanRef-8 v3, GPL6883;
39 ischemic-stroke + 24 control whole-blood samples). Validation: GSE58294.
The repo ships the raw data: GSE16561_series_matrix.txt (expression) and
GPL6883.annot (platform annotation). So the training pipeline is fully self-contained
and reproducible. This is a third-party-tool-style reproduction (P16): the repo is the
authors' own glue around standard R packages (limma, glmnet, e1071/caret, pROC, CIBERSORT) —
equally valid.
Pipeline-derived results (IN SCOPE — attempted)
| # | Result | Reported value | Pipeline / script |
|---|---|---|---|
| C1 | DEG count (limma) | 188 total (103 up, 85 down) | 02.Differential expression analysis.R |
| C2 | LASSO feature count | 12 genes | 07.LASSO.R (glmnet, seed 123) |
| C3 | SVM-RFE feature count | 34 genes | 08.SVM-RFE.R (caret rfe, seed 123) |
| C4 | Hub genes (LASSO ∩ SVM-RFE) | 6: ANTXR2, BAZ2B, C5AR1, PDK4, PPIH, STK3 | 09.Venn.R |
| C5 | Per-gene training ROC AUC | BAZ2B .892 / C5AR1 .966 / PDK4 .891 / STK3 .966 | 14...ROC.R |
| C6 | Diagnostic model AUC (train, GSE16561) | 0.999 | 14...ROC.R |
Lower priority / harder 20% (attempt if time; else noted)
| # | Result | Reported | Why harder |
|---|---|---|---|
| C7 | CIBERSORT immune cells up/down in IS | monocyte/M0/neutrophil/mast-act up; Tfh/CD8 down | needs LM22 ref.txt (not in repo); qualitative |
| C8 | GO/KEGG/GSEA top terms | qualitative term lists | qualitative, org.Hs.eg.db + msigdb |
| C9 | Validation model AUC (GSE58294) | 0.940 | external GEO download + re-annotation |
OUT OF SCOPE (not pipeline-derived → not attempted)
- qPCR / wet-lab confirmation of biomarker expression (manual lab work).
Known code/paper discrepancy to flag (possible-fabrication audit point)
02...R filters DEGs on raw P.Value < 0.05 (diffSig=allDiff[(abs(logFC)>logFCfilter & P.Value < 0.05),]), but the paper Methods state "adjusted P-value < 0.05". These can
give very different DEG counts. We compute both and report which (if either) gives 188.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.