Severe COVID-19 Shares a Common Neutrophil Activation Signature with Other Acute Inflammatory States.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- ✓Overall, the reproduction was clean
- Every checked point held up.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough and reproduces 1:1. Target = the tow-sample-tests analysis in github.com/lschimke/COVID19-and-HLH-paper@3b2fbb9 (non-parametric two-sample Satterthwaite t-approx, ICU vs Non-ICU COVID-19). The repo ships input data (my_data2.csv, 503 vars x 102 samples), the unmodified script.R (set.seed(2021)), and the expected output tables. Running the shipped script on the shipped data in a clean conda env (R 4.0.5 = paper's exact version; nparcomp 3.0, MVN 5.9, ggplot2 3.3.6, npmv 2.4.1 from CRAN) on «our HPC» SLURM regenerated Supplementary Tables S1 and S2 BYTE-IDENTICAL (same SHA256, max abs diff 0.0 across all 503 rows of both tables) and the headline claim of 212 significant variables (p<0.05) exactly. FigS1/FigS2 regenerated (visual, not byte-compared). NOT attempted (out of 80/20 scope, recorded in scope.md): the single-cell Seurat pipeline on GSE152418 (partly stochastic, no shipped numeric reference) and the PCA/CCA/MLR/randomForest/heatmap visualisations (ship example data + plotting code but no pinnable expected value; randomForest also stochastic). Minor wording note: paper says '504 observed variables' but 503 are tested (the 504th column is the Group label) — wording off-by-one, not a result discrepancy. No fabrication signal: all compared values fully derivable from the deposited data+code.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 88assessed: 2026-06-15 ⛓ 77c0fb2b067d
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15👤 1 human curator(s) · Level L2 2026-06-15
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusSevere COVID-19 clinically overlaps with other hyperinflammatory states such as HLH; the authors test whether these conditions share common transcriptomic/immunological pathways and gene signatures (particularly cytokine/chemokine and neutrophil-mediated immunity) that associate with and stratify COVID-19 severity.
- ★ COVID-19 and HLH share immunological pathways, yielding 237 common up-regulated DEGs dominated by cytokine/chemokine signaling and neutrophil-mediated immune responses. finding
- ★ A cluster of neutrophil-associated genes reflects a generalized hyperinflammatory state, being also dysregulated in Kawasaki disease and bacterial pneumonia. finding
- ★ These gene signatures stratify COVID-19 ICU from non-ICU patients and are dysregulated at the protein level, forming an interconnected network pointing to neutrophil hyperactivation in ICU patients. finding
- ★ scRNAseq shows these genes are specifically upregulated across leukocyte populations including lymphocyte subsets and immature neutrophils. finding
- ★ An AI/random forest model confirms a strong association of these genes with COVID-19 severity (ICU admission). method
- ★ Integrative multi-omics meta-analysis of publicly available transcriptome data across 1596 individuals from 11 datasets identifies shared inflammatory signatures. method
- Seven fHLH/IEI-associated genes, though not commonly dysregulated across datasets, enrich neutrophil degranulation and granule cellular components. mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Bulk RNA-seq / transcriptome meta-analysis (DEG analysis) | Human peripheral blood lymphocytes, PBMCs, and nasopharyngeal swabs from COVID-19, HLH, MIS-C, Kawasaki disease, influenza, seasonal coronavirus, bacterial pneumonia patients and healthy controls (1596 individuals, 11 datasets) | none (observational disease vs control) | Differentially expressed genes (log2 CPM; log2FC>1/<-1, adj p<0.05) | NetworkAnalyst 3.0, limma-voom pipeline |
| Single-cell RNA-seq analysis | Human leukocytes/PBMCs (Schulte-Schrepping et al. dataset, EGAS00001004571) | none | Cell-type-specific gene expression (UMAP, dotplot, heatmap) | Seurat pipeline |
| Plasma proteomics (LFQ mass spectrometry) | Plasma from COVID-19_ICU vs COVID-19_nonICU patients (Overmyer et al.) | none | Differentially expressed proteins (LFQ abundance) | LFQ quantification; nonparametric MANOVA |
| Protein–protein interaction / interactome network analysis | DEGs common to COVID-19 and HLH | none | Direct physical protein interactions / network | Integrated Interactions Database (IID v2021-05); NAViGaTOR 3.0.14 |
| Pathway/enrichment analysis | Common DEG and DEP gene lists | none | Enriched biological processes, cellular components, signaling pathways | ClusterProfiler R, Enrichr |
| Correlation analysis (CCA, PCA, correlograms) | COVID-19_PBL dataset (GSE157103; 100 COVID-19 + 26 SARS-CoV-2-negative controls) | none | Linear relationship between cytokine/chemotaxis and neutrophil gene signatures | R packages CCA, whitening, factoextra, corrgram |
| Machine learning classification (random forest) | COVID-19_nonICU vs COVID-19_ICU transcriptomes | none | ICU-admission classification / variable importance (gini, node count, min depth); OOB error, ROC | randomForest R package v4.6.14 |
- ▲ 239 unique common DEGs identified between HLH and all COVID-19 datasets, with 237 up-regulated and 2 down-regulated (GNLY, MYOM2). 237 up / 2 down of 239
- – Total of 21,583 DEGs obtained from seven COVID-19 cohorts and one HLH cohort. 21,583 DEGs
- ▲ Among the 237 common up-regulated DEGs, 25 assigned to cytokine, 34 to chemotaxis, and 58 to neutrophil signatures. 25/34/58 genes
- ▲ Common up-regulated DEGs include neutrophil-activation genes (CD177, MPO, ELANE), chemokine genes (CCL4, CCL2, CXCR2), and cytokine genes (IL1R2, IL1B, IL6, TNF).
- ▲ Top enriched biological processes encompass neutrophil activation, degranulation, migration, and response to IL-1; cellular components include secretory/azurophil/specific granules.
- – Seven fHLH-associated genes (AP3B1, LYST, PRF1, RAB27A, STX11, STXBP2, UNC13D) enrich neutrophil degranulation and granule components despite not being commonly dysregulated. 7 genes
- count 1596 individuals (Total cohort across 11 datasets analyzed)
- count 11 datasets (Number of transcriptome datasets integrated)
- count 239 unique common DEGs (Common DEGs between HLH and all COVID-19 datasets)
- count 237 up-regulated DEGs (Up-regulated common DEGs focus of analysis)
- count 21,583 DEGs (Total DEGs from seven COVID-19 cohorts and one HLH cohort)
- count 100 COVID-19 + 26 controls (GSE157103 COVID-19_PBL dataset for CCA)
- count 25 transcripts (Genes associated with COVID-19 severity used in PCA)
- other 5000 trees, mtry=3, 75% train/25% test (Random forest model parameters for ICU classification)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is an integrative bioinformatics/meta-analysis of publicly available transcriptome (bulk and single-cell RNA-seq) and plasma proteome data from 1596 individuals across 11 datasets, comparing COVID-19 with HLH, other inflammatory syndromes, and respiratory infections. Differential expression was computed per dataset with the limma-voom pipeline using fixed fold-change and adjusted p-value cut-offs, and shared genes were intersected and explored through enrichment, protein-protein interaction networks, multivariate correlation (PCA, CCA, multilinear regression), nonparametric multivariate testing of proteins, and a random forest classifier for ICU prediction. Results were reported largely as DEG/DEP lists, enrichment dot plots, heatmaps, network diagrams, and classifier performance (OOB error, ROC) rather than as conventional per-comparison significance tables with dispersion statistics.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| limma-voom differential expression (moderated linear model on log2 CPM) | DEGs between groups within each transcriptome dataset (e.g., COVID-19 vs control, HLH vs control) | per-dataset sample sizes from 11 datasets totaling 1596 individuals; e.g., GSE157103 with 100 COVID-19 and 26 controls | not stated |
| Seurat differential expression for single-cell RNA-seq | scRNAseq dataset EGAS00001004571 (Schulte-Schrepping et al.) across leukocyte populations | — | not stated |
| Nonparametric MANOVA (multivariate analysis of variance) | differences in plasma protein expression between COVID-19_ICU and COVID-19_nonICU (Overmyer et al. proteomics) | — | na |
| Nonparametric inference for multivariate data (npmv / nparcomp) | follow-up comparison of differentially expressed proteins between COVID-19_ICU and nonICU | — | na |
| Canonical Correlation Analysis (CCA) | linear relationship between cytokine/chemotaxis and neutrophil-mediated immunity gene signatures (COVID-19_PBL, GSE157103) | 100 COVID-19 and 26 control individuals | not stated |
| Principal Component Analysis (PCA) and multilinear regression | 25 transcripts associated with COVID-19 severity; combinations of genes | — | not stated |
-
DEGs were defined using fixed cut-offs of |log2 fold-change| > 1 together with adjusted p-value < 0.05.↳ Could also: A ranked or continuous approach (e.g., reporting the full statistic with FDR and using gene-set enrichment such as GSEA on ranked lists rather than a hard fold-change threshold) could also be used. — Continuous/ranked methods retain genes with moderate but coherent changes and reduce dependence on an arbitrary fold-change boundary, which can complement the threshold-based list.
-
Shared genes were identified by intersecting per-dataset DEG lists (Venn/Circos) across studies generated on different platforms.↳ Could also: A formal meta-analysis model (e.g., random-effects effect-size combination across studies, or batch-aware joint modeling) could also be applied. — A pooled statistical model provides combined effect estimates with confidence intervals and explicitly accounts for between-study heterogeneity, complementing the overlap-counting approach.
-
Multiplicity was handled per dataset via an adjusted p-value threshold, without a stated correction across the multivariate, correlation, and CCA analyses.↳ Could also: Reporting the specific adjustment method (e.g., Benjamini-Hochberg FDR) and extending multiplicity control to the correlation/multivariate family could also be done. — Naming the method and covering all test families makes the family-wise or false-discovery scope fully transparent across the analyses.
-
Plasma protein differences between ICU and nonICU groups were assessed with nonparametric MANOVA and nonparametric multivariate inference.↳ Could also: Per-protein moderated tests (e.g., limma on proteomics) with FDR control, accompanied by effect sizes and confidence intervals, could also be reported. — Per-protein estimates with intervals convey the magnitude and precision for individual proteins alongside the global multivariate result.
-
The random forest ICU classifier was evaluated with a single 75/25 train/test split, OOB error, and ROC.↳ Could also: Repeated k-fold or nested cross-validation with confidence intervals on AUC could also be used. — Resampling-based validation gives a more stable performance estimate and uncertainty bounds that are less sensitive to a single partition.
-
Group summaries were displayed via box plots and heatmaps without a specified dispersion statistic.↳ Could also: Explicitly reporting SD, IQR, or 95% confidence intervals alongside the plots could also be done. — Stating the dispersion measure makes the spread and uncertainty of each group directly interpretable, which is especially helpful for smaller per-group n.
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Reproduction scope — pmid-35269470
Paper: Schimke LF et al. (2022) Severe COVID-19 Shares a Common Neutrophil Activation Signature with Other Acute Inflammatory States. Cells 11(5):847. DOI 10.3390/cells11050847 · PMCID PMC8909161.
Code: https://github.com/lschimke/COVID19-and-HLH-paper @ commit
3b2fbb9bc74324fb9af93a858d7de4afc4d67d8b (HEAD of main, pushed 2021-07-30).
Public, no license file (re-use for reproduction only; not redistributed).
Data: the repo is organized as one folder per analysis method; each folder
ships its own input data + scripts + (mostly) expected output figures/tables.
The GEO accession GSE152418 is used only by the single-cell folder.
Repo structure (analysis units)
| Folder | Method | Ships input | Ships expected output | Determinism |
|---|---|---|---|---|
tow-sample-tests |
non-param. MANOVA + Satterthwaite two-sample t-approx (npmv, nparcomp) | ✅ my_data2.csv (503 vars, ICU/Non-ICU) | ✅ TabS1.csv, TabS2.csv, FigS1/S2 | ✅ set.seed(2021), deterministic tests |
PCA |
PCA + ggplot | ✅ example datafile PCA.txt | ✖ (figure only, no value) | deterministic |
random forest |
randomForest classifier | ✅ my_data.csv | ✖ (no shipped metric) | ✖ stochastic (RF) |
CCA |
canonical correlation (CCApackage) | ✅ demo csv/xlsx | ✖ | deterministic |
Multilinear regression |
MLR (2- & 3-group) | ✅ example txt | ✖ | deterministic |
Correlograms / Circular Heatmaps / DotPlots / BoxPlot |
ComplexHeatmap / ggplot visualisations | ✅ example txt/xlsx | ✖ (plots only) | deterministic |
single cell analysis |
Seurat 10x PBMC pipeline on GSE152418 | ✖ (needs GEO download) | ✖ | partly stochastic (UMAP/clustering) |
In scope (attempted)
Primary — tow-sample-tests. This is the cleanest reproducible unit: it
ships the exact input data, the exact scripts (with a fixed seed), AND the exact
expected output tables (TabS1.csv = full two-sample-test result for all 503
variables; TabS2.csv = log2 fold-change + p-value). The deterministic statistical
tests (npmv::nonpartest, nparcomp::npar.t.test method t.app) should
regenerate these tables 1:1. Pinnable claim from the repo README/TabS1:
"From the 504 observed variables only 212 had significantly mean differences
between ICU and Non-ICU patients." (Tested variables = 503; significant at
p<0.05 = 212.)
Pipeline: R 4.0.5 → reshape2/dplyr reshape → MVN:::descriptives →
npmv::nonpartest (NP-MANOVA) → per-variable nparcomp::npar.t.test →
subset p<0.05 → log2 fold change → ggplot2 FigS1/FigS2 + write.table TabS1/S2.
Out of scope (not attempted) — and why
- Single-cell (GSE152418, Seurat): main figures of the paper, but the pipeline is partly stochastic (UMAP/Louvain seeds, package-version sensitive) and ships no expected numeric reference in the repo → not 1:1 verifiable in the 80/20 budget. Recorded as not-attempted, not a drop.
- PCA / CCA / MLR / heatmaps / boxplots / dotplots / random forest: ship example data and plotting code but no shipped expected value/figure with a pinnable number to compare against (random forest is additionally stochastic). Visual-only reproductions are not auditable as exact/within-tol.
- All wet-lab / clinical / proteomic measurements (the upstream generation of the 503-variable matrix): external, not pipeline-derived → out of scope.
Faithfulness notes
- R version pinned to the paper's 4.0.5 in the conda env where it solves; numeric outputs of these tests are version-robust (3-decimal rounding).
- The shipped TabS1/TabS2 are treated as the expected reference; we run the shipped script on the shipped input in a fresh copy and compare byte/numeric.
- All heavy steps run on «our HPC» (SLURM); «host» holds results only.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Re-running the authors' own shipped script.R on their deposited my_data2.csv in the paper's exact R 4.0.5 environment regenerated Supplementary Tables S1 and S2 byte-identically and the headline claim of 212 significant variables (p<0.05) exactly. There is no deviation on any compared value, no fabrication signal, and the data/code are on the authors' side and fully sufficient. The only blemish is cosmetic — the paper says '504 observed variables' while 503 are tested (the 504th column is the Group label) — an off-by-one in wording. Note the reproduction was scoped (80/20) to the two-sample-tests target; the stochastic single-cell Seurat and PCA/RF visualizations were not attempted, but the scoped claims reproduce 1:1.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.