Inflammatory Bone Marrow Mesenchymal Stem Cells in Multiple Myeloma: Transcriptional Signature and In Vitro Modeling.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🔴Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🔴Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Wang et al. 2023 (Cancers) is a SECONDARY META-ANALYSIS pooling ~17 public expression datasets to define an inflammatory-MSC signature in multiple myeloma; it ships NO own analysis code (its only GitHub link is a third-party scRNA-seq repo cited for cell-type annotation) and a vague data-availability statement. Its headline numbers (667 up/844 down bulk DE; 279-gene iMSC signature) are NOT derivable from any single shipped artifact + accession (undocumented multi-dataset pooling + quantile normalization + sample selection) = the honest 20%, NOT attempted. We DID reproduce, end-to-end on «our HPC», the paper's STATED DE method (two-sample t-test p<0.05 & FC>2) on the room's pinned component dataset GSE113736 (clean Affy microarray, 12 MM-MSC vs 12 normal-donor MSC): 46 DE genes (12v12) / 17 (4v4 subject means). Directionally, inflammatory chemokines (IL8/CXCL8, IL1B, CXCL2/3) trend up consistent with the paper's theme, but IL6 itself is flat and most genes fall below FC>2. Outcome=partial (faithful method-on-paper's-data reproduction of one in-scope dataset; headline meta-analysis numbers not reproducible 1:1 from shipped info). NOT a drop: pinned data clean + one analysis runnable; NOT a 1:1: no GSE113736-specific number exists in the paper to match. No result fabricated.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-15 ⛓ b40876a1d807
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusThe authors test whether the MM-specific inflammatory MSC (iMSC) transcriptional signature, lost during in vitro expansion of bone marrow MSCs, can be reactivated in vitro by cytokine stimulation or coculture with immune/cancer cells to enable functional modeling of iMSCs.
- ★ Reanalysis of scRNA-Seq identified 279 iMSC signature genes upregulated in BM MSCs of MM patients vs healthy donors, covering 82% of the top 50 previously defined iMSC genes finding
- ★ iMSC signature expression is positively associated with minimal residual disease: elevated in MRD-positive and attenuated in MRD-negative patients finding
- ★ In vitro expansion of BM MSCs generally results in loss of the MM-specific iMSC signature and downregulation of proliferation-related genes finding
- ★ Cytokine stimulation with IL-1β and TNF-α strongly reactivates iMSC signature expression, with lesser activation by TGF-β1 and no change with IFN-γ finding
- ★ Coculture with neutrophils, pro-inflammatory macrophages, and activated T cells reactivates iMSC signature genes in distinct gene subsets finding
- ★ TNF-α stimulation substantially activates IL1B transcription and IL-1β cellular response, indicating IL-1β mediates part of the TNF-α response mechanism
- Meta-analysis of public expression data with GSEA and CytoSig scoring is a viable approach to characterize iMSC signature across studies method
- iMSCs from MM patients are distinct from inflammatory CAFs in solid tumors, sharing only immune-related genes finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| single-cell RNA-seq (reanalysis) | FACS-isolated BM MSCs from newly diagnosed MM patients and healthy donors (E-MTAB-9139) | none (disease vs healthy) | upregulated iMSC signature genes (FindAllMarkers, Wilcoxon) | CellRanger v3.0.2, 10x Genomics; Seurat 4.2.1 |
| bulk RNA-seq (reanalysis) | FACS-isolated in vivo BM MSCs from MM patients vs HDs (E-MTAB-9285) | none (disease vs healthy) | differentially expressed genes; GSEA enrichment | — |
| bulk RNA-seq (reanalysis, paired) | BM MSCs paired at diagnosis vs during MRD (E-MTAB-9285) | none (MRD status) | DE genes; iMSC signature/TNF-NF-κB GSEA | — |
| meta-analysis of transcriptome data | in vitro expanded BM MSCs from MM vs HD (6 studies: GSE113736, GSE137369, GSE146649, GSE46053, GSE78235, GSE36474) | in vitro expansion | quantile-normalized fold changes; GSEA | — |
| meta-analysis of transcriptome data | HD BM MSCs stimulated by cytokines (GSE129165, GSE161762, GSE33755, GSE35331, GSE77814, E-MTAB-5420, E-MTAB-5421) | IL-1β, IFN-γ, TGF-β1, TNF-α stimulation | expression fold changes; GSEA; CytoSig cytokine response score | CytoSig webserver |
| transcriptome coculture analysis | HD BM MSCs cocultured with neutrophils (GSE62782) | neutrophil coculture | iMSC signature/TNF-NF-κB GSEA; CytoSig | — |
| transcriptome coculture analysis | BM MSCs cocultured with pro- or anti-inflammatory macrophages (GSE93970) | macrophage coculture | iMSC signature/TNF-NF-κB GSEA; CytoSig | — |
| transcriptome coculture analysis | mouse BM MSCs with CD3/CD28-activated T cells (GSE75749); BM MSCs cocultured with MM cell lines MM.1S (GSE46053) or INA-6 (GSE87073) | activated T cell / MM cell coculture | DE genes; Metascape enrichment | — |
- ▲ 279 iMSC signature genes identified, representing 82% of the top 50 previously defined iMSC-specific genes 82% overlap; FC>1.5
- – 667 upregulated and 844 downregulated genes in in vivo BM MSCs of MM vs HDs 667 up / 844 down; FC>2
- ▲ MRD-positive BM MSCs show elevated iMSC signature genes (IL6, CXCL3) and upregulated TNF-α/NF-κB signaling vs diagnosis
- ▼ MRD-negative BM MSCs show decreased iMSC signature genes (IL6, CXCL3) and downregulated TNF-α/NF-κB signaling (~600 DE genes) ~600 DE genes
- ▼ Inflammatory/immune gene upregulation (CCL2, CXCL2/3/5/8, PTGS2) generally lost during in vitro expansion; proliferation genes downregulated
- ▲ TNF-α and IL-1β produce dramatic upregulation of TNF-NF-κB and iMSC signature genes; TGF-β1 lesser; IFN-γ none
- ▲ Neutrophils upregulate iMSC genes (CCL2, CXCL3, CXCL5, CXCL8, IL6) with IL-1β and TNF-α ranked top driver cytokines by CytoSig
- ▲ 649 genes upregulated in BM MSCs by CD3/CD28-activated T cells in a mouse model 649 genes; FC>2
- count 279 upregulated iMSC signature genes (scRNA-Seq MM BM MSCs vs HDs, adj p<0.05, FC>1.5)
- count 667 upregulated and 844 downregulated genes (bulk RNA-Seq in vivo BM MSCs, MM vs HD, p<0.05, FC>2)
- other 82% (of top 50 previously defined iMSC genes covered by the 279 genes)
- count ~600 differentially expressed genes (MRD-negative patients after treatment)
- count 649 genes upregulated (BM MSCs by activated T cells, FC>2, adj p<0.05 (GSE75749))
- count 10^-5 (MRD defined as cancer cells at rate no more than 1 in 10^5 normal BM cells)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This study conducted a meta-analysis of publicly available bulk and single-cell RNA-seq datasets to characterize the inflammatory MSC (iMSC) transcriptional signature in multiple myeloma. Differential expression was identified with Wilcoxon rank-sum tests (scRNA-seq via Seurat) and with t-tests or paired t-tests (bulk RNA-seq), each using fold-change thresholds alongside p-value cutoffs. Pathway-level evidence relied on Gene Set Enrichment Analysis (GSEA) against MSigDB hallmark gene sets, and cytokine contributions were estimated with the CytoSig webserver. Expression changes across heterogeneous public datasets were harmonized by quantile normalization of GSEA rank files before meta-analytic comparison.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Wilcoxon rank-sum test (Seurat FindAllMarkers) | scRNA-seq: identification of upregulated genes in BM MSCs of MM patients vs. healthy donors (E-MTAB-9139) | — | not stated |
| Two-sample t-test | Bulk RNA-seq: differentially expressed genes in MM patients vs. healthy donors (E-MTAB-9285), threshold p < 0.05 and FC > 2 | — | not stated |
| Paired t-test | Bulk RNA-seq: MRD-positive or MRD-negative samples vs. paired diagnosis samples from the same patients (E-MTAB-9285), threshold p < 0.05 and FC > 2 | — | not stated |
| Gene Set Enrichment Analysis (GSEA) | Pathway enrichment across all datasets for iMSC signature genes and MSigDB hallmark gene sets; applied to in vivo, in vitro expansion, cytokine stimulation, and coculture datasets | — | na |
| Hierarchical clustering (correlation-based) | Grouping of cytokine stimulation samples by similarity of expression-change profiles (Figure 3A) | — | na |
| CytoSig score | Estimation of individual cytokine contributions to expression changes in BM MSCs after neutrophil coculture, macrophage coculture, and cytokine stimulation (Figures 3F, 4D, 4H) | — | na |
-
Bulk RNA-seq differential expression was assessed with a standard t-test at a nominal p < 0.05 with no stated multiple-testing correction across all genes↳ Could also: Negative-binomial model-based methods such as DESeq2 or limma-voom with Benjamini-Hochberg FDR correction could also have been applied — Model-based approaches explicitly account for count-data overdispersion and provide genome-wide FDR control, which is relevant when tens of thousands of genes are tested simultaneously
-
Single-cell differential expression was computed with a Wilcoxon rank-sum test via Seurat's FindAllMarkers with an adjusted p-value threshold↳ Could also: Pseudo-bulk approaches (aggregating counts per donor then applying DESeq2 or edgeR) or mixed-effects models such as MAST could also have been used — Pseudo-bulk and mixed-effects methods account for within-donor correlation among cells, which can otherwise inflate the effective sample size and produce overly narrow p-values in single-cell comparisons with multiple donors
-
Cross-study expression harmonization relied on quantile normalization of GSEA rank files↳ Could also: Explicit batch-correction methods such as ComBat, limma removeBatchEffect, or surrogate variable analysis (SVA) could also have been applied — These frameworks model study-of-origin as a covariate and can reduce technical variance more explicitly while preserving biological signal, which is particularly relevant when integrating nine or more heterogeneous public datasets
-
Effect sizes were expressed as fold change with a hard dichotomous threshold (FC > 1.5 or FC > 2)↳ Could also: Shrinkage-estimated log2 fold changes (e.g., apeglm in DESeq2) or standardized effect sizes such as Cohen's d could also have been reported alongside p-values — Shrinkage estimation stabilizes fold-change estimates for low-count or high-variance genes and provides a continuous measure of effect that is not dependent on an arbitrary cutoff
-
Pathway analysis was performed with GSEA using pre-ranked gene lists to produce a single summary enrichment score per gene set per comparison↳ Could also: Gene set variation analysis (GSVA) or single-sample GSEA (ssGSEA) could also have been applied to generate per-sample enrichment scores — Per-sample scoring allows visualization of enrichment variability across individual samples and enables conventional statistical comparison of enrichment distributions between groups, complementing the aggregate ranking approach
-
Per-group sample sizes for each comparison were not reported in the text, and no power analysis or sample size justification was described↳ Could also: Explicit reporting of per-group n for every comparison and, where feasible, a post-hoc sensitivity or power calculation could also have been included — Reporting per-group n and power supports assessment of whether individual studies in the meta-analysis were adequately sized to detect the effect magnitudes reported, particularly for the paired MRD analysis where patient numbers appear small
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
Downstream reach in the literature
16 downstream papers · 13 datasetsHow widely the datasets deposited by this paper are reused across the whole literature (Europe PMC), beyond our assessed set. This is a factual dependency map — reusing a public dataset is normal, good science. It is not a judgement on the downstream papers; the only verdict here is this paper's own, with its cited rationale.
- Transcriptomic profile induced in bone marrow mesenc... 2014 · 43 cites
- Host-Derived Matrix Metalloproteinase-13 Activity Pr... 2021 · 16 cites
- Selective inhibition of matrix metalloproteinase-2 i... 2017 · 13 cites
- Identification of a novel cuproptosis-related gene s... 2023 · 9 cites
- Cytokine Activation Reveals Tissue-Imprinted Gene Pr... 2022 · 17 cites
- Transcriptome profiles acquired during cell expansio... 2020 · 14 cites
- SLAMF8 regulates osteogenesis and adipogenesis of bo... 2024 · 5 cites
- Current status of cell-based therapies for respirato... 2020 · 178 cites
- Interferon-γ and Tumor Necrosis Factor-α Polarize Bo... 2016 · 73 cites
- Deciphering the combinatorial landscape of immunity. 2020 · 9 cites
- Co-Inflammatory Roles of TGFβ1 in the Presence of TN... 2017 · 25 cites
- Co-Inflammatory Roles of TGFβ1 in the Presence of TN... 2017 · 25 cites
- Pericentromeric Non-Coding DNA Transcription Is Asso... 2022 · 9 cites
- Persistent Inflammatory Stimulation Drives the Conve... 2021 · 39 cites
- IL1β induces mesenchymal stem cells migration and le... 2012 · 145 cites
- Identification of key genes in osteosarcoma by meta‑... 2019 · 13 cites
- Neutrophils trigger a NF-κB dependent polarization o... 2015 · 60 cites
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-37958322 (Wang et al. 2023, Cancers; "Inflammatory BM-MSC in Multiple Myeloma")
Nature of the paper
Secondary meta-analysis / re-analysis of ~17 public expression datasets (GEO + ArrayExpress) to define an "inflammatory MSC (iMSC)" transcriptional signature in multiple myeloma. NOT a primary-data paper with its own pipeline repo.
Code/data availability (verbatim findings)
- Data Availability Statement: "The data presented in this study are available in this article." (no per-result accession→analysis mapping)
- The ONLY GitHub link (github.com/MyelomaRotterdam/de-Jong-et-al.-2021) is a THIRD-PARTY scRNA-seq repo from a different study, cited as the "cell type annotation source" — it is NOT the analysis code that produced this paper's numbers.
- => Paper ships NO runnable analysis code for its reported DE/GSEA results. (SCREENING taxonomy: borderline no_code / docs_insufficient.)
Datasets used by the paper (from Methods/Data section)
Primary MSC: E-MTAB-9139, E-MTAB-9285. Expanded MSCs: GSE113736, GSE137369, GSE146649, GSE46053, GSE78235, GSE36474, GSE80608, GSE196297, GSE108159. Plus cytokine-stim, immune-coculture, MM-coculture sets. Quantile-normalized & pooled.
Pipeline-derived reported results (candidate claims)
| # | result | method (as stated) | in scope? |
|---|---|---|---|
| C1 | 667 up / 844 down genes, MM vs HD | bulk RNA-Seq t-test p<0.05 & FC>2 | source dataset = E-MTAB-9139/9285 (RNA-seq), undocumented sample selection → NOT cleanly reproducible (the "20%") |
| C2 | 279 up genes = iMSC signature; 82% of top-50 iMSC genes | pooled multi-dataset meta-analysis, quantile-norm | NOT reproducible (combination logic unspecified) |
| C3 | scRNA DE | Seurat 4.2.1 Wilcoxon adj-p<0.05 FC>1.5 | own raw scRNA not deposited as such → out |
What we DO attempt (room pinned to GSE113736)
R1 — MM-MSC vs HD-MSC differential expression on GSE113736, the room's pinned
dataset and the cleanest in-scope component (microarray: 12 MM-MSC vs 12 ND-MSC,
processed matrix on GEO). We apply the paper's STATED bulk DE criteria (two-group
t-test, p<0.05, FC>2 i.e. |log2FC|>1) as a faithful "third-party-method-on-paper's-
data" reproduction. The paper reports no GSE113736-specific DE count (it pools
this set), so R1 cannot be a 1:1 number-match; it is a concrete, auditable data
point on the pinned dataset, graded partial. We compare its magnitude/direction
to the paper's analogous bulk claim (C1) for context only.
Explicitly NOT attempted (and why)
- C1/C2 headline numbers: require undocumented multi-dataset pooling + sample selection + quantile normalization that the paper does not ship → the "last 20%".
- scRNA (C3): raw 10x data + annotation chain not reproducibly specified here.
- We do not fabricate a match to 667/844 or 279.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a secondary meta-analysis whose central quantitative claims (667 up/844 down bulk DE; 279-gene iMSC signature) are not derivable from any single shipped artifact + accession — they require an undocumented pooling of ~17 datasets with quantile normalization and unspecified sample selection, and the paper ships no own analysis code (its lone GitHub link is a third-party scRNA-seq annotation repo). The gap is squarely on the authors'/data-availability side, not our methodology. We faithfully applied the paper's stated DE method (t-test p<0.05 & FC>2) to the clean in-scope component dataset GSE113736, getting 46 DE genes (12v12) / 17 (4v4); inflammatory chemokines (IL8/CXCL8, IL1B, CXCL2/3) trend up consistent with the theme, but IL6 — named by the paper — is flat (-0.07, p=0.72). No fabrication; overall a solid, explainable partial reproduction whose headline numbers simply cannot be matched 1:1.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.