Omics Playground: a comprehensive self-service platform for visualization, analytics and exploration of Big Omics Data.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓No authors-side cause for any deviation
- ✓The central claim held under reproduction
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Software/platform paper (Omics Playground). Described well enough to reproduce its pipeline, but it reports NO pinnable numeric result for its example datasets (confirmed across full text + every figure caption), so a classical 1:1 numeric match is not possible. Mis-mined metadata corrected: real code is bigomics/omicsplayground (+playbase/playdata), NOT pinin4fjords/shinyngs (wrong tool); GSE72056 is a genuine example dataset. Per BRIEF P16 we (a) reran the platform's DE pipeline on its own example dataset GSE10846 (Fig 3A) on «our HPC» and recovered the established DLBCL ABC/GCB signature 12/12 direction-correct with ~4388 consensus DEGs, and (b) verified the paper's concrete software-content claims against current source: DE method set matches exactly, '>50,000 gene sets' holds (123,832), but '30 collections' does not map to the current 2,050 (version/definition drift). NOT attempted (80/20): full Shiny app + Docker deployment, CNV inference (Fig 2D), survival tree (Fig 3D), L1000 drug connectivity (Fig 2H), scRNA-seq t-SNE cell typing (Fig 2A) — illustrative panels with no reported values and high UI/manual effort. No fabrication detected.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 61assessed: 2026-06-15 ⛓ b2806c0ad802
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusTertiary analysis (data visualization and interpretation) of big omics data is the current bottleneck for life scientists lacking bioinformatics skills; the paper presents Omics Playground, a self-service platform, as a consolidated, code-free solution to this need.
- ★ Omics Playground is a user-friendly, interactive self-service bioinformatics platform for in-depth analysis, visualization and interpretation of transcriptomics and proteomics data without coding resource
- ★ The platform focuses strongly on tertiary analysis (interpretation) while also supporting secondary analysis (statistical testing) method
- ★ Differential expression is computed with up to seven algorithms (including t-test, limma, edgeR, DESeq2) and combined via meta-analysis to yield reliable hits across methods method
- ★ Gene set enrichment is computed over more than 50,000 gene sets using seven methods (Spearman correlation, GSVA, ssGSEA, Fisher's exact test, GSEA, camera, fry) method
- ★ The platform uniquely combines a drug connectivity map (L1000 CMap), immune cell profiling, and modules for both biomarker selection and survival analysis resource
- ★ Omics Playground was specifically devised to support single-cell RNA-seq data in addition to traditional gene expression experiments resource
- Data import/precomputation is performed offline so statistics are precomputed, minimizing UI latency and enabling real-time interaction method
- The platform handles microarray, RNA-seq and LC-MS/MS proteomics data for two species (human and mouse) resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| RNA-seq / microarray gene expression analysis | human and mouse samples | none (analysis platform) | differentially expressed genes between contrasts (logFC, meta.q) | Omics Playground v1.0 (R/Shiny) |
| LC-MS/MS proteomics analysis | human and mouse samples | none | protein/gene expression and enrichment | Omics Playground v1.0 |
| Single-cell RNA-seq analysis | human and mouse | none | single-cell profiling, clustering (PCA/tSNE) | Omics Playground v1.0 |
| Gene set enrichment analysis | transcriptomics/proteomics datasets | none | enrichment scores across >50,000 gene sets / 30 collections | GSVA, ssGSEA, GSEA, camera, fry, Fisher's exact test, Spearman correlation |
| Drug connectivity map (CMap) | contrast expression profiles | in silico drug-profile correlation | top similar/opposite drug signatures | L1000 database (>5000 perturbagens) |
- – Omics Playground offers the second largest number of gene set databases among compared platforms, with only BioJupies offering more
- – Top features in clustering heatmaps are divided into five clusters and functionally annotated using >42 reference databases
- – Meta-analysis combines DEG/GSE methods so meta.q reflects the highest q value and stars indicate number of methods with significant q (q<0.05)
- – Drug C-Map shows the top 10 similar and opposite drug profiles by correlating contrast profiles against the L1000 database
- count >50,000 gene sets (total gene sets/pathways available for enrichment, in 30 collections)
- count >5000 perturbagens (drug profiles in L1000 CMap database for connectivity mapping)
- count >42 reference databases (databases used for functional annotation of clusters)
- count 7 methods (GSE analysis methods combined; up to 7 DEG algorithms)
- pvalue q < 0.05 (significance threshold for stars qualifier in DEG/GSE tables)
- count top N = 50, 500 features (top features selected for heatmap hierarchical clustering)
Statistical methods review
Model: opusA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software/methods paper describing 'Omics Playground', a self-service bioinformatics platform, rather than an empirical study that applies statistical tests to compare experimental groups. Accordingly, it does not report its own hypothesis tests, sample sizes, or p-values; instead it enumerates the statistical and enrichment methods the platform makes available to users and describes how multiple method results are combined via meta-analysis. Differential expression is offered through several algorithms whose results are consolidated (a meta.q value taken as the highest q across methods, with a 'stars' count of how many methods reach q < 0.05), and gene-set enrichment is offered through several complementary methods.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| t-test (standard and Welch) | differential expression analysis between contrasts (platform option) | — | not stated |
| limma (no trend, trend, voom) | differential expression analysis between contrasts (platform option) | — | not stated |
| edgeR (QLF, LRT) | differential expression analysis between contrasts (platform option) | — | not stated |
| DESeq2 (Wald, LRT) | differential expression analysis between contrasts (platform option) | — | not stated |
| Fisher's exact test | gene-set/pathway overlap enrichment (KEGG graph, gene set enrichment module) | — | not stated |
| GSEA, ssGSEA, GSVA, camera, fry, Spearman rank correlation | gene set enrichment analysis and drug connectivity (C-Map) scoring | — | not stated |
-
Differential expression significance is consolidated across methods using meta.q (the highest q value among methods) and a count of methods reaching q < 0.05.↳ Could also: A single pre-specified primary method (e.g. limma-voom or DESeq2) with a formally defined consensus/rank-aggregation rule could also be used. — Pre-specifying one primary method, or a documented aggregation statistic, can make the family-wise or false-discovery interpretation of the combined result more explicit and reproducible across users.
-
FDR/q-value thresholds are applied for multiplicity within a contrast.↳ Could also: The specific FDR procedure (e.g. Benjamini-Hochberg) could also be named explicitly, and family-wise methods such as Bonferroni or Holm could be offered for users seeking stricter control. — Naming the exact correction and its family clarifies what error rate is controlled and helps users match the method to their tolerance for false positives versus false negatives.
-
Several enrichment methods (GSEA, ssGSEA, GSVA, camera, fry, Fisher's exact, Spearman) are offered and combined.↳ Could also: A documented strategy for reconciling agreement/disagreement among enrichment methods (e.g. reporting concordance or a rank-based meta score) could also accompany the combined output. — An explicit concordance summary helps users interpret cases where competitive (camera, GSEA) and self-contained or overlap-based (fry, Fisher) tests answer subtly different questions.
-
As a tool description, the paper does not report sample sizes, dispersion measures (SD/SEM/CI), or exact p-values for any worked example.↳ Could also: A reproducible benchmark dataset with reported n, effect sizes and confidence intervals could also be included to illustrate the platform's output. — A worked example with full reporting (n, dispersion, exact p, effect sizes) would let readers see how results are summarized and assess the methods on known data.
-
Batch and unwanted-variation correction is offered via limma/ComBat and sva.↳ Could also: Including the batch/surrogate variables as covariates directly in the differential-expression model could also be used instead of, or alongside, removing them beforehand. — Modeling batch within the test (rather than correcting then testing) propagates the associated uncertainty into the inference, which some workflows prefer for accurate standard errors.
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
Drug CMap module identifies top 10 most similar and most opposite drug signatures by correlating contrast expression profiles against the L1000 database.other omics-playground 2019×1papers★ This paper is the founder (earliest)
-
Clustering heatmap top features are partitioned into five clusters and functionally annotated against more than 42 reference databases.other omics-playground 2019×1papers★ This paper is the founder (earliest)
-
Omics Playground provides the second largest number of gene set databases among compared bioinformatics platforms, exceeded only by BioJupies.other omics-playground 2019×1papers★ This paper is the founder (earliest)
-
Meta-analysis integrates multiple DEG and GSE methods; meta.q is the highest q-value across methods, with significance stars indicating count of methods reaching q<0.05.other omics-playground 2019×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-33575569 (Omics Playground, Akhmedov et al. 2019, NAR Genom Bioinform)
Paper type
Software / platform descriptor. Omics Playground is a self-service Shiny web platform that wraps standard omics methods. The paper demonstrates the platform on three public example datasets and reports no standalone numeric result (no DEG counts, p-values, gene names, or statistics in text/figure captions) for those datasets. Verified by reading PMC7671354 text + all figure captions.
- Real code: https://github.com/bigomics/omicsplayground (license: custom/NOASSERTION,
active, default branch
master). NOTE: the room's minedcode_url(pinin4fjords/shinyngs) is mis-mined — wrong tool; shinyngs is an unrelated RNA-seq Shiny package by J. Manning. Corrected here. - Compute backend (core functions): https://github.com/bigomics/playbase (+ bundled
resources in
playdata). - Example datasets (all public GEO):
- GSE72056 — melanoma scRNA-seq (Tirosh 2016) — used in Fig 2 (t-SNE cell types, CNV inference, enrichment, biomarkers).
- GSE114716 — Ipilimumab — Fig 2H (drug enrichment).
- GSE10846 — DLBCL microarray (Lenz 2008, Affy U133 Plus 2, 414 samples, ABC/GCB subtypes in metadata) — Fig 3: (A) volcano plots from eight DE methods, (B) cluster heatmap, (C) variable-importance, (D) survival tree.
In scope (pipeline-derived, attempted)
| result | pipeline | dataset | approach |
|---|---|---|---|
| Fig 3A: differential expression by multiple statistical methods | t-test (Welch) + limma (the microarray-appropriate subset of the platform's DE methods: t-test, limma, edgeR, DESeq2) + the platform's meta aggregation (meta.q, stars = #methods with q<0.05) |
GSE10846 | Download via GEOquery on «our HPC», define ABC vs GCB contrast from GEO metadata, run DE, report consensus DEG counts + top genes, check recovery of the canonical ABC/GCB subtype signature (external biological ground truth: MME/LMO2/SERPINA9/MYBL1/BCL6 = GCB; IRF4/PIM1/FOXP1/CCND2/NEK6 = ABC). |
| Software-content claim: ">50,000 gene sets across 30 collections"; DE & enrichment method lists | repo/playdata inspection | n/a | Count bundled gene-set collections; confirm method lists match the 8 volcano methods (Fig 3A) and 7 enrichment methods. |
Out of scope (not pipeline-derived / last-20% / not attempted)
- Full Shiny platform UI + Docker deployment — running the entire interactive app to regenerate every figure panel is the hard last 20%; not attempted (80/20).
- CNV inference (Fig 2D), survival tree (Fig 3D), drug-connectivity L1000 (Fig 2H) — these require the platform's bundled reference resources and many manual UI choices; high effort, no reported number to match → not attempted.
- scRNA-seq t-SNE cell typing (Fig 2A) — illustrative, no reported value; GSE72056 used only as a secondary dataset-shape sanity check if time permits.
Honesty note (auditability)
The paper provides no pinnable numeric claim for direct 1:1 matching
(no_expected_result in the screening taxonomy). Per BRIEF rule P16, we reproduce
the platform's described pipeline on the paper's own example data and grade
against the established external biology for that dataset. All grades are provisional
for human audit; we do NOT assert a paper number was reproduced where none exists.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a software/platform paper with no pinnable numeric results, so reproduction verified software-content claims against current source plus a pipeline rerun. The DE method set matches exactly and >50,000 gene sets holds (123,832); the only real deviation, 30 → 2,050 collections (C3), is explained version/definition drift on the authors-resource side, not a contradiction or fabrication. The central platform claim is functionally confirmed — the Fig 3A pipeline recovers the canonical DLBCL ABC/GCB signature 12/12 direction-correct (q≤4.4e-3). Overall yellow: a solid, fabrication-free reproduction limited by the absence of numeric endpoints and self-defined cohort/method choices.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.