huSA: a comprehensive database for multi-dimensional resolution of bulk, single cell and spatial transcription profiles in skin diseases.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🔴A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
huSA is a database/resource paper with NO deposited analysis code (the only code link, parallel-fastq-dump, is a generic SRA->FASTQ tool) and NO deposited processed atlas (data availability = website only). Its headline numbers (4,653,696 cells; 1,434/1,502/63 samples; 95,753-fibroblast integrated atlas; cNMF K=43) are cross-project construction aggregates over 38+ projects and are NOT independently verifiable from any obtainable artifact -- NOT ATTEMPTED (out of scope, the whole-paper 'last 20%'). We reproduced the one tractable, data-backed target: the paper's worked case study on GEO GSE173706 (psoriasis scRNA, Fig 6). Running the documented downstream pipeline (scanpy 1.10.4: QC nFeature>200/mito<10%/genes>=3cells -> Scrublet -> normalize/HVG/scale/PCA -> Harmony -> leiden -> marker annotation) on the 33 deposited per-cell count matrices: 96,088 cells loaded -> 93,219 after QC -> 26 global clusters -> ALL 10 reported major lineages recovered (qualitative match, 1:1 on architecture). Reported subtype COUNTS (6 KC/5 FB/5 EC) are NOT 1:1 reproducible: they are curated marker-merged annotations at an unspecified resolution, and unsupervised subclustering over-splits (KC 11-16, FB 10-16, EC 10-11) -- the subjective last 20%, not chased. Documented deviation: used GEO deposited matrices instead of re-running Cell Ranger v8 + GRCh38-2024-A from raw FASTQ on 33 libraries, so counts are not byte-identical to a huSA re-run. Verdict: PARTIAL -- the pipeline is described well enough to recover the case-study cell-type architecture on the paper's own data, but the paper ships no code and no per-dataset numeric claim, so exact 1:1 grading is impossible and the database-wide claims are unverifiable (flagged, not fabricated). No fabrication signal found for the GSE173706 case study.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-14 ⛓ 2c8716ae7eed
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusThe authors hypothesized that a comprehensive, integrative database unifying bulk, single-cell, and spatial transcriptomic data across multiple skin diseases with standardized analysis pipelines would enable robust, scalable exploration of skin disease biology that is not possible with fragmented, inconsistently analyzed individual datasets.
- ★ huSA is a publicly accessible database integrating data from 17 skin diseases and 63 independent datasets, including 1434 scRNA-seq, 63 spatial transcriptomics, and 1502 bulk RNA-seq samples resource
- ★ The database provides a standardized scRNA-seq pipeline including cell-type annotation, differential gene expression, cell-cell interaction, pathway/metabolic module enrichment, transcription factor regulatory inference, and differentiation state assessment method
- ★ Data from identical skin diseases were integrated to enhance biological signal detection, enabling both single-dataset and cross-dataset analysis method
- ★ huSA embeds 'cellxgene' and 'Cirrocumulus' platforms for interactive, customizable single-cell and spatial gene expression visualization resource
- ★ Demonstration analyses confirmed that results from single datasets or aggregated multi-dataset integrations exhibit high reliability and biological relevance finding
- ★ huSA v1.0 encompasses 4,653,696 high-quality single cells from 1434 scRNA-seq samples resource
- Existing general and disease-specific single-cell databases lack skin-specific focus, often miss cell-type annotations, and lack cross-tissue comparative analysis finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| scRNA-seq | human skin disease tissues (skin lesions, blood, kidney, synovium) across 17 skin diseases | disease vs healthy (none/other) | UMI count matrix, cell clustering/annotation, DEGs, differentiation potential | Cell Ranger v8.0.0 (GRCh38-2024-A reference); scanpy v1.10.1 |
| Bulk RNA-seq | human skin disease tissues | disease vs control (other) | DEGs, GO, KEGG, GSEA, GSVA | RNAseqStat2 package |
| Spatial transcriptomics | human skin disease tissues | none/other | spatial domain recognition and clustering | OmicVerse package; GraphST; mclust/leiden/louvain |
| Transcription factor regulon analysis | scRNA-seq samples | none | TF co-expression modules, regulon activity (AUCell), Regulon Specificity Score | pySCENIC v0.12.1; SCENIC R package v1.3.1 |
| Cell-cell interaction analysis | scRNA-seq samples, distinct cell types | none | predicted intercellular communication (ligand-receptor) | CellPhoneDB v5.0.1; OmicVerse v1.6.3 |
| Cell developmental trajectory / differentiation inference | scRNA-seq cell types under disease and healthy states | disease vs healthy | CytoTRACE2 differentiation/stemness scores (0-1) | CytoTRACE2 (Omicverse) |
| Cell-type specific metabolic flux analysis | single cells from scRNA-seq | none | metabolic flux across 168 modules and 70 pathways | scFEA v1.1.2 |
| cNMF gene program analysis / fibroblast abundance testing | prurigo nodularis (PN) vs healthy control (HC) fibroblasts | disease vs healthy | gene programs (cNMF), differential neighbourhood abundance, Ro/e | cNMF (K=43); MiloR via pertpy v0.10.0 |
- – huSA integrates 1434 scRNA-seq, 63 spatial transcriptomics, and 1502 bulk RNA-seq samples across 17 skin diseases and 63 datasets
- – Database encompasses 4,653,696 high-quality single cells 4,653,696 cells
- – In single-cell sequencing, raw sequencing datasets and expression matrices accounted for 62.69% and 37.31% respectively 62.69% / 37.31%
- – Single-dataset and integrative multi-dataset analyses both yielded reliable and biologically relevant results
- – cNMF identified 43 gene programs (K=43) in PN samples as optimal compromise between factor stability and model error K=43
- – scFEA enables flux estimation across 168 metabolic modules and 70 pathways 168 modules / 70 pathways
- count 4 653 696 high-quality single cells (single cells in huSA v1.0)
- count 17 skin diseases, 63 independent datasets (datasets/diseases covered)
- count 1434 scRNA-seq, 63 spatial, 1502 bulk RNA-seq samples (sample counts by modality)
- other 62.69% raw sequencing vs 37.31% expression matrices (proportion of scRNA-seq data formats)
- other K = 43 gene programs (cNMF hyperparameter selected for PN)
- count 168 metabolic modules and 70 pathways (scFEA metabolic architecture)
- other about 16% of body mass (skin as largest organ (background))
- other 58% of psoriasis patients relapse annually; AD 7-year relapse rate 75.9% (disease burden background statistics)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This paper describes the construction of huSA, a computational database integrating bulk RNA-seq, scRNA-seq, and spatial transcriptomics from 17 skin diseases across 63 independent datasets. Statistical analyses consist of standardized bioinformatic pipelines applied uniformly to all ingested data, including cell clustering, differential gene expression, transcription factor regulon scoring, permutation-based cell-cell interaction testing, compositional abundance testing, and metabolic flux estimation. Results are communicated primarily as proportions, enrichment scores, Ro/e ratios, and permutation-derived p-value thresholds rather than classical inferential summaries with effect sizes or confidence intervals. Demonstration analyses were used to confirm consistency between single-dataset outputs and multi-dataset integrations.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Chi-square test (to derive expected cell numbers for Ro/e ratio calculation) | Distribution of fibroblast clusters across disease phenotypes | — | not stated |
| Permutation test (CellPhoneDB, 1000 iterations, p < 0.05 threshold) | Cell-cell interaction significance across all scRNA-seq samples | — | not stated |
| Differential neighbourhood abundance testing (MiloR via pertpy v0.10.0, kNN graph-based) | Fibroblast subpopulation abundance differences between prurigo nodularis and healthy controls | — | not stated |
| Pearson correlation | Gene expression similarity within a cell type across integrated disease datasets | — | not stated |
| Differential gene expression analysis via RNAseqStat2 (specific underlying test not named) | Bulk RNA-seq DEG identification across disease datasets | — | not stated |
| AUCell enrichment scoring and Regulon Specificity Score (RSS; pySCENIC/SCENIC) | Transcription factor regulon activity per cell type in all scRNA-seq datasets | — | na |
-
Cross-dataset batch correction was performed using Harmony operating on PCA embeddings↳ Could also: scVI (variational autoencoder on raw counts) or Scanorama could also be applied for multi-dataset integration — scVI models the count-generating process directly and can jointly capture biological and technical variation; this may be preferred when datasets derive from heterogeneous protocols, as it preserves count-level uncertainty rather than correcting only the low-dimensional embedding
-
Fibroblast cluster enrichment across diseases was quantified using Ro/e ratios with chi-square-derived expected values↳ Could also: Compositional methods such as scCODA or Dirichlet-multinomial regression could also model cell-type proportions — These approaches formally account for the compositional and correlated nature of cell-type proportions and provide uncertainty estimates and posterior probabilities for condition-associated changes, complementing the descriptive Ro/e ratio
-
Cell-cell interactions were inferred with CellPhoneDB using a single ligand-receptor database and permutation scoring↳ Could also: LIANA (aggregating multiple ligand-receptor resources and scoring methods) or NicheNet could also be applied — LIANA produces consensus scores across methods and databases, increasing robustness to database choice; NicheNet additionally links predicted sender-receiver interactions to downstream target gene expression, adding mechanistic specificity to prioritization
-
The number of cNMF gene programs (K) was selected as K=43 by visually identifying the best compromise between stability and model error across a tested range of 20–50↳ Could also: Quantitative selection criteria such as the cophenetic correlation coefficient across bootstrap replicates, or cross-validation-based reconstruction error, could also be used to choose K — Objective, reproducible metrics for rank selection reduce analyst subjectivity and allow readers to independently evaluate whether the chosen factorization rank is well-supported by the data
-
Differential gene expression for scRNA-seq clusters was performed through Scanpy (the specific underlying statistical test is not named in the text)↳ Could also: Pseudobulk approaches (e.g., DESeq2 or edgeR applied to per-donor aggregated counts) could also be used for cluster-level differential expression — Pseudobulk methods aggregate counts per biological replicate before testing, which better accounts for within-donor correlation and has been shown in benchmarks to control type-I error more accurately than cell-level tests when multiple donors contribute to each condition
-
Spatial domain recognition was performed using GraphST followed by mclust, leiden, or louvain clustering↳ Could also: BayesSpace or BANKSY could also be used for spatially-aware domain segmentation — BayesSpace and BANKSY explicitly incorporate spatial neighborhood structure into their statistical models, potentially yielding sharper domain boundaries and providing uncertainty quantification for domain assignments, which may improve interpretability in heterogeneous tissue sections
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-41719583 (huSA: human skin atlas database)
What kind of paper this is
huSA is a database / resource paper (journal Database (Oxford)). Its primary "result" is an interactive web atlas (https://humanskinatlas.com), built by ingesting and uniformly re-processing public transcriptomic datasets of skin disease. It is not a single-experiment analysis paper.
Reported headline tallies (construction counts):
- 4,653,696 high-quality single cells; 1,434 scRNA-seq samples from 38 projects, 17 diseases
- 1,502 bulk RNA-seq samples; 63 spatial transcriptomics samples from 5 projects
- 11 skin major cell categories / 53 subtypes; 8 blood categories / 25 subtypes
- Integrated fibroblast atlas: 95,753 fibroblasts (composition %); 10 FB subsets; cNMF K=43 in PN
Code & data availability (as published)
- Code availability: none deposited. The only code link mined from the paper is
github.com/rvalieris/parallel-fastq-dump— a generic third-party SRA→FASTQ converter, NOT the analysis pipeline. The actual multi-tool pipeline (Cell Ranger v8.0.0 + GRCh38-2024-A, scanpy 1.10.1, Scrublet, Harmony, pySCENIC 0.12.1, CellPhoneDB 5.0.1, scFEA 1.1.2, CytoTRACE2, GraphST/Omicverse, RNAseqStat2) is described only in prose — no scripts, no parameter files, no notebooks. - Data availability: website only ("huSA 1.0 is available online"). No deposited processed atlas / super-series. Source datasets are the original public accessions.
In scope (pipeline-derived, attempted)
The brief gives one concrete accession — GEO GSE173706 — which is also the paper's worked case-study / pipeline-validation dataset (paper Fig. 6B–6E; "Single Cell and Spatial Sequencing ... in Psoriasis", Gudjonsson/Modlin/Pellegrini). It is 33 scRNA-seq samples (8 normal skin, 11 peri-lesional/PN, 14 psoriatic/PP), each shipping a raw gene×cell count CSV in GEO supplementary (GSE173706_RAW.tar, 264 MB).
Reproduction target (case study, Fig. 6): apply the documented huSA scRNA-seq downstream pipeline (QC → doublet removal → normalize/HVG/scale/PCA → Harmony → leiden clustering → marker-based lineage annotation) to GSE173706 and check recovery of the reported cell-type architecture:
- reported: 6 keratinocyte subtypes, 5 fibroblast subtypes, 5 endothelial subtypes, plus T-cell subtypes (CD4 naive, CD8 Tem/Trm/Tex, Treg, cycling) and myeloid (Langerhans, macrophage, mast, NK, cDC1, cDC2).
- reproduced (clear data points): total cells loaded, cells passing the documented QC, n leiden clusters, and the major lineages + KC/FB/EC subtype counts recovered.
Documented deviation (honest 80/20)
huSA re-aligned raw FASTQ with parallel-fastq-dump → Cell Ranger v8.0.0 + GRCh38-2024-A. We instead use the GEO-deposited per-cell raw count matrices (original authors' alignment) and run the documented downstream steps. Re-running Cell Ranger v8 on 33 10x libraries is the heavy "last 20%" the brief says to skip; the deposited matrices are a faithful substitute for the downstream analysis being reproduced. This means cell counts are NOT expected to be byte-identical to a huSA Cell-Ranger-v8 re-run, and is flagged in AUDIT.md.
Out of scope (not attempted, with reason)
- The 4.65M-cell / 1,434+1,502+63-sample headline tallies and the 95,753-fibroblast integrated atlas / cNMF K=43 — these are cross-project aggregates over 38+ projects, not reproducible from any single obtainable accession without redoing the entire multi-dataset construction (the whole paper). No deposited code/values to verify.
- pySCENIC / CellPhoneDB / scFEA / CytoTRACE2 / spatial (GraphST) / bulk (RNAseqStat2) sub-analyses — described in prose only, no per-result parameters or expected values.
- Exact subtype counts (6/5/5) are clustering-resolution + manual-annotation
dependent — the subjective "last 20%"; we report recovery qualitatively, graded
partial, and do not tune resolution to force a match.
Honest reprodu
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a resource/database paper with no deposited analysis code and no deposited processed atlas, so its headline construction tallies are unverifiable (flagged, not fabricated). The reproduction sensibly targeted the one tractable, data-backed item — the GSE173706 psoriasis case study (Fig 6) — and running the documented scanpy pipeline on the paper's own deposited matrices fully recovers the qualitative cell-type architecture (all 10 major lineages, with KC/FB/EC heterogeneity). The deviations are confined to the curated subtype counts (6/5/5), which are marker-merged annotations at an unspecified resolution; unsupervised subclustering over-splits (KC 11–16, FB 10–16, EC 10–11), a methodology/underspecification gap rather than an authors' fabrication. Overall solid with explainable deviations — yellow, no fabrication signal for the case study.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.