Preparation of mouse pancreatic tumor for single-cell RNA sequencing and analysis of the data.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
PARTIAL PROGRESS, not partial agreement: substantial scoping + setup done, but NO reproduced numeric value was produced before the finalize request, so nothing is graded against the paper yet. What is established: (1) This is a STAR Protocols methods paper demonstrating the Barski-lab/datirium CWL scRNA-seq pipeline on its companion research paper's data (Gabitova-Cornell, Cancer Cell 2021, PMID 32976774); its Fig 12-16 are illustrative pipeline outputs, with NO stand-alone numeric claim in text. (2) The brief's code URL github.com/datirium/workflows is 404/gone, but the cited code is fully recovered and pinned: Zenodo 10.5281/zenodo.5339177 = Barski-lab/scRNA-Seq-Analysis v1.0.1 (commit b95b2ba), Seurat tool Docker biowardrobe2/seurat:v0.0.8 (Seurat 4.0.1); pipeline + all adjusted parameters captured (tutorial Table 1: min genes 300, max 6200, min UMI 500, novelty>=0.8, mito<=5% pattern '^mt-', 3000 HVG, 20 PCs, resolution 0.5, FindAllMarkers logfc>=0.25/minpct 0.1/wilcox/onlypos). So this is NOT a repo_gone drop. (3) Data PRJNA657051 = GEO GSE156210 = SRP277429 = the protocol's 5 samples (3 KPPC + 2 KPPCN). GEO supplementary ships only NORMALIZED (log) already-cell-filtered matrices (KPC 12579x9574, KPCN 12579x8197; per-sample 3777/3282/2515 + 6111/2086 = 17771 cells) which are the companion paper's MANUAL-analysis end product, NOT raw counts and NOT the protocol CWL output -> usable only as a comparison target. (4) Reproduction plan (set up, not finished): re-align the 5 raw SRA runs with STARsolo 2.7.10a (CellRanger4 emulation; Cell Ranger 4.0.0 is proprietary; P16 third-party-tool reproduction) on refdata-gex-mm10-2020-A, then compare per-sample 'Estimated Number of Cells' to the deposited counts; clustering at res 0.5 as a qualitative secondary. UNRESOLVED LINCHPIN: the 10x barcode (CB+UMI) read recoverability from SRA. ENA serves only 2x150bp biological reads (no 28bp barcode read); SRA .sra technical reads must be checked. The first probe used an invalid sra-tools flag (--skip-technical=no, rejected by fastq-dump 3.0.10) and parsed help text, so its 'OK' marker is a false positive -> read structure is still UNKNOWN. If only 2x150 cDNA is recoverable, the alignment cannot be demultiplexed and this would become drop_reason data_unavailable (read structure not preserved). NOT ATTEMPTED / hard 20%: bit-exact Cell Ranger 4.0.0 run (~400GB very deep data, proprietary); exact Fig 16 marker-table values (screenshot only). Infrastructure left on «infra» for continuation: conda env aln (STAR+sra-tools), mm10 STAR index (building under «job»), 10x whitelists, recovered code at repo/v101/. To continue: fix the SRA probe (use 'fasterq-dump --include-technical' or 'fastq-dump --split-files' without the bad flag) on SRR12450157, confirm a ~28bp barcode read, then run STARsolo per sample.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-15 ⛓ a84c4d285c43
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opus- ★ A protocol combining mechanical and enzymatic dissociation of mouse pancreatic tumor yields a single-cell suspension with preserved cell characteristics suitable for single-cell RNA sequencing. method
- ★ Sequential dead cell removal, optional red blood cell lysis, and CD45+ leukocyte depletion enrich for viable, non-immune tumor cells prior to library generation. method
- ★ Converting the scRNA-Seq analysis into Common Workflow Language (CWL) pipelines executed on the SciDAP platform guarantees reproducibility and portability of the bioinformatics analysis. method
- ★ Open-source CWL pipelines run inside Docker containers provide a graphical, code-free, reproducible downstream analysis workflow for biologists. resource
- Achieving well-separated single cells with viability ≥90% at 700–1200 cells/μL is critical for high-quality 10X Genomics library generation. method
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Tumor tissue dissociation (mechanical + enzymatic) | KPC mouse (LSL-KrasG12D;Trp53f/f;Pdx1-Cre) pancreatic tumor | none | single-cell suspension cell number and viability | gentleMACS Dissociator; Tumor Dissociation Kit, mouse (Miltenyi 130-096-730) |
| Magnetic dead cell removal | Mouse pancreatic tumor single-cell suspension | none | live (unlabeled) cell fraction | Dead Cell Removal Kit (Miltenyi 130-090-101); MACS LS columns |
| CD45+ leukocyte depletion (magnetic separation) | Mouse pancreatic tumor single-cell suspension | CD45+ cell depletion | CD45-negative cell fraction | CD45 MicroBeads, mouse (Miltenyi 130-052-301); MACS LS columns |
| Cell counting / viability | Mouse pancreatic tumor single-cell suspension | none | total cell number and viability (% live) | Hemocytometer with Trypan Blue exclusion |
| Single-cell 3' RNA-seq library preparation | Mouse pancreatic tumor single cells | none | barcoded scRNA-seq libraries | Chromium Single Cell 3' Library, Gel Bead & Multiplex Kit and Chip Kit V3 (10X Genomics PN-1000092) |
| cDNA library QC (electrophoresis + fluorometric quantification) | scRNA-seq cDNA library | none | cDNA size profile and concentration | Agilent Bioanalyzer High Sensitivity DNA Kit; Qubit 2.0 dsDNA HS Assay |
| scRNA-seq data processing and clustering (bioinformatics) | Mouse pancreatic tumor scRNA-seq data | none | cell clusters / cellular populations | CellRanger 4.0.0; Seurat 4.0.1; CWL pipelines on SciDAP |
- – Final single-cell suspension target concentration for loading 700–1200 cells/μL
- – Required cell viability for downstream library preparation ≥90% (minimum >70%)
- – Estimated cells loaded per library/well 7,000–12,000 cells
- – Permissible total cDNA amount range for a high-quality library 1–1,900 ng
- other 700–1,200 cells/μL (final cell concentration for loading)
- other ≥90% viability (>70% minimum) (required cell viability)
- count 7,000–12,000 cells (cells loaded per library)
- other 20,000 read pairs/cell (50,000 for 3' v2) (10X recommended sequencing depth)
- count 1–1,900 ng (permissible total cDNA amount)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a methods protocol paper describing wet-lab procedures for mouse pancreatic tumor dissociation and single-cell RNA sequencing library preparation, followed by a bioinformatics pipeline using CellRanger and Seurat executed via CWL on the SciDAP platform. No primary inferential statistical analyses are presented within this protocol paper itself; statistical analyses of data generated by this protocol were reported in the companion paper (Gabitova-Cornell et al., 2020). The paper's quantitative guidance is limited to cell viability thresholds (>70–90%) and target cell concentration ranges (700–1,200 cells/μL) for quality control.
-
CellRanger was used for read alignment and UMI/cell barcode counting from 10x Genomics data↳ Could also: STARsolo (part of STAR 2.7+) or Salmon/Alevin could also be used for alignment and quantification of 10x Chromium scRNA-seq data — STARsolo and Alevin offer open-source, platform-independent alternatives that produce compatible count matrices; Alevin additionally provides uncertainty quantification on transcript-level counts, which may be informative for downstream differential expression
-
Seurat was chosen as the scRNA-seq clustering and dimensionality reduction framework↳ Could also: Scanpy (Python/AnnData ecosystem) or Bioconductor's SingleCellExperiment/scran/scater stack could also be used for the same clustering and visualization steps — Scanpy offers a Python-native workflow with comparable graph-based clustering and UMAP support, which may integrate more readily with Python-centric pipelines; scran implements pooling-based normalization that some benchmarks suggest performs well with sparse count data
-
Physical dead cell removal was performed using MACS Dead Cell Removal MicroBeads prior to library preparation↳ Could also: Computational filtering within Seurat or equivalent (thresholding on mitochondrial gene fraction, log-library size, and gene detection count) could also be applied as an alternative or complement — Computational filtering is more widely documented in published scRNA-seq workflows and allows the analyst to tune thresholds post-hoc; physical removal reduces doublet and ambient RNA artifacts at the source but may introduce cell-type-specific bias if certain live populations bind non-specifically
-
CD45+ leukocyte depletion was performed magnetically to enrich for non-immune tumor cells↳ Could also: Retaining all cells and performing computational cell-type deconvolution (e.g., SingleR, CellTypist, or marker-gene-based annotation in Seurat) could also distinguish immune from non-immune populations — Computational depletion preserves immune cell data for tumor microenvironment analysis, which is often scientifically valuable in pancreatic cancer studies; physical depletion reduces sequencing cost when immune populations are not of interest but forecloses immune-compartment analyses
-
CWL pipelines were used to package and execute the bioinformatics workflow for reproducibility↳ Could also: Snakemake or Nextflow (nf-core/scrnaseq) pipelines could also containerize and execute equivalent scRNA-seq workflows with comparable reproducibility guarantees — Nextflow's nf-core/scrnaseq is a widely adopted, community-maintained pipeline with built-in MultiQC reporting and direct support for 10x Chromium data; Snakemake integrates tightly with conda environments and is common in academic bioinformatics settings
-
Cell viability and concentration were assessed manually using a hemocytometer with Trypan Blue exclusion, counted twice with two aliquots↳ Could also: Automated cell counters (e.g., Countess, NucleoCounter, or flow cytometry with viability dyes such as 7-AAD or DAPI) could also be used for cell enumeration and viability assessment — Automated counters reduce inter-operator variability and provide objective, logged records of cell counts; some instruments also estimate cell diameter distributions that can flag aggregates, which is relevant to the protocol's emphasis on single-cell quality
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-34927097
Paper: Surumbayeva et al., Preparation of mouse pancreatic tumor for single-cell RNA sequencing and analysis of the data. STAR Protocols 2021. PMID 34927097 / PMC8649395 / DOI 10.1016/j.xpro.2021.100989.
Nature of paper: This is a STAR Protocols methods paper, not a primary research article. It demonstrates a reusable scRNA-seq analysis pipeline on the data of its companion research paper (Gabitova-Cornell et al., Cancer Cell 2021, PMID 32976774, "Cholesterol pathway inhibition induces TGFβ signaling…"). The protocol's figures (Fig 12–16) are illustrative example outputs of the pipeline; the protocol prints no stand-alone quantitative claim (no "N cells / K clusters" in text).
Code (recoverable, but brief's URL is dead)
- Brief code URL
github.com/datirium/workflows→ 404 / repository gone. - The cited, pinned code survives on Zenodo: DOI 10.5281/zenodo.5339177 =
Barski-lab/scRNA-Seq-Analysisv1.0.1 (commitb95b2ba, title "Star Protocols"). Mirror still live: github.com/Barski-lab/scRNA-Seq-Analysis @ v1.0.1. → NOT arepo_gonedrop: the exact archive is recoverable. - Pipeline = 4 CWL workflows: Cell Ranger mkref → Cell Ranger count → Cell Ranger
aggr → Seurat Cluster (
tools/seurat-cluster.cwl, Dockerbiowardrobe2/seurat:v0.0.8, Seurat 4.0.1).
Data
- SRA PRJNA657051 (= GEO GSE156210, = SRP277429). 5 runs, mouse, 10x 3' v3:
- SRR12450154 KPPC 1, SRR12450155 KPPC 2, SRR12450156 KPPC 3, SRR12450157 KPPCN 1, SRR12450158 KPPCN 2.
- Raw FASTQ is huge: ~480M read pairs/sample (≈40+40 GB each), ~400 GB total.
- Read layout caveat: ENA serves
_1/_2both 150 bp, ~equal size — NOT the standard 10x layout (R1=28 bp CB+UMI, R2=cDNA). The 28 bp barcode read is likely only recoverable from the.sravia SRA Toolkit--include-technical. This is a feasibility linchpin verified at run time.
- GEO supplementary ships normalized (log) count matrices, NOT raw counts:
GSE156210_KPC_Advanced_Count_Matrix.txt.gz(12579 genes × 9574 cells) and…_KPCN_Advanced…(12579 × 8197). These are the research paper's manual-analysis end product (already cell-filtered + normalized), per-sample prefixes: KPC = KPCAdvC3 (3777) + KPCAdvD3 (3282) + KPCAdvH2 (2515); KPCN = KPCNAdv1 (6111) + KPCNAdv2 (2086). Total 17,771 cells across the 5 samples. → They are NOT valid input to the protocol's Seurat tool (which needs raw-count MEX from Cell Ranger), and represent a different (manual) analysis. Useful only as the comparison target for cell counts.
IN SCOPE (pipeline-derived, attempt — 80/20)
- Per-sample post-filter cell counts (Fig 12 / tutorial Fig 7A). Re-align each
SRA run with a Cell Ranger-4-faithful caller and apply the protocol's QC
(Table 1: min genes 300, max 6200, min UMI 500, novelty ≥0.8, mito ≤5%
^mt-), compare cell counts to the deposited per-sample counts above. Primary gradeable claim.- Aligner: STARsolo 2.7.x with
--clipAdapterType CellRanger4 --soloCellFilter EmptyDrops_CRas the open, faithful proxy for Cell Ranger 4.0.0 (proprietary). P16: applying an established third-party tool to the paper's data is a valid repro. mm10 reference = 10xrefdata-gex-mm10-2020-A; v3 whitelist 3M-february-2018.
- Aligner: STARsolo 2.7.x with
- Clustering at resolution 0.5 (Fig 13–14): number of clusters and the cell-type marker structure for the 14 genes of interest (Clec3b, Il6, Lgals7, Pdgfra, Vim, Tgfb1, Ptprc, Epcam, Cldn4, Krt7, Sox9, Cdh1, Upk3b, Mki67). Qualitative — protocol prints no exact cluster count; secondary.
OUT OF SCOPE / hard 20% (skip, with reason)
- Bit-exact Cell Ranger 4.0.0 run (proprietary binary; ~400 GB very deep data; multiple >12 h SLURM walls). STARsolo proxy substitutes the cell-calling step.
- Exact reproduction of the marker-gene table values (Fig 16) — stochastic + screenshot only, no machine-readable repor
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Incomplete reproduction of a methods paper, no authors' defect. The reproduction recovered the (404'd) code from Zenodo, pinned all Seurat QC parameters, and set up the environment/mm10 index, but produced no reproduced numeric value before finalize, so none of C1-C8 are graded. The comparison target itself is weak: GEO ships only the companion paper's manual normalized, pre-filtered matrices (per-sample 3777/3282/2515/6111/2086, total 17771), not the protocol's own CWL output, and the central blocker is data structure — whether the 28bp 10x barcode read survives in SRA is still unknown after an inconclusive probe. All deviations sit on the input/method/availability side (our incomplete pipeline + restricted data form), not on the authors; no fabrication is indicated, hence yellow across the board rather than any red.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.