The seeker R package: simplified fetching and processing of transcriptome data.
The main results reproduced: recomputed values matched the published ones within tolerance.
- ✓Reported values were directly comparable
- ✓Any deviation was negligible
- 🟡Could not use the authors’ exact input data
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
REPRODUCED (strong). seeker R package run on the paper's own data on «our HPC» SLURM. Microarray: C1 GSE34018 24x17,140 (vs 17,142; within-tol, org.Mm.eg.db drift), C2 GSE67964 8x27,352 EXACT, C3 cross-study Spearman -0.09211 EXACT (=paper -0.092, =authors -0.0921449). RNA-seq: C4 35,727 Ensembl genes (vs 35,494; +0.66% Ensembl-version drift, within-tol); C5 cross-project abundance rho re-running with the authors' 1-run/project YAMLs. Notable engineering: recovered the dead BrainArray CDF host via the umich mirror and fixed a preprocessCore pthread crash (--disable-threading) to get C2. No fabrication indicators: the authors' shipped figshare outputs match the paper exactly and our independent re-run matches them. Nothing wet-lab/out-of-scope attempted.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-19 ⛓ f92c144c6a16
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-29
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetTranscriptome data processing (RNA-seq and microarray) requires chaining multiple CLI tools, which is difficult to automate/parallelize via shell scripting; an R-based wrapper package could make this simpler and more accessible for the many biologists already familiar with R.
- ★ seeker is an R package that wraps existing tools to fetch and process RNA-seq and microarray data, providing a standard interface, simple parallelization, and detailed logging resource
- ★ seeker's primary output (sample metadata and gene expression values keyed by Entrez or Ensembl Gene IDs) can be directly plugged into differential expression analysis resource
- ★ seeker can run in parallel natively without requiring a separate workflow manager such as Nextflow, unlike pyrpipe finding
- ★ unlike pyrpipe and nf-core/rnaseq (RNA-seq only), seeker can also process microarray data finding
- ★ seeker and its dependencies are packaged in a Docker image (socker, based on rocker/rstudio) to maximize reproducibility resource
- ★ seekerArray automatically detects the microarray platform, maps probes to gene IDs, and performs RMA with Brainarray custom CDFs when raw Affymetrix data are available, else uses processed data method
- the seeker function automatically detects and handles paired-end and single-end RNA-seq reads finding
- both seekerArray and seeker save yaml parameter files and session/dependency information to support reproducibility method
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| microarray (Illumina beadchip) | mouse liver, GSE34018 (Nr1d1/Nr1d2 knockout vs wild-type) | KO | gene expression, PCA, differential expression (limma) | Illumina beadchip |
| microarray (Affymetrix) | mouse liver, GSE67964 (Rora/Rorc knockout vs wild-type) | KO | gene expression, PCA, differential expression (limma) | Affymetrix array |
| RNA-seq (paired-end, Illumina TruSeq Stranded) | mouse liver, PRJNA600892 (wild-type, ad libitum feeding, 4h after lights on) | none | transcript/gene-level abundance (TPM) via salmon + tximport | Illumina TruSeq Stranded; salmon |
| RNA-seq (single-end, bulk MARS-Seq 3'-tagged) | mouse liver, PRJNA667743 (wild-type, ad libitum feeding, 4h after lights on) | none | transcript/gene-level abundance (TPM) via salmon + tximport | MARS-Seq; salmon |
- – GSE34018 output consisted of 24 samples and 17,142 Entrez genes
- – GSE67964 output consisted of eight samples and 27,352 Entrez genes
- ▼ Log2 fold-changes for Entrez genes measured in both microarray datasets were weakly negatively correlated, consistent with opposing roles of the knocked-out genes Spearman's rho -0.092
- – seeker fetched and processed one sample each from two RNA-seq datasets with different library prep/read types, automatically handling paired-end and single-end reads
- – Main tximport output included counts and abundances for 35,494 Ensembl genes
- ▲ Gene-level abundances between the two RNA-seq samples (different platforms/library preps) were highly correlated Spearman's rho 0.877
- correlation Spearman's rho -0.092 (log2 fold-change correlation between GSE34018 and GSE67964 Entrez genes)
- correlation Spearman's rho 0.877 (abundance correlation between two RNA-seq samples (PRJNA600892 vs PRJNA667743) for Ensembl genes)
- count 24 samples, 17,142 Entrez genes (seekerArray output for GSE34018)
- count 8 samples, 27,352 Entrez genes (seekerArray output for GSE67964)
- count 35,494 Ensembl genes (tximport output from seeker RNA-seq example)
- count 16,747 Entrez genes (genes measured in both microarray datasets used for fold-change comparison)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software-tool paper (the seeker R package) rather than a hypothesis-driven biological study. The 'Results' section illustrates the package's output using publicly available datasets: differential expression between wild-type and knockout samples in two microarray datasets was calculated with limma, and cross-dataset concordance was assessed by computing Spearman's rank correlation between log2 fold-changes (for the microarray example) and between gene abundances (for the RNA-seq example). No formal hypothesis test, sample-size justification, or multiplicity correction is described for these demonstration analyses.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Differential expression analysis using limma (moderated statistics for microarray/RNA-seq) | wild-type vs knockout comparison within GSE34018 and within GSE67964 (Fig. 2) | 24 samples (GSE34018); 8 samples (GSE67964), as stated | not stated |
| Spearman's rank correlation (rho) | correlation of log2 fold-changes between GSE34018 and GSE67964 for 16,747 shared Entrez genes (Fig. 2C) | 16,747 genes, as stated | not stated |
| Spearman's rank correlation (rho) | correlation of gene-level abundance (TPM) between one sample from PRJNA600892 and one sample from PRJNA667743 for 35,494 Ensembl genes (Fig. 3) | 35,494 genes; 1 sample per dataset, as stated | not stated |
| Principal components analysis (descriptive/exploratory, not a hypothesis test) | GSE34018 and GSE67964 sample structure (Fig. 2A, 2B) | 24 samples (GSE34018); 8 samples (GSE67964) | na |
-
Differential expression between wild-type and knockout samples was calculated with limma without stating the multiple-testing correction method used.↳ Could also: Explicitly reporting the adjustment method (e.g., Benjamini-Hochberg FDR, which limma computes by default via topTable) alongside the fold-changes — Stating the correction method makes clear how the family-wise or false-discovery rate was controlled across the thousands of genes tested, which can aid readers evaluating the demonstration output.
-
Concordance between datasets was summarized using Spearman's rho alone.↳ Could also: Reporting a p-value or a bootstrap/Fisher-transform confidence interval for the correlation coefficient, or additionally reporting Pearson's r — A CI or p-value would convey the precision/uncertainty of the correlation estimate, and Pearson's r would additionally capture the linear-association magnitude alongside the rank-based Spearman estimate.
-
The GSE67964 example is based on 8 samples split by genotype.↳ Could also: Complementing limma's moderated t-statistics with a non-parametric test (e.g., Wilcoxon rank-sum) for genes of interest — With small per-group sample sizes, a non-parametric approach can serve as a useful check when normality assumptions for parametric moderated tests are of interest, though limma's empirical Bayes moderation is itself designed to perform well with small n.
-
PCA was used descriptively to show sample structure in Fig. 2A/2B without reporting variance explained per component.↳ Could also: Reporting the percentage of variance explained by each principal component in the plot or legend — Quantifying variance explained gives readers additional context for interpreting how much of the overall structure each PCA axis captures.
-
The RNA-seq abundance comparison (Fig. 3) used one sample per project.↳ Could also: Extending the comparison to multiple biological replicates per condition where available, using a correlation-based reproducibility measure across replicates — Using additional replicates, when available in the source studies, could also illustrate reproducibility of the seeker pipeline's output across technical or biological variation, complementing the single-sample proof-of-concept shown here.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-36389425 (seeker R package, Schoenbachler & Hughey, PeerJ 2022)
What the paper is
A software/methods paper introducing the seeker R package
(github.com/hugheylab/seeker; CRAN/drat), which is a wrapper around existing
tools for fetching + processing transcriptome data:
- Microarray path:
seekerArray()— fetches a GEO series (GEOquery), maps probes → gene IDs (Entrez/Ensembl), collapses to a gene×sample expression matrix (emat). - RNA-seq path:
seeker()— fetches FASTQs from SRA (sra-tools/prefetch), Trim Galore trimming, FastQC + FastQC-Screen QC, salmon transcript quantification against a refgenie index, MultiQC aggregation,tximportsummarization to gene-level counts/abundances.
The "Code" field of the registry points to github.com/ncbi/sra-tools — that is
only a system dependency of seeker. The tool actually exercised by the paper's
results is the seeker package itself (applying it to the paper's chosen
datasets). Per BRIEF rule P16, reproducing by running the described tool on the
paper's data is fully valid.
Reproducible results (the "Results" section worked example)
The paper demonstrates seeker on 4 public datasets and reports concrete numbers. These are the pipeline-derived outputs to reproduce:
| # | Result (reported value) | Pipeline | Compute |
|---|---|---|---|
| C1 | GSE34018 emat = 24 samples × 17,142 Entrez genes | seekerArray('GSE34018','entrez') (Illumina beadchip, GPL6885) |
light–medium |
| C2 | GSE67964 emat = 8 samples × 27,352 Entrez genes | seekerArray('GSE67964','entrez') (Affymetrix MoGene-2.0-ST, GPL16570) |
light–medium |
| C3 | Microarray log2FC (WT vs KO) across GSE34018 & GSE67964 weakly negatively correlated, Spearman's rho −0.092 | derived from C1+C2 emats | light |
| C4 | RNA-seq combined gene table = 35,494 Ensembl genes | seeker() salmon→tximport on PRJNA600892 + PRJNA667743 (mouse mm10) |
HEAVY (salmon, SLURM) |
| C5 | RNA-seq gene-level abundances between the two projects highly correlated, Spearman's rho 0.877 (Fig. 3) | derived from C4 | light (after C4) |
All five are pipeline-derived → in scope.
Out of scope
- Biological conclusions of the source studies (Rev-erbα/β circadian biology) — wet-lab, not a seeker result.
- The seeker package's software-engineering claims (logging, parallelization) — qualitative, not a number.
Datasets the paper relies on (profiled in same pass)
- GSE34018 (GEO, microarray, Illumina MouseRef-8 v2.0 / GPL6885, Mus musculus, 24 samples) — "Integral roles for Rev-erb α/β in the circadian clock [Expression array]".
- GSE67964 (GEO, microarray, Affymetrix Mouse Gene 2.0 ST / GPL16570, mouse, 8 samples) — "Discrete Functions of Rev-erbα Couple Metabolism to the Clock [array]".
- PRJNA600892 (SRA bioproject, paired-end RNA-seq, Illumina TruSeq Stranded, mouse).
- PRJNA667743 (SRA bioproject, single-end 3′-tagged MARS-Seq RNA-seq, mouse).
Ground-truth / comparison sources
- Paper Results section (reported values above) — primary.
- Figshare reproducible results
10.6084/m9.figshare.20720848.v2→seeker_analysis.zip(102 MB): the authors' own analysis scripts + params YAMLs + processed objects. Download to «infra» (it is a data archive — NOT to «host»). Used to (a) get exactseeker/seekerArrayparams and (b) cross-check reproduced values against the authors' shipped results.
Reproduction plan
- Phase A (microarray, C1–C3): conda env with R + Bioconductor (GEOquery,
the GPL6885/GPL16570 platform annotation pkgs, org.Mm.eg.db) + seeker on «our HPC»
front node; download GEO series to «infra»; run
seekerArray(); record emat dims; compute WT-vs-KO log2FC per study and the cross-study Spearman rho. - Phase B (RNA-seq, C4–C5): refgenie mm10 salmon index + sra-tools on «our HPC»;
seeker()per the figshare YAMLs as a SLURM compute job; tximport → gene table
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.