Viewing RNA-seq data on the entire human genome.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Reported values were directly comparable
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓The central claim held under reproduction
- 🟡Could not use the authors’ exact input data
- 🟡A deviation arose in the data or preprocessing
- 🟡The deviation was non-trivial in magnitude
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH: yes. The paper's primary computational result is the rnaseqview pipeline's per-gene expression table for SRA SRR562646, and the repo ships its own output as an answer key. We re-ran the authors' actual Python-2 pipeline on «our HPC» (sam-dump SRR562646.sra | samtools sort -n | awk '$7=="="' | htseq-count -s no -i gene - GRCh37.gtf -> count/longest-transcript*1000) with NCBI Annotation Release 105 (GRCh37.p13) and htseq 0.6.1 (paper's version). RESULT = PARTIAL, faithful: Spearman = Pearson(log) = 0.98 between our regenerated table and the shipped one, 60.5% of common genes bit-identical to 4 decimals (e.g. A1CF 0.40144520273 exact, A2ML1 within 0.3%). NOT 1:1 byte-exact because the authors' exact 2017 Annotation Release 105 GFF build is no longer served by NCBI (their hard-coded ARCHIVE URLs 404); we used the earliest surviving re-archive (105.20190906), shifting per-gene longest-exon denominators for ~40% of genes. AUDITABILITY WINS: the paper's '31,148 annotations' is NOT fabricated -- it exactly equals the annotation count in the shipped JSON; and we flag a terminology inaccuracy -- the code normalizes by longest-transcript length x1000, NOT 'TPM' as the paper/README state. NOT ATTEMPTED (out of scope): UI/performance metrics (399KB gzip JSON, <285ms download, 830-1044ms render -- browser/network dependent), the secondary GSE40705 GEO example, and the interactive Ideogram.js figures (visual). Per 80/20 we did not chase the byte-identical 2017 GFF. Three pipeline bugs from tool-version drift were fixed (GTF URL-rot, sratools driver wrapper on bare accession, samtools 1.9 /dev/stdout temp-file); heavy data kept on «infra».
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-16 ⛓ 3465e545315e
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-16
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusDepicting RNA-seq data across all chromosomes at once (in an ideogram) can intuitively highlight global patterns of gene expression in a way single-chromosome browser views cannot; the paper presents RNA-Seq Viewer, a web application to do this from SRA/GEO data.
- ★ RNA-Seq Viewer is a web application that visualizes genome-wide expression data from NCBI's SRA and GEO databases using an ideogram across the entire human genome. resource
- ★ A backend Python data pipeline extracts aligned RNA-seq data from SRA/GEO, quantifies and normalizes expression to TPM, and transforms it into the JSON annotation format used by Ideogram.js. method
- ★ Ideogram.js relies only on JavaScript, HTML and CSS (with D3.js and Crossfilter), enabling client-side faceted search with no server-side code, simplifying embedding in web pages. method
- Compact JSON and space-efficient data structures minimize web-page load time for genome-wide annotation sets. method
- Only genome assemblies GRCh37 and GRCh38 are supported, using NCBI Homo sapiens Annotation Releases 105 and 107 respectively. method
- The prototype was created by a small team during a three-day NCBI hackathon at Brandeis University. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| RNA-seq read counting / gene expression quantification | human (Homo sapiens), SRA run SRR562646; BAM/SAM alignments | none | per-gene read counts normalized to TPM (normalized by mature transcript length using longest transcript) | HTSeq-count v0.6.1p1; sam-dump v1.3 (SRA Toolkit) |
| Genomic coordinate/gene-type annotation parsing and JSON formatting | human genome assemblies GRCh37/GRCh38 | none | chromosome name, start/stop coordinates, gene type, and TPM per gene formatted into Ideogram.js JSON | NCBI Homo sapiens Annotation Release 105/107 GFF files; Python pipeline |
| Web-page load/render performance benchmarking | Chrome v51 on Mac OS X laptop, 2.9 GHz Intel Core i5 | network-throttled (14 Mb/s, 50 ms latency) | download time and interactive load/render time for genome-wide histogram of gene features | Chrome Developer Tools / DevTools Timeline |
- – Gzip-compressed set of human gene feature annotations for SRR562646 is 399 KB and downloads in under 285 ms on an average US Internet connection. 399 KB; <285 ms
- – An uncached interactive genome-wide histogram of expression loads and renders in Ideogram between 830 ms and 1044 ms after start of navigation. 830-1044 ms
- – The web application prototype provides interactive filtering of genes by expression level and gene type across the whole genome or within a single chromosome.
- – Pipeline output JSON contains feature annotations for every human gene with symbols, coordinates, expression levels and gene types. 31,148 gene features
- count 31,148 human gene feature annotations (annotations output by pipeline for SRA run SRR562646)
- other 399 KB (gzip-compressed annotation set size for SRR562646)
- other <285 ms (download time on 14 Mb/s, 50 ms latency connection)
- other 830-1044 ms (uncached genome-wide histogram load+render time in Ideogram)
- other 14 Mb/s download bandwidth, 50 ms latency (network-throttled test conditions for average US Internet connection)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
-
Expression was normalized to TPM using HTSeq-count raw counts divided by longest transcript length↳ Could also: Dedicated quantification tools such as Salmon or Kallisto (both mentioned in the Discussion) could be used directly to produce TPM estimates, incorporating effective transcript length and GC-bias correction — Alignment-free or quasi-mapping quantifiers are faster, require less memory, and account for sequence-composition biases, which may improve TPM accuracy, especially for multi-isoform genes
-
Only the longest transcript was used as the gene length proxy for TPM normalization↳ Could also: Using the effective length (e.g., mean or abundance-weighted average across isoforms) as implemented in tools like featureCounts or Salmon would also be standard — Longest-transcript length can over- or under-estimate effective gene length; isoform-aware or effective-length approaches can improve comparability across genes with many alternative transcripts
-
Performance benchmarking (load and render time) was measured once on a single machine under a single network condition↳ Could also: Repeated measurements across multiple machines, browsers, and network profiles with reported mean and range would also be standard for software benchmarking — A single-run measurement on one configuration gives a point estimate with no indication of variability; repeated trials with summary statistics allow readers to better judge expected performance in their own environments
-
Only properly paired reads were retained after filtering the BAM/SAM file before HTSeq-count quantification↳ Could also: Reporting the fraction of reads retained versus discarded after filtering, and testing sensitivity of TPM estimates to this filter, would also be informative — Documenting read-level attrition rates is common practice in RNA-seq pipelines and helps users understand how much data is used and whether the filtering step materially affects downstream values
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
- Accelerating rare disease diagnostics by linki... L1 No data access
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-28979763 (rnaseqview / "Viewing RNA-seq data on the entire human genome")
- Paper: F1000Research 2017, 6:1715 (v1). DOI 10.12688/f1000research.9762.1. PMCID PMC5605993.
- Code: https://github.com/NCBI-Hackathons/rnaseqview @ commit 65fe6db (2017-07-24).
- Data: SRA
SRR562646(Homo sapiens, RNA-Seq, Illumina, ~9.40M reads, 1.43 Gbase; aligned to GRCh37 in SRA).
Pipeline (from counting/counter.py, mapper.py, formatter/formatter.py) — Python 2
For an SRR accession the authors' counter.py runs:
sam-dump SRR562646 | samtools sort -n -O sam - -o /dev/stdout \
| awk '$7=="="' | htseq-count -s no -i gene - GRCh37.gtf > SRR562646_counts.tsv
then normalizes: for each gene, norm = raw_count / (longest transcript length of that gene) * 1000
(_get_size sums exon lengths +1 per exon per transcript, keeps the longest transcript per gene).
GTF is built by mapper.g37_map: download NCBI Annotation Release 105
ref_GRCh37.p13_top_level.gff3.gz + chr_accessions_GRCh37.p13, map RefSeq accessions to chrom
names, drop region/comment lines. formatter.py then bins expression to 1+int(percentile/16)
and attaches gene coords/type from gene_lookup_GRCh37.tsv → JSON for Ideogram.js.
Reported tool versions: HTSeq-count 0.6.1p1, sam-dump 1.3, NCBI Annotation Release 105 (GRCh37).
IN SCOPE (pipeline-derived, has shipped answer key)
- C1: per-gene normalized expression table
data/SRR562646_norm.tsv(31,195 genes; 18,063 nonzero). Reproduce by re-running the authors' pipeline on SRR562646; compare per-gene values (Spearman/Pearson, fraction matching within tolerance). - C2: gene-count / annotation count. Paper text: "31,148 human gene feature annotations".
Shipped
SRR562646_norm.tsvhas 31,195 gene rows. Check our regenerated table's gene count. - C3 (cheap):
data/SRR562646.jsonformatter output (binned expression levels). Reproduce by runningformatter.pyon our norm.tsv and comparing to shipped JSON.
OUT OF SCOPE / not attempted
- Browser rendering performance metrics (399 KB gzip JSON, <285 ms download, 830–1044 ms render) — UI/performance, not a deterministic pipeline output; hardware/network dependent.
- Ideogram.js interactive visualization (Figures 1–2) — manual/visual, not a numeric pipeline result.
- The GSE40705 example (
data/GSE40705.json) — secondary GEO example; SRR562646 is the primary.
Possible-inaccuracy note
Paper/README say expression is normalized to "TPM units", but the code (normalize()) computes
raw_count / longest_transcript_length * 1000 — a single-gene length normalization, NOT TPM
(no library-size / sum-of-(count/length) scaling). Flag for human review.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The authors' own Python-2 pipeline was re-run on the identical SRA reads SRR562646 and compared to the repo's shipped answer key: the per-gene expression table reproduces at Spearman = Pearson(log) = 0.98 with 60.5% of common genes bit-identical to 4dp, and the reported 31,148 annotations matches the shipped JSON exactly (not fabricated). The deviation lies on the input/reference side, not the computation: the exact 2017 Annotation Release 105 GFF is no longer served by NCBI, so a later snapshot shifts longest-transcript denominators for ~40% of genes (gene count 36155 vs 31195; A1BG 1.08 vs 3.36). Severity is moderate (rank/magnitude hold) and there is no significance to flip, so the core method claim is confirmed; a separate terminology inaccuracy (calling length-scaling 'TPM') is flagged but is not a numeric defect.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.