PrimerSeq: Design and visualization of RT-PCR primers for alternative splicing using RNA-seq data.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
PrimerSeq's core pipeline (splice-graph construction from a real GTF + RNA-seq junction reads, EM-based isoform abundance estimation, Primer3-based flanking-exon primer design) was reproduced end-to-end on real data: the paper's TJP1 alternative-splicing case study (Figure 2) was rebuilt from scratch using GEO/ENA raw FASTQ (ERR030886 heart, ERR030873 testes from GSE30611/Illumina Human Body Map 2.0), aligned with HISAT2 to hg19 (96.5%/94.8% overall alignment rates), and run through the patched PrimerSeq CLI against the real UCSC hg19.refGene GTF (chr15 subset, properly sorted). Reproduced PSI values (heart 86.60%, testes 47.22%) are within a few percentage points of the paper's reported RNA-seq PSI values as read from Figure 2 (~82.5% heart, ~45.5% testes) -- note these reference numbers come from visually reading a figure image in an earlier turn of this same session, not from machine-readable article text (PMC text search confirmed the article body reports no numeric PSI values, only 'PSI estimates matched well between RNA-seq and RT-PCR'), so treat them as approximate and flag for human verification against the actual published figure. The bundled repository example (FN1 gene) was also fully reproduced (PSI 48.39%) after fixing FOUR genuine CLI wiring bugs in primer.py that only affected the argparse-based command-line entry point (never the GUI-driven PrimerApp.py): (1) target-file parsing, (2) output-path wiring, (3) a no-op job_id basename split ('.split("/\")' on a literal 2-char string that never appears in a Unix path), (4) a missing 'short_isoform' default (only ever set by a GUI checkbox). These are documented, minimal, input/output-only patches -- no algorithmic logic was touched. A genuine coordinate-convention discovery was also required: PrimerSeq's gtf.py stores exon starts as (raw GTF start - 1) internally (0-based), but the CLI target-file parser does not apply this shift, so target-file start coordinates must be supplied as (true GTF start - 1) to match; this is undocumented and would trip up any user following the README literally. RT-PCR wet-lab validation values (paper's Figure 2B, read in an earlier turn as ~78%/52%) are out of scope per the reproduction brief (wet-lab, not pipeline-derived) and are not graded as claims here, only mentioned as external context. Produced BAM files (heart 5.17GB, testes 5.57GB) are notably larger than the paper's stated BAM sizes (3.5GB/2.7GB) -- plausibly explained by aligner choice (HISAT2 here vs. an unspecified/likely TopHat aligner circa 2014), retained secondary/multi-mapped alignments, and/or possible compression-level differences, but this was not further investigated and is flagged for human review rather than asserted. GSE30611 dataset profiling: the accession itself is the full Illumina Human Body Map 2.0 (48 samples/GSM records across 16 tissues per its GEO summary), but the paper only uses 2 of those 48 samples (heart, testes) as its case-study data source -- both were located, downloaded in full via their ENA run accessions, and successfully aligned, so the 2 samples the paper actually relies on are complete and deliver what was promised. Nothing was dropped. No result was force-fit: the reproduction stopped at the one worked case study the paper presents in detail (TJP1/Figure 2), since PrimerSeq is a primer-design tool rather than a paper reporting a benchmark table across many genes, and its "main quantitative result" essentially is this one worked example plus the RT-PCR concordance check.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-07-31
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-07-31no human curator yet
- Last updated
- 2026-07-31
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusThe authors address the bottleneck between transcriptome-wide RNA-seq discovery of alternative splicing (AS) events and their RT-PCR validation, positing that an ideal RT-PCR primer design tool for AS analysis should incorporate user-provided RNA-seq transcriptome profiles and arbitrary transcript annotations rather than relying on static, species-restricted pre-defined annotations.
- ★ PrimerSeq is a user-friendly stand-alone software with a GUI for systematic design and visualization of RT-PCR primers for alternative splicing analysis using user-provided RNA-seq data. resource
- ★ To the best of the authors' knowledge, PrimerSeq is the only software that incorporates RNA-seq data in the design and visualization of RT-PCR primers for AS analysis. resource
- ★ PrimerSeq uses the biconnected components algorithm on the splice graph to define generalized AS events (alternative splicing modules, ASMs), enabling primer design for all AS types including exon skipping, alternative splice sites and mutually exclusive exons. method
- ★ PrimerSeq uses RNA-seq read counts and an expectation maximization algorithm to estimate isoform abundance and PSI, and places primers on flanking exons with constitutive splicing or inclusion level above a user-defined threshold (95% by default). method
- ★ PrimerSeq visualizes RNA-seq read density juxtaposed with the expected sizes and relative ratios of RT-PCR products, allowing direct comparison of experimental results with RNA-seq predictions. method
- ★ Primers designed by PrimerSeq were experimentally validated: all five randomly-chosen differential AS events had successful target amplification by RT-PCR with product sizes consistent with the design. finding
- Implementing PrimerSeq as stand-alone local software (rather than web-based) is necessary due to the large size of RNA-seq data files and the goal of supporting a diverse range of organisms. method
- Novel isoforms supported by RNA-seq reads can optionally be added to the supplied transcript annotations and visualized before performing RT-PCR. method
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| RNA-seq (computational analysis of publicly-available data) | Human heart and testes tissue (Illumina Human Body Map 2.0 Project, NCBI GEO GSE30611) | none (tissue comparison: heart vs. testes) | Differential alternative splicing events and percent-spliced-in (PSI/ψ) exon inclusion levels | Illumina; MATS version 3.0.6.beta for differential AS detection |
| RT-PCR with TBE-PAGE gel resolution | Total RNA from human heart (Applied Biosystems, Foster City, CA, USA) and human testes (Clontech, Mountain View, CA, USA) | none | Presence/size of PCR amplification products and PSI (relative ratio of splice-form products) | 5% TBE-PAGE gel |
| In silico RT-PCR primer design (PrimerSeq software) | Human genome/transcriptome (FASTA genome, GTF annotation, BAM RNA-seq alignments, BigWig read density); UCSC Known Genes annotation | none | Primer pairs on flanking constitutive exons, expected PCR product sizes and predicted relative ratios | Python/wxPython GUI; Primer3 v2.3.4; NetworkX; SAM-JDK v1.77; BigWig API r39; UCSC In-Silico PCR |
- – All five randomly-chosen AS events (from the top 100 differential AS events detected by MATS between human heart and testes) had successful primer design as evidenced by target amplification during RT-PCR. 5 of 5 events
- – In all five events, RT-PCR primers successfully amplified the target regions and PCR product sizes were consistent with the PrimerSeq design results. 5 of 5 events
- – For the TJP1 (tight junction protein 1) AS event, PSI estimates matched well between RNA-seq and RT-PCR.
- – For HNRPLL, RNA-seq data suggested a novel isoform not present in the UCSC Known Genes annotation, which was confirmed by RT-PCR in testes as a 114 bp PCR product. 114 bp PCR product
- – RNA-seq BAM file sizes for the Illumina Body Map 2.0 human heart and testes datasets are large, making web-based processing impractical and motivating the stand-alone implementation. 3.5 GB (heart) and 2.7 GB (testes)
- – Indexing of BAM, FASTA and BigWig files provides significant speed improvements for handling large datasets.
- count five (AS events chosen at random from the top 100 differential AS events for primer design and RT-PCR validation)
- count top 100 (Differential AS events detected by MATS between human heart and testes from which five were randomly selected)
- other PSI > 95% (default user-defined threshold) (Minimum exon inclusion level required for an exon to be selected as a flanking exon for primer placement)
- other 114 bp (Size of the RT-PCR product in testes supporting a novel HNRPLL isoform absent from the UCSC Known Genes annotation)
- other 3.5 GB and 2.7 GB (Sizes of the human heart and testes RNA-seq BAM files in the Illumina Body Map 2.0 dataset (GEO GSE30611))
- other 5% (TBE-PAGE gel concentration used for resolving RT-PCR products)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software application-note describing PrimerSeq, a tool for RT-PCR primer design and visualization for alternative splicing (AS) analysis using RNA-seq data. The 'statistical' component is limited to a proof-of-concept validation: differential AS events between human heart and testes RNA-seq data were identified using the MATS Bayesian framework, and five events from the resulting top-100 list were selected at random and validated qualitatively by RT-PCR. No formal hypothesis tests, p-values, or effect sizes are reported in the text for the validation itself; results are described narratively (e.g., 'PSI estimates matched well between RNA-seq and RT-PCR').
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| MATS (Multivariate Analysis of Transcript Splicing) Bayesian framework for differential alternative splicing detection | Identification of differential AS events between human heart and testes RNA-seq data (Illumina Human Body Map 2.0, GSE30611) | — | not stated |
-
Differential AS events were identified with MATS applied to RNA-seq data from single heart and testes samples, without a description of biological replicates.↳ Could also: A replicate-based differential splicing approach (e.g., MATS, rMATS, or a GLM-based framework applied to multiple biological replicates per tissue) could also be used. — Additional biological replicates would allow explicit modeling of biological variability and reporting of variance estimates or confidence intervals around the splicing differences.
-
Agreement between RNA-seq-predicted PSI and RT-PCR results for the five validated events was described qualitatively (e.g., 'matched well') rather than with a numeric comparison.↳ Could also: A quantitative concordance measure, such as a correlation coefficient or Bland-Altman agreement analysis between RNA-seq PSI and RT-PCR PSI, could also be reported. — This would provide an explicit numerical summary of platform agreement that complements the visual/qualitative comparison shown in the figures.
-
The manuscript refers to a 'top 100 differential AS events' list from MATS without stating the specific p-value or FDR threshold used to define it.↳ Could also: Explicitly reporting the significance threshold (e.g., a Benjamini-Hochberg FDR cutoff) used to generate the ranked event list could also be included. — Stating the exact statistical threshold would let readers know precisely how the tested event set was defined and enable comparison with cutoffs used in similar studies.
-
RT-PCR validation was reported as successful amplification and product sizes consistent with expectations, based on gel images, without band intensity quantification.↳ Could also: Densitometric quantification of RT-PCR gel bands (e.g., with ImageJ) to derive a numeric PSI estimate from RT-PCR could also be performed. — Numeric quantification would allow a direct statistical comparison (e.g., correlation) between RNA-seq-derived and RT-PCR-derived PSI values rather than relying on visual/qualitative comparison alone.
-
The heart-versus-testes comparison appears to be based on one RNA sample per tissue.↳ Could also: Incorporating multiple independent biological replicates per tissue could also be used. — This would enable estimation of biological variability and application of standard variance-based statistics (e.g., SD, SEM, or t-tests) when comparing PSI values between tissues.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
PrimerSeq reproduces. Using the authors' own deposited data (GSE30611, ERR030886 heart / ERR030873 testes) and their own released code, the TJP1 case study of Figure 2A came out at PSI 86.60% (heart) and 47.22% (testes) against the figure-read reference of ~82.5% / ~45.5%, and the bundled FN1 example ran to completion at PSI 48.39% with a valid primer pair — magnitude, direction and the heart≫testes contrast all hold. The residual few-percentage-point gap sits on our side (self-chosen HISAT2 alignment and hg19.refGene annotation, since the paper names no aligner) and is partly unquantifiable because the paper publishes no numeric PSI anywhere in its text — the reference values exist only as pixels in Figure 2A, which is a reporting weakness on the authors' side but not a validity problem. The one substantive defect is on the authors' side and is engineering, not science: the released CLI is broken in four places (primer.py no-op basename split, missing -o wiring, incomplete target parsing, KeyError on the GUI-only short_isoform) plus an undocumented 0-based/1-based target-coordinate shift in gtf.py, so the deposited code cannot be run headlessly by anyone following the README. Overall yellow: the science is confirmed, the artifact is not turnkey and the reference numbers are figure-derived.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.