transXpress: a Snakemake pipeline for streamlined de novo transcriptome assembly and annotation.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓The central claim held under reproduction
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
transXpress is a tools/pipeline paper; demonstration assembly of Piper longum (3 SRA runs, 2x150 PE, 67,298,239 reads). REPRODUCED. C1 read counts EXACT at file level (md5-verified FASTQ). Anti-fabrication cross-check vs the authors' Zenodo deposit (10.5281/zenodo.7380017): 15 of 16 Table 2 values reproduce EXACTLY to the digit (transcripts, ORFs, full-length %, median len, mean ORF aa, BUSCO embryophyta_odb10, bowtie2 mapping for both Trinity & rnaSPAdes). The ONE exception is rnaSPAdes max transcript length (C7s): the deposited assembly contains a 16,826 bp transcript vs the reported 15,665 -> paper UNDER-reports vs its own data (not inflation; likely a measurement/version difference) - flagged for human. Independent re-run of the pipeline (Trinity 2.13.2 / rnaSPAdes 3.13.0) on the paper's own reads is still computing as a 2nd evidence line; de novo is non-deterministic so within-tol/partial expected there, not exact. NOT attempted: Table 1 (qualitative tool comparison), C10 SignalP/C11 TMHMM targeting/TM-domain %s (license-gated, not core Table 2). All grades provisional, human-auditable.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-30
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-30no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper does not test a scientific hypothesis but addresses the software engineering question of whether a unified, reproducible, Snakemake-based pipeline can streamline de novo transcriptome assembly and annotation for non-model organisms while supporting distributed HPC computing, an unmet need in existing tools.
- ★ transXpress is a Snakemake pipeline that streamlines de novo transcriptome assembly, quantification, and annotation for non-model organisms method
- ★ transXpress supports two assemblers (Trinity and rnaSPAdes) and allows parallel execution on heterogeneous cluster computing hardware method
- ★ transXpress produces standardized, annotated output FASTA files that can be mined using SequenceServer for rapid gene/protein discovery resource
- ★ Among existing pipelines, only Pincho supports both de novo assembly and annotation, but it lacks support for distributed HPC computing, limiting utility for large datasets finding
- ★ transXpress integrates quality control (FastQC/MultiQC/Trimmomatic), expression quantification (kallisto), differential expression (edgeR), and functional annotation (BLAST, Pfam, Rfam, SignalP, TargetP, TMHMM, BUSCO) into one workflow method
- Application of transXpress to Piper longum RNA-seq data demonstrates its utility for generating annotated transcriptomes in a non-model medicinal plant finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| RNA-seq read quality control | Piper longum (leaf, spike, root) RNA-seq reads | none | read quality metrics before/after trimming | FastQC, MultiQC, Trimmomatic |
| de novo transcriptome assembly | Piper longum leaf/spike/root pooled RNA-seq reads | assembler choice (Trinity vs rnaSPAdes) | number of assembled transcripts/genes, contig length stats, ORF counts and lengths | Trinity v2.13.2, rnaSPAdes v3.13.0, TransDecoder |
| transcriptome completeness assessment | assembled P. longum transcriptomes (Trinity and rnaSPAdes) | none | BUSCO completeness/duplication/fragmentation/missing scores | BUSCO, embryophyta_odb10 lineage |
| transcript expression quantification | P. longum RNA-seq reads vs assembled transcriptome | none | transcript-per-million (TPM) expression levels | kallisto |
| read-to-transcript alignment | P. longum RNA-seq reads vs assembled transcriptome | none | % of reads aligned to transcriptome | Bowtie2 |
| differential expression analysis | P. longum leaf, spike, root tissue samples | tissue type (leaf vs spike vs root) | differentially expressed transcripts, hierarchically clustered heatmap | edgeR |
| protein functional annotation | predicted P. longum ORF/protein sequences (n=131,098 for Trinity) | none | homology hits, protein domains, ncRNA hits, signal/targeting peptides, transmembrane helices | BLAST+, hmmer3 (Pfam-A), cmscan (Rfam), SignalP 6.0, TargetP 2.0, TMHMM (Python reimplementation) |
- – Trinity assembled 268,313 unique transcripts; rnaSPAdes assembled 296,600 unique transcripts from P. longum reads 268,313 vs 296,600 transcripts
- ▲ Trinity assembly showed higher BUSCO completeness than rnaSPAdes C:95.2% (Trinity) vs C:84.1% (rnaSPAdes)
- ▲ Bowtie2 read alignment rate to the assembled transcriptome was higher for Trinity than rnaSPAdes 87.5% vs 83.3%
- ▼ TransDecoder predicted average ORF lengths shorter than the related P. nigrum genome-derived proteins, consistent with partial transcripts 282 aa (Trinity) / 255 aa (rnaSPAdes) vs 449 aa (P. nigrum)
- – A large proportion of assembled transcripts were 5'-partial, limiting targeting peptide prediction 22% of transcripts
- – Targeting peptides were predicted in a subset of protein sequences by TargetP, most commonly ER signal peptides followed by chloroplast transit peptides 11.8% of protein sequences
- – A substantial fraction of predicted proteins contained transmembrane domains ~19% of protein sequences
- – Differential expression analysis of root, leaf, and spike samples produced a hierarchically-clustered heatmap of expression differences
- count 67,298,239 total raw reads (16,901,456 leaf + 22,900,035 spike + 27,496,748 root) (input RNA-seq read counts for P. longum assembly)
- count 268,313 (Trinity) vs 296,600 (rnaSPAdes) assembled transcripts (assembly output size comparison)
- count 131,098 (Trinity) vs 118,984 (rnaSPAdes) predicted protein ORFs (TransDecoder ORF prediction)
- other BUSCO: C:95.2% [S:10.5%, D:84.7%], F:2.7%, M:2.1% (Trinity) vs C:84.1% [S:18.6%, D:65.5%], F:11.1%, M:4.8% (rnaSPAdes) (transcriptome completeness, embryophyta_odb10 lineage)
- fold_change 87.5% (Trinity) vs 83.3% (rnaSPAdes) reads aligned (Bowtie2 alignment rate)
- mean average ORF length 282 aa (Trinity) / 255 aa (rnaSPAdes) vs 449 aa in P. nigrum genome (predicted ORF length comparison to related species genome)
- other 22% of transcripts 5'-partial (proportion of incomplete transcripts, likely due to oligo(dT) priming)
- other 11.8% of protein sequences with targeting peptides; ~19% with transmembrane domains (TargetP and TMHMM predictions on Trinity-derived proteins)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software/methods paper describing transXpress, a Snakemake pipeline for de novo transcriptome assembly and annotation, rather than a hypothesis-driven biological study. The pipeline's output is demonstrated on a single Piper longum RNA-seq dataset (one sample each from leaf, spike, and root), with results reported mainly as descriptive assembly/annotation statistics (e.g., transcript and ORF length summaries, BUSCO completeness percentages, read-alignment rates) and a differential expression step using edgeR summarized as a hierarchically-clustered heatmap. The text does not report p-values, effect sizes, or explicit multiple-testing correction methods.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| edgeR differential expression analysis (specific statistical test/model not further specified in text) | Comparison of expression across leaf, spike, and root P. longum tissue samples, visualized as a hierarchically-clustered heatmap (Fig. 4) | One RNA-seq sample per tissue (leaf, spike, root); no replicates described | not stated |
-
Differential expression across the three tissues was assessed with edgeR using one RNA-seq sample per tissue, without biological replicates described.↳ Could also: A design with biological replicates per tissue, paired with edgeR's or DESeq2's replicate-based dispersion estimation — Replicates allow estimation of biological variability and calculation of significance measures (p-values, FDR-adjusted q-values) for expression differences, which is harder to characterize robustly from single, unreplicated samples.
-
The differential expression result is presented as a hierarchically-clustered heatmap (Fig. 4) without an accompanying numerical table of fold-changes or significance values in the text.↳ Could also: Reporting a supplementary table of log2 fold-changes and FDR-adjusted p-values (e.g., edgeR's topTags output) alongside the heatmap — This would let readers assess the magnitude and statistical support for specific transcript-level differences in addition to the overall clustering pattern shown visually.
-
Assembly and annotation metrics (transcript/ORF lengths, BUSCO completeness) are reported as single-value descriptive statistics (min/median/mean/max or percentages) without measures of variability.↳ Could also: Including variance or resampling-based uncertainty estimates (e.g., bootstrapped BUSCO scores across read subsamples) — This can convey how sensitive these summary metrics are to sequencing depth or assembler stochasticity, complementing the point estimates already reported.
-
Trinity and rnaSPAdes assemblies of the same dataset are compared side by side in Table 2 using descriptive statistics only.↳ Could also: A formal paired comparison across multiple matched datasets (e.g., a paired t-test or Wilcoxon signed-rank test on assembly quality metrics) — This could provide a statistical basis for generalizing about systematic differences between assemblers, since the current comparison is based on a single dataset.
-
No specific multiple-testing correction method is named in the text for the transcriptome-wide expression comparison.↳ Could also: Explicitly stating the correction method used (e.g., Benjamini-Hochberg FDR, edgeR's common default) — Naming the method explicitly clarifies for readers how the false discovery rate is controlled across the many simultaneous transcript-level comparisons.
-
Software versions are given for the two assemblers (Trinity v2.13.2, rnaSPAdes v3.13.0) but not for edgeR, kallisto, or other analysis tools.↳ Could also: Reporting version numbers for all statistical/quantification packages used — Version reporting supports exact reproducibility, since default parameters or algorithms in statistical packages can change between releases.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-37016291 (transXpress)
Paper: Fallon TR et al. (2023) transXpress: a Snakemake pipeline for streamlined de novo transcriptome assembly and annotation. BMC Bioinformatics. PMID 37016291.
Nature of the paper: This is a software/tools paper describing a Snakemake pipeline (transXpress) that wraps existing third-party tools (Trinity, rnaSPAdes, TransDecoder, BUSCO, kallisto, edgeR, BLAST+, HMMER, SignalP/TargetP, etc.). The quantitative "results" are a demonstration run of the pipeline on a public RNA-seq dataset of Piper longum (long pepper). Per BRIEF rule P16, running this pipeline / its component tools on the paper's own data is a fully valid reproduction.
Datasets the paper relies on
| Tissue | SRA run | Reported reads (Table 2) |
|---|---|---|
| Leaf | SRR10362954 | 16,901,456 |
| Spike | SRR10362953 | 22,900,035 |
| Root | SRR10583928 | 27,496,748 |
| Total | 67,298,239 |
Reads = Illumina stranded paired-end 2×150 bp. (BRIEF named only SRR10362954, but the paper's demonstration assembly used all three combined — Table 2.)
Authors also deposited the full output archives (Trinity + rnaSPAdes runs) on Zenodo: https://doi.org/10.5281/zenodo.7380017 — usable to cross-check the reported Table 2 numbers against the authors' own shipped outputs (fabrication check).
In scope (pipeline-derived, quantitative — Table 2)
Each is a number produced by a bioinformatic tool from the raw reads:
| ID | Result (Table 2) | Trinity v2.13.2 | rnaSPAdes v3.13.0 | Tool |
|---|---|---|---|---|
| C1 | total raw reads (3 runs) | 67,298,239 | (same) | SRA / seqkit |
| C2 | assembled transcripts | 268,313 | 296,600 | Trinity / rnaSPAdes |
| C3 | predicted protein ORFs | 131,098 | 118,984 | TransDecoder |
| C4 | full-length ORFs (%) | 54.7% | 60.4% | TransDecoder |
| C5 | median transcript length (bp) | 577 | 363 | seqkit/Trinity stats |
| C6 | mean ORF length (aa) | 282 | 255 | TransDecoder |
| C7 | max transcript length (bp) | 15,159 | 15,665 | assembly stats |
| C8 | BUSCO complete (%) | 95.2% | 84.1% | BUSCO |
| C9 | reads mapped back (%) Bowtie2 | 87.5% | 83.3% | bowtie2 |
| C10 | targeting peptides (% of proteins) | 11.8% | — | SignalP/TargetP |
| C11 | transmembrane domains (% proteins) | ~19% | — | TMHMM reimpl. |
Reproduction priority:
- Quick MINIMUM (clear, low compute): C1 (read counts — direct from SRA), plus cross-checking C2–C9 against the Zenodo deposited outputs (delivers-promised).
- Core (heavy compute on «our HPC»): independently regenerate C2, C5, C7 by running rnaSPAdes and/or Trinity on the 3 runs; then C3,C4,C6 (TransDecoder), C8 (BUSCO), C9 (Bowtie2).
- Harder / license-gated: C10 (SignalP 6.0 + TargetP 2.0 require manual license download), C11 (TMHMM reimplementation). Attempt only if envs build cleanly.
Out of scope
- Table 1 (qualitative feature comparison vs 16 other pipelines) — not a computed numeric result; literature survey.
- Biological interpretation of annotations (which genes, pathways) — descriptive.
- No runtime/memory benchmarks are reported in the paper, so none to reproduce.
Reproduction notes / expected fidelity
- De novo assembly is version- and thread-sensitive: exact transcript counts (C2) depend on Trinity/rnaSPAdes version, k-mer settings, read normalization, and CPU count. Exact match to 268,313 / 296,600 is unlikely; within-tol or partial is the realistic, honest expectation. Read counts (C1) sho
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a pipeline-demonstration paper, and 15 of 16 Table 2 values reproduce exactly to the digit against the authors' own Zenodo deposit, with read counts md5-verified 1:1 from public SRA — a strong reproduction with no fabrication signal. The single mismatch is C7s (rnaSPAdes max transcript length): the deposit contains a 16,826 bp transcript vs the reported 15,665 bp, i.e. the paper under-reports its own data, pointing to a version/measurement difference rather than inflation. The deviation lies on the authors' side but is peripheral and conservative; the central claim holds fully. Caveat: the exactness is a cross-check against the authors' deposit — the fully independent re-run was still running at submission.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.