TRAPID 2.0: a web application for taxonomic and functional analysis of de novo transcriptomes.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
TRAPID 2.0 is a web-application tool paper; its headline annotation numbers (C3-C7) and performance metric (C8) are produced only by the live web server / authors' cluster and are out of scope as standalone compute by design. The reproducible upstream pipeline ran end-to-end on «our HPC» (Trimmomatic 0.36 + MIRA 4.0.2, «job», mira_exit=0): C1 read count reproduces within ~3% (raw deposit 559,011, post-trim 542,211 vs reported 526,527 - same data, underspecified trim), and C2a MIRA assembly reproduces closely (56,143 contigs / N50 393 bp vs 53,569 / 391 bp). C2b (contigs+singletons = 143,308) MISMATCHES: MIRA's default est/accurate run reported 0 singletons and discarded 292,309 unassembled reads as debris, whereas the paper retained 89,739 singletons; the singleton-retention parameter is not documented, so 143,308 is not reproducible from the methods. No fabrication concern - every in-scope value is derivable from the shipped data, deltas trace to underspecified parameters and assembler non-determinism. NOT attempted: the full TRAPID self-host (CakePHP/MySQL/SGE + DIAMOND/Kaiju/Infernal/eggNOG-mapper + pico-PLAZA 3.0 / NCBI-nr, hundreds of GB), MMETSP 26-sample validation, and ciliate genetic-code analysis (all web-server driven).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 42assessed: 2026-06-18 ⛓ 19de7ec472d2
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-25
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe authors aimed to develop TRAPID 2.0, an improved web application, to overcome the computational challenges of processing fragmented and contaminant-containing de novo assembled (meta)transcriptomes, providing fast structural, functional, and taxonomic annotation together with downstream comparative analyses.
- ★ TRAPID 2.0 is a web application performing global characterization of de novo transcriptomes via structural, functional, and taxonomic annotation in an initial processing phase, followed by an exploratory phase of downstream analyses. resource
- ★ TRAPID 2.0 supports analysis of assembled (meta)transcriptomes with mixed prokaryotic and/or eukaryotic composition due to improved processing speed, updated reference databases, and integrated protein-based taxonomic classification. method
- ★ Core gene family completeness assessment allows estimation of gene space completeness along a user-defined evolutionary lineage. method
- ★ Using the ciliate nuclear genetic code instead of the standard genetic code for ORF prediction improves the 'best hit recovery ratio' for ciliate transcriptomes with stop codon reassignment. finding
- Splitting the Kaiju taxonomic classification index for memory efficiency results in performance nearly identical to a regular (non-split) Kaiju MEM run. finding
- Functional annotation transfer from gene families requires the annotation label to be present in at least 50% of GF members (majority vote), validated by benchmarking on five model species. method
- ★ Application of TRAPID 2.0 to diatom-dominated Antarctic metatranscriptomes reveals functional variation between phytoplankton communities from different sampling sites. finding
- TRAPID 2.0 offers unique features relative to comparable transcriptome annotation tools. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Taxonomic classification (protein-based k-mer classification) | 42 MMETSP re-assembled unicellular eukaryote transcriptomes | none | taxonomic assignment of transcripts | Kaiju |
| Similarity search / gene family assignment / functional annotation | Query transcriptomes vs. reference proteomes (PLAZA 4.5, pico-PLAZA 3.0, PLAZA diatoms 1.0, eggNOG 4.5) | none | top protein hit, gene family assignment, GO/KO terms, protein domains | DIAMOND; eggNOG-mapper v1 |
| Non-coding RNA identification | Query transcript sequences | none | RNA family/clan assignment and GO term annotation | Infernal (cmsearch) with Rfam 14.1 models |
| Ribosomal RNA taxonomic classification | Putative SSU/LSU rRNA transcripts | none | taxonomic classification of rRNA sequences | SINA v1.5.0 with SILVA 132 |
| ORF prediction / best hit recovery ratio comparison | 16 ciliate MMETSP samples (825,773 sequences; 257,454 with similarity hits) | genetic code choice (standard translation table 1 vs ciliate nuclear translation table 6) | ratio of predicted ORF length to best similarity search hit length | TRAPID 2.0 pipeline / eggNOG 4.5 reference |
| Multiple sequence alignment | Two MMETSP0018 (Uronema sp. Bbcil) transcripts assigned to eggNOG orthologous group 0IF5I plus 4 Alveolata reference sequences | genetic code used for translation (standard vs ciliate) | MSA visual comparison of translated sequences | MAFFT; MSA R package (TEXshade) |
| Metatranscriptome assembly and expression quantification | Diatom-rich community metatranscriptomes, 3 western Antarctic Peninsula sampling sites (SRX727358, SRX727361, SRX727362) | sampling site (environmental) | assembled contigs, TPM expression values, expressed transcript subsets | Trimmomatic, FastQC, MIRA v4.0.2, BWA mem v0.7.17, FeatureCounts (subread v1.6.2) |
| Subset functional enrichment analysis | Diatom (Bacillariophyta)-assigned transcript subsets expressed per sampling site | sampling site comparison | enriched functional annotation labels (hypergeometric test, BH-corrected q-value) | TRAPID 2.0 with pico-PLAZA 3.0 reference |
- – Splitting Kaiju's reference index for memory efficiency yields nearly identical classification performance to a non-split run.
- ▲ Using the ciliate nuclear genetic code improved the best hit recovery ratio compared to the standard genetic code for ciliate transcriptomes.
- – Global metatranscriptome assembly from 526,527 combined reads produced 53,569 contigs (N50 = 391 bp); 208,592 reads unused by MIRA, of which 89,739 sequences >200 bp were retained as bona fide transcripts. N50=391 bp
- – Final combined metatranscriptome dataset used for analysis contained 143,308 sequences with N50 of 320 bp. N50=320 bp
- – Functional enrichment analysis of diatom-specific transcript subsets revealed functional variation between the three phytoplankton communities across sampling sites.
- count 825,773 sequences (total transcripts across 16 ciliate MMETSP samples processed)
- count 257,454 transcripts (transcripts with protein similarity search hits used for best hit recovery ratio computation)
- other ≥90% of species in clade (threshold for a gene family to be considered 'core' for completeness analysis)
- other ≥50% of GF members (majority vote) (threshold for transferring functional annotation labels from gene families)
- count 526,527 reads (total retained reads combined from 3 Antarctic sampling site libraries for assembly)
- count 53,569 contigs, N50 391 bp (MIRA global metatranscriptome assembly output)
- count 143,308 sequences, N50 320 bp (final metatranscriptome dataset (contigs + retained singletons) used for downstream analysis)
- other TPM ≥ 2 (threshold to consider a transcript expressed in a given sampling site library)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
TRAPID 2.0 is a bioinformatics software paper presenting a web application for de novo transcriptome annotation and exploration. Statistical methods appear in three contexts: (i) hypergeometric tests with Benjamini–Hochberg FDR correction for subset functional enrichment analysis; (ii) bootstrap-based branch support estimation in phylogenetic tree inference; and (iii) a descriptive ratio metric (best hit recovery ratio: predicted ORF length / best similarity-search hit length) used to compare the impact of standard versus ciliate nuclear genetic codes on ORF prediction across 257,454 sequences. Assembly quality and expression levels are reported as N50 lengths and TPM values, respectively, with no inferential summary statistics accompanied by dispersion measures in the main text.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Hypergeometric distribution test with Benjamini–Hochberg q-value correction | Subset functional enrichment analysis: functional annotation labels (e.g. GO terms) in a user-defined transcript subset tested against the whole-experiment background | not stated in main text (whole experiment used as background; subset sizes not given) | not stated |
| Ultrafast bootstrap approximation (UFBoot), 1000 replicates | Branch support estimation in IQ-TREE maximum-likelihood phylogenetic trees | 1000 replicates | not stated |
| Standard non-parametric bootstrap, 100 replicates | Branch support estimation in FastTree2, PhyML, and RAxML phylogenetic trees | 100 replicates | not stated |
| Best hit recovery ratio — descriptive metric, no formal significance test stated | Evaluation of standard (NCBI translation table 1) vs. ciliate nuclear (table 6) genetic code effect on ORF prediction quality, across 16 ciliate MMETSP samples | 257,454 transcript sequences with protein similarity search hits (from 825,773 total sequences across 16 samples) | na |
| TPM threshold (TPM ≥ 2) for binary expression calling — not an inferential test | Definition of expressed transcript subsets per diatom sampling site | 143,308 input sequences; 526,527 reads mapped across 3 libraries | na |
-
The best hit recovery ratio was compared descriptively between standard and ciliate nuclear genetic codes across 257,454 sequences, with no formal inferential test reported↳ Could also: A paired non-parametric test (e.g., Wilcoxon signed-rank test on per-transcript ratio differences) or a permutation test could also quantify this comparison — A formal test would yield a p-value and a standardized effect size, allowing readers to gauge whether the improvement attributable to the alternative genetic code exceeds chance variation across the large sequence set
-
Subset functional enrichment used the hypergeometric distribution with a user-supplied background (all transcripts in the experiment)↳ Could also: Fisher's exact test (one-tailed; mathematically equivalent to the hypergeometric test) or rank-based gene set enrichment analysis (GSEA) using a ranked TPM list could also be applied — Fisher's exact test is widely familiar and gives identical results; GSEA avoids the need to set a binary expression threshold by using the full ranked list, which can improve sensitivity for moderate but consistent enrichment signals
-
A fixed TPM threshold of ≥ 2 was used to dichotomize transcripts as expressed or unexpressed per sampling site↳ Could also: Count-based differential expression methods (e.g., DESeq2 or edgeR) could also model expression differences across sampling sites with statistical testing — Count-based methods explicitly model overdispersion in RNA-Seq count data and provide FDR-controlled p-values for differential expression between sites, whereas a fixed TPM threshold treats all above-threshold transcripts equivalently regardless of count-level uncertainty
-
Branch support in IQ-TREE was estimated using UFBoot (1000 replicates); other programs used standard non-parametric bootstrap (100 replicates)↳ Could also: Approximate likelihood-ratio tests (aLRT) or Bayesian posterior probabilities (e.g., via MrBayes) could also estimate branch support — aLRT is computationally cheaper than bootstrap resampling; Bayesian posterior probabilities have a direct probabilistic interpretation and can be preferable when bootstrap values are known to be overconfident in certain tree topologies
-
Core gene family status was defined by a fixed 90% species-representation threshold within a clade, with a weighted completeness score computed as a ratio↳ Could also: A sensitivity analysis across a range of thresholds (e.g., 80%, 95%) or the widely used BUSCO framework for universal single-copy ortholog completeness assessment could also be applied — Threshold sensitivity analyses would reveal how completeness scores change with stringency; BUSCO provides a community-standard benchmark that facilitates direct cross-study comparisons
-
Functional annotation labels were transferred to a transcript from its gene family using a 50% majority-vote threshold across GF members↳ Could also: Weighted voting (weights proportional to sequence similarity or phylogenetic distance) or probabilistic annotation transfer (e.g., PANNZER2-style scoring) could also be applied — Weighting by sequence similarity would give higher confidence to annotation labels supported by closely related family members, which may improve precision in large and functionally heterogeneous gene families
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
TRAPID 2.0 is a web-application descriptor paper; its reproducibility is limited by design, not by an authors' defect or fabrication. The one number we could check (C1: 526,527 reported vs 559,011 raw, +6.2%) deviates moderately with a benign explanation (raw vs Trimmomatic-trimmed). The headline case-study figures (C2–C7: assembly size, gene-family/taxonomy/GO percentages) are not derivable from shared data because the MIRA assembly was never deposited and annotation is web-server-only and non-deterministic. Net: solid-but-incomplete partial reproduction, deviations explainable and on the data-availability/our-methodology side — no significance flip or fabrication concern.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.