RNAdetector: a free user-friendly stand-alone and cloud-based system for RNA-Seq data analysis.
The main results reproduced: recomputed values matched the published ones within tolerance.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Reproduced RNAdetector's small-ncRNA case study (PMID 34082707) by running the AUTHORS' OWN tool (docker alaimos/rnadetector:v0.0.3 via Apptainer) + their Human_hg38_genome reference on the paper's own data SRP183064 (12 miRNA-Seq runs). Pipeline: TrimGalore->HISAT2(hg38)->featureCounts->metaseqR(edgeR+limma,Simes,qvalue). DE counts reproduce WITHIN ~4-5%: C1 410 vs 426 (p<0.05), C2 373 vs 357 (FDR<0.05); class breakdown 319 miRNA / 87 snoRNA (EXACT) / 3 tRF / 1 tsRNA (EXACT) vs reported 337/87/1/1. Near-1:1 reproduction using the authors' own tool+reference+data; residual diffs = tool-version drift. C4 (MITHrIL 166 pathways) attempted next. No fabrication signs.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-18 ⛓ 12240309bfb9
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-24
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetExisting RNA-Seq analysis pipelines require technical expertise (command-line usage, dependency management, limited OS/species/ncRNA support) that creates a bottleneck for non-expert users in small research and biomedical laboratories, motivating the need for a comprehensive, flexible, easy-to-use, GUI-based, cross-platform tool.
- ★ RNAdetector is a free, cross-platform, user-friendly RNA-Seq analysis software usable locally (stand-alone) or in cloud environments via a GUI resource
- ★ RNAdetector supports quantification, normalization, and differential expression analysis of mRNAs and multiple ncRNA classes (miRNAs, piRNAs, snoRNAs, lncRNAs, t-UCRs, circRNAs, tRNA-derived ncRNAs) for human, mouse, and C. elegans finding
- ★ The pipeline offers flexible tool choice at each step (alignment via HISAT2/STAR/SALMON/BWA; quantification via HTseq/FeatureCounts/SALMON/CIRI2/CIRIquant; differential expression via DESeq2/edgeR/LIMMA or meta-analysis combination) method
- ★ RNAdetector integrates miRNA-sensitive topological pathway analysis using MITHrIL method
- ★ RNAdetector is distributed as a Docker container requiring no additional dependency installation, deployable as stand-alone desktop software (Windows, macOS, Linux) or on cloud providers (Google Cloud, Azure, AWS) via docker-compose or Kubernetes resource
- An offline interactive genome browser based on JBrowse 2 is integrated to visualize read coverage depth resource
- Additional/custom species can be analyzed by uploading user-provided FASTA genomes/transcriptomes and GTF/BED annotations method
- ★ RNAdetector fills the gap for labs lacking bioinformatics expertise, reducing reliance on outsourcing to bioinformatics facilities or expensive commercial software finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| small RNA-Seq (case study, public dataset SRP183064) | human samples (NCBI SRA dataset) | none | differential expression of miRNAs and pathway impact | RNAdetector (HISAT2, featureCounts, edgeR, limma, MITHrIL) |
- – Case study workflow completed trimming (Trim Galore), alignment to HG38 (HISAT2), read counting (featureCounts), normalization (edgeR), and differential expression (limma and edgeR) successfully
- – Meta-analysis combining limma and edgeR statistical significance was performed using Simes correction/combination method
- – MITHrIL pathway analysis using miRNA LogFC values identified impacted pathways
- pvalue FDR or adjusted p-values < 0.01 (threshold used to consider pathways impacted in MITHrIL pathway analysis of the case study)
- other Minimum: 6 cores, 16 GB RAM, 1 Tb hard drive; Recommended: 8+ cores, 32+ GB RAM, 2+ Tb hard drive (system requirements for stand-alone version of RNAdetector)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software description paper introducing RNAdetector, a GUI-based RNA-Seq analysis pipeline; formal inferential statistics are confined to a single case-study demonstration (SRP183064, small RNA-Seq). Differential expression was assessed with both limma and edgeR, their results were combined via the Simes correction and combination method as a meta-analysis, and downstream pathway impact was evaluated with MITHrIL using LogFC values, with significance defined as FDR < 0.01. No hypothesis-driven study design with pre-specified power or sample size is described, as the paper's purpose is to validate software functionality rather than to answer a biological question.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| limma (moderated empirical-Bayes linear model, internal test not explicitly named) | Differential expression analysis in the SRP183064 case study | — | not stated |
| edgeR (exact or likelihood-ratio test, internal test not explicitly named) | Normalization and differential expression analysis in the SRP183064 case study | — | not stated |
| Simes correction and combination method (p-value combination / meta-analysis) | Combining statistical significance across limma and edgeR results | — | not stated |
| MITHrIL topological pathway perturbation scoring | Pathway analysis using LogFC values from differential expression step; significance threshold FDR < 0.01 | — | not stated |
-
Differential expression results from limma and edgeR were combined using the Simes p-value combination method as a meta-analysis↳ Could also: Fisher's combined probability test, Lancaster's method, or a rank-aggregation approach (e.g., RobustRankAggreg) could also be used to synthesize results across tools — Different combination rules weight individual p-values differently and have varying power under correlation between tests; reporting which specific Simes variant was used (original Simes vs. Hommel) would help readers judge assumptions
-
Low-count features were removed by filtering out genes with zero counts across all samples before normalization↳ Could also: A count-based filtering threshold (e.g., CPM > 1 in at least k samples, as recommended in edgeR and DESeq2 vignettes) could also be applied — A minimum-expression filter retains more features than a zero-only filter while still removing unreliably quantified genes, potentially improving power by reducing the multiple-testing burden
-
Pathway significance was defined by a single FDR threshold of < 0.01↳ Could also: A threshold of FDR < 0.05 is also widely used; additionally, reporting the distribution of perturbation accumulation (Acc) scores alongside FDR would give readers a sense of effect magnitude — Combining a significance threshold with an effect-size measure (pathway perturbation score) allows distinguishing statistically significant but small-effect pathways from those with both strong statistical and biological signals
-
The case study uses edgeR for normalization (TMM) before applying limma for testing↳ Could also: DESeq2's internal normalization (median-of-ratios) with its Wald or LRT test could also serve as a single self-contained tool, as the paper itself offers it as an option — Using one tool's normalization with a different tool's testing framework is valid but introduces an extra step; an integrated single-tool workflow may be simpler to document and reproduce
-
The number of samples, biological replicates, and experimental groups in the SRP183064 case study are not described in the text↳ Could also: Reporting sample size, replicate structure, and group definitions would be standard practice even in a software demonstration context — These details let readers assess whether the case study represents a typical use case (e.g., small-n RNA-Seq, which is where tool choice and normalization assumptions matter most) and facilitate re-analysis
-
Effect sizes are conveyed only as LogFC values passed into pathway analysis; no dispersion measures (SD, SEM, CI) or volcano/MA-plot summaries of DE results are reported in the text↳ Could also: Reporting the number of differentially expressed features, a summary of LogFC distribution, and a measure of spread (e.g., IQR of LogFC) would also contextualize the scale of expression changes — Quantitative summaries of DE results, beyond a significance threshold, help readers evaluate whether the software produces biologically plausible output on the demonstration dataset
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-34082707 (RNAdetector)
Paper: La Ferlita et al. 2021, RNAdetector: a free user-friendly stand-alone and cloud-based system for RNA-Seq data analysis. BMC Bioinformatics. PMID 34082707.
RNAdetector is a software tool (Docker GUI + R/PHP backend) that wraps standard RNA-Seq pipelines. The paper's quantitative content is a case study demonstrating the small-ncRNA workflow on a public miRNA-Seq dataset.
Data
- Accession: SRP183064 (SRA) = GEO context: small/miRNA-Seq, human colorectal cancer.
- 12 single-end miRNA-Seq runs: 6 CRC tumour (CT3–CT8) + 6 adjacent-normal (CN3–CN8), paired by patient. (ENA filereport confirms 12 runs, read_count 10.7M–19.1M.)
RNAdetector internal pipeline (from paper Methods + repo)
- Trimming: Trim Galore
- Small-ncRNA quantification: alignment + counting against a curated small-ncRNA transcriptome (miRBase miRNAs, snoRNA, tRFdb tRNA-fragments, tsRNA, piRNA, …).
- Differential expression: DESeq2 (default), edgeR, limma.
- Pathway impact: MITHrIL algorithm.
In-scope pipeline-derived claims (to reproduce)
| id | reported | paper loc | pipeline |
|---|---|---|---|
| C1 | 426 DE small ncRNAs at p<0.05 (CRC vs normal) | Results / case study | quant + DESeq2 |
| C2 | 357 DE small ncRNAs at FDR<0.05 | Results / case study | DESeq2 |
| C3 | breakdown of C2: 1 tRF-3 (tRFdb-3033a), 1 tsRNA (ts-112), 87 snoRNAs, 337 miRNAs | Results | annotation join |
| C4 | 166 pathways significantly impacted (FDR<0.01) | Results | MITHrIL |
Out of scope (not attempted)
- The GUI/usability/cloud-deployment claims (wet-lab/manual/UX, not pipeline-derived).
- MITHrIL pathway claim (C4) is attempted only if the small-ncRNA DE list reproduces AND MITHrIL is runnable; MITHrIL is a separate Java tool with its own DB — flagged as harder, secondary.
Reproduction strategy
Per BRIEF rule 2 (P16): applying the described tools to the paper's own data is valid. Primary: run RNAdetector's actual small-ncRNA pipeline if its reference DB is obtainable; otherwise reproduce the DE small-ncRNA count with an equivalent standard pipeline (Trim Galore → align to small-ncRNA reference → count → DESeq2 at the paper's thresholds). The headline reproducible numbers are the DE counts (C1, C2) and the breakdown (C3).
CORRECTION (after reading paper Results + repo, this run)
The paper's case-study method is NOT plain DESeq2. Verbatim from Results/Methods:
- Trimming: Trim Galore
- Alignment: HISAT2 to human genome HG38 (repo default config says hg19, but the case study text says HG38; reference packages.json offers Human_hg38_genome).
- Quantification: featureCounts against the bundled small-ncRNA GTF annotation.
- Normalization: edgeR (TMM).
- DE statistics: edgeR + limma, combined via metaseqR/PANDORA Simes meta-analysis, keeping only ncRNAs DE by BOTH approaches. adjust.method = qvalue.
- Thresholds: 426 DE at p<0.05; 357 at FDR<0.05.
- Pathways: 166 significantly impacted at FDR<0.01 via MITHrIL.
- Reference DBs: miRBase (miRNA), GENCODE (snoRNA/lncRNA), tRFdb+tRFexplorer (tRNA-derived).
Reproduction approach (this run) — run the ACTUAL tool
Pull alaimos/rnadetector:v0.0.3 as an Apptainer SIF (bundles all tools + /rnadetector/scripts
- metaseqR + MITHrIL). Download
Human_hg38_genomereference package (HISAT2 index + sncRNA GTF) from https://rnadetector.s3.eu-west-1.amazonaws.com/Human_hg38_genome.tar.bz2. Download 12 fastqs from ENA. Run the repo's own scripts: trim_galore.bash -> hisat.sh -> featurecounts.bash -> harmonize -> de_analysis.R (metaseqR edger+limma, simes, qvalue) -> pathway_analysis.R (MITHrIL). Compare DE counts/breakdown (C1 426, C2 357, C3 breakdown) and pathways (C4 166).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.