A consensus approach to vertebrate de novo transcriptome assembly from RNA-seq data: assembly of the duck (Anas platyrhynchos) transcriptome.
The main results reproduced, with only marginal, non-material deviations.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Reproduction of Moreton et al. 2014 (duck transcriptome, PMID 25009556) single-k-mer (SK) assemblies from a 30M-read-pair subsample of ERR515179. Oases SK (k=23) completed successfully end-to-end and produced an assembly of the same order of magnitude as reported (118,288 transcripts / 158.3 Mbp / N50=3,121bp vs. paper Table 2B's 78,640 / ~125Mbp / N50=3,144bp) - N50 in particular is within 1% of the paper's value, though transcript count and total length are both ~30-50% higher, plausibly reflecting the reduced (subsampled) input depth, assembler/library version drift since 2014, and/or the absence of a paper-side redundancy-filtering step not fully specified in the methods. ABySS SK (k=35) did NOT converge: across 3 independent SLURM jobs (8h00m05s + 12h00m04s + 12h00m13s = 32h00m22s cumulative wall-clock on up to 64 CPU cores at ~100% sustained utilization), the single monolithic ABYSS-P unitig-assembly stage never completed even once, and - critically - each resubmission restarted from absolute zero (re-reading FASTQ, reloading 289M k-mers) because raw ABYSS-P has no intra-stage checkpoint and the cluster's real per-job wall-time ceiling for this account is a hard 12h (SLURM QOS MaxWall, confirmed via sacctmgr, not a choice we made). This is reported as a genuine COMPUTE-DURATION limitation (partial/inconclusive), not a scientific reproduction failure: the underlying assembler algorithm and parameters were correctly configured and run at full available cluster scale, but the tool/infrastructure combination (no checkpointing x hard 12h ceiling) cannot complete this stage as configured. As an independent side-observation (not attributable to our reproduction in either direction): the paper's own Table 2B numbers do not match the summary statistics of the assemblies the authors themselves deposited on Figshore for either Oases SK or ABySS SK, suggesting Table 2B may reflect a post-processing/filtering step not fully documented in the methods. Overall verdict: PARTIAL reproduction - Oases SK qualitatively/order-of-magnitude reproduced (N50 near-exact), ABySS SK inconclusive due to a documented, hard compute-duration ceiling rather than a methodological mismatch.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-08-03
- Rubric version
- not recorded
- Assessed by
- —
- Last updated
- 2026-08-03
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusBecause no single tool or parameter set is clearly optimal for de novo transcriptome assembly of a vertebrate lacking a reference genome, the authors test whether a consensus pipeline — comparing three assemblers, single vs. multiple k-mer strategies, full vs. sub-sampled read sets, and merging assemblies across tools — yields a more robust transcriptome, using Pekin duck (Anas platyrhynchos domestica) RNA-seq as the test case.
- ★ Multiple k-mer (MK) assemblies are more complete than single k-mer (SK) assemblies, showing higher reads-mapped-back-to-transcripts (RMBT) and higher CEGMA complete-gene percentages for all three tools. finding
- ★ A random sub-sample of reads (30 million pairs) is sufficient to produce a robust assembly by multiple methods without requiring very high memory capacity, enabling rapid comparison of assembly characteristics. finding
- ★ RMBT and CEGMA together provide useful metrics for determining the completeness of a de novo transcriptome assembly. method
- ★ Merging single k-mer assemblies from different tools with CAP3 yields smaller assemblies composed of longer transcripts (higher N50, higher mean transcript length), indicating fewer fragmented transcripts. finding
- ★ A consensus pipeline is proposed comprising: comparison of three assembly tools, single and multiple k-mer approaches, evaluation of read-number influence, and cross-tool assembly merging. method
- ★ Merging six SK assemblies (two k-mer values per tool, including the largest k-mers) outperforms merging three SK assemblies on RMBT, CEGMA, N50, mean transcript length, and number of long transcripts. finding
- Among SK assemblies, Oases produced the highest total base pairs, mean transcript length, N50, RMBT percentage, and number of long transcripts, and assembled faster than ABySS; CLC was quicker but gave lower N50. finding
- ★ The duck transcriptomes generated by all compared assembly methods plus the final merged consensus assembly are released as a public resource (figshare doi 10.6084/m9.figshare.1032613; reads at ENA PRJEB6385). resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Illumina paired-end RNA-seq (poly-A selected mRNA library prep and sequencing) | Anas platyrhynchos domestica (Pekin duck) embryo fibroblasts grown in tissue culture | none | 100 bp paired-end sequence reads (412 million paired-end reads generated) | Illumina TruSeq RNA Sample Preparation kit; Agilent BioAnalyser 2100; Illumina cBot with HiSeq v3 flow cell; Illumina HiSeq 2000 |
| Read quality/adapter trimming, duplicate removal, and length filtering | Duck fibroblast RNA-seq reads | other (limit = 0.05, max 2 ambiguous nucleotides, minimum length 20 nt; reads kept if >50 bp) | Number of surviving read pairs and orphan reads at each processing stage | CLC Genomics Workbench Version 6 |
| De novo transcriptome assembly, single k-mer (SK) | All 277 million quality-filtered duck reads | other (k-mer sweep: odd k = 21–79 for Velvet-Oases; k = 25, 34, 62 for CLC) | Number of transcripts, total Mbp, mean transcript length, N50, number of transcripts >1 kb | Velvet v1.2.08 + Oases v0.2.08 (-short, -shortPaired, -ins_length 305, -min_trans_lgth 200); CLC de novo assembly tool; Ubuntu server 24 cores Xeon X5690 3.46 GHz, 192 GB RAM |
| De novo transcriptome assembly, single k-mer (SK) | Random sub-sample of 30 million post-quality-filtered paired duck reads | other (odd k = 21–79 for Oases and ABySS; every other odd k = 21–63 for CLC) | Number of transcripts, total Mbp, mean transcript length, N50, number of transcripts >1 kb | Velvet v1.2.09 + Oases v0.2.08; ABySS v1.3.5 (default parameters, 32 threads); CLC Genomics Workbench; CentOS server 32 cores AMD Opteron 6386SE 2.8 GHz, 192 GB RAM |
| De novo transcriptome assembly, multiple k-mer (MK) merge of SK assemblies | Duck reads (both all-reads and 30 million-pair sub-sample sets) | other (merge k = 27 default; Oases MK all-reads k = 33–79 since k = 21–31 failed; CLC MK all-reads k = 25, 34, 62) | Number of transcripts, total Mbp, mean transcript length, N50, RMBT, transcripts >1 kb, CEGMA score | Oases oases_pipeline.py (Oases-M) |
| Redundancy removal / clustering of assembled transcripts | Each duck de novo assembly | other (shorter transcripts sharing >99% identity removed; non-redundant sequences >200 bp kept) | Non-redundant transcript set used for all downstream statistics | cd-hit-est Version 4.6 |
| Reads mapped back to transcripts (RMBT) alignment validation | Duck assemblies; reads unselected in the random sub-sampling for sub-sample assemblies, entire read set for all-read assemblies | none | Percentage of reads aligning back to assembled transcripts | Bowtie2 Version 2.1.0, end-to-end mode |
| CEGMA core eukaryotic gene completeness assessment and CAP3 cross-tool assembly merging | Duck sub-sample assemblies (3 SK: Oases k = 23, ABySS k = 35, CLC k = 25; 6 SK adding Oases k = 79, ABySS k = 79, CLC k = 61) | other (attempted 6-assembly merge of 3 SK + 3 MK failed from memory exhaustion; only robust contigs retained) | Percentage of 458 core eukaryotic genes complete/partial; number of CAP3 contigs and robust contigs, total Mbp, mean length, N50, RMBT | CEGMA tool (HMMs of 458 core genes); CAP3 VersionDate 12/21/07, default settings |
- ▲ CLC MK assembly of the sub-sample scored best on completeness metrics, with the highest RMBT and highest CEGMA scores of the individual assemblies RMBT 95.06%; CEGMA 94.8% complete / 99.2% partial
- ▲ MK assemblies outperformed the corresponding SK assemblies on RMBT for every tool in the sub-sample comparison Oases 90%→94%; CLC 87%→95%; ABySS 59%→87%
- ▲ MK assemblies outperformed SK assemblies on CEGMA complete-gene percentage for every tool in the sub-sample comparison Oases 87.5%→92.7%; CLC 79.8%→94.8%; ABySS 33.1%→83.5%
- ▼ The ABySS SK assembly had markedly the lowest CEGMA and RMBT scores of all assemblies RMBT 59%; CEGMA 33.1% complete / 65.3% partial
- ▲ Merging six SK assemblies gave better metrics than merging three SK assemblies RMBT 77%→84%; N50 4006→4155 bp; mean length 2463→2689 bp; transcripts >1 kb 17,101→18,104
- ▲ N50 values increased in the merged consensus assemblies relative to the best individual SK/MK assemblies merged maximum N50 4155 bp vs. individual maximum 3659 bp
- ▼ RMBT percentages were generally lower for merged assemblies than for individual SK and MK assemblies, as expected since only robust contigs were retained merged 77–84% vs. individual up to 95%
- – For ABySS sub-sample assemblies, N50 rose with increasing k-mer up to k = 55 and then deteriorated, attributed to k exceeding half the average read length peak at k = 55 (average read length 89 bp)
- count 412 million paired-end reads generated; 411,488,930 raw sequences in pairs (Illumina HiSeq 2000 100 bp run of duck embryo fibroblast library)
- count 277,000,683 reads after trimming, duplicate removal and >50 bp filter (274,607,074 in pairs + 2,393,609 orphans) (Table 1, effect of quality filtering; these reads used for the all-read assemblies)
- count 30 million paired reads (size of the random sub-sample used for the tool/k-mer comparison)
- other RMBT 95.06%; CEGMA 94.8% complete, 99.2% partial (best-scoring individual assembly, CLC MK on the 30 M sub-sample)
- other 81.9% of core genes complete, 89.1% present in partial form (CEGMA analysis of the final 6 SK merged consensus assembly)
- count 1,208,328 transcripts, 2601 Mbp, mean 2153 bp, N50 3487 bp, RMBT 96%, 763,694 transcripts >1 kb (Oases MK assembly using all reads (Table 2A), the largest assembly produced)
- count 40,805 CAP3 contigs, 24,834 robust contigs, 67 Mbp, mean 2689 bp, N50 4155 bp, RMBT 84%, 18,104 transcripts >1 kb (final 6 SK merged consensus assembly (Table 3))
- count 458 core genes; contigs filtered at >200 bp and >99% identity (CEGMA HMM gene set and cd-hit-est/length thresholds applied before all statistics)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a methods/benchmarking study comparing de novo transcriptome assemblies produced by different tools (Velvet-Oases, CLC Genomics Workbench, ABySS) and assembly strategies (single k-mer vs. multiple k-mer, all reads vs. a random 30-million-read sub-sample). Assemblies were evaluated using descriptive quality metrics (number of transcripts, total assembly length, mean transcript length, N50, percentage of reads mapped back to transcripts (RMBT), and CEGMA core-gene completeness) reported as single point values per assembly in summary tables, rather than through formal inferential hypothesis testing.
-
Assembly quality was assessed from one assembly per tool/k-mer/read-set combination, with metrics such as N50 and RMBT% reported as single values.↳ Could also: Generate multiple assemblies from independent random sub-samples or bootstrap resamples of the reads and report the resulting variability (e.g., mean and SD/range) of N50, RMBT%, and CEGMA scores — This would give a sense of how much assembly metrics fluctuate due to the random read-sampling process itself, complementing the single-assembly comparisons presented.
-
Differences between tools and k-mer strategies (e.g., MK vs. SK RMBT or CEGMA scores) were compared by inspecting table values without a formal statistical test.↳ Could also: Apply a statistical test (e.g., paired comparison across replicate assemblies, or a proportion test for RMBT percentages given known read totals) when replicate data are available — A formal test could help quantify whether observed differences exceed the variability expected from chance or resampling alone.
-
RMBT percentages are reported as point estimates for each assembly.↳ Could also: Report a confidence interval for the mapping rate (e.g., a binomial/Wilson interval based on the number of reads mapped and total reads considered) — Given the large but finite number of reads used, a confidence interval would convey the precision of the estimated mapping rate alongside the point value.
-
N50 was used as the primary summary metric of transcript length/contiguity for selecting a 'best' single k-mer assembly per tool.↳ Could also: Report additional distributional descriptors of transcript length (e.g., median length, length histograms, or N25/N75) alongside N50 — N50 can be disproportionately influenced by a small number of very long transcripts; complementary length metrics can give a fuller picture of assembly contiguity.
-
The 'best' single k-mer assembly for each tool was chosen using a heuristic rule (maximizing N50 while keeping total assembly length long).↳ Could also: Use a predefined multi-criteria composite scoring approach that combines N50, RMBT%, and CEGMA completeness with explicit weights — A composite, pre-specified scoring rule can make the trade-offs among competing quality metrics explicit and reproducible across future comparisons.
-
CEGMA completeness was reported as a single complete/partial percentage per assembly.↳ Could also: Use a tool that also reports per-gene detection across assemblies (e.g., BUSCO gene-by-gene tables), enabling a gene-level paired comparison between assemblies — Gene-level detection data would allow assessment of which specific core genes are gained or lost between assembly strategies, beyond an aggregate percentage.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.