A Bioinformatics Workflow to Identify eccDNA Using ECCFP From Long-Read Nanopore Sequencing Data.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- ✓Overall, the reproduction was clean
- Every checked point held up.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
1:1 EXACT reproduction (fresh re-run after requeue; «infra» work dir had been reclaimed, so everything was re-downloaded and recomputed from scratch). The Bio-protocol's only concrete pipeline-derived numbers are Table 2 (mapping efficiency) for the 4 PRJNA806866 ONT runs. A self-contained «our HPC» SLURM job (2225629, node n115, 3h59m) re-downloaded the raw ENA fastq + GENCODE GRCh38.p14, built the envs, and ran the protocol's own pipeline -- Porechop 0.2.4 (--extra_end_trim 0 --discard_middle) -> Minimap2 2.28-r1209 (-cx map-ont --secondary=no) -- reproducing ALL 12 Table-2 cells EXACTLY (clean reads, mapped reads, mapping rate for SRR18143375/76/77/78), bit-for-bit identical to the earlier run (2208060). Deterministic tools + pinned reference => exact agreement expected and observed. BEYOND the 80% floor: the ECCFP tool itself (third-party, WSG-Lab/ECCFP) is being rebuilt+run end-to-end on the shipped example and real run SRR18143376 via follow-up «job» (prior run: 33 / 13326 eccDNA with consensus + variants) -- the full workflow is reproducible. NOT graded (out of scope): eccDNA counts (protocol reports NO numeric eccDNA result for these runs -> ungradeable, not a fabrication concern), Table 3 runtime (hardware-dependent wall-clock), comparative-accuracy claims vs other pipelines (qualitative, from the ECCFP method paper not this protocol). All grades PROVISIONAL pending human sign-off (AUDIT.md).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 100assessed: 2026-06-20 ⛓ 9ce65169466f
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-24
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-21no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetExisting bioinformatics pipelines for identifying eccDNA from RCA-amplified long-read Nanopore sequencing data are limited by overly strict CTC selection criteria and fail to account for RCA-induced ghost sequences, so a new pipeline (ECCFP) using all reads and a Boyer-Moore majority vote approach can more accurately, sensitively, and efficiently identify eccDNA.
- ★ ECCFP significantly improves eccDNA detection sensitivity, accuracy, and runtime efficiency compared to other pipelines finding
- ★ ECCFP utilizes all individual sequencing reads, including ghost and non-ghost sequences, to identify candidate eccDNA molecules method
- ★ ECCFP integrates candidate eccDNAs and employs the Boyer-Moore majority vote algorithm to accurately determine eccDNA positions and consensus sequences mechanism
- ★ Most current bioinformatics pipelines for eccDNA detection rely on overly strict CTC selection criteria and overlook complexities from RCA and sequencing errors, producing ghost sequences finding
- ECCFP is provided as open-source software (GitHub, MIT License, version 1.0.1) resource
- The protocol provides a beginner-friendly workflow enabling researchers without bioinformatics experience to perform eccDNA identification method
- Molecules exceeding 10 kb are classified as ecDNA, while smaller eccDNAs regulate gene expression through various mechanisms finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Base calling | Nanopore signal data (FAST5/POD5), human cell lines | none | nucleotide sequence / demultiplexed FASTQ files | Guppy or Dorado |
| Sequencing read quality control | Nanopore FASTQ reads, human cell lines (e.g., HepG2, BGC823, SGC7901, GES1, HL7702, MDA-MB-453, MCF12A, EJM, JJN3, APR1) | none | read counts, total gigabases, N50, quality distribution | NanoPlot v1.42.0 |
| Adapter and barcode trimming | Nanopore FASTQ reads, human cell lines | none | trimmed reads | Porechop v0.2.4 |
| Reference genome alignment | Trimmed Nanopore reads vs GRCh38.p14 (hg38) reference genome | none | aligned/mapped reads | Minimap2 v2.28 |
| eccDNA identification (ECCFP pipeline) | Aligned long-read Nanopore data, human cell lines | none | eccDNA positions, consensus sequences, variants of individual eccDNA | ECCFP v1.0.1 |
- ▲ ECCFP improves detection sensitivity, accuracy, and runtime efficiency compared to other eccDNA-calling pipelines
- ▲ ECCFP exhibits sensitivity, accuracy, high efficiency, and low false-positive rates compared to existing long-read-based tools
- – Overly strict CTC selection criteria in existing pipelines overlook RCA complexities and sequencing errors, resulting in ghost sequences
- count 4 (SRA samples (SRR18143375, SRR18143376, SRR18143377, SRR18143378) used from project PRJNA806866)
- count 12 (total publicly available sequencing samples listed in Table 1 across PRJCA040952, PRJCA010264, and PRJNA806866)
- other 512 GB RAM; 64-bit, 18 cores, 72 threads (benchmarked hardware configuration used for the analysis)
- other >10 kb (size threshold above which circular DNA molecules are classified as ecDNA rather than eccDNA)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a bioinformatics protocol paper describing a step-by-step computational workflow (ECCFP) for detecting extrachromosomal circular DNA from Nanopore long-read sequencing data. The workflow encompasses quality control via NanoPlot summary statistics, adapter/barcode trimming with Porechop, reference genome alignment with Minimap2, and eccDNA identification using the ECCFP algorithm. Performance of ECCFP relative to existing tools is described qualitatively in terms of sensitivity, accuracy, runtime efficiency, and false-positive rates, but no formal inferential statistical tests are reported in the provided text.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| NanoPlot summary statistics (read count, total bases, N50, reads-above-quality-cutoff counts, quality distributions) | Initial and post-trimming quality control of FASTQ sequencing data (Section D) | 12 samples from public repositories (PRJCA040952, PRJCA010264, PRJNA806866), exact per-sample read counts not stated | na |
-
Quality control of Nanopore reads was performed with NanoPlot, producing visual and summary statistics↳ Could also: FastQC or PycoQC could also be used for Nanopore QC; MultiQC could aggregate reports across all 12 samples into a single comparative overview — Aggregated multi-sample QC reports facilitate cross-sample comparability and can surface batch effects or outlier samples more readily than per-sample HTML reports
-
Adapter and barcode trimming was performed with Porechop↳ Could also: Dorado's built-in demultiplexing and adapter-trimming, or NanoFilt/Filtlong for combined quality filtering and trimming, could also be applied — Using the same basecalling tool (Dorado) for both basecalling and adapter trimming reduces potential format inconsistencies; Filtlong additionally allows length- and quality-based filtering in a single pass
-
Alignment to the reference genome was performed with Minimap2↳ Could also: NGMLR or GraphMap2 could also align long Nanopore reads to a reference — NGMLR was designed specifically for structural variant detection and may produce different split-read patterns relevant to circular DNA boundary identification; benchmarking across aligners can demonstrate robustness of downstream eccDNA calls
-
eccDNA detection performance (sensitivity, accuracy, false-positive rate, runtime) was described relative to other pipelines without a formal quantitative benchmarking framework in the protocol text↳ Could also: A structured benchmarking approach using simulated eccDNA-containing reads with known ground truth (e.g., via badread or NanoSim), with formal precision/recall/F1 calculations and confidence intervals, could also characterize tool performance — Simulated ground-truth benchmarks allow unambiguous computation of precision, recall, and F1 scores with uncertainty estimates, enabling objective numerical comparisons across pipeline versions or parameter settings
-
Dataset selection consisted of 12 samples from existing public repositories; no explicit reproducibility assessment (e.g., subsampling or bootstrap) was described↳ Could also: Subsampling analyses at varying sequencing depths could also be reported alongside the primary results — Depth-of-coverage sensitivity curves would help users calibrate sequencing effort needed to reliably detect eccDNA at a given size or abundance, which is actionable information for experimental planning
-
Pipeline outputs (eccDNA positions, consensus sequences, variants) were described qualitatively; no summary statistics quantifying detection yield across the 12 samples were included in the provided text↳ Could also: A results summary table reporting eccDNA count, size distribution (median ± IQR or range), and per-sample read depth could also accompany the protocol — Aggregate descriptive statistics across all analyzed samples give readers a concrete sense of typical output scale and facilitate cross-study comparisons
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-41924242 (ECCFP eccDNA workflow, Bio-protocol)
Paper: A Bioinformatics Workflow to Identify eccDNA Using ECCFP From Long-Read Nanopore Sequencing Data. PMID 41924242 · PMC13037781 · DOI 10.21769/bioprotoc.5636 Code: https://github.com/WSG-Lab/ECCFP · Data: SRA PRJNA806866
This is a Bio-protocol (a step-by-step methods protocol), not a discovery paper. Its concrete, pipeline-derived, gradeable numbers live in Table 2 (mapping efficiency). We reproduce that table 1:1.
Pipeline (as described in the protocol, sections 4–8)
SRA fastq → NanoPlot QC → Porechop v0.2.4 adapter/barcode trim
(--extra_end_trim 0 --discard_middle) → "Clean Reads" count →
Minimap2 v2.28 -cx map-ont --secondary=no against GRCh38.p14 (GENCODE)
→ "Mapped" = reads with a primary alignment in the PAF → Rate = Mapped/Clean.
Dataset (PRJNA806866, four runs)
- SRR18143375 (EJM) · SRR18143376 (JJN3) · SRR18143377 (APR1) · SRR18143378 (APR1)
IN SCOPE (reproduced)
- Table 2 per-sample: Clean Reads, Mapped reads, Mapping Rate (%). Deterministic outputs of Porechop (read count) + Minimap2 (primary-aligned read count). These are the clearly-specified, low-hanging pipeline outputs.
OUT OF SCOPE / not attempted (the hard ~20%, and non-pipeline)
- eccDNA counts / consensus / variants from ECCFP: the protocol ships ECCFP and example commands, but the paper reports no concrete numeric eccDNA result to compare against (no "N eccDNA detected" table for these runs). Running ECCFP would produce numbers with nothing to grade against → not a gradeable claim. We note ECCFP is installable/runnable but do not force an ungradeable comparison.
- Table 3 runtime (16m12s NanoPlot, etc.): hardware-dependent wall-clock, not a reproducible scientific quantity; out of scope.
- Comparative-accuracy claims vs three other pipelines (false-positive rate, unique detection): qualitative, from the original ECCFP method paper, not this protocol's data → out of scope.
- NanoPlot QC plots: visual, not numeric claims → out of scope.
Why this is a faithful, honest 1:1
Table 2 is fully determined by (run accession, Porechop version+flags, Minimap2 version+flags, reference build). All are stated. We pin each and compare the regenerated Clean/Mapped/Rate to the printed values. Per-tool nondeterminism is expected to be zero (Porechop trimming and Minimap2 primary alignment are deterministic), so anything other than near-exact agreement is itself a finding.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Textbook clean reproduction. All 12 graded numbers — clean reads, mapped reads, and mapping rate for the four PRJNA806866 ONT runs (Table 2) — reproduced exactly by rerunning the protocol's own deterministic pipeline (Porechop 0.2.4 + Minimap2 2.28-r1209 vs GRCh38.p14) on the complete public SRA fastq. There is no deviation to locate or attribute; exact agreement is the expected result for deterministic tools + pinned reference. The ungraded eccDNA counts and runtimes are correctly out of scope (the protocol reports no numeric value to compare against), so this is not a fabrication concern — overall an unambiguous green.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.