E3RC: A step-by-step computational protocol for exploring enhancer RNA expression and regulation using conventional RNA-seq data.
The main results reproduced, with only marginal, non-material deviations.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
REPRODUCED (qualitative). STAR Protocols method paper E3RC; real code github.com/bioinfo202408/E3RC HEAD 090f73b (BRIEF code link was a link-mining false positive). The paper prints NO numeric results (its demonstration figures carry no axis numbers), so grading is shape/peak-ranking, not 1:1 numeric. Ran the FULL E3RC pipeline end-to-end on the paper's OWN data (GSE98150, all 18 SRR runs from ENA) on «our HPC», repo scripts run VERBATIM where they parse. RESULTS: DP1 enhancer annotation (module 1) = 725100 enhancers, with 3 documented repo bugs faithfully carried (abs_summit missing col; blacklist filter column-swap removes only 297; gene filter chr-prefix mismatch + shipped GFF is the Ensembl Regulatory Build w/o gene features). DP2a = 12129 bidirectional eRNAs identified (verbatim identify_bidirectional_enhancers.R). DP2b (headline Fig 2A) = per-stage expressed-eRNA counts MII 3283 / 2-cell 8067 / 4-cell 7638 / 8-cell 4809 / Morula 6070 / ICM 6835: the 2-cell stage is the CLEAR MAXIMUM, reproducing the paper's central claim that eRNA number peaks at the 2-cell stage (bar plot regenerated by the repo's own plot_eRNA_number.R, artifacts/eRNA_number_Fig2A.png). Data-handling note: a prior duplicate-job race left 2 samples (SRR5479527, SRR5479529) silently failed (truncated ENA download + single-end trim) because the repo-style map script was not fail-fast; caught via samtools quickcheck of all 18 BAMs, re-ran hardened, all 18 valid. Deviations (all to make verbatim scripts RUN, not to alter results): substituted real Ensembl v102 gene GTF for the q95 background (shipped GFF has no genes); stripped a deeptools-version header off-by-one cell in the matrix label row; computed the per-stage matrix over the 12129 identified eRNAs (the R script's rowname assignment requires it). NOT attempted: wet-lab/manual steps and downstream Fig 2B/Fig 3 TF-eRNA network panels (out of the core in-scope 80/20 floor; the central Fig 2A claim is the headline result and it reproduced). All grades PROVISIONAL; a human reviewer signs off.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 62assessed: 2026-06-21 ⛓ edde7280b7c0
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-21
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-21no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetEnhancer RNA (eRNA) expression and regulation, conventionally studied via nascent RNA-seq, can instead be comprehensively identified, quantified, and regulatory-network-mapped using conventional (non-nascent) bulk RNA-seq data, for which no comprehensive tool previously existed.
- ★ E3RC is a computational framework for identifying and quantifying eRNAs and characterizing their expression and transcriptional regulation using conventional RNA-seq data. resource
- ★ Prior to this work, no comprehensive tool existed for identifying and quantifying eRNAs and delineating their upstream and downstream regulatory networks. finding
- ★ E3RC comprises five main modules: constructing enhancer annotation, preprocessing RNA-seq data, identifying/quantifying eRNAs, constructing TF-eRNA regulatory networks, and building eRNA-target regulatory networks. method
- ★ A comprehensive enhancer annotation is built by integrating eRNAbase, FANTOM5, Ensembl, and eRNAFinder databases with multi-step filtering against blacklist regions and known genes (UCSC refGene, NCBI RefSeq, GENCODE). method
- E3RC is demonstrated using a public mouse preimplantation embryo RNA-seq dataset (GSE98150) spanning MII oocyte through inner cell mass stages. method
- E3RC is versatile and applicable to other biological contexts beyond the demonstrated mouse embryo dataset. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| bulk RNA-seq | mouse preimplantation embryos (MII oocyte, 2-cell, 4-cell, 8-cell, morula, ICM stages) | none (developmental stage comparison) | eRNA expression/quantification across developmental stages | — |
| computational enhancer annotation integration/filtering | mouse genome (mm10/GRCm38) | none | non-redundant enhancer region catalog | — |
| TF motif enrichment analysis (AME) | eRNA regulatory regions, mouse | none | upstream transcription factor binding predictions for eRNAs | MEME Suite AME v.5.5.7 |
- – For the 8-cell stage, replicates 1-2 were used due to consistent transcriptional profiles, while replicate 3 showed a distinct pattern and was excluded.
- – The complete E3RC workflow executes within 16 hours on recommended hardware specifications. 16 h
- count 18 SRA samples across 6 developmental stages (MII oocyte, 2-cell, 4-cell, 8-cell, morula, ICM) (GSE98150 mouse embryo RNA-seq dataset used to demonstrate E3RC)
- other 16 hours (runtime of complete E3RC workflow on 8 CPU cores/16GB RAM recommended system)
- count 5 modules (number of main modules comprising the E3RC pipeline)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a computational protocol (STAR Protocols) describing E3RC, a bioinformatics pipeline for identifying and quantifying enhancer RNAs (eRNAs) from conventional RNA-seq data, applied as a worked example to a public mouse preimplantation embryo dataset (GSE98150) across six developmental stages. The protocol focuses on data processing steps (read trimming, alignment with HISAT2, duplicate removal, BAM merging/quantification), annotation integration/filtering, motif enrichment (MEME Suite AME) for TF-eRNA network construction, and expression correlation analysis for eRNA-target networks. Results are described as being visualized (e.g., heatmaps, clustering, correlation plots) rather than reported through explicit formal hypothesis-testing statistics in the provided text.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Motif enrichment analysis (MEME Suite AME) | Construction of TF-eRNA regulatory network (identifying upstream TFs of eRNAs) | — | not stated |
| Expression correlation analysis (specific coefficient not named) | eRNA-target regulatory network construction, described as 'plots the expression correlation' | — | not stated |
-
For the 8-cell stage, replicate 3 was excluded from analysis based on the observation that it 'showed distinct patterns' relative to replicates 1-2.↳ Could also: A quantitative outlier-detection approach (e.g., pairwise correlation thresholds, PCA/hierarchical clustering of replicates with a pre-specified distance cutoff) could also be used to identify and justify replicate exclusion. — A pre-specified quantitative criterion makes replicate-exclusion decisions explicit and reproducible across users applying the pipeline to their own data.
-
Biological replicates for each developmental stage are merged (BAM files combined) prior to eRNA quantification and downstream visualization.↳ Could also: Replicates could also be kept separate and modeled with a count-based framework (e.g., DESeq2 or edgeR treating replicate as a factor) to estimate per-stage dispersion. — Retaining replicate-level information allows variance across replicates to be estimated directly, which supports computing confidence intervals or significance values for stage-to-stage expression differences.
-
eRNA-target gene relationships are summarized by 'plotting the expression correlation' without naming a specific correlation coefficient in the provided text.↳ Could also: Reporting a specific coefficient such as Pearson's r (for linear association) or Spearman's rho (for rank-based/non-linear association), together with the associated p-value or confidence interval, could also be used. — Naming the coefficient and providing an interval/p-value gives readers a standardized way to judge the strength and reliability of the eRNA-target association shown in the plot.
-
TF-eRNA regulatory links are inferred via sequence-based motif enrichment using MEME Suite AME.↳ Could also: Complementary approaches such as HOMER motif enrichment, or orthogonal validation with TF ChIP-seq/ATAC-seq footprinting data, could also be used alongside AME. — Combining sequence-based motif enrichment with independent binding evidence can corroborate predicted TF-eRNA regulatory relationships from more than one angle.
-
eRNA expression is compared descriptively across six developmental stages without a stated formal statistical test for differential expression in the provided text.↳ Could also: A count-based differential expression framework (e.g., DESeq2 Wald test or limma-voom) with multiple-testing correction (e.g., Benjamini-Hochberg FDR) could also be applied to assign significance and FDR-adjusted p-values to stage-wise eRNA expression changes. — This would let users of the pipeline attach statistical significance and control for multiple comparisons when many eRNAs are tested simultaneously across stages.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-40716058 (E3RC protocol)
Paper: Yu H et al. "E3RC: A step-by-step computational protocol for exploring enhancer RNA expression and regulation using conventional RNA-seq data." STAR Protocols 2025. PMID 40716058 · PMCID PMC12318280 · DOI 10.1016/j.xpro.2025.103989.
Type: STAR Protocols method paper. Its "results" are expected-outcome figures, not printed numbers. This shapes what can be graded.
Code & data (resolved — BRIEF link was junk)
- BRIEF.md listed
github.com/git-guides/install-git— that is a link-mining false positive (an "install git" how-to cited in the protocol's prerequisites), NOT the code. Real code: https://github.com/bioinfo202408/E3RC (public, no license file, default branchmain, HEAD090f73b20d1f2e648d8fdb1ad06676436f759fcf, committed 2025-05-30). Zenodo archive: https://doi.org/10.5281/zenodo.15727536. - Pinned conda env shipped as
E3RC.yml(R 4.1, Python 3.6.15, HISAT2 2.1.0, Trim Galore 0.6.5, Picard 2.18.2, samtools 1.15.1, deepTools 3.3.2, MEME 5.5.7, bedtools 2.30, DESeq2 1.32). Repo also ships the enhancer annotation source files andpicard.jar. - Example data: GEO GSE98150 (Wang/Gao lab, mouse early-embryo RNA-seq subset of
the H3K9me3 study). RNA-seq samples used by the protocol (SRA SRR ids):
- MII oocyte: SRR5479518, SRR5479519 (2 rep)
- 2-cell: SRR5479520–SRR5479523 (4 rep)
- 4-cell: SRR5479524–SRR5479527 (4 rep)
- 8-cell: 3 rep, protocol uses rep 1–2 (SRR5479528, SRR5479529)
- Morula: SRR5479531, SRR5479532 (2 rep)
- ICM: SRR5479533–SRR5479536 (4 rep)
Pipeline modules (from Methods + repo scripts/)
- Enhancer annotation construction —
scripts/constructe_annotation_scripts/: format FANTOM5 / eRNAbase / eRNAFinder BEDs + Ensembl mm10 v102 GFF,merge_enhancers.py(merge summits ≤100 bp, keep length >146 bp, std chromosomes), thenfilter_enhancers_overlap_blacklist.py(mm10 ENCODE blacklist v2) andfilter_enhancers_overlap_knowngenes.py(±1 kb of Ensembl genes). → final enhancer set. - QC + mapping — Trim Galore (paired, q20, --illumina) → HISAT2 mm10 (
--dta) → keep NH:i:1 unique reads → Picard MarkDuplicates (REMOVE_DUPLICATES) → sorted/indexed BAM. - eRNA identification + quantification — merge stage replicates, BAM→BigWig (deepTools), bidirectional-transcription scoring at the 95th percentile of background + TPM>0.001 → per-stage eRNA expression matrix.
- TF–eRNA regulatory network — MEME AME motif enrichment (adj p<0.01) + Pearson r>0.7, FDR<0.01 co-expression.
- eRNA–target-gene network — top-5% r*FDR pairs, genomic distance <1 Mb.
In scope (pipeline-derived, attempted)
- DP1 — Enhancer annotation count (module 1). Fully deterministic; depends only on shipped annotation files; no GSE download, light compute. We reproduce the final count. ⚠ The paper prints no enhancer count → no reported value to match; we report the derived value as an auditable artifact and grade partial (pipeline reproduced, no printed target).
- DP2 — Expressed-eRNA count per developmental stage (Figure 2A) (modules 2–3). The
headline computational claim: "the number and abundance of eRNA peaked at the 2-cell
stage."
plot_eRNA_number.Rcounts eRNAs with expression >0 per stage column. We reproduce the per-stage count vector and test the 2-cell peak (shape match — the paper gives no per-stage numbers, only the bar plot).
Out of scope / not attempted (80/20 — the hard last 20%)
- Modules 4–5 (TF–eRNA and eRNA–target-gene networks, Figure 3): depend on the full module 1–3 output plus motif DBs; many tunable thresholds; no printed value to grade against. Deferred — documented, not attempted.
- Exact figure aesthetics (colors, JS-divergence specificity, single-eRNA ENH019874 expression panel): visualization, not a gradable computational quantity.
Gradability note
Because the paper reports no numeric expecte
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
STAR Protocols method paper (E3RC) that prints no numeric results, so all grading is qualitative/shape. The full pipeline ran end-to-end on the authors' own data (GSE98150, all 18 SRR runs) and reproduced the headline Fig 2A claim — eRNA number peaks at the 2-cell stage (8067, clear max). Deviations are on our/technical side (substituted Ensembl v102 GTF, stripped a header cell) and three carried repo code bugs, all needed to make verbatim scripts run rather than to change results. Reproduction is solid but inherently qualitative (q2 yellow, q8 yellow); the core conclusion holds and values are derivable, with no fabrication signal.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.