A2TEA: Identifying trait-specific evolutionary adaptations.
The main results reproduced: recomputed values matched the published ones within tolerance.
- ✓Same input data as the authors
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Any deviation was negligible
- 🟡Reported values were only indirectly comparable
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
REPRODUCED (essentially 1:1). A2TEA is the authors' own Snakemake workflow (tgstoecker/A2TEA.Workflow v1.1.0, commit 4b3ea27). The paper is a methods/software paper that reports NO numeric results for its example dataset (figures are WebApp screenshots), so the reproduction target is the authors' OWN published reference object (Zenodo 7089608, MD5-verified intact). I ran the shipped 3-species Poaceae drought test config (Zenodo 7089022; 2000 prot/sp, 2M reads/sample, 16 RNA-seq units, 2 hypotheses) end-to-end on «our HPC» via the authors' docker container (--use-conda --use-singularity for the upstream STAR/kallisto->DESeq2 + OrthoFinder2 + expansion; then host conda without singularity for the per-OG MSA/tree checkpoint stage, since per-job container entry spawns thousands of squashfuse processes that the README itself warns about). Comparing my A2TEA_finished.RData vs the reference object: HOGs 762=762, expanded OGs 26=26 (hyp1) & 52=52 (hyp2), DEGs Oryza 580=580 & Zea 42=42 (Hordeum 7105 vs 7106, off by ONE gene at the padj boundary), species tree topology identical (Oryza basal, barley+maize sister), 478=478 seqs, HOG_DE 45535=45535 rows. Net: 5 claims EXACT, 2 within-tol, 1 partial (C1 has no stored reference value), 0 mismatch. No fabrication concern. Not attempted: Brassicaceae 2nd example, full PRJNA656507 data, CAFE5, WebApp.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 100assessed: 2026-06-19 ⛓ 00523843c75d
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-30
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetCombining comparative genomics analysis of gene family (orthologous group) expansion with cross-species differential expression data from stress experiments can reveal trait-specific evolutionary adaptations and prioritize candidate genes for crop improvement.
- ★ A2TEA integrates gene family expansion analysis with differential expression data across species to identify genes that were targets of evolutionary adaptation to a given stress/treatment mechanism
- ★ A2TEA.Workflow is a Snakemake pipeline that integrates orthologous group prediction, phylogeny, protein function annotation, and RNA-seq-based differential expression analysis into a single processing pipeline method
- ★ A2TEA.WebApp is an R Shiny web application that allows interactive exploration, filtering, highlighting, and export of A2TEA.Workflow results resource
- ★ The pipeline is independent of the underlying species and can be applied to any set of species, not only crops finding
- As of release v1.1, RNA-seq data is not required for every species, allowing species lacking treatment-specific expression data to still be included in the phylogenetic analyses method
- AHRD can optionally be run during the workflow to infer protein functional annotation (descriptions and GO terms) for species lacking user-provided functional information resource
- The workflow generates GeneTonic-compatible input files to support functional enrichment interpretation resource
- The WebApp includes a bookmarking system that lets users store, subset, and re-load selected genes/OGs as smaller standalone .RData files resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| RNA-seq differential expression analysis | user-provided plant species (e.g. Arabidopsis_thaliana) | control vs. treatment/stress | differentially expressed genes (DEGs) | STAR (alignment-based) or kallisto (pseudoalignment); DESeq2 for differential expression |
| Orthologous group (gene family) prediction | multi-species proteomes (protein FASTA per species) | none | orthologous groups (OGs) and expanded OGs per hypothesis | OrthoFinder |
| Phylogenetic analysis | multi-species protein sequences within orthologous groups | none | species tree and gene trees | STAG/STRIDE (via OrthoFinder) |
| Sequence similarity search / reciprocal best hits | OG genes and extended hits across species | none | BLAST/DIAMOND hit tables; additional related OGs | BLAST/DIAMOND |
| Multiple sequence alignment | genes within expanded OGs and additional OGs | none | MSA used as input for phylogenetic tree construction | — |
| Protein function annotation | protein sequences per species | none | human-readable functional descriptions and GO terms | AHRD |
| Gene Ontology (GO) term enrichment analysis | gene/OG sets selected in the WebApp | none | enriched GO terms | A2TEA.WebApp (GeneTonic-compatible output) |
- – Pipeline efficiency was demonstrated on real-world datasets
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a software tool article describing A2TEA, a Snakemake-based bioinformatics pipeline paired with an R Shiny web application that integrates comparative genomics (orthologous group/gene family expansion detection) with RNA-seq differential expression analysis (via DESeq2) and GO term enrichment. The article describes the statistical and computational methods built into the pipeline for general use rather than presenting the results of a specific hypothesis-driven experiment with its own reported statistics; users supply their own RNA-seq and genomic data, and the pipeline computes differential expression, gene family expansion, and enrichment statistics on those user-provided datasets.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| DESeq2 differential expression analysis (negative-binomial model, default Wald test) | Identification of differentially expressed genes (DEGs) between control and treatment RNA-seq samples per species | — | not stated |
| Gene Ontology (GO) term enrichment analysis | WebApp analysis tab for functional enrichment of gene sets/OGs | — | not stated |
| Set size comparisons and tests (unspecified test type) | WebApp analysis tab comparing orthologous group (OG) set sizes across hypotheses/species | — | not stated |
-
Differential expression is computed via DESeq2, which by default uses a negative-binomial generalized linear model with a Wald test for pairwise comparisons.↳ Could also: edgeR (quasi-likelihood F-test) or limma-voom — These are widely used alternative RNA-seq DE frameworks with different dispersion estimation and testing approaches; they could also be applied and may be preferred for particular sample sizes or study designs, and comparing tools is a common way to assess robustness of DEG calls.
-
The pipeline reports a 'number of significant differentially expressed genes (DEGs)' without the text specifying the significance threshold or multiple-testing correction used.↳ Could also: Explicit reporting of the adjusted-p-value method (e.g., Benjamini-Hochberg FDR, which is DESeq2's default) and chosen alpha threshold alongside DEG counts — Stating the correction method and cutoff explicitly alongside DEG counts would let readers directly gauge the false-discovery control applied when comparing DEG numbers across species or hypotheses.
-
GO term enrichment analysis is offered as a dedicated analysis tab in the web application.↳ Could also: Gene Set Enrichment Analysis (GSEA), a rank-based method using the full ranked gene list rather than a hard significance cutoff — GSEA can also detect coordinated, modest expression shifts across a gene set that might not individually pass a DEG significance threshold, complementing over-representation-style enrichment.
-
Orthologous group set-size comparisons across species/hypotheses are described as including 'tests,' without the specific test named in the text.↳ Could also: Fisher's exact test or the hypergeometric test for comparing counts/proportions between OG sets, with correction for multiple comparisons if many OGs or species pairs are tested — These are standard approaches for comparing category or set sizes and would also allow explicit p-values or confidence intervals to accompany the set-size comparisons shown in the app.
-
Gene family 'expansion' is defined operationally via user-set ratio/minimum-gene-count cutoffs between species groups rather than via a formal statistical model.↳ Could also: A likelihood-based gene family evolution model such as CAFE, which estimates gene birth/death rates under a phylogenetic model — A birth-death modeling approach could also provide formal p-values or likelihood ratios for lineage-specific expansion/contraction events, complementing the cutoff-based expansion definition used here and offering an alternative, model-based line of evidence.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-37224329 (A2TEA: Identifying trait-specific evolutionary adaptations)
- Paper: Stöcker et al., F1000Research 2022, 11:1137 (v2). PMID 37224329 · PMCID PMC10186066 · DOI 10.12688/f1000research.126463.2
- Code: https://github.com/tgstoecker/A2TEA.Workflow (MIT; v1.1.0 "No transcriptomes required", 2023-03-12; repo frozen, dev moved to groupschoof/A2TEA.Workflow)
- Type: Authors' own code (P16 not applicable) — a Snakemake workflow + R/Shiny WebApp.
What A2TEA is
A Snakemake pipeline that combines differential gene expression (RNA-seq: trimming →
STAR/kallisto → DESeq2) with comparative genomics (OrthoFinder2 orthogroups +
hierarchical orthogroups (HOGs) + gene-family expansion detection + per-OG MSA &
tree + optional CAFE5) and functional enrichment (topGO/GO), and integrates them to
prioritise expanded, stress-responsive gene families. Single output object:
tea/A2TEA_finished.RData containing nested objects: HYPOTHESES.a2tea (S4 per
hypothesis), HOG_level_list, HOG_DE.a2tea (DESeq2 results), A2TEA.fa.seqs,
SFA/SFA_OG_level, all_speciesTree.
Reproduction strategy (THE key decision)
The paper reports NO specific quantitative numbers for the example dataset — it is a software/methods paper that directs readers to interactive exploration in the WebApp ("Try a demo A2TEA.RData file"). Figures 3–4 are screenshots of the example result object, not numeric tables.
Therefore the 1:1 reproduction target is the authors' own published reference result object, which IS what the paper's figures show:
- Reference output: Zenodo record 7089608, file
a2tea_poaceae_reduced_testdata.RData(2.4 MB, MD5596670c8d2119a42bd275ada9e10d82a), "Example result object of A2TEA.Workflow output investigating drought stress in Poaceae". - Reproduction: run the shipped test config (3 Poaceae species, drought) through
the workflow on «our HPC» → produce my own
A2TEA_finished.RData→ extract derived quantities and compare mine vs the authors' reference object (and vs the few exact facts the paper does state). This tests whether the shipped code + shipped test data reproduce the authors' shipped reference output — a clean, auditable reproduction.
Test dataset (shipped config; input = Zenodo 7089022 via get_test_data.sh)
Subsampled: peptide FASTA → 2000 proteins/species; reads → 2M/sample.
- Species (3):
Zea_mays(B73, SE reads),Hordeum_vulgare(Scarlett, PE),Oryza_sativa(Chilbo, PE). Functional annotation supplied as AHRD TSVs (so the AHRD submodule is NOT needed). - Samples (16 units): barley Scarlett 3 control + 3 treatment (PE); maize B73 3+3 (SE);
rice Chilbo 2 control + 2 treatment (PE). (per
config/samples.tsv) - Hypotheses (2):
- "Expanded in barley compared to rice and maize" — expanded_in=Hordeum_vulgare, compared_to=Zea_mays;Oryza_sativa, min_expansion_factor=2, min_expansion_difference=10, Nmin_expanded_genes=2, ploidy_normalization=YES.
- "Expanded in barley compared to maize" — compared_to=Zea_mays, factor=2, ploidy_norm=NO. Expansion defined as: N(genes species A) ≥ 2 × N(genes species B).
In scope (pipeline-derived, attempted)
| # | Result | Pipeline | Compared against |
|---|---|---|---|
| C1 | N OrthoFinder orthogroups (and N genes assigned) | OrthoFinder2 | reference RData |
| C2 | N HOGs analysed (HOG_level_list / HOG_DE.a2tea) |
OrthoFinder HOGs | reference RData |
| C3 | N expanded OGs, hypothesis 1 | A2TEA expansion calc | reference RData |
| C4 | N expanded OGs, hypothesis 2 | A2TEA expansion calc | reference RData |
| C5 | N DEGs (DESeq2, padj<.05) per species | STAR/kallisto→DESeq2 | reference RData |
| C6 | Species tree topology (3 taxa) | OrthoFinder STAG/STRIDE | reference RData + biology |
| C7 | N hypotheses = 2; expansion ≥2× | config/methods | paper (exact, definitional) |
| C8 | Per-OG MSAs + gene trees present for expanded OGs |
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.