COXPRESdb v8: an animal gene coexpression database navigating from a global view to detailed investigations.
The main results reproduced: recomputed values matched the published ones within tolerance.
- Nothing in this column.
- 🟡Could not use the authors’ exact input data
- 🟡Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH; PRIMARY RESULT REPRODUCES ~1:1. COXPRESdb v8 (NAR 2023) database paper, two pipeline components in scope. (A) CoexPub SciBERT binary sentence classifier — reproduced essentially exactly. No author training code was published, so this is a faithful re-implementation of the documented method (allenai/scibert_scivocab_uncased, lr=1e-5, epochs=3, GENEAAA/GENEBBB special tokens) on the exact open Zenodo 7069129 data (700 sentences, 300/300 train + 50/50 test, all QC checks pass). Test accuracy 0.85 hit on the nose (seed0; mean 0.854±0.010 over 5 seeds), confusion matrix 46/11/4/39 vs reported 46/10/4/39 — the ±1 is fully explained by the paper's own confusion matrix summing to 99 while the deposited test set has 100 sentences. Compute on «our HPC» std node («job», 41 min). (B) function_score.pl partial-AUC (Table 1 KEGG/GO scores) — third-party Perl pipeline verified RUNNABLE via smoke test on synthetic data keyed to the shipped dummy pathway/paralog files («job», rc=0), but the exact Table 1 values are NOT reproducible: KEGG annotation is license-blocked (paid FTP), GO annotation is not shipped, and the exact COXPRESdb v8 coexpression matrices were not obtained — graded uncheckable, not a fabrication signal. NOT ATTEMPTED: database-scale deposit counts (build statistics, out of scope). No fabrication concern: every reproduced number is derivable from the shipped data/code. All grades PROVISIONAL — human reviewer signs off in AUDIT.md.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-19 ⛓ e20a9e4050c4
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-29
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThe paper asks whether adding new navigation and interpretation tools (CoexMap, pathway enrichment summaries, CoexPub) plus an integrated union-type coexpression measure can make the complex, dense gene coexpression information in COXPRESdb more accessible and useful across overview, summary, and individual-gene-pair scales.
- ★ COXPRESdb version 8 adds CoexMap (UMAP-based genome-scale coexpression visualization), KEGG pathway enrichment summaries, and CoexPub (literature-linking tool) as new analysis features. resource
- ★ Union-type coexpression (average of RNAseq- and microarray-based z-scores) scores more consistently with KEGG and GOBP functional annotations than either individual platform, supporting its use as the default representative coexpression measure. finding
- ★ Interspecies similarity among coexpression platforms reflects evolutionary relationships among the 12 species analyzed. finding
- ★ The database recapitulates known biology: human TASL (CXorf21) shows strong coexpression with TLR7, consistent with prior experimentally validated colocalization in B cells. finding
- Newly added domestic cat coexpression data clusters closest to canine coexpression among the platforms, as expected from phylogeny. finding
- ★ CoexMap reveals that one-to-one orthologous (housekeeping) genes occupy a distinct dense region of the coexpression map compared to randomly selected genes. finding
- For species with very large sample numbers (human, mouse), coexpression is computed on partitioned subsets and integrated with a coverage-based shrinkage penalty to keep computation tractable. method
- Coexpression z-scores follow an approximately normal distribution except above a value of 3, suggesting z=3 as a threshold for biologically meaningful co-regulation. finding
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| RNAseq-based gene coexpression calculation | 12 animal species (human, mouse, rat, dog, cat, chicken, monkey, zebrafish, fly, nematode, budding yeast, fission yeast) | none | gene-gene coexpression z-scores (Mutual Rank of Pearson correlation) | Illumina RNAseq, quantified with Matataki |
| Microarray-based gene coexpression calculation | same 12 species | none | gene-gene coexpression z-scores | Affymetrix microarray, RMA method, Combat batch correction |
| Pathway/GO-based quality scoring of coexpression data | coexpression datasets per species/platform | none | KEGG score and GOBP score (consistency with functional annotation) | KEGG pathway database, Gene Ontology Biological Process |
| UMAP dimensionality reduction (CoexMap) | human gene coexpression network | none | 2D visualization of gene positions in coexpression space | UMAP |
| Literature/text-mining association (CoexPub) | PubTator Central gene-publication corpus | none | machine-learning-ranked sentences linking a gene pair | PubTator Central, machine learning model |
| Comparative orthologous-gene coexpression analysis | 656 one-to-one orthologous genes across 12 species | none | Pearson correlation coefficients between coexpression platforms | OrthoFinder |
- ▲ Union-type coexpression consistently scored higher on KEGG/GOBP consistency metrics than RNAseq or microarray coexpression alone.
- ▲ TASL (CXorf21) shows strongest coexpression with GAPT and TLR7 in human union-type coexpression data. z=9.8 (GAPT), z=9.0 (TLR7)
- – CoexPub identified five publications describing a functional relationship between TASL and TLR7, including the original colocalization study. n=5
- – Newly added cat RNAseq coexpression (Fca-r) clusters closest to dog coexpression in the platform similarity matrix.
- – About 40% of all gene pairs lack microarray coexpression values, causing union-type coexpression to resemble RNAseq coexpression more closely. ~40%
- – One-to-one orthologous genes cover the dense, housekeeping-associated region on the left side of the CoexMap, distinct from randomly selected genes.
- – Coexpression z-scores are approximately normally distributed except above a value of 3. threshold z=3
- count 656 one-to-one orthologous genes (orthologs identified across 12 species via OrthoFinder)
- count 214,840 gene pairs (gene pairs from orthologs used for cross-platform Pearson correlation)
- other z-score 9.8 for GAPT–TASL coexpression (strongest coexpressed gene with TASL in human union coexpression)
- other z-score 9.0 for TLR7–TASL coexpression (second-strongest coexpressed gene, consistent with prior colocalization study)
- count 5 relevant publications (CoexPub literature hits linking TASL and TLR7)
- other ~40% of gene pairs lack microarray coexpression values (explains similarity of union-type to RNAseq coexpression)
- count >200,000 RNAseq runs each for human and mouse (scale requiring partitioning into 7 subsets for computation)
- count 235,187 samples (Hsa-r.c6-0) (human RNAseq coexpression dataset sample count, Table 1)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a database resource paper describing COXPRESdb version 8, an animal gene coexpression database. The primary quantitative approach is computational/correlational rather than hypothesis-testing: gene coexpression values are derived from Mutual Rank of Pearson correlation coefficients across large public RNAseq and microarray datasets, aggregated via subsampling/bagging (Subagging) of principal components, standardized to z-scores, and validated against KEGG/GOBP functional annotation consistency (KEGG and GOBP scores). Results are reported as correlation matrices, hierarchical clustering of platforms, and dimensionality-reduction (UMAP) visualizations rather than classical group-comparison statistics.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Pearson correlation coefficient | similarity among the 35 coexpression platforms across 12 species using 214,840 one-to-one orthologous gene pairs (Figure 1) | 214,840 gene pairs from 656 one-to-one orthologous genes | not stated |
| Mutual Rank (MR) transformation of Pearson correlation coefficients, with logit transformation | core gene-to-gene coexpression value calculation for each gene pair | — | not stated |
| Hierarchical clustering (average linkage method) | clustering of the 35 coexpression platforms by similarity (Figure 1) | — | na |
| KEGG and GOBP consistency scoring (quality/enrichment scoring against pathway and GO annotations) | quality assessment of each coexpression dataset/platform (Table 1, Supplementary Figures S3-S4) | e.g. 1743 genes associated with informative KEGG terms | not stated |
| UMAP dimensionality reduction | CoexMap genome-scale visualization of coexpression space (Figure 3) | — | na |
-
Coexpression similarity between platforms was quantified using Pearson correlation coefficients on the coexpression z-scores.↳ Could also: Spearman or Kendall rank correlation — Rank-based correlation measures are also commonly used when the linearity or normality of the underlying relationship is uncertain, and can be less sensitive to outlier gene pairs.
-
Platforms were grouped using hierarchical clustering with the average linkage method.↳ Could also: Alternative linkage criteria (e.g. Ward's method, complete linkage) or other clustering approaches such as k-means — Different linkage/clustering choices can also be used to explore whether the same evolutionary/similarity groupings emerge under alternative clustering assumptions.
-
Batch effects across experiments were corrected using ComBat.↳ Could also: Other batch-correction methods such as surrogate variable analysis (SVA), RUV (remove unwanted variation), or limma's removeBatchEffect — These alternative approaches could also be applied to adjust for unwanted technical variation and may be preferred depending on whether batch structure is fully known or must be inferred from the data.
-
Genome-scale coexpression relationships were visualized using UMAP dimensionality reduction.↳ Could also: t-SNE or PCA-based visualization — These are commonly used alternative dimensionality-reduction techniques that could also convey global structure, with different trade-offs between preserving local versus global distances.
-
Reliability of coexpression estimates from subsets with fewer contributing experiments was addressed with a data-coverage shrinkage penalty rather than a formal statistical significance/multiple-testing framework.↳ Could also: An empirical or model-based false discovery rate (FDR) framework applied to gene-pair coexpression significance — An FDR-based approach could also be used to explicitly quantify and control the expected proportion of false-positive coexpression calls across the many gene pairs tested, complementing the shrinkage-based reliability weighting.
-
Coexpression data quality was benchmarked using KEGG pathway and GOBP annotation consistency scores.↳ Could also: Gene Set Enrichment Analysis (GSEA) style permutation-based enrichment testing — A permutation-based enrichment framework could also be used to generate a null distribution and associated significance estimates for how well coexpression neighborhoods recapture known pathway membership.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-36350658 (COXPRESdb v8, NAR 2023)
COXPRESdb v8 is a database update paper. Most of its content (web UI, DB build, data integration) is infrastructure, not a reproducible analytic pipeline. Two components are genuine bioinformatic pipelines with reported numeric results.
IN SCOPE
A. CoexPub text classifier (PRIMARY target)
- What: Fine-tuned SciBERT binary sentence classifier that decides whether a
sentence describes a functional relationship between two genes (the two target
genes masked as
GENEAAA/GENEBBB). - Data: Zenodo 10.5281/zenodo.7069129 "CoexPub learning datasets" — exact
train/test files:
train_positive_300.txt(300),train_negative_300.txt(300),test_positive_50.txt(50),test_negative_50.txt(50). One sentence per line. - Method (Methods §CoexPub): SciBERT pre-trained model,
learning_rate=1e-5,epochs=3. Target genes masked as GENEAAA/GENEBBB. Paper text also says "250 of 300 sentences were used for training and the rest for test" (slight inconsistency with the 300/50 file split — see AUDIT). - Reported result: test accuracy 0.85, confusion matrix 46 TP, 10 FP, 4 FN, 39 TN (sums to 99).
- Code availability: NO published CoexPub training script. Reproduction is a faithful re-implementation of the described method on the shipped data.
- Compute: SciBERT fine-tuning → 1 GPU SLURM job on «our HPC».
B. Function score / partial AUC (SECONDARY target)
- What:
function_score.pl(GitHub takeshiobayashi/coex-function-score) computes a partial AUC (FPR 0–1%, scaled ×10000) of a coexpression matrix against pathway annotation = the "KEGG score" / "GO score" columns of Table 1. - Reported (Table 1, examples): Human Hsa-r.c6-0 KEGG 6.105 / GO 2.800; Mouse Mmu-r.c6-0 KEGG 6.953 / GO 3.150.
- Inputs: (1) per-species coexpression matrix from COXPRESdb/ATTED-II bulk download; (2) pathway annotation — KEGG requires a paid FTP license (README), GO is open.
- Feasibility: Script runnable on shipped dummy data (smoke test). Matching exact Table 1 values requires the exact v8 coexpression matrices + KEGG (license-blocked). GO score is the only openly-reproducible variant; even then exact-value match is unlikely (matrix version, ortholog exclusion list). Treated as partial / smoke-test.
OUT OF SCOPE (not pipeline-reproducible)
- Database scale counts (species=12, RNAseq samples Human 235,187 / Mouse 214,753, microarray counts) — these are deposit/build statistics, reported only, recorded as claims but not re-derivable without the full DB build (out of scope to recompute).
- Web interface, network viewer, ortholog tables, cross-links.
Datasets to profile
- zenodo:10.5281/zenodo.7069129 — CoexPub learning datasets (4 text files).
- (pointer) COXPRESdb/ATTED-II bulk coexpression matrices — referenced for B, KEGG license-blocked; profiled as pointer only if not downloaded.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.