TransOrGAN: An Artificial Intelligence Mapping of Rat Transcriptomic Profiles between Organs, Ages, and Sexes.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓No relevant deviation in data/preprocessing
- ✓Any deviation was negligible
- 🟡Reported values were only indirectly comparable
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
Described well enough to reproduce the MODEL-INDEPENDENT results 1:1. The brief's code link (the-gan-zoo) is wrong; the authors' real repo is github.com/TingLi2016/TransOrGAN (commit 33a4629), which ships analysis scripts but NOT data or the trained model ('by request'). Data is public: GEO GSE53960 (Rat BodyMap). From it we rebuilt the 288-sample x 40064-gene matrix (9 sex-common organs), log2(x+1), and computed the all-pairwise cosine + RMSE among the REAL profiles -- exactly what the authors' control_cosine.py / mse.py call the 'control group'. Results: 288 samples and 40064 genes EXACT; control cosine 0.9399+/-0.0212 vs reported 0.940+/-0.021 (train)/0.939+/-0.020 (test) EXACT to 3 decimals; control RMSE 1.884+/-0.361 vs reported 1.900+/-0.340 within-tol. This confirms dataset, log2 preprocessing, and the baseline metric end-to-end with no fabrication concern. NOT attempted (the ~20%): the TransOrGAN generated-profile metrics (cosine 0.984 / RMSE 0.910) and the 87-pathway / sex-gene cosines -- these need the trained autoencoder+cGAN/CycleGAN whose weights are not shipped and whose retraining is non-deterministic, so they are not 1:1 reproducible here and are out of 80/20 scope.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 84assessed: 2026-06-15 ⛓ 8f1aa40b4a30
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-15
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan a generative adversarial network framework (TransOrGAN) accurately infer/map rat transcriptomic profiles between different organs, sexes, and developmental ages, thereby offering an alternative that reduces animal use in toxicology safety assessment?
- ★ TransOrGAN can infer transcriptomic profiles between any 2 of 9 organs with an average cosine similarity of 0.984 between synthetic and real profiles finding
- ★ TransOrGAN can infer female transcriptomic profiles from male profiles with an average cosine similarity of 0.984 finding
- ★ TransOrGAN can infer juvenile, adult, and aged transcriptomic profiles from adolescent animals with cosine similarities of 0.981, 0.983, and 0.989 respectively finding
- ★ A GAN framework combining conditional GAN and cycle-consistent GAN can translate transcriptomic profiles across organ/sex/age within a single model method
- A five-layer autoencoder reduces 40,064-gene transcriptomic profiles to a 128-dimensional representation used for GAN training and reconstruction method
- TransOrGAN provides an approach to reduce animal usage and enable integrated whole-organism toxicity assessment irrespective of sex or age resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| bulk RNA-seq transcriptomic profiling (rat BodyMap dataset) | Fischer 344 rat, 9 nonsex-specific organs (adrenal gland, brain, heart, kidney, liver, lung, muscle, spleen, thymus), both sexes, 4 developmental stages | none | gene expression of 40,064 AceView-annotated genes across 288 samples | GEO accession GSE53960 |
| GAN-based in silico organ-to-organ transcriptomic translation (TransOrGAN: cGAN + CycleGAN with autoencoder) | rat transcriptomic profiles (9 organs, 2 sexes, 4 developmental stages) | computational condition mapping (source vs target organ/sex/age) | cosine similarity, RMSE, and UMAP between generated and real profiles | Adam optimizer; autoencoder (40064-2048-128) deep learning framework |
| toxicity pathway-related gene concordance analysis | rat transcriptomic profiles restricted to toxicity-related pathway gene sets | none | cosine similarity within 87 toxicity-related pathways and GO0009410 gene set between generated and real profiles | CTD database; DAVID gene ID conversion tool |
| sex hormone-related gene concordance analysis | rat transcriptomic profiles restricted to 19 sex hormone-related genes | none | cosine similarity of sex hormone gene expression between generated and real profiles | — |
- – Synthetic vs real transcriptomic profiles across any 2 of 9 organs cosine similarity 0.984
- – Inference of female profiles from male profiles cosine similarity 0.984
- – Inference of aged-animal profiles from adolescent animals cosine similarity 0.989
- – Inference of adult-animal profiles from adolescent animals cosine similarity 0.983
- – Inference of juvenile-animal profiles from adolescent animals cosine similarity 0.981
- – Optimized generator selected at training epoch with minimum generator loss 8576 epochs
- mean cosine similarity 0.984 (average organ-to-organ synthetic vs real profile similarity)
- mean cosine similarity 0.984 (male-to-female inference)
- mean cosine similarity 0.981 / 0.983 / 0.989 (adolescent-to-juvenile/adult/aged inference)
- count 288 transcriptomic profiles, 40,064 genes (rat BodyMap samples used)
- count 64,800 pairwise transcriptomic profiles (total organ pairs across sex/developmental stage)
- count 63,504 training / 1296 test pairwise profiles (train-test split (252 vs 36 rats))
- count 135,783 gene-pathway relationships, 11,584 genes, 2352 pathways (CTD download; 87 toxicity-related pathways retained)
- count 19 of 36 sex hormone-related genes shared (sex hormone gene analysis)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
TransOrGAN is a generative adversarial network (GAN) framework trained and evaluated on rat RNA-seq data from 288 samples across 9 organs, 2 sexes, and 4 developmental stages. Model fidelity was assessed primarily through cosine similarity and RMSE between generated and real transcriptomic profiles, supplemented by UMAP for visual inspection of separation. A t-test was used to compare TransOrGAN output similarity against a background distribution of pairwise correlations from real transcriptomic data. Results were reported as average cosine similarity values across the test set.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| t-test (type—one- vs two-tailed, paired vs unpaired—not specified) | Comparison of TransOrGAN-generated profile similarity scores vs. background distribution of pairwise correlations of real transcriptomic data (Figures 5–7) | — | not stated |
| Cosine similarity (evaluation metric, not inferential test) | Generated vs. real transcriptomic profiles across all organ, sex, and age translation tasks; also applied to 87 toxicity-related pathway gene sets and 19 sex hormone-related genes | 1296 pairwise profiles in test set | na |
| RMSE (evaluation metric, not inferential test) | Quantifying magnitude of difference between generated and real transcriptomic profiles in the test set | 1296 pairwise profiles in test set | na |
-
Model accuracy was summarized as average cosine similarity across test-set profile pairs, with no dispersion measure reported↳ Could also: Report mean ± SD or 95% CI for cosine similarity across the 1296 test-set pairs, or provide per-condition breakdowns — A dispersion measure would reveal whether high average performance is uniform across all organ/sex/age translation tasks or driven by a subset, which is informative for understanding model reliability
-
A t-test was used to compare TransOrGAN output to the background distribution of real pairwise correlations, but the variant (one- vs two-tailed, paired vs unpaired) was not specified↳ Could also: A non-parametric alternative such as the Mann-Whitney U test, with full specification of directionality and pairing — Cosine similarity distributions over high-dimensional genomic data may depart from normality; a non-parametric test makes no distributional assumption, and full specification supports reproducibility
-
Cosine similarity was the primary fidelity metric for comparing generated and real transcriptomic profiles↳ Could also: Pearson or Spearman correlation coefficient between generated and real gene-expression vectors — Pearson/Spearman correlation is the more widely reported benchmark in transcriptomics GAN evaluation literature, enabling direct comparison to other published models
-
Cosine similarities were computed across 87 toxicity-related pathways and ranked by average value, without adjustment for the number of pathways tested↳ Could also: Apply Benjamini-Hochberg FDR correction if inferential tests are performed across the 87 pathways simultaneously — Testing or ranking across many gene sets simultaneously increases the chance of spurious high-similarity rankings; FDR control is standard practice in pathway-level transcriptomic analysis
-
A single train/test split was used to estimate generalization performance (252 training rats, 36 test rats)↳ Could also: Leave-one-biological-replicate-out or k-fold cross-validation stratified by organ/sex/age condition — With only 4 biological replicates per condition, a single split may yield a variance-sensitive performance estimate; cross-validation would provide a more stable assessment across the limited sample size
-
UMAP was used solely as a visual tool to inspect clustering of generated vs. real profiles in low-dimensional space↳ Could also: Supplement UMAP with a quantitative metric such as average nearest-neighbor distance between generated and real profile clusters, or a silhouette score — A numerical clustering quality metric would complement the visual UMAP assessment and enable objective comparison across model variants or training epochs without reliance on visual interpretation
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-37200521 (TransOrGAN)
Paper: Li, Roberts, Liu, Tong (2023). TransOrGAN: An AI Mapping of Rat Transcriptomic Profiles between Organs, Ages, and Sexes. Chem Res Toxicol.
Real code repo: https://github.com/TingLi2016/TransOrGAN (the brief's listed
the-gan-zoo link is WRONG — that is a generic catalog of GAN papers, not this
code. The authors' actual repo is TingLi2016/TransOrGAN, named in the paper's Code
Availability statement.) Repo ships 8 analysis scripts; data + trained model are
"provided by request", NOT committed.
Data: GEO GSE53960 (Rat BodyMap, Fischer-344). GSE53960_RAW.tar (84 MB) =
320 per-sample AceView expression files GSM*_SEQC_<Organ>_<Sex>_<Age>_<Rep>.txt.gz.
Pipeline-derived results (what the paper computes)
TransOrGAN = autoencoder (40064 genes -> 128-dim) + cGAN/CycleGAN translating a source transcriptomic profile to a target (organ/age/sex) condition. Evaluation metrics: cosine similarity and RMSE between generated vs real target profiles, with a control group = pairwise similarity among real profiles.
| Result | Pipeline | In scope? | Why |
|---|---|---|---|
| Dataset structure: 288 samples / 9 sex-common organs / 2 sexes / 4 ages / 4 reps | GEO parse | YES | Directly verifiable from GSE53960 file naming |
| Gene count: 40,064 AceView genes | GEO parse | YES | header-excluded line count of each sample file |
| Control-group cosine 0.940±0.021 (train) / 0.939±0.020 (test) | control_cosine.py = all-pairwise cosine of REAL log2 profiles |
YES | Needs only public data + log2; NO trained model |
| Control-group RMSE 1.900±0.340 (test) | mse.py::control_rmse = all-pairwise RMSE of REAL log2 profiles |
YES | Same — model-independent |
| Generated cosine 0.984 / RMSE 0.910 | autoencoder + GAN -> decoded_prediction.csv |
NO (the 20%) | Trained model only "by request"; retraining a GAN is non-deterministic, will not byte-match, out of 80/20 scope |
| Pathway (87 CTD) / sex-gene cosines | needs decoded_prediction.csv (model output) |
NO | Depend on the trained model |
| UMAP figures | needs model output | NO | qualitative + model-dependent |
Reproduction strategy
Reproduce the model-independent control/baseline values (the denominator of
the paper's improvement claim) and the dataset dimensionality, exactly as the
authors' own control_cosine.py / mse.py compute them, from public GSE53960.
The control-group method (all-pairwise similarity of real log2 profiles) is fully
specified and deterministic. The model-generated values are explicitly NOT
attempted (trained weights not shipped; GAN training non-deterministic).
Out of scope (out-of-pipeline): nothing wet-lab here; the only excluded items are the trained-model outputs above.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
The model-independent control results reproduce essentially 1:1 from public data: n=288 and 40064 genes exact, control cosine 0.9399 vs reported 0.940/0.939 (exact to 3 decimals), control RMSE 1.884 vs 1.900 (within ~1%), with no fabrication concern. The gap is on data-availability / authors' side, not a defect in any computed value: the paper's central generated-profile metrics (cosine 0.984, RMSE 0.910) could not be tested because the trained autoencoder+GAN weights are 'by request' and not deposited, and retraining is non-deterministic. Severity of the actual deviations is negligible, but the headline GAN claim remains unverified (limited, not refuted), so overall this is a solid-but-partial reproduction with an explainable coverage gap.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.