DAGFormer: A graph-based domain adaptation approach for single-cell cancer drug response prediction.
The main result did not reproduce in this reproduction attempt. Where our recomputation produced values that differ from the published ones, those discrepancies are listed below. This is a single automated attempt — not peer review and not a finding of error or misconduct — and differences can also arise from data access, undocumented parameters or the computing environment. The verdict can be contested via “report an error”.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- Nothing in this column.
- 🔴Could not use the authors’ exact input data
- 🔴Reported values were only indirectly comparable
- 🟡A deviation arose in the data or preprocessing
- 🔴A deviation was attributed to the published material
- 🔴Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🔴Overall, the reproduction showed a material discrepancy
This paper has a computational component, but its primary data is legally or ethically access-restricted — identifiable patient cohorts, rare-disease genomes, or controlled-access biobanks that cannot be openly shared. The reproduction therefore could not be attempted. That is a neutral verdict: it does not mean the result is wrong or that the authors fell short — only that, for legitimate privacy reasons, it cannot be independently checked from public data. We deliberately do NOT assign a 0–100 score here, because a low number would wrongly read as a failed reproduction.
▸Reproduction agent’s raw note
DROP (data_unavailable). DAGFormer is described well enough to run (clear entrypoint python main_GT.py --drug <DRUG>, 10 drugs, default hyperparameters tabulated, metrics = target-domain best AUC/AUPR/accuracy), but the pipeline's INPUT is gone. The model reads preprocessed preprocessNormData/<DRUG>/{Source_exprs_resp_z,Target_expr_resp_z}.<DRUG>.tsv; the repo (commit ff42d11) ships ZERO data files (13 code files only) and the only data pointer — a README Google Drive folder — returns HTTP 404 via 5 independent methods (gdown folder API, curl HEAD, curl GET, WebFetch, uc?id=&export=download final hop), i.e. deleted, not merely private. The paper's Data Availability statement claims the datasets are in the GitHub repo, but they are not (flagged as a possible-fabrication/availability discrepancy for human review; link rot is the benign explanation). Raw public data (GSE149215, GSE108383, GDSC, CCLE) exists, but the repo provides no raw->processed script, so rebuilding the exact z-scored input would require re-inventing unspecified LOBIco labels, the GDSC<->scRNA shared-gene list, per-cell target labels and normalization — that is the explicit 'do not chase the last 20%' case and would not be a 1:1 reproduction. NOT ATTEMPTED: any training/metric reproduction (no input), and any from-raw reconstruction (out of scope, non-faithful). No result fabricated; no compute spent. Repo cloned to «infra» at reproductions/pmid-41417875/DAGFormer @ ff42d11.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessmentassessed: 2026-06-14 ⛓ 2f0253548a7d
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-14
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-15no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: opusCan integrating bulk RNA-seq and scRNA-seq data through a graph-based domain adaptation framework that models intercellular interactions improve prediction of cancer drug responses at the single-cell level, while overcoming batch effects and intratumoral heterogeneity?
- ★ DAGFormer, a graph-based domain adaptation framework integrating bulk and scRNA-seq data, predicts single-cell drug responses more accurately than existing methods. method
- ★ Constructing cellular neighbor graphs via diverse topological strategies captures intercellular interactions that influence single-cell drug response. method
- ★ Graph Domain Adaptation (GDA) with a domain discriminator and feature extractor in adversarial training bridges graph-level distribution gaps and mitigates batch effects between bulk and scRNA-seq data. mechanism
- ★ A dual-domain collaborative decoupling-fusion decoder disentangles shared and modality-specific representations, preserving general and unique biological signals. method
- ★ DAGFormer outperformed existing methods across ten independent scRNA-seq datasets, demonstrating robustness in cancer drug response prediction. finding
- Drug response is stratified into pre-treatment (inherent resistance) and post-treatment (acquired resistance) scenarios to elucidate distinct resistance mechanisms. method
- Python code and datasets are publicly available as a resource. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| bulk RNA-seq (source domain, drug response prediction) | Pan-Can cancer cell lines (GDSC database) | drug treatment (e.g., Gefitinib, Afatinib, AR-42, Cetuximab, Etoposide, NVP-TAE684, PLX4720, Sorafenib, Vorinostat) | binary drug sensitivity/resistance label (sensitive vs resistant) from IC50 | GDSC; RMA normalization; LOBIco binarization |
| scRNA-seq (target domain, single-cell drug response prediction) | JHU006 and SCC47 head/neck cell lines (CCLE) | drug treatment (Gefitinib, Afatinib, AR-42, Cetuximab, NVP-TAE684, Sorafenib, Vorinostat) | per-cell drug sensitivity/resistance classification | 10x Genomics |
| scRNA-seq (target domain, post-treatment/acquired resistance) | PC9 lung adenocarcinoma cell line | Etoposide (untreated parental vs drug-tolerant survivors) | per-cell drug sensitivity/resistance classification | 10x Genomics (GSE149215) |
| scRNA-seq (target domain, PLX4720 sensitivity) | A375 and 451Lu melanoma cell lines | PLX4720 (BRAF inhibitor) | per-cell drug sensitivity/resistance classification | SMART-seq (GSE108383) |
- ▲ DAGFormer achieved superior performance over existing methods across ten independent scRNA-seq datasets in single-cell drug response prediction.
- – Graph-based cellular relationship construction effectively addressed batch effects between bulk and scRNA-seq derived cell graphs and accurately predicted single-cell drug responses.
- – Post-treatment (acquired resistance) prediction was evaluated for agents such as Etoposide and PLX4720.
- – Pre-treatment (inherent resistance) prediction was evaluated for agents including Gefitinib and Cetuximab.
- other TP53 mutations occur in approximately 96% of cases (high-grade serous ovarian cancer, motivating drug resistance background)
- count ten independent scRNA-seq datasets (benchmarking of DAGFormer)
- count 714 resistant / 115 sensitive cell lines, 10610 genes (Gefitinib GDSC source domain dataset)
- count 811 resistant / 53 sensitive cell lines, 9738 genes (Etoposide GDSC source domain dataset)
- count 746 resistant / 629 sensitive cell lines, 11937 genes (PLX4720 GDSC source domain dataset)
- count 764 resistant / 629 sensitive single cells (Etoposide PC9 target domain (GSE149215))
- other SMOTE then Random UnderSampler to near 1:1 sensitive:resistant ratio (training-set class imbalance correction)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
DAGFormer is a computational deep-learning benchmarking paper that frames drug response prediction as a binary classification task (sensitive vs. resistant), training on bulk RNA-seq cell-line data (GDSC) and evaluating on 10 independent scRNA-seq target datasets covering 9 drugs. Class imbalance in the training set was addressed with a two-step SMOTE oversampling and Random UnderSampler pipeline targeting a 1:1 sensitive-to-resistant ratio. Gene expression features were standardized via Z-score normalization. Performance is compared against existing methods across datasets; the provided text does not describe formal inferential hypothesis tests or dispersion measures for reported metrics.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Binary classification (LOBIco — logistic optimization binary classification algorithm for converting IC50 to sensitive/resistant labels) | Source domain GDSC cell-line IC50 values converted to binary drug response labels | 714–811 resistant and 37–629 sensitive cell lines per drug (Table 1) | not stated |
| SMOTE oversampling followed by Random UnderSampler (class-balancing preprocessing, not an inferential test) | Training set construction from GDSC source domain data | Resulting 1:1 ratio; exact post-balancing n not stated | not stated |
| Benchmarking comparison of classification performance across competing models (specific metric names and inferential tests not stated in provided text) | 10 scRNA-seq target domain datasets (Tables 1–2, Figs 4–5) | 33–764 resistant and 33–629 sensitive single cells per dataset (Table 2) | not stated |
-
IC50 values were converted to binary sensitive/resistant labels via LOBIco before model training↳ Could also: Continuous IC50 values could be used directly as regression targets (e.g., with a mean-squared-error loss), or ordinal thresholding could be applied at multiple cutoffs to assess label-threshold sensitivity — Retaining the continuous pharmacological signal avoids information loss from binarization and allows assessment of how much predictive performance depends on the chosen IC50 threshold
-
Class imbalance was addressed by combining SMOTE oversampling with Random UnderSampler in the training set↳ Could also: Class-weighted loss functions (assigning higher weight to the minority class) or cost-sensitive learning could also handle imbalance without generating synthetic training samples — Synthetic sample generation (SMOTE) can introduce interpolation artifacts in high-dimensional gene expression space; cost-sensitive approaches operate directly on the observed data distribution
-
Benchmarking compared model performance across 10 datasets without reported inferential statistical tests↳ Could also: Paired Wilcoxon signed-rank tests or permutation tests applied to per-dataset metric differences across models would also allow a statement about whether observed performance gaps are systematic — Summarizing raw metric differences across datasets without a significance test leaves open whether observed advantages are within the range of dataset-to-dataset variability
-
Performance results appear to be reported as point estimates (specific metrics not named in provided text)↳ Could also: Bootstrapped 95% confidence intervals around performance metrics (e.g., AUROC) could also be reported for each dataset — Point estimates alone do not convey the uncertainty around the metric given the finite and sometimes small target-domain sample sizes (as low as n=66 total cells for some datasets)
-
Z-score standardization (StandardScaler) was applied to gene expression data↳ Could also: Quantile normalization or batch-effect correction methods (e.g., ComBat, Harmony) could also be applied before graph construction to further reduce cross-platform distributional differences between bulk and scRNA-seq data — Z-score normalization aligns means and variances but does not explicitly address higher-order distributional differences that arise from different sequencing platforms and library preparation methods
-
The two experimental scenarios (pre-treatment inherent resistance and post-treatment acquired resistance) were evaluated separately↳ Could also: A formal interaction test (e.g., model × scenario interaction in a two-way design) could also be used to assess whether relative model performance differs between the two resistance scenarios — Separate reporting by scenario shows within-scenario rankings but does not directly test whether the performance advantage of any model is consistent across both drug-resistance mechanisms
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-41417875 (DAGFormer)
Paper: DAGFormer: A graph-based domain adaptation approach for single-cell cancer
drug response prediction. Yan F, Du Z, Huang YA. PLoS Comput Biol 2025.
PMID 41417875 · PMCID PMC12795466 · DOI 10.1371/journal.pcbi.1013832
Code: https://github.com/yanfen-git/DAGFormer @ commit ff42d11be059c5ff593cc5252ecdb34dc9bad593
Named data: geo:GSE149215 (PC9 / Etoposide), geo:GSE108383 (A375,451Lu / PLX4720), GDSC (source), CCLE.
What the paper reports (pipeline-derived → in principle in scope)
DAGFormer is a graph domain-adaptation deep model (DGL/PyTorch). For each of 10 drugs it
trains on a GDSC bulk source domain and predicts single-cell drug response on a scRNA-seq
target domain, reporting per-drug ACC / AUC(AUROC) / AUPR on the target domain plus an
overall average. These are pipeline-derived (a single training script main_GT.py produces
them) → would be in scope for a 1:1 reproduction.
Reported anchors (PMC full text):
- Overall average: ACC 0.902, AUC 0.935, AUPR 0.938 (ten independent runs).
- Etoposide (PC9, GSE149215): ACC 0.835, AUC 0.843, AUPR 0.908.
- Sorafenib: AUC 0.987, AUPR 0.988.
- Remaining per-drug values are embedded in Figures 4–5, not tabulated.
The pipeline and its required input
main_GT.py --drug <DRUG> calls load_data_drug('source'|'target', DRUG) which reads ONLY:
./preprocessNormData/<DRUG>/Source_exprs_resp_z.<DRUG>.tsv(GDSC bulk, z-scored; cols: response, logIC50, genes)./preprocessNormData/<DRUG>/Target_expr_resp_z.<DRUG>.tsv(scRNA-seq, z-scored; col0 = binary response, genes) It then builds a cosine k-NN graph (k=15), SMOTE-balances the source, and trains the GDA model. The reported metrics are thetarget best AUC/AUPR/accprinted at the end of training.
Blocking finding (why this is a DROP)
The pipeline's actual input — the preprocessed preprocessNormData/<DRUG>/*.tsv files — is
not obtainable:
- The repo (commit ff42d11) ships NO data —
git ls-files= 13 files, all code (*.py,*.ipynb,*.R,my_model.png); there is nopreprocessNormData/, no.tsv/.csv. - The only data pointer is a Google Drive folder link in the README
(
/drive/folders/1y4_xWRmhIs1noyDmWz9CKL1oDWLGkO2Y) which returns HTTP 404 via FIVE independent methods (gdown folder API; curl HEAD; curl GET → Google "Error 404"; WebFetch; and theuc?id=…&export=download→drive.usercontent.google.com/downloadfinal hop = 404). 404 (not a "Request access"/403 page) ⇒ the folder is deleted/withdrawn, not merely private. - The paper's Data Availability statement says "Python code and the datasets used in our studies are made available at https://github.com/yanfen-git/DAGFormer" — but the datasets are absent from that repo. (Possible-discrepancy note for the human auditor: stated availability ≠ actual availability.)
Could it be reconstructed from public raw data? (the hard >20% — not attempted, with reason)
GSE149215, GSE108383, GDSC, CCLE raw data ARE public. But the repo provides no script that
converts raw GEO/GDSC into the *_resp_z.<DRUG>.tsv the model consumes. A faithful rebuild would
require re-deriving, all unspecified in the shipped code: GDSC IC50 → LOBIco binary
sensitivity labels (per-drug thresholds), the exact GDSC↔scRNA shared-gene list (sets the real
feature dimension), the per-cell target labels (parental=sensitive / drug-surviving=resistant
barcode assignments), and the QC/normalization/z-score parameters. Re-inventing this is NOT a 1:1
reproduction — any resulting number would diverge for preprocessing reasons unrelated to the
method — and falls squarely in the "don't chase the last 20%" zone. Therefore not attempted.
Outcome
drop · drop_reason = data_unavailable. The dataset the pipeline consumes is withdrawn (dead Drive link) and is not shipped with the code; the code cannot regenerate it from public raw data. No comput
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
This is a clean data-availability drop: DAGFormer's code is internally consistent and runnable, but the preprocessed input TSVs are absent from repo @ff42d11 and the only data link (a Google Drive folder) is HTTP 404, so none of the 8 reported metrics (overall ACC=0.902/AUC=0.935/AUPR=0.938, etoposide, sorafenib AUC=0.987) could be put to test. The blocker sits on the authors'/data-availability side — the paper states the datasets are in the GitHub repo, but they are not — which is more than mere access restriction and is flagged for human review (link rot is the likely benign cause). No numbers are internally suspect or 'too perfect'; the central claim is simply not confirmable, not refuted. Severity of any deviation is unknown because nothing ran.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.