Gene-expression patterns in peripheral blood classify familial breast cancer susceptibility.
The main results reproduced: recomputed values matched the published ones within tolerance.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
- ✓Same input data as the authors
- ✓Reported values were directly comparable
- ✓No relevant deviation in data/preprocessing
- ✓No authors-side cause for any deviation
- ✓Reported values are derivable from the shared data
- ✓Any deviation was negligible
- ✓The central claim held under reproduction
- ✓Overall, the reproduction was clean
- Every checked point held up.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
DESCRIBED WELL ENOUGH -> 1:1 REPRODUCED. The authors' own repo (srp33/BCSP) ships the complete classifier pipeline self-contained: mlflex.jar + Weka/libsvm jars + the fully-processed, ComBat batch-corrected ARFF expression matrices and class labels for GSE47862. We ran the authors' ML-Flex experiments utahontario1 + utahontario2 unchanged (FS = Weka SVMAttributeEval / SVM-RFE; CLF = R e1071 RBF-SVM; train Utah n=124 / test Ontario n=73; 10-fold inner CV; feature sweep 25-300) in a conda env (openjdk8 + R4.5.3 + e1071 1.7-17) built inside the «our HPC» compute job. Reproduced the two headline AUCs to 2-3 decimals: Utah 10-fold CV AUC 0.763 (reported 0.76) and Ontario external-validation AUC 0.729 (reported 0.73, pooling both batches as the BCSP postprocess does). NOT ATTEMPTED (out of scope / hard 20%): upstream CEL->matrix preprocessing and the demographic-correlation gene-EXCLUSION step (its variable file is absent from the repo -> 'contact the authors'; the shipped batch ARFFs already encode it, so the classifier reproduces but this step is not independently re-derivable); the permutation p-value (1000 iters) and 95% CIs; secondary comparisons (1vs35/24vs35), pathway analyses, GSE12517/Visvader datasets, and clinical likelihood-ratio numbers. This is a faithful re-run of the authors' code+config on their own inputs, not an independent reimplementation. No fabrication indicators: both headline values are fully derivable from shipped artifacts and matched. Provisional grades; human reviewer decides ground truth.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 83assessed: 2026-06-16 ⛓ df38c4a5a935
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-16
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-16no human curator yet
- Last updated
- 2026-09-19
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetGene-expression levels in peripheral blood mononuclear cells reflect underlying genetic and epigenetic variation and can be used to identify women at highest risk of developing familial breast cancer, independent of BRCA1/2 mutation status.
- ★ A multigene expression biomarker from PBMCs accurately classifies familial breast cancer (FBC) status finding
- ★ The biomarker generalizes to an independent external cohort (Ontario) when trained on the Utah cohort finding
- ★ Individuals classified as 'high risk' have substantially greater odds of developing FBC than those classified as 'low risk' finding
- ★ SVM-RFE was used to select genes whose PBMC expression best differentiates FBC patients from controls method
- ComBat was applied to correct batch effects arising from different processing facilities/times method
- ★ Genes upregulated in PBMCs of FBC patients are also upregulated in normal breast tissue from women with strong family history of breast cancer mechanism
- ★ Peripheral blood gene-expression patterns can reflect BRCA1/2 mutation status and genomic variation relevant to breast cancer risk mechanism
- Raw/processed data (GSE47862) and analysis scripts (GitHub) were made publicly available resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| Microarray gene-expression profiling | PBMCs from Utah cohort (n=124 women, with/without family history and BRCA1/2 status) | none (case-control by FBC status) | genome-wide gene-expression levels | Affymetrix GeneChip Human Exon 1.0 ST |
| Microarray gene-expression profiling | PBMCs from Ontario cohort (n=73 women, external validation) | none (case-control by FBC status) | genome-wide gene-expression levels | Affymetrix GeneChip Human Exon 1.0 ST |
| SVM-RFE feature selection | Utah PBMC gene-expression dataset | none (computational) | ranked gene subsets differentiating FBC vs. control | Weka SVMAttributeEval module |
| SVM classification (cross-validation and train/test) | Utah (10-fold CV) and Ontario (train on Utah, test on Ontario) | none (computational) | probabilistic FBC risk score, AUC, sensitivity/specificity | e1071 R package / LIBSVM |
| Total lymphocyte enumeration test | Blood samples from 22 Utah cohort patients | none | counts of CD4+ T-cells, CD8+ T-cells, CD3+ T-cells, B-cells, NK-cells | — |
| Health-assessment survey | 63 Utah cohort patients | none | epidemiological/demographic/clinical covariates for confounder adjustment | — |
| Gene Set Enrichment Analysis | Normal breast cells (GSE17072, Lim et al.) from prophylactic mastectomy/reduction mammoplasty in high-risk and control women | none | enrichment of top 250 PBMC FBC-associated genes in normal breast tissue expression | GSEA software |
- ▲ Cross-validated Utah biomarker predicted FBC status with high accuracy AUC = 0.76 [0.67-0.85]
- ▲ External validation in Ontario cohort confirmed predictive accuracy AUC = 0.73 [0.59-0.86]
- ▲ High-risk classified individuals had greater odds of developing FBC than low-risk classified individuals 2.4-fold greater odds
- – Permutation testing confirmed biomarker accuracy was not due to chance in both cohorts p = 0.001 (Utah); p = 0.002 (Ontario)
- – Predictive accuracy peaked at 250 genes and remained consistent at 150+ genes peak at 250 genes
- ▲ Top genes upregulated in PBMCs of FBC patients were also upregulated in normal breast tissue of high-risk women
- – Similar prediction accuracy observed for BRCA1/2 carriers and BRCAX (family history without known mutation) individuals
- other AUC = 0.76 [95% CI 0.67-0.85] (Utah cohort cross-validated FBC classification)
- other AUC = 0.73 [95% CI 0.59-0.86] (Ontario external validation cohort)
- pvalue p = 0.001 (permutation test, Utah biomarker significance)
- pvalue p = 0.002 (permutation test, Ontario external validation significance)
- fold_change 2.4 times greater odds (odds of FBC for high-risk vs low-risk classified individuals)
- pvalue p = 0.064 (ANOVA testing age differences across Utah patient subgroups)
- pvalue p = 0.28 (t-test comparing ages between FBC and non-FBC groups)
- count 2,201,005 probes (microarray probes retained after quality filtering, summarized to gene-level values)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
The study used Support Vector Machines (SVM) with SVM-Recursive Feature Elimination (SVM-RFE) to classify familial breast cancer (FBC) status from peripheral-blood PBMC gene-expression profiles in two independent cohorts (Utah n=124; Ontario n=73). Internal validation was performed via 10-fold cross-validation repeated 1,000 times in the Utah cohort; external validation was conducted by training on all Utah samples and applying the model to the Ontario cohort. Predictive performance was summarized as AUC with 95% confidence intervals derived from 10,000 bootstrap iterations, and permutation tests (class-label shuffling) were used to assess significance of the AUC estimates.
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| One-way analysis of variance (ANOVA) | Comparison of age at blood draw across six patient subgroups in the Utah cohort | 117 Utah participants with available ages (Table 2) | not stated |
| Two-sample t-test | Comparison of age between individuals who developed FBC and those who did not (Utah cohort) | — | not stated |
| Multifactor analysis of covariance (ANCOVA) | Per-gene filtering to exclude genes whose expression correlated with any of >20 clinical, demographic, or treatment variables at p < 0.01 | Up to 63 Utah participants with health survey data; 22 with lymphocyte count data | not stated |
| SVM classification with 10-fold cross-validation (repeated 1,000 times) and permutation test (p = 0.001) | Internal validation of FBC prediction; primary AUC result for Utah cohort (AUC = 0.76, 95% CI 0.67–0.85) | 124 Utah participants | na |
| SVM classification with train/test split and permutation test (p = 0.002) | External validation; Utah-trained model applied to Ontario cohort (AUC = 0.73, 95% CI 0.59–0.86) | 73 Ontario participants | na |
| Bootstrap resampling (10,000 iterations) for 95% CI on AUC | Confidence intervals on AUC for both Utah and Ontario predictions | 124 (Utah); 73 (Ontario) | na |
| Gene Set Enrichment Analysis (GSEA) with gene-based permutation | Assessment of whether top 250 PBMC genes up-regulated in FBC were also up-regulated in normal breast cells (GSE17072, Lim et al.) | — | na |
-
SVM with SVM-RFE was used for both feature selection and classification as two sequential steps↳ Could also: Penalized logistic regression (LASSO or elastic net) could also have been used — LASSO/elastic net simultaneously performs feature selection and model fitting within a single regularized framework, yields directly interpretable log-odds coefficients, and is a widely used baseline for high-dimensional small-n binary classification; it also facilitates calibration assessment more naturally than SVM probability estimates
-
Hyperparameter C was tuned via nested cross-validation but performance was ultimately estimated by the same 10-fold outer loop↳ Could also: A fully nested (double) cross-validation design—where an inner loop tunes hyperparameters and a separate outer loop estimates performance—could also be reported explicitly as the primary evaluation — Fully nested CV provides a less optimistic performance estimate when the same fold structure is used for both tuning and evaluation; making this structure explicit aids reproducibility and comparison with other studies
-
Predictive performance was summarized primarily by AUC (discrimination)↳ Could also: Calibration metrics such as the Brier score or a calibration plot (observed event rates vs. predicted probabilities) could also have been reported — For a clinical risk-estimation tool intended to guide individual decisions, calibration—how closely predicted probabilities match observed rates—is as clinically relevant as discrimination; reporting both gives a more complete picture of utility
-
Genes whose expression correlated with clinical/demographic covariates were removed by applying a fixed p < 0.01 threshold per gene across >20 variables without a stated multiple-testing correction↳ Could also: Surrogate variable analysis (SVA) or inclusion of known covariates as explicit terms within the classification model could also address confounding — Hard filtering permanently removes genes; SVA or covariate-adjusted modeling retains potentially informative genes while adjusting for confounders, and avoids the implicit family-wise error inflation from thousands of unadjusted per-gene tests
-
ComBat was applied to all samples as a preprocessing step before model training and evaluation↳ Could also: Batch could also be modeled as an explicit covariate within the classification pipeline, or ComBat could be applied only within training folds and then applied to held-out folds — Applying ComBat to the full dataset before cross-validation allows information from held-out samples to influence batch correction in training folds; restricting correction to training data at each fold keeps the pipeline strictly prospective and avoids this form of data leakage
-
Age balance across groups was verified post-hoc with ANOVA (p=0.064) and a t-test (p=0.28), and treated as confirmation that age was not a confounder↳ Could also: Age could also be included directly as a covariate in the classification model, or propensity-score matching on age could be used to construct balanced comparison groups — A non-significant age difference in a small sample does not exclude meaningful confounding; direct adjustment within the model or matched-sample design ensures that classification accuracy reflects the gene-expression signal rather than residual age imbalance
Result convergence & founder nodes
Findings this paper shares with others that ran a comparable experiment. A node’s strength is how many independent papers report it (replication breadth) — not how often it is cited, so a heavily-replicated but under-cited founder still stands out.
-
High PBMC expression classifier score associates with 2.4-fold greater odds of developing familial breast cancermicroarray human pbmc up 2015×1papers★ This paper is the founder (earliest)
-
PBMC multigene expression signature classifies familial breast cancer susceptibility with AUC 0.76 in discovery cohort (Utah, cross-validation)microarray human pbmc 2015×1papers★ This paper is the founder (earliest)
-
PBMC multigene expression classifier discriminates familial breast cancer with similar accuracy in BRCA1/2 mutation carriers and BRCAX womenmicroarray human pbmc none 2015×1papers★ This paper is the founder (earliest)
Citation network
Where this publication sits in the reproducibility-weighted citation graph — what it is built on, and what is built on it. Citation data from OpenAlex.
No assessed neighbours yet — the network grows as more papers are assessed.
Data lineage
The datasets this paper uses (text-mined from the full text via Europe PMC), and which other assessed papers stand on the same data. A shared dataset is a factual link — not a judgement.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-26538066
Paper: Piccolo SR et al. (2015) Gene-expression patterns in peripheral blood classify familial breast cancer susceptibility. BMC Med Genomics 8:72. DOI 10.1186/s12920-015-0145-6 · PMCID PMC4634735.
Code: https://github.com/srp33/BCSP (authors' own repo) — cloned to «infra» at
«path».
Data: GEO GSE47862 (Affymetrix Human Exon 1.0 ST, peripheral blood). The repo
ships the fully-processed, gene-summarized, batch-corrected expression matrices
and class labels used as direct input to the classifier, so no GEO download or
CEL-level processing is required for the classification result.
The classification task
Binary: BRCACancer (women with a family history of breast cancer who carry BRCA1/2 or BRCAX mutations and have/had cancer) vs Control (everyone else).
- Utah cohort: 39 cases / 85 controls (n=124)
- Ontario cohort: 28 cases / 45 controls (n=73; two array batches of 36+37)
Authors' pipeline (ML-Flex)
- Independent vars:
ML-Flex/InputData/utahontario{1,2}.batch.arff(ComBat batch-corrected, gene-level Entrez expression, ~25k genes). - Dependent var:
classes.0124v35.arff. - Feature selection: Weka
SVMAttributeEval(= SVM-RFE, removes 10%/iter) → config aliasweka_svm_weights. - Classifier: R
e1071SVM, radial-basis-function kernel → aliasr_svm_rbf. - Design: TRAIN = Utah (124), TEST = Ontario (73), with 10-fold inner CV; feature-count sweep 25..300 (optimum ~250 genes).
- Self-contained:
mlflex.jar+lib/{weka,libsvm,commons-math}.jarshipped; only Java + R/e1071 needed (built in a conda env inside the «our HPC» job).
IN SCOPE (pipeline-derived, attempted)
| id | reported result | location |
|---|---|---|
| C1 | Utah 10-fold CV AUC = 0.76 (95% CI 0.67–0.85) | Abstract / Results / Fig 2 |
| C2 | Ontario external-validation AUC = 0.73 (95% CI 0.59–0.86; perm p=0.002) | Abstract / Results / Fig 3 |
| C3 | optimal performance at ~250 genes | Results |
Method to reproduce C1/C2: run the authors' ML-Flex experiments utahontario1
and utahontario2 exactly as shipped, then compute AUC from the emitted
prediction-probability files (Utah = inner-CV training predictions; Ontario =
combined test predictions across the two batches).
OUT OF SCOPE (not attempted — the hard ~20% / non-pipeline)
- CEL→matrix preprocessing (RMA/normalization, gene summarization): the demographic/clinical-correlation gene-exclusion file is NOT in the repo ("Please contact the authors" — README). The shipped batch ARFFs already encode this step, so the classifier is reproducible but the upstream exclusion is not re-derivable. Documented, not attempted.
- Permutation p-value (
*_Permute, NUM_ITERATIONS=1000) and 95% CIs — heavy / bootstrap; secondary to the point estimates. - Secondary comparisons (
1vs35,24vs35), pathway analyses, the GSE12517 treatment-response and Visvader external datasets, likelihood-ratio / odds clinical-utility numbers (derived from the same predictions but downstream).
Possible-fabrication watch
Both headline AUCs are derivable from shipped data+code, so they are checkable. The point estimates (0.76 / 0.73) are what we test 1:1.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Every item that counted toward this verdict, and the exact part of the reproduction that produced it.
Faithful 1:1 re-run of the authors' own BCSP/ML-Flex pipeline on their shipped, batch-corrected GSE47862 ARFFs reproduces both headline AUCs to 2-3 decimals (Utah CV 0.763 vs 0.76; Ontario validation 0.729 vs 0.73), and the central conclusion holds. The only deviations are the C3 peak feature-count (125 vs ~250 genes) within a flat AUC plateau and the upstream gene-exclusion step being non-re-derivable (exclusion file not deposited) — a transparency/availability gap on the authors' side, not a discrepancy in the reproduced values. No fabrication indicators; severity negligible.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at [email protected].
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.