Corpus 1,272 assessed · 1,173 scored · 643 reproduced ≥75 · 168 flagged ·∅ 74.1/100
← New search

Transcriptome profiling of radish (Raphanus sativus L.) root and identification of genes involved in response to Lead (Pb) stress with next generation sequencin

PLoS One · 2013
L1 27/100 3/4
⚑ Flagged for review — a reproduced result did not match the reported value

Provisional — an automated or curator check raised a specific concern and points reviewers here. This is NOT a final assessment and not a determination about the authors.

Why this verdict

The main result did not reproduce in this reproduction attempt. Where our recomputation produced values that differ from the published ones, those discrepancies are listed below. This is a single automated attempt — not peer review and not a finding of error or misconduct — and differences can also arise from data access, undocumented parameters or the computing environment. The verdict can be contested via “report an error”.

Reproduced on the brainbox compute brainarbeit.com
Scoring basis — itemised

Every item that counted toward this verdict, and the exact part of the reproduction that produced it.

Supporting (toward a concern)
Content-critical question only partially held
+2 pts
From: Q5 · Derivability / plausibility 🟡
Content-critical question only partially held
+2 pts
From: Q7 · Core claim 🟡
Content-critical question only partially held
+2 pts
From: Q8 · Severity of the miss (overall human judgment) 🟡
Minor / cosmetic deviation
+1 pts
From: Q3 · Location of the main deviation 🟡
Minor / cosmetic deviation
+1 pts
From: Q4 · Cause of the deviation 🟡
Minor / cosmetic deviation
+1 pts
From: Q2 · Endpoint comparability 🟡
Concordant (toward reproduced)
Code + data deposited & functional
-2 pts
From: Data & code availability Available & functional
Total score +7
✓ What held up
  • Nothing in this column.
What did not (or only partly)
  • 🟡Could not use the authors’ exact input data
  • 🟡Reported values were only indirectly comparable
  • 🟡A deviation arose in the data or preprocessing
  • 🟡A deviation was attributed to the published material
  • 🟡Reported values were not (fully) derivable from the shared data
  • 🔴The deviation was non-trivial in magnitude
  • 🟡The central claim did not (fully) hold under reproduction
  • 🟡Overall, the reproduction showed a material discrepancy
How its reproducibility compares
27/100
Reproducibility score
2.7 SD below mean
vs. all fields · 1173 studies
🎯 Scores higher than 1% of all assessed papers rank 1157 of 1173 scored

A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.

Reproduction agent’s raw note

Continued a prior worker's already-staged pipeline for pmid-23840502 (radish Pb-stress transcriptome). Reused pre-downloaded SRA reads (SRX256970/SRR802994=CK, SRX263753/SRR803021=Pb1000), pre-pulled containers, and pre-staged Swiss-Prot/GO reference data. Trinity de novo assembly required multiple 12h SLURM windows (walltime fallback chain trinity2 to trinity3 to trinity4) due to Butterfly-phase runtime on this dataset, and completed successfully. All ten in-scope SLURM pipeline stages (trinity assembly, rsem reference build, rsem quantification x2, diamond annotation, go annotation, edgeR differential expression, goatools GO enrichment) completed with State=COMPLETED, ExitCode=0:0 per sacct. Assembly stats, RSEM quantification (bowtie2 index/alignment via trinity.sif since rsem.sif has no bundled aligner, then rsem-calculate-expression --bam), edgeR differential expression (no biological replicates in the original design -- single CK vs single Pb1000 library -- so a fixed-BCV=0.1 exactTest was used as the closest available approximation since the paper's exact DE method is unspecified), diamond blastx vs Swiss-Prot (single-database annotation, narrower than the paper's combined Nr/Swiss-Prot/COG/KEGG/GO pipeline), and goatools GO enrichment (matching the cited code repository) were all run as SLURM jobs on the HPC cluster and their outputs verified directly from job logs and output files. KEGG pathway enrichment (paper Tables 2-3) was NOT attempted: no KEGG mapping data was staged and it falls outside the cited goatools repository's scope. The 314823 pre-existing radish ESTs and 17187 NCBI unigenes that the paper additionally incorporated into its assembly were NOT included here (only the two SRA read sets were assembled de novo), which is an expected source of divergence in absolute assembly statistics. TheGO enrichment claim, previously checkpointed as 'NOT AVAILABLE'/error, has been corrected in this pass: the goatools computation itself succeeded (774 significant terms for the up-regulated study set, 702 for the down-regulated study set), and only a separate downstream summary-writer script failed on an oversized CSV field; the real numbers were recovered from the job's full stdout log and cross-verified two independent ways. All numeric comparisons remain provisional pending human review, especially DEG counts and GO term counts, given the documented methodological substitutions (DE tool, single annotation DB, no EST/unigene integration). Not attempted in this reproduction: KEGG pathway enrichment/mapping (paper Tables 2-3), qRT-PCR validation of DEGs (wet-lab, out of scope), and any dose-response/time-course analysis beyond the single CK-vs-Pb1000 comparison (wet-lab experimental design, out of scope).

These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.

✎ I am an author of this paper

Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.

Reason for the rerun

We email you a confirmation link first. The rerun is an objective re-measurement — it cannot change the verdict in your favour, only ask us to look again.

Provenance — full disclosure

When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.

Reproduced
2026-07-31
Rubric version
v1.0
Assessed by
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-07-31
no human curator yet
Last updated
2026-07-31

Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.

Deep full-text extraction

Model: opus
Founding hypothesis

Little is known about how radish (Raphanus sativus L.) responds to lead (Pb) stress at the molecular level; this study asks which genes and pathways in radish roots are differentially regulated under Pb stress, using NGS-based de novo transcriptome sequencing to elucidate the molecular regulation mechanisms and critical genes involved.

Core claims
  • A de novo radish root transcriptome of 68,940 assembled transcripts including 33,337 unigenes was generated, providing the first comprehensive molecular characterization of the radish root response to Pb stress. resource
  • 4,614 differentially expressed genes (DEGs) were detected between untreated control (CK) and Pb-treated (Pb1000) radish roots. finding
  • Upregulated DEGs under Pb stress are predominantly involved in defense responses in the cell wall (callose deposition, cell wall thickening) and glutathione metabolism-related processes. mechanism
  • Downregulated DEGs are mainly involved in carbohydrate metabolism-related pathways and cell wall organization/modification. mechanism
  • Candidate defense and detoxification genes — signaling protein kinases, transcription factors, metal transporters and chelate-compound biosynthesis enzymes — were identified as Pb-responsive. finding
  • qRT-PCR validation of 22 selected genes showed expression patterns highly concordant with the Solexa/RNA-seq analysis. method
  • Illumina paired-end Solexa RNA-seq with Trinity de novo assembly, Bowtie/RSEM quantification and FPKM normalization is an effective approach for transcriptome profiling in a species lacking a reference genome. method
  • Expression of selected Pb-responsive genes varies with both Pb concentration and treatment duration, e.g. GST3-1 and GSHB1 are downregulated at 200 and 500 mg L−1 but upregulated at 1000 mg L−1 Pb(NO3)2. finding
Experimental setups
Assay System Perturbation Readout Platform
RNA-seq (Illumina paired-end Solexa transcriptome sequencing, de novo) Radish (Raphanus sativus L.) fleshy taproot/root, two cDNA libraries (CK and Pb1000) Pb(NO3)2 at 1000 mg L−1 vs untreated control Clean paired-end reads (2×101 bp), assembled transcripts/unigenes, FPKM expression levels, differentially expressed genes Illumina HiSeq 2000
De novo transcriptome assembly and expression quantification (bioinformatics) Radish root cDNA sequence reads combined with 314,823 radish ESTs and 17,187 NCBI unigenes none Contig/transcript length distribution, N50/N90, unigene counts; read mapping and expression estimation Trinity (assembly), Bowtie (mapping), RSEM (expression), FPKM normalization
Functional annotation and enrichment analysis (GO, COG, KEGG/KOBAS) Assembled radish root transcripts and DEG sets none (in silico on Pb-treated vs control DEGs) GO term assignments, COG functional categories, KEGG pathway assignments, enriched GO terms and pathways (hypergeometric test with Bonferroni correction) BLASTX (E-value cut-off 1e−5), goatools, KOBAS, COG and KEGG databases
Quantitative real-time PCR (qRT-PCR) validation Radish roots (CK and Pb1000 samples) Pb(NO3)2 at 1000 mg L−1 vs untreated control Relative expression ratio (Pb1000/CK) of 22 selected DEGs compared with RNA-seq log fold change
qRT-PCR dose-response and time-course expression profiling Radish roots Pb(NO3)2 at 0, 200, 500 and 1000 mg L−1 for 72 h; and 1000 mg L−1 for 0, 24, 48 and 72 h Relative transcript levels of six genes (GST3-1, GGT1, GSHB1 upregulated; SPDS, LAX2-1, LAX2-2 downregulated)
Key results
  • A total of 4,614 DEGs (1,398 unigenes) were detected between Pb1000 and CK libraries, comprising 2,154 upregulated and 2,460 downregulated transcripts 4,614 DEGs (2,154 up / 2,460 down)
  • De novo assembly yielded 68,940 assembled transcripts including 33,337 unigenes, contig length 306–16,101 bp, average 1,226.63 bp, N50 = 1,262, N90 = 437 68,940 transcripts; 33,337 unigenes; N50 1,262 bp
  • GO enrichment identified 270 significantly enriched terms for upregulated transcripts versus only 107 for downregulated ones (Bonferroni-corrected P ≤ 0.05) 270 up vs 107 down GO terms
  • Upregulated DEGs were enriched in xenobiotics biodegradation/metabolism pathways (drug metabolism-cytochrome P450, metabolism of xenobiotics by cytochrome P450) and glutathione metabolism; 13 pathways significantly enriched 13 enriched pathways; glutathione metabolism 20 genes, corrected P = 6.49E-05
  • Downregulated DEGs were enriched in carbohydrate metabolism pathways: pentose and glucuronate interconversions, starch and sucrose metabolism, amino sugar and nucleotide sugar metabolism; six pathways significantly enriched 6 enriched pathways; pentose and glucuronate interconversions 26 genes, corrected P = 4.49E-12
  • Glutathione pathway genes were strongly induced by Pb, e.g. GSTU11 (comp2239_c0_seq1) rose from 0.67 to 193.06 FPKM and GSHB1 from 51.37 to 454.81 FPKM GSTU11 log FC 8.18 (qRT-PCR 56.49-fold); GSHB1 log FC 3.15 (qRT-PCR 3.25-fold)
  • Auxin transporter-like genes and spermine synthase were repressed by Pb, e.g. LAX3 fell from 199.38 to 4.81 FPKM and CCD8 from 180.13 to 0 FPKM LAX3 log FC −5.38 (qRT-PCR 0.13); CCD8 log FC −38.82 (qRT-PCR 0.01)
  • SPDS, LAX2-1 and LAX2-2 were downregulated at all Pb concentrations after 72 h, but SPDS was upregulated at 24 and 48 h and LAX2-1 at 24 h, indicating time-dependent responses
Key statistics
  • count 4,614 DEGs (1,398 unigenes); 2,154 upregulated and 2,460 downregulated transcripts (DEGs between Pb1000 and CK libraries, log FC >2 or <−2, P<0.005, FDR<0.001)
  • count 26,381,880 (CK) and 22,535,988 (Pb1000) clean paired-end reads; 2,636,208,012 and 2,251,686,397 nucleotides; 48,918,868 total clean reads (Illumina HiSeq 2000 sequencing output for the two libraries)
  • count 68,940 assembled transcripts, 33,337 unigenes, whole dataset 84,563,748 bp, average contig 1,226.63 bp, N50 1,262, N90 437 (De novo Trinity assembly overview (Table 1))
  • pvalue 1.34E-14 (corrected 4.49E-12) (Pentose and glucuronate interconversions [ko00040], most enriched downregulated pathway (26 of 104 background genes))
  • pvalue 4.98E-10 (corrected 1.29E-07) (Drug metabolism–cytochrome P450 [ko00982], most enriched upregulated pathway (16 of 65 background genes))
  • fold_change log FC 8.1793185 (RNA-seq); 56.49 (qRT-PCR Pb1000/CK) (GSTU11 glutathione-S-transferase, FPKM 0.67 → 193.06)
  • count 46,966 transcript sequences (68.12%) assigned at least one GO term; 22,518 sequences assigned to 23 COG categories; 19,480 of 48,864 annotated sequences mapped to 296 KEGG pathways (Functional annotation of assembled transcripts)
  • count 22 candidate DEGs validated by qRT-PCR; 6 genes profiled across 4 Pb concentrations (0, 200, 500, 1000 mg L−1) and 4 time points (0, 24, 48, 72 h) (qRT-PCR validation and dose/time-course experiments)

Statistical methods review

Model: sonnet

A neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.

The study used Illumina RNA-seq to compare a de novo assembled transcriptome from two radish root cDNA libraries, one untreated (CK) and one Pb-treated (Pb1000), each apparently represented by a single sequencing library. Differentially expressed genes (DEGs) were called using a log fold-change threshold (|logFC|>2) combined with a significance cutoff (P<0.005) and false discovery rate (FDR<0.001), and enrichment of GO terms and KEGG/KOBAS pathways among DEGs was assessed with a hypergeometric test and Bonferroni correction. A subset of 22 DEGs was validated by qRT-PCR, described as showing general agreement in expression direction/magnitude with the RNA-seq results.

Replicationunclear Sample sizeNot described; the RNA-seq comparison appears based on one CK library and one Pb1000 library rather than multiple biological replicates, and no replicate number is given for the qRT-PCR assays GroupsUntreated (CK) vs Pb(NO3)2-treated (Pb1000, 1000 mg/L) radish roots; later dose (0/200/500/1000 mg/L) and time-course (0/24/48/72 h) qRT-PCR comparisons Pairingna Randomization/blindingnot stated Dispersionnone Exact p-valuesyes Effect sizesyes Confidence intervalsno Multiplicity correctionFalse discovery rate (FDR) thresholding for DEG calling; Bonferroni correction for GO term and pathway enrichment tests
Statistical tests used
Test Applied to n Assumptions
Fold-change + significance/FDR thresholding for DEG calling (|log FC|>2, P<0.005, FDR<0.001) Comparison of CK vs Pb1000 root transcriptomes (4,614 DEGs) Two sequencing libraries (one CK, one Pb1000), no stated biological replicate number not stated
Hypergeometric test with Bonferroni correction GO term enrichment among up- and down-regulated DEGs All annotated DEGs vs background transcriptome not stated
Hypergeometric-based pathway enrichment (KOBAS) KEGG pathway enrichment among up- and down-regulated DEGs (Tables 2-3) DEG counts per pathway vs background gene counts per pathway not stated
qRT-PCR expression comparison (descriptive, no formal statistical test stated) Validation of 22 selected DEGs, and dose/time-course expression of 6 selected genes (Figure 5) Not stated (no replicate number given for qRT-PCR reactions) not stated
Approaches that could also have been used
  • DEGs between CK and Pb1000 were identified from what appears to be one library per condition rather than multiple biological replicates.
    Could also: Designing the RNA-seq comparison with several biological replicates per condition and analyzing with a replicate-aware tool (e.g., DESeq2 or edgeR, which model biological variance via negative binomial dispersion) — Replicate-based variance modeling would let the FDR/P-value estimates capture biological variability between individual plants, which a single-library comparison cannot directly reflect
  • DEGs were called using a combined logFC (>2 or <-2) plus P-value and FDR threshold.
    Could also: Applying a dedicated count-based differential expression model (e.g., DESeq2 Wald test or edgeR exact/GLM test) directly to raw read counts — These tools jointly estimate dispersion and fold-change shrinkage, which can refine significance calls particularly for genes with low counts or high variance
  • GO term enrichment used a hypergeometric test with Bonferroni correction.
    Could also: Using a Benjamini-Hochberg FDR correction for GO enrichment — Because GO terms are hierarchically related and tests are non-independent, an FDR-based correction is often preferred over the more conservative Bonferroni approach for controlling the proportion of false discoveries while retaining more true positives
  • qRT-PCR validation of 22 genes was described narratively as showing general agreement with RNA-seq, without a stated formal statistical comparison.
    Could also: Reporting a quantitative concordance measure, such as a Pearson or Spearman correlation coefficient between RNA-seq and qRT-PCR fold-changes, or a paired t-test/Wilcoxon signed-rank test on paired fold-change values — A quantitative correlation or paired test would give a numerical summary of agreement between the two platforms in addition to the qualitative description
  • Expression levels (FPKM, fold-change) are reported as single point estimates without measures of dispersion.
    Could also: Reporting variability (e.g., SD or CI) from biological replicates, or bootstrap/posterior credible intervals from a count-based DE model — Dispersion measures would convey the uncertainty around each expression estimate alongside the point value
  • The dose-response and time-course qRT-PCR data (Figure 5) are presented descriptively.
    Could also: A two-way ANOVA (dose x time) or mixed-effects model with post-hoc comparisons — Such a model would allow formal statistical testing of dose and time effects and their interaction across the multiple treatment combinations shown in Figure 5
Software: Trinity (de novo assembly) · Bowtie (read mapping) · RSEM (expression quantification) · goatools (GO enrichment) · KOBAS (KEGG pathway enrichment)

What was reproduced

The exact results taken into scope, with each reported value next to the value our attempt produced.

assembly_n_transcripts
Reported
68940 assembled transcripts (Trinity de novo assembly)
Reproduced
104011
did not match
assembly_n_unigenes
Reported
33337 unigenes
Reproduced
56894
did not match
assembly_N50
Reported
N50 = 1262 bp
Reproduced
1539
partial
assembly_mean_length
Reported
mean contig length 1226.63 bp (range 306-16101 bp)
Reproduced
1042.45
partial
functional_annotation_rate
Reported
70.88 percent (48864/68940) annotated across Nr, Swiss-Prot, GO, COG, KEGG combined (BLAST e-5)
Reproduced
67.14 percent (69826/104011) via diamond blastx vs Swiss-Prot only, a single-DB subset of the full pipeline described in the paper (the paper's 70.88 percent covers combined Nr/Swiss-Prot/GO/COG/KEGG); no Nr/COG/KEGG DB was staged for this reproduction
partial
DEG_total
Reported
4614 DEGs (1398 unigenes); 2154 up / 2460 down; abs(log2FC)>2, P<0.005, FDR<0.001
Reproduced
25169 DE transcripts (14417 unigenes); 10064 up / 15105 down -- edgeR exactTest with fixed BCV=0.1 (the exact DE tool/method used in the paper is undocumented/unavailable; the same thresholds were applied here)
did not match
GO_enrichment_terms
Reported
270 up / 107 down terms
Reproduced
goatools find_enrichment.py (BH FDR<0.05) run separately per DE-direction study set against a population of 69384 annotated background transcripts. Up-regulated study set (n=10064 DE transcripts): 774 significant terms total = BP 464 (282 enriched + 182 purified) + CC 87 (13 enriched + 74 purified) + MF 223 (159 enriched + 64 purified); enriched-only subset (closest analogue to the paper's 'up' count) = 454. Down-regulated study set (n=15105 DE transcripts): 702 significant terms total = BP 433 (251 enriched + 182 purified) + CC 102 (40 enriched + 62 purified) + MF 167 (96 enriched + 71 purified); enriched-only subset (closest analogue to the paper's 'down' count) = 387.
did not match

Assessments & scoring basis

Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.

🤖 AI curator · claude (ai-curator room) · v1.0 L1 27/100

An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.

🟡1. Data identity
🟡2. Endpoint comparability
🟡3. Location of the main deviation
🟡4. Cause of the deviation
🟡5. Derivability / plausibility
🔴6. Severity of the deviation
🟡7. Core claim
🟡8. Severity of the miss (overall human judgment)
Scoring basis — itemised

Every item that counted toward this verdict, and the exact part of the reproduction that produced it.

Supporting (toward a concern)
Content-critical question only partially held
+2 pts
From: Q5 · Derivability / plausibility 🟡
Content-critical question only partially held
+2 pts
From: Q7 · Core claim 🟡
Content-critical question only partially held
+2 pts
From: Q8 · Severity of the miss (overall human judgment) 🟡
Minor / cosmetic deviation
+1 pts
From: Q3 · Location of the main deviation 🟡
Minor / cosmetic deviation
+1 pts
From: Q4 · Cause of the deviation 🟡
Minor / cosmetic deviation
+1 pts
From: Q2 · Endpoint comparability 🟡
Concordant (toward reproduced)
Code + data deposited & functional
-2 pts
From: Data & code availability Available & functional
Total score +7

All ten in-scope pipeline stages completed cleanly (sacct ExitCode 0:0), but no reported number was matched 1:1: 104011 vs 68940 transcripts, 56894 vs 33337 unigenes, N50 1539 vs 1262 bp, 25169 vs 4614 DE features, 774/702 vs 270/107 GO terms. The deviations sit overwhelmingly on the input and method side — the paper's assembly incorporated 314823 ESTs and 17187 NCBI unigenes that were never deposited, its read-cleaning and DE tool are unspecified (forcing a fixed-BCV=0.1 no-replicate exactTest), and annotation here used Swiss-Prot alone (67.14 percent) against the paper's five-database 70.88 percent. This is a shared defect: our substitutions are the proximate cause, but the paper's underspecification is what made them necessary, so the numbers are not derivable from the deposited data without being fabrication-suspect. The qualitative core — a large Pb-responsive, GO-enrichable DE set skewed toward down-regulation — survives, so q7 is limited rather than refuted; KEGG (Tables 2-3) and qRT-PCR were not tested at all.

🤝
Reproduced automatically — and fairly

Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.

Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.

🚩 Report an error in this record

Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.

Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.