Corpus 1,272 assessed · 1,173 scored · 643 reproduced ≥75 · 168 flagged ·∅ 74.1/100
← New search

RNA-sequence analysis of primary alveolar macrophages after in vitro infection with porcine reproductive and respiratory syndrome virus strains of differing vir

PLoS One · 2014
41/100 3/4
⚑ Flagged for review — a reproduced result did not match the reported value

Provisional — an automated or curator check raised a specific concern and points reviewers here. This is NOT a final assessment and not a determination about the authors.

Why this verdict

The main result did not reproduce in this reproduction attempt. Where our recomputation produced values that differ from the published ones, those discrepancies are listed below. This is a single automated attempt — not peer review and not a finding of error or misconduct — and differences can also arise from data access, undocumented parameters or the computing environment. The verdict can be contested via “report an error”.

Reproduced on the brainbox compute brainarbeit.com
How its reproducibility compares
41/100
Reproducibility score
1.9 SD below mean
vs. all fields · 1173 studies
🎯 Scores higher than 4% of all assessed papers rank 1124 of 1173 scored

A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.

Reproduction agent’s raw note

Partial reproduction. Upstream pipeline steps (sickle trimming, TopHat2 alignment, Cufflinks/Cuffmerge/Cuffcompare assembly) reproduce within/near the paper's reported ranges for 7 of 8 samples, with the 8th (SRR1205844) a well-explained outlier (extreme viral load). Condition labels (Mock/LV/Lena) are not present in SRA metadata; they were inferred via viral-reference alignment, cleanly separating 3 LV + 3 Lena + 2 Mock samples -- which explains the paper's stated 9 libraries vs 8 SRA runs as a missing 3rd Mock replicate. Downstream differential-expression results (edgeR gene-level, Cuffdiff isoform/TSS/promoter-level) show the correct DIRECTION and RANKING (LV/Lena vs Mock >> LV vs Lena) but substantially lower significant-hit COUNTS than reported: edgeR DE genes are ~50-100x lower (6/2/1 vs 446/153/241); Cuffdiff isoform and TSS counts are ~30-65% of reported magnitude; Cuffdiff promoter-switching is nearly null (0/0/1 vs 14/6/7). Diagnostics (BCV=0.25, reasonable nominal p<0.05 hit counts before FDR correction, no MDS structure suggesting recoverable pig identity) indicate this is a genuine, well-substantiated STATISTICAL POWER shortfall -- not a pipeline execution failure -- stemming from two structural SRA-metadata gaps that cannot be recovered from the public deposition: (1) the original paired per-animal (3 pigs x 3 conditions) design removes inter-animal variance via blocking, which is unrecoverable without pig-identity labels, forcing an unpaired analysis here; (2) the Mock group has only 2 of the presumed 3 replicates. Wet-lab or purely descriptive results (e.g. clinical/histopathology findings) were out of scope and not attempted. All pipeline mechanics were verified sound at each stage (mapping rates, cuffcompare Sn/Sp>=92%, HTSeq counts, edgeR dispersion diagnostics) before attributing the DE-count gap to study design/power rather than execution.

These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.

✎ I am an author of this paper

Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.

Reason for the rerun

We email you a confirmation link first. The rerun is an objective re-measurement — it cannot change the verdict in your favour, only ask us to look again.

Provenance — full disclosure

When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.

Reproduced
2026-08-03
Rubric version
not recorded
Assessed by
Last updated
2026-08-03

Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.

Deep full-text extraction

Model: opus
Founding hypothesis

The study asks how primary porcine alveolar macrophages (PAMs) transcriptionally respond in vitro to European PRRSV strains of differing virulence, testing whether the low-virulence Lelystad (LV, subtype 1) and high-virulence Lena (subtype 3) strains induce distinct gene, isoform, transcription start site and promoter usage programs at 12 h post-infection.

Core claims
  • Infection of PAMs with either LV or Lena affects signaling pathways directly linked to the innate immune response, including IRF activation, RIG1-like receptors, TLRs and PKR pathways. finding
  • Interferon signaling is the pathway most strongly modulated during PAM infection and is crucial for transcriptional regulation upon PRRSV infection. finding
  • IFN-β1 and IFN-αω, but not IFN-α, are up-regulated following infection with either the LV or Lena strain. finding
  • Canonical pathways including the interplay between innate and adaptive immune responses, cell death, and TLR3/TLR7 signaling are down-regulated by both strains, with Lena triggering stronger down-regulation than LV. finding
  • PRRSV infection of PAMs produces a complex pattern of transcriptional and post-transcriptional regulation detectable as changes in gene expression, isoforms, alternative transcription start sites and differential promoter usage. finding
  • RNA-Seq applied to primary PAMs allows simultaneous characterization of gene expression, splice variants, TSSs and differential promoter usage, unlike the microarray studies used previously in this system. method
  • The RNA-Seq dataset from LV-, Lena- and mock-infected PAMs is deposited in the NCBI Sequence Read Archive (SRX352447) as a community resource. resource
  • The 'Role of Pattern Recognition Receptors in Recognition of Bacteria and Viruses' pathway response involved IRF7 but not IRF3. mechanism
Experimental setups
Assay System Perturbation Readout Platform
RNA-Seq (2×100 bp paired-end) Primary pulmonary alveolar macrophages from three 3-week-old piglets (Landrace x Large White sows × Pietrain boars), cultured in vitro Infection at MOI 0.5 with PRRSV LV strain or Lena strain vs. mock inoculation; cells collected 12 h post-infection Transcript abundance (counts/FPKM) for genes, isoforms, transcription start sites and promoter usage Illumina HiSeq 2000; TruSeq Sample Prep Kits (Illumina)
Immunoperoxidase staining (viral antigen detection) PRRSV-infected primary PAM cultures (duplicate of each infection/control) LV or Lena infection at MOI 0.5; fixed in acetone-100% methanol at −20°C at 12 h post-infection Percentage of viral antigen-positive cells (3 microscopic fields, minimum 200 cells per field) Monoclonal anti-nucleocapsid antibody P3/27, HRP-labeled goat anti-mouse secondary, 3-amino-9-ethylcarbazole substrate; Olympus light microscope
Quantitative PCR LV- and Lena-infected primary PAMs LV or Lena infection Threshold cycle (Ct) values confirming infection
Immunoperoxidase monolayer assay (IPMA) 3-week-old piglets from a PRRS-negative herd none (screening) PRRSV-negative status of donor piglets
PCR 3-week-old piglets from a PRRS-negative herd none (screening) PCV2-negative status of donor piglets
Microcapillary electrophoresis (RNA quality control) Total RNA from LV-, Lena- and mock-infected PAMs none RNA integrity/quality Agilent 2001 Bioanalyzer with RNA 6000 Nanochips
Spectrophotometric RNA quantification Total RNA from PAMs (TRIzol + RNeasy column extraction) none RNA concentration NanoDrop ND-1000
Bioinformatic differential expression and pathway analysis Nine cDNA libraries (3 LV, 3 Lena, 3 mock) mapped to Sus scrofa genome Sscrofa10.2.71 Pairwise comparisons LV vs mock, Lena vs mock, LV vs Lena (paired design by pig) Differentially expressed genes/isoforms/TSSs/promoters (FDR<0.05, FC≥1.5) and enriched canonical pathways FASTQC, Sickle, TopHat v2.0.8, Cufflinks v2.1.1/Cuffmerge/Cuffcompare/Cuffdiff, HTSeq-count, edgeR, Ingenuity Pathway Analysis (IPA)
Key results
  • 446 genes were differentially expressed between LV-infected and mock-infected PAMs; 153 between Lena and mock; 241 between LV and Lena. 446 / 153 / 241 genes
  • 'Interferon Signaling' was the top canonical pathway in LV vs mock and included 12 up-regulated genes (IFIT1, IFIT3, IFITM1, IFNβ1, IRF1, JAK2, MX1, OAS1, PSMB8, SOCS1, STAT1, STAT2). −log(p-value) = 1.08E+01; ratio = 3.87E-01; 12 genes
  • 14 up-regulated genes (ADAR, DDX58, DHX58, IFIH1, IFIT2, IFNβ1, IL10, IRF7, ISG15, NFKBIA, STAT1, STAT2, TNF, ZBP1) were involved in the 'Activation of IRF by Cytosolic Pattern Recognition Receptors' pathway in LV vs mock. −log(p-value) = 9.97E+00; ratio = 2.64E-01; 14 genes
  • IFN-β1 and IFN-αω were up-regulated after infection with either LV or Lena, whereas IFN-α was not.
  • Pathways covering innate–adaptive immune interplay, cell death and TLR3/TLR7 signaling were down-regulated by both strains, more strongly by Lena than by LV.
  • Differentially affected isoforms, TSSs and promoters differed between comparisons: isoforms 187 (LV vs mock), 72 (Lena vs mock), 34 (LV vs Lena); TSSs 240 / 93 / 50; promoters 14 / 6 / 7. 187/72/34 isoforms; 240/93/50 TSSs; 14/6/7 promoters
  • The percentage of PAMs infected was 21% for LV and 16% for Lena on average, corroborated by qPCR Ct values. 21% vs 16%
  • The 'Role of PKR in Interferon Induction and Antiviral Response' and 'Retinoic Acid Mediated Apoptosis Signaling' pathways were up-regulated and shared four genes (CASP8, BID, IFNβ1, IRF1). −log(p-value) = 7.03E+00; ratio = 2.5E-01 (PKR pathway)
Key statistics
  • count 446 differentially expressed genes (LV vs mock), 153 (Lena vs mock), 241 (LV vs Lena) (Significance threshold FDR<0.05 and fold change ≥1.5)
  • pvalue −log(p-value) = 1.08E+01, ratio 3.87E-01 (Interferon Signaling, top IPA canonical pathway, LV vs mock (right-tailed Fisher's Exact Test))
  • pvalue −log(p-value) = 9.97E+00, ratio 2.64E-01 (Activation of IRF by Cytosolic Pattern Recognition Receptors, LV vs mock)
  • pvalue −log(p-value) = 8.55E+00, ratio 1.78E-01 (Role of Pattern Recognition Receptors in Recognition of Bacteria and Viruses, LV vs mock)
  • pvalue −log(p-value) = 7.03E+00, ratio 2.5E-01 (Role of PKR in Interferon Induction and Antiviral Response, LV vs mock)
  • count 29,900 annotated genes and 58,347 isoforms expressed in infected and non-infected PAMs (Total expressed features across all libraries)
  • mean 14,483,298 (LV), 17,381,381 (Lena), 10,473,546 (mock) reads per strand (Mean sequencing depth per group; 11–17% of reads filtered out, 73–87% of passing paired reads mapped to Sscrofa10.2.71)
  • other Ct = 15.6 (LV) and 20.5 (Lena); 21% and 16% infected cells respectively (qPCR and immunoperoxidase quantification of infection at 12 h post-infection, MOI 0.5)

Statistical methods review

Model: sonnet

A neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.

The study compared RNA-Seq transcriptomes of porcine alveolar macrophages from 3 pigs, each split into LV-infected, Lena-infected, and mock-infected fractions (paired design, mock subtracted per pig). Gene-level differential expression was tested with edgeR (negative binomial generalized linear model), while isoform, transcription-start-site, and differential promoter usage were tested with Cuffdiff; features were called significant at FDR < 0.05 and fold change ≥ 1.5. Pathway/functional enrichment was assessed in Ingenuity Pathway Analysis (IPA) using a right-tailed Fisher's Exact Test, and results were reported mainly as FPKM values, fold changes, gene counts, and pathway p-values.

Replicationbiological Sample size3 pigs, each contributing paired LV-infected, Lena-infected, and mock-infected PAM fractions (9 libraries total); infection-rate assessment was done in duplicate per condition GroupsLV-infected vs mock, Lena-infected vs mock, LV-infected vs Lena-infected Pairingpaired Randomization/blindingnot stated Dispersionunclear Exact p-valuesno Effect sizesyes Confidence intervalsyes Multiplicity correctionFalse discovery rate (FDR) thresholding (FDR < 0.05) combined with a fold-change cutoff (≥1.5); the specific FDR algorithm (e.g., Benjamini-Hochberg) is not explicitly named
Statistical tests used
Test Applied to n Assumptions
edgeR generalized linear model (negative binomial, conditional weighted likelihood for overdispersion) Gene-level differential expression: LV vs mock, Lena vs mock, LV vs Lena 3 pigs (paired, mock-subtracted per pig) not stated
Cuffdiff statistical testing Differential isoform, TSS, and promoter usage: LV vs mock, Lena vs mock, LV vs Lena 3 replicates per condition (9 libraries total) not stated
Right-tailed Fisher's Exact Test (IPA) Canonical pathway and biological function enrichment among differentially expressed genes 413 genes mapped to the IPA database (LV vs mock) not stated
Bayesian inference method for confidence intervals on FPKM estimates (Cufflinks) Transcript/gene abundance estimation not stated
Approaches that could also have been used
  • Gene-level differential expression across the three pairwise comparisons was tested with edgeR's negative-binomial generalized linear model.
    Could also: DESeq2 (also negative-binomial based) or limma-voom could also be used for the same count data. — These are widely used alternative RNA-Seq DE frameworks with different dispersion-estimation and shrinkage strategies, and comparing results across tools can illustrate how method choice affects the called gene set.
  • Significance for genes, isoforms, TSSs, and promoter usage was defined using a combined FDR < 0.05 and fold-change ≥ 1.5 threshold.
    Could also: Reporting the continuous adjusted p-value and fold-change/log-fold-change with confidence intervals for all tested features (not only those passing the cutoff) could also be used. — This would let readers apply their own significance and effect-size thresholds and see the full distribution of effects rather than only the features that passed the chosen cutoff.
  • A paired design was used in which mock values were subtracted from treatment values per pig before computing group averages and variance.
    Could also: A mixed-effects (or generalized linear mixed) model with pig included as a random effect could also be used on the raw counts. — This approach can incorporate the pairing directly into the model rather than through pre-subtraction, which may be useful when variances differ between conditions or when additional covariates need to be modeled.
  • Pathway and functional enrichment among differentially expressed genes was assessed with a right-tailed Fisher's Exact Test in IPA on an already-thresholded gene list.
    Could also: Gene Set Enrichment Analysis (GSEA), which uses the full ranked gene list rather than a hard significance cutoff, could also be used. — Rank-based enrichment methods can capture coordinated but individually sub-threshold expression changes across a pathway, complementing an over-representation test applied to a fixed gene list.
  • Differential isoform, TSS, and promoter usage were assessed using Cuffdiff's built-in statistical testing.
    Could also: DEXSeq or a similar count-based differential exon/isoform usage tool could also be applied to the same alignments. — These tools use an alternative statistical framework for testing usage differences and can be run alongside Cuffdiff as a cross-check on splicing- and promoter-usage calls.
  • Confidence intervals for FPKM estimates were obtained via a Bayesian inference method within Cufflinks.
    Could also: Bootstrap-based confidence intervals (as used by tools like kallisto/sleuth) could also be used for transcript abundance uncertainty. — Bootstrapping offers an alternative, model-light way to quantify quantification uncertainty and can be a useful complement to model-based Bayesian intervals.
Software: FASTQC · Sickle · TopHat v2.0.8 · Cufflinks/Cuffmerge/Cuffcompare v2.1.1 · HTSeq · edgeR · Cuffdiff · Ingenuity Pathway Analysis (IPA)

What was reproduced

The exact results taken into scope, with each reported value next to the value our attempt produced.

sickle_trim_pct_removed
Reported
Sickle quality/length trimming (Q20, min length 50) removed ~11-17% of reads (paper Methods)
Reproduced
7/8 samples: 8.27-8.73% of reads removed; 1/8 (SRR1205844): 0% removed (all reads already passed Q20/length-50 filters cleanly)
partial
tophat2_overall_mapping_rate
Reported
73-87% overall read mapping rate to Sscrofa10.2.71 (paper Results/Methods)
Reproduced
7/8 samples: 80.4%, 83.4%, 83.6%, 82.4%, 85.4%, 86.1%, 87.7% (within/near reported range); 1/8 (SRR1205844): 57.7%, a genuine outlier explained by that sample carrying by far the highest viral RNA load observed (6,973,697 LV-aligned reads via bowtie2), which displaces host-genome-mapping reads rather than indicating a pipeline problem
within tolerance
cufflinks_merged_transcriptome_size
Reported
~29,900 genes / 58,347 isoforms assembled genome-wide (paper Results)
Reproduced
25,211 genes / 32,728 isoforms in the cuffmerge+cuffcompare merged assembly (cuffcompare vs Ensembl release-71 annotation: transcript-level Sn=94.0%, Sp=92.9%, locus-level Sn/Sp>=99.8%, indicating a high-quality assembly that is nonetheless smaller in raw isoform count than reported, likely reflecting cufflinks/cuffcompare version differences and/or stricter merging in this run)
partial
edger_de_genes_LV_vs_Mock
Reported
446 DE genes (FDR<0.05, |FC|>=1.5)
Reproduced
6 DE genes (FDR<0.05, |FC|>=1.5); 372 genes at nominal p<0.05 before FDR correction, showing the underlying signal exists but does not survive multiple-testing correction at this sample size/design
did not match
edger_de_genes_Lena_vs_Mock
Reported
153 DE genes (FDR<0.05, |FC|>=1.5)
Reproduced
2 DE genes (FDR<0.05, |FC|>=1.5); 55 genes at nominal p<0.05
did not match
edger_de_genes_LV_vs_Lena
Reported
241 DE genes (FDR<0.05, |FC|>=1.5)
Reproduced
1 DE gene (FDR<0.05, |FC|>=1.5); 235 genes at nominal p<0.05
did not match
cuffdiff_isoform_de_counts
Reported
Significant DE isoforms: LV-vs-Mock=187, Lena-vs-Mock=72, LV-vs-Lena=34
Reproduced
LV-vs-Mock=130, Lena-vs-Mock=26, LV-vs-Lena=17 (same order of magnitude and same ranking as reported, running consistently 30-65% of the reported counts, likely reflecting the same reduced statistical power as the gene-level edgeR result: no recoverable per-pig pairing and only 2 Mock replicates vs the original 3-pigs-x-3-conditions paired design)
partial
cuffdiff_tss_de_counts
Reported
Significant DE TSS groups: LV-vs-Mock=240, Lena-vs-Mock=93, LV-vs-Lena=50
Reproduced
LV-vs-Mock=134, Lena-vs-Mock=28, LV-vs-Lena=18 (same ranking, ~30-56% of reported magnitude)
partial
cuffdiff_promoter_de_counts
Reported
Significant differential promoter use: LV-vs-Mock=14, Lena-vs-Mock=6, LV-vs-Lena=7
Reproduced
LV-vs-Mock=0, Lena-vs-Mock=0, LV-vs-Lena=1 -- promoter-switching test (sqrt(JS) distance) essentially null in this reproduction, the most power-sensitive of the Cuffdiff tests and the one most degraded by losing per-pig pairing
did not match
condition_assignment_inference
Reported
Paper describes a 3-pigs x 3-conditions (Mock/LV/Lena) design (9 libraries)
Reproduced
SRA metadata for SRX352447 carries no condition/pig labels; inferred condition per run via bowtie2 alignment to a combined LV(M96262.2)+Lena(JF802085.1) reference and idxstats read counts. Clean separation: infected samples carried 6,578-6,973,697 reads to their dominant strain vs 277-1,336,907 to the other; Mock samples carried only 277-703 reads to either (background level). Result: 3 LV + 3 Lena + 2 Mock = 8 samples, consistent with the reported design missing exactly its 3rd Mock replicate (n_reported=9 vs n_observed=8)
within tolerance

Assessments & scoring basis

Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.

No assessment has been recorded yet.
🤝
Reproduced automatically — and fairly

Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.

Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.

🚩 Report an error in this record

Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.

Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.