A bioinformatic pipeline for simulating viral integration data.
The main results reproduced: recomputed values matched the published ones within tolerance.
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
1:1 (described well enough, reproduced exactly). simvi is a deterministic Snakemake simulator of viral/vector integration; the Zenodo v0.0.2 deposit (= github aehrc/vector-integration-simulation-pipeline commit af917d5) ships a fixed-seed test config plus reference example_results for self-comparison. Ran the test config on «our HPC» (SLURM «job», node n127) via snakemake --use-conda with the repo's PINNED env yamls (art=2016.06.05, bwa=0.7.17, samtools=1.10, python=3.7). Results: sim_ints BED 4/4 byte-identical (SHA256) and sim_reads FASTQ 8/8 byte-identical to the shipped reference; sorted BAM 4/4 with identical alignment-record counts (710/710/560/720) and byte differences confined to the samtools @PG header (run command/path) -> within-tol by design. Config structure exact: 2 conditions (frag_len 250/500) x 2 replicates = 4 runs, int_num=3 integrations/run (confirmed in annotated tsv), seeds 12345+123*k (determinism confirmed). NOT attempted (out of scope): the prose-described production-scale worked example (100 integrations, AAV2 into full human chr1) -- its exact config + full reference genomes are not part of the reproducible shipped test artifact; and any downstream Isling detection-benchmark results (different repo). Overall: clean, auditable, deterministic reproduction. Grades remain provisional pending human sign-off (AUDIT.md).
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-25
- Rubric version
- not recorded
- Assessed by
- —
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnet- ★ A snakemake-based pipeline was developed to simulate integration of a viral or vector genome into a host genome, including sub-genomic fragment integration, structural variation, and host-site deletions. method
- ★ The pipeline generates simulated sequencing reads (via ART) from the integrated reference and annotates which reads cross each integration junction. method
- ★ Primary outputs are simulated fastq reads, a table of integration properties/locations, and a BED file of junctions with supporting reads. resource
- ★ This simulation method was used to evaluate the performance of viral integration detection tools (Isling). finding
- ★ Example dataset simulates wild-type AAV2 (NC_001401.2) integration into human chromosome 1 (GRCh38, NC_000001.11), with 100 integrations per replicate. resource
- ★ Simulation parameters (sub-genomic fragment probability, rearrangement/deletion probability, junction gap/overlap probability, host deletion probability) were set to reflect known biological properties of AAV integration. method
- Code and example data are publicly available via GitHub and the CSIRO Data Access Portal. resource
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| in silico integration simulation | AAV2 viral genome (GenBank NC_001401.2) into human chromosome 1 (GenBank NC_000001.11) | simulated integration with configurable sub-genomic fragment, rearrangement, deletion, junction gap/overlap, and host deletion probabilities | location and structural properties of simulated integrations (int-info.tsv) | Python/Biopython script (insert_virus.py) in Snakemake pipeline |
| simulated paired-end next-generation sequencing | host reference containing simulated integrations and episomal sequences | none (read simulation with specified read length, fold-coverage, fragment length/SD, sequencing system error profile) | simulated paired-end fastq reads | ART (art_illumina) |
| junction-crossing read annotation and alignment | simulated reads aligned to integrated reference (SAM/BAM) | none | reads crossing each integration junction; BED3 file of supported junction locations | Python script using Pysam |
- – Pipeline reproduces complex integration characteristics: sub-genomic fragment integration, structural variation, and host-genome deletions at integration sites
- – Example dataset simulated 100 integrations per replicate of AAV2 into human chr1 100 integrations/replicate
- – Probability of sub-genomic (rather than whole-genome) fragment integration set to 0.5, with minimum fragment length 50 bp p=0.5, min 50 bp
- – Probabilities of rearrangement and deletion of the integrated fragment both set to 0.1 p=0.1 each
- – Probabilities of a gap or overlap at each host/virus junction both set to 0.2, with mean junction length of 1 bp p=0.2 each; mean=1 bp
- – Probability of a host-genome deletion at each integration site set to 0.2, with deletion length drawn from Poisson distribution (mean 20 bp) p=0.2; mean=20 bp
- count 100 integrations per replicate (AAV2-into-chr1 example dataset)
- other p_whole/sub-genomic fragment probability = 0.5 (probability whole vs. sub-genomic viral genome integrated)
- other min sub-genomic fragment length = 50 bp (minimum length parameter for sub-genomic fragments)
- other p_rearrange = 0.1, p_delete = 0.1 (structural variation probabilities of integrated fragment)
- other p_gap = 0.2, p_overlap = 0.2 (host/virus junction type probabilities)
- mean lambda_junction mean = 1 bp (Poisson-distributed junction length)
- other p_host_deletion = 0.2 (probability of host deletion at integration site)
- mean lambda_host_deletion mean = 20 bp (Poisson-distributed host deletion length)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a Data in Brief data-descriptor article accompanying a bioinformatics tool (Isling) rather than a hypothesis-testing study. It describes a Snakemake/Python simulation pipeline that generates synthetic viral integration data (host+virus reference with integrations, simulated reads via ART, and annotation tables), with event properties (fragment length, junction type, deletions) drawn from user-set probabilities and Poisson distributions. No experimental groups are compared and no inferential statistical tests are reported; results are described as pipeline outputs (tables, fasta/fastq/BED files) rather than statistically analyzed findings.
-
Integration event properties (fragment length, junction length, host deletion length) are modeled using a Poisson distribution with a user-set mean (lambda).↳ Could also: A negative binomial distribution (which allows variance to differ from the mean) could also be used to parameterize these lengths. — When empirical integration/deletion length data show more variability than a Poisson model implies (overdispersion), a negative binomial or other overdispersed count distribution can offer additional flexibility in matching simulated data to observed biological variability.
-
The pipeline uses fixed probability parameters (e.g., p_whole, p_rearrange, p_delete, p_gap, p_overlap, p_host_deletion) set per condition to control simulated integration characteristics.↳ Could also: Parameters could also be drawn from prior distributions (a hierarchical or Bayesian simulation approach) rather than fixed per-condition values. — Sampling parameters from distributions rather than fixing them can generate a wider and more continuous range of simulated scenarios, which some users might find useful for stress-testing detection tools across a broader parameter space.
-
Multiple replicates of each parameter combination (condition) are generated using different random seeds, and the paper does not report a benchmark statistical comparison (e.g., of detection tool performance) using this simulated data in the current article.↳ Could also: When simulated replicates are later used to benchmark integration-detection tools (as referenced in the companion Isling paper), performance metrics across replicates could also be summarized with confidence intervals or compared using paired tests (e.g., paired t-test or Wilcoxon signed-rank test) across conditions. — Since replicates share the same underlying condition, paired analyses can account for condition-level variability when comparing detection accuracy between methods, and reporting confidence intervals alongside point estimates conveys the precision of any such benchmark comparisons.
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-35496474
Paper: Scott et al. (2022) "A bioinformatic pipeline for simulating viral integration data." Data in Brief 42:108161. DOI 10.1016/j.dib.2022.108161.
Nature of the paper: A Data in Brief descriptor of a simulation pipeline (not a wet-lab study). It describes a Snakemake workflow that simulates integration of a virus/vector genome into a host genome and emits a ground-truth set of integration sites plus simulated sequencing reads, used to benchmark viral-integration detection software (companion tool "Isling").
Code: github.com/aehrc/vector-integration-simulation-pipeline (Snakemake).
"Data": Zenodo 10.5281/zenodo.6403449 = the v0.0.2 source release ZIP
(780 KB) — i.e. a snapshot of the same code repo, including the example_results/
that ship with the repo. CSIRO DAP 10.25919/m529-q062 = code + example dataset.
There is no separate experimental dataset; the deposit is the software.
In scope (pipeline-derived, reproducible)
The pipeline is deterministic: the test config pins initial_seed: 12345 and the
same seed drives both integration simulation (insert_virus.py) and read
simulation (art_illumina). The repo ships example_results/ and the README
instructs users to "compare these results to those in example_results." This is
the canonical, self-contained reproduction target.
- R1 — Integration sites: running
snakemake --configfile test/config/simulation.ymlregeneratessim_ints/cond{0,1}.rep{0,1}.int-info.bed. Compare byte/field-exact against the shippedexample_results/.../sim_ints/. - R2 — Simulated reads: regenerate
sim_reads/cond*.rep*[12].fqand compare against shippedexample_results/.../sim_reads/*.fq(art_illumina is seeded). - R3 — Config-implied counts: 2 conditions (frag_len 250 & 500) x 2 replicates = 4 simulation runs; int_num = 3 integrations per replicate. (Paper's worked example elsewhere cites 100 integrations/replicate for AAV2 into chr1 — that is the illustrative production config, NOT the shipped test config; the reproducible shipped artifact is the test config with int_num 3.)
- R4 (softer) — Aligned reads:
sim_reads/*.sorted.bamregenerated via bwa. BAM is alignment-tool/version dependent, so treated as within-tol, not exact.
Out of scope
- The production-scale worked example (100 integrations, AAV2/chr1, full human genome) — parameters are described in prose but the exact config + reference genomes for that run are not shipped as the reproducible test artifact; not attempted as an exact 1:1.
- Any downstream Isling detection-benchmark results (different paper/repo).
- Hardware-specific runtime/throughput claims (Dell PowerEdge C6525 etc.) — not reproducible numbers, environment-specific.
Pipeline named per result
All in-scope results come from the single Snakemake pipeline
vector-integration-simulation-pipeline (rules: simulate integrations
[insert_virus.py] -> simulate reads [art_illumina] -> align [bwa] -> summarize).
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.