miRbiom: Machine-learning on Bayesian causal nets of RBP-miRNA interactions successfully predicts miRNA profiles.
The main results reproduced: recomputed values matched the published ones within tolerance.
- ✓Reported values were directly comparable
- 🟡Could not use the authors’ exact input data
- 🟡A deviation arose in the data or preprocessing
- 🟡A deviation was attributed to the published material
- 🟡Reported values were not (fully) derivable from the shared data
- 🟡The deviation was non-trivial in magnitude
- 🟡The central claim did not (fully) hold under reproduction
- 🟡Overall, the reproduction showed a material discrepancy
A 0–100 reproducibility-quality score from the per-question grades, shown as a z-score: standard deviations above (+) or below (−) the mean of comparable assessments.
▸Reproduction agent’s raw note
INTERIM. C1 reproduced EXACTLY: ran authors' shipped validation.py on shipped benchmarking/test.csv (1075 miRNAs) in era-correct py2.7.15/pandas0.24.2/numpy1.16.5 on a «our HPC» compute node -> RMSE 1.0670394705731083, RMAPE 8.943718334276047, Accuracy 91.06, all bit-identical to README. Repo cloned at pinned commit; example RNA-seq + ground-truth tables present. C2 (regenerate predictions independently via xgb.py) is blocked: the 146MB trained-model tarball host scbb.ihbt.res.in is currently offline (connection timeout on 80+443), no mirror/cache. Model TRAINING (CLIP-seq ~10.8TB + TCGA) was always out of scope (not shipped). Polling the model host for recovery to complete C2.
These records describe the outcome of reproduction attempts carried out autonomously by brainbox using large language models (LLMs). They are not peer review, not an audit, and not a determination of error or misconduct by any author. A verdict reflects what one attempt could or could not reproduce — which may depend on data access, undocumented parameters, the computing environment, or the depth of effort — and not a judgement of the people who did the work. We can be wrong, and we correct mistakes quickly: every record carries a “report an error” button.
Assessment versions
Every reproduction run is kept as an immutable version — anchored to the data as it stood, with a tamper-evident chain hash. A rerun (e.g. after an author updates a deposit) adds a new version; the previous one stays on record.
-
v1 current initial assessment Score 50assessed: 2026-06-18 ⛓ f1df5071da34
✎ I am an author of this paper
Updated or fixed a deposit, or is there an erratum? Ask us to re-run the metrics. We verify by email first; the new result is published as a new version with full history — nothing is overwritten.
Provenance — full disclosure
When this reproduction was carried out, which methodology version was used, and by whom — so the record can be audited and checked independently.
- Reproduced
- 2026-06-25
- Rubric version
- v1.0
- Assessed by
-
🤖 AI curator · claude (ai-curator room) · v1.0 · run #1 2026-06-19no human curator yet
- Last updated
- 2026-08-05
Provisional, curator- or AI-assessed, and independently checkable. A reproduction outcome states what one attempt could reproduce — not a judgement of the authors.
Deep full-text extraction
Model: sonnetThat RNA binding proteins (RBPs) beyond the canonical Drosha/DGCR8/Dicer complex form conditional causal networks with miRNAs, and that modeling these RBP-miRNA interaction networks (via Bayesian causal networks and machine learning) can accurately explain and predict spatio-temporal human miRNA biogenesis/expression profiles directly from RNA-seq data, without separate miRNA profiling experiments.
- ★ RBPs beyond Drosha/DGCR8/Dicer are involved in regulating miRNA biogenesis and explain its spatio-temporal nature mechanism
- ★ Bayesian causal networks (BNA) built from RBP-miRNA interaction data across many conditions can reason/explain miRNA formation method
- ★ The Bayesian network components can be molded into an XGBoost machine learning system to quantitatively predict miRNA formation levels and profiles method
- ★ miRbiom predicts expression of 1,204 human miRNAs directly from RNA-seq data alone, without needing miRNA-seq or array experiments finding
- ★ miRbiom achieves consistently high average accuracy (91%) when tested across a large number of experimentally established conditions finding
- ★ miRbiom is implemented as an interactive open-access web server enabling miRNA profile prediction and downstream functional analysis resource
- RBP-miRNA interactions vary across cell types/conditions, indicating multiple regulatory paths beyond the canonical uniform biogenesis model mechanism
| Assay | System | Perturbation | Readout | Platform |
|---|---|---|---|---|
| CLIP-seq (PAR-CLIP, HITS-CLIP, eCLIP, iCLIP, CLEAR-CLIP, irCLIP) | Human, multiple cell lines/tissues (e.g., HepG2, K562) | none (interaction mapping) | RBP binding sites on pri-/pre-miRNA regions | ENCODE and GEO databases |
| RNA-seq | Human tissues/cell lines across 47 experimental conditions (e.g., thyroid, heart, blood, cancer tissues) | disease/tissue/treatment (varies by dataset, e.g., tumor vs normal, infection, siRNA/shRNA transfection) | Gene and pre-miRNA expression (RPKM/FPKM) | Bowtie-based Seqmap alignment, rSeq quantification |
| sRNA-seq/miRNA-seq | Human tissues/cell lines, same 47 conditions | disease/tissue/treatment (varies by dataset) | Mature miRNA expression (TPM) | miRDeep2 (mapper.pl, quantifier.pl) |
| Bayesian network analysis (BNA) | Computational model using expression data of pre-miRNAs, mature miRNAs, RBPs, and PPI-linked genes | none (computational modeling) | Causal network structure explaining miRNA:RBP associations | — |
| XGBoost machine learning | Computational model built on Bayesian network components | none (computational modeling) | Predicted mature miRNA expression/profile | XGBoost |
| Protein-protein interaction (PPI) network mapping | Human RBPs and interacting partners | none | RBP interaction partners (up to 8 steps) | STRING database |
- – miRbiom achieved high average accuracy across a large number of experimentally established datasets and conditions 91%
- – Prediction models were successfully developed for 1,204 human miRNAs, detectable directly from RNA-seq data alone 1,204 miRNAs
- – RBP-miRNA interactions and their contribution to miRNA biogenesis varied across cell types/conditions
- – Large-scale multi-platform data (CLIP-seq, RNA-seq, sRNA-seq) integrated to build causal networks >25TB data, 47 conditions
- other 91% average accuracy (miRbiom prediction performance across tested experimental conditions)
- count 1,204 human miRNAs modeled (Number of miRNAs for which expression prediction models were built)
- count 1,881 pre-miRNAs, 2,588 mature miRNAs (miRBase version 21 human miRNA sequences used)
- count 1,230 CLIP-seq samples for 155 RBPs (~10.8Tb) (CLIP-seq data collected from ENCODE and GEO)
- count >25TB of data across 47 experimental conditions (Total high-throughput data volume and conditions studied)
- count 21 experiments (~15.6TB) for RNA-seq/sRNA-seq (Source NGS data collected from GEO/SRA)
- count 138 RBPs across 82 CLIP-seq conditions; 32 conditions common to all three data types (Overlap of CLIP-seq, RNA-seq, and sRNA-seq experimental conditions used for network/model building)
Statistical methods review
Model: sonnetA neutral, descriptive read of the statistical approach — what was done, and (for shared learning, not as criticism) what could also have been done.
This is a computational bioinformatics study that processed >25 TB of public CLIP-seq, RNA-seq, and miRNA-seq data across 47 experimental conditions to reconstruct Bayesian causal networks of RBP-miRNA interactions, which were then modeled using XGBoost gradient boosting to predict expression profiles for 1,204 human miRNAs. Batch effects across heterogeneous public datasets were removed with a two-way ANOVA linear model implemented in the limma R package. Binding site validity was established by requiring concordance of a read-depth threshold criterion with at least one of two independent peak-callers (Piranha or PEAKachu). Overall predictive performance is summarized in the abstract as a single average accuracy figure (91%).
| Test | Applied to | n | Assumptions |
|---|---|---|---|
| Two-way ANOVA linear model (via limma R package) | Batch effect correction applied to RNA-seq and sRNA-seq normalized expression values across all experimental conditions prior to network analysis | — | not stated |
| Bayesian Network Analysis (BNA) | Reconstruction of directed causal RBP-miRNA association networks; performed separately for each experimental condition | 32 experimental conditions common to CLIP-seq, RNA-seq, and sRNA-seq data | not stated |
| XGBoost gradient boosting (machine learning model) | Quantitative prediction of mature miRNA expression levels for 1,204 miRNAs from RBP/network-component expression features derived from RNA-seq | — | not stated |
| Peak calling — Piranha (binomial/negative-binomial scoring) and PEAKachu (adaptive mode) | Identification and validation of RBP binding sites on pri-miRNA and pre-miRNA regions from CLIP-seq reads | 1,230 CLIP-seq samples from 155 RBPs | not stated |
-
Batch effects were removed using a two-way ANOVA linear model via the limma R package applied to already-normalized expression values (RPKM/FPKM/TPM)↳ Could also: ComBat-seq (negative-binomial model, sva package) could also be applied to raw count matrices before normalization — ComBat-seq is designed specifically for count-based RNA-seq data and corrects batch variation in a manner that preserves count-data distributional properties; operating on raw counts before normalization can reduce over-correction of biological variance compared to correcting already-normalized values
-
CLIP-seq binding sites were validated by requiring a read-count threshold (≥5 reads, ≥2 supporting samples) concordant with at least one of two peak-callers↳ Could also: The IDR (Irreproducible Discovery Rate) framework could also be used to quantify peak reproducibility across biological or technical replicates with a calibrated false-discovery rate — IDR provides a probabilistic, statistically grounded measure of reproducibility, yielding an FDR-like threshold rather than a fixed read-count cutoff, which may generalize more consistently across datasets with varying sequencing depth
-
Bayesian Network Analysis was chosen to learn directed causal network structure from multi-platform expression data↳ Could also: Constraint-based algorithms (e.g., PC algorithm), score-based continuous methods (e.g., NOTEARS), or LASSO-regularized regression networks could also infer directed regulatory relationships — Each approach encodes different assumptions: constraint-based methods rely on conditional independence tests, NOTEARS uses continuous optimization with an acyclicity constraint, and LASSO-based methods impose sparsity; comparing results across approaches can help assess robustness of inferred edges
-
XGBoost was used to map network-component expression features to miRNA expression levels↳ Could also: Random forests, elastic-net regularized regression, or feed-forward neural networks could also model this feature-to-output relationship — Random forests provide built-in permutation-based feature importance without requiring learning-rate tuning; elastic-net regression offers more interpretable linear coefficients; comparing multiple learners on a held-out test set would allow model selection to be grounded in cross-validated generalization performance
-
Overall predictive performance is summarized as a single average accuracy value (91%)↳ Could also: Pearson or Spearman correlation between predicted and observed expression values, RMSE, R², or precision-recall AUC could also quantify model performance for continuous expression prediction — For quantitative expression-level prediction, correlation and error-based metrics capture the magnitude and direction of prediction errors across the full dynamic range, information that a single accuracy threshold (which depends on a binarization cutoff) does not convey
-
Expression data were normalized to RPKM/FPKM for RNA-seq and TPM for sRNA-seq before Bayesian network inference↳ Could also: Variance-stabilizing transformation (VST) or regularized log transformation (rlog) from DESeq2 could also be applied to stabilize variance across the expression range before network learning — VST and rlog reduce the dependency of variance on mean expression level, which can improve the reliability of correlation-based edges inferred at low-expression ranges where RPKM/FPKM values are noisiest
What was reproduced
The exact results taken into scope, with each reported value next to the value our attempt produced.
Scope — pmid-34637468 (miRbiom)
Paper: Pradhan UK, Sharma NK, Kumar P, Kumar A, Gupta S, Shankar R (2021) miRbiom: Machine-learning on Bayesian causal nets of RBP-miRNA interactions successfully predicts miRNA profiles. PLoS ONE 16(10): e0258550. PMID 34637468 · PMC8509996 · DOI 10.1371/journal.pone.0258550
Repo: https://github.com/SCBB-LAB/miRbiom @ commit
b169921ac5e1a70f11fa6e5fa613525942bb15e4 (pushed 2021-11-17).
Trained models: xgb_all.tar.gz (146 MB, md5 26e2178cebf77254f7d68c35ced79edb)
from https://scbb.ihbt.res.in/miRbiom-webserver/SC/xgb_all.tar.gz → 3621 files
(≈1,204 miRNAs × 3: <miR>_final_input_new.sav XGBRegressor + 2 feature lists).
What the shipped tool actually does (the reproducible pipeline)
xgb.py <rnaseq_expression.txt>:
- Reads a 2-column RNA-seq gene-expression profile (Ensembl gene ID + value).
- For each of 1,204 mature miRNAs, loads that miRNA's feature gene list
(
xgb_all/<miR>_gene_ense_feature_new) and its pickled XGBoost regressor (xgb_all/<miR>_final_input_new.sav, objectivereg:linear). - If ≥40% of the miRNA's feature genes are expressed (>0), predicts the miRNA's
relative expression; emits
miR , predicted , miRBase-id , mirTarBase-id , log2(pred).benchmarking/validation.py <test.csv>: given a (miRNA, Predicted, Actual) table, prints RMSE, MAPE / RMAPE, and Accuracy(%) = 100 − MAPE.
IN SCOPE (pipeline-derived, attempted)
- C1 — Benchmark metric reproduction (shipped data). Run the authors'
validation.pyon the shippedbenchmarking/test.csv(example1's predicted vs actual miRNA expression) → reproduce README's statedRMSE 1.0670…,MAPE 8.94…,Accuracy 91.06%. Closest pinnable numeric claim. - C2 — End-to-end prediction. Run
xgb.pyon the shippedexample1/2/3.txtRNA-seq inputs with the downloaded trained models, build the (miRNA,Predicted, Actual) table against the shipped*_miRNA_actual_expression.txt, runvalidation.py→ compare reproduced accuracy to the paper's average 91% (range 87–94%) (Abstract / Results) and README's 91.06% for example1. - Dataset profiling — GSE56862 (the harvested GEO accession) and the repo-shipped example/model data.
OUT OF SCOPE (not attempted — not reproducible from shipped artifacts)
- Model training itself: the Bayesian causal-network construction from CLIP-seq (~10.8 TB, 1,230 samples / 155 RBPs) and XGBoost training on TCGA (17,737 RNA-seq + 11,717 miRNA-seq samples). Neither the CLIP-seq processing, the Bayesian DAG estimation, nor the training feature matrices/labels are shipped — only the trained models are. So training-derived numbers (54.73%/44.67% static-PPI, 41.83%/35.56% swapped-net, p<1e-48, the 431-sample validation split) are not reproducible here.
- Wet-lab / webserver-only components.
Known code issues (minimal faithful patches, documented in AUDIT.md)
xgb.pyusessys.argvandmath.logbut never importssysormath→ crashes as shipped. Patch = addimport sysandimport math(no logic change).- Python 2.7 +
printstatements; models pickled under Py2 with xgboostreg:linear→ require an era-correct env (python=2.7, xgboost=0.90, scikit-learn=0.20) built with conda on the «our HPC» front node.
Assessments & scoring basis
Each contributor’s verdict, the per-question basis, and the auditable, itemised worksheet behind it.
An automated assessment. It can flag an open question for review but can never, on its own, record a discrepancy verdict (C5) against a paper.
Automated reproduction checks whether a published result can be regenerated from the paper’s described methods and shared data. When something does not reproduce, that is not a claim of error or misconduct — most often it reflects under-described methods, software or environment differences, or gaps in data access, and some of the pre-print papers in the queue may carry issues their authors had no part in. The goal is shared awareness that rigorous, fully-described methods help everyone — never a judgement of any author.
Are you an author? We would genuinely like to hear from you — to clarify the record, add data or code, re-run the pipeline after an accession update, and publish your response right next to the assessment. Everything here is open and auditable.
🚩 Report an error in this record
Spotted something wrong — a verdict you’d contest, a data or value error, or a private detail that slipped through? Tell us, with a short justification. Authors and readers are equally welcome to write in; we review every report.
Prefer email, or the form below not working? Contact us at support@doesitreproduce.com.
Reproduction footprint
claude-opus-4-8Measured resources invested to assess this paper — sanitised (machine class only, no job ids/paths). Compute = HPC accounting (SLURM); tokens = the AI agent's session.