Researchers’ decisions about whether, when, how or where to report studies or results are often influenced by the P value, magnitude or direction of the study results (‘non-reporting biases’). A consequence is bias in meta-analyses, because the available evidence differs systematically from the missing evidence.

In this web clinic, the presenters introduced ROB-ME ("Risk Of Bias due to Missing Evidence"), a comprehensive tool for assessing the risk of bias that arises when entire studies, or particular results within studies, are missing from a meta-analysis because of the P value, magnitude, or direction of the study results.

The session was delivered in October 2024 and below you will find the videos from the webinar, together with the accompanying slides to download [PDF]. Recordings from other Methods Support Unit web clinics are available here. 

Part 1: Introduction
Part 2: Overview of the ROB-ME tool
Part 3: Example
Part 4: Questions & answers

Presenter Bios

Matthew Page is a Senior Research Fellow and Deputy Head of the Methods in Evidence Synthesis Unit at Monash University. His research aims to improve the quality of systematic reviews of health and medical research. He co-led the PRISMA 2020 statement, is an associate scientific editor for the Cochrane Handbook for Systematic Reviews of Interventions and is a co-convenor of the Cochrane Bias Methods Group.

Julian Higgins is Professor of Evidence Synthesis at the University of Bristol, where he co-directs the NIHR Bristol Evidence Synthesis Group and heads the Bristol Appraisal and Review of Research (BARR) group. His research interests span all areas of systematic review and meta-analysis. Among his methods contributions are a Bayesian approach to network meta-analysis, the I-squared statistic to quantify inconsistency across studies in a meta-analysis, simple prediction intervals for random-effects meta-analysis and risk-of-bias assessment tools for clinical trials and other study designs. He has long been an active contributor to Cochrane, is a former member of the Cochrane Collaboration Steering Group, the Cochrane Editorial Board and the Cochrane Scientific Committee, and is currently co-convenor of the Cochrane Bias Methods Group. He has co-edited the Cochrane Handbook for Systematic Reviews of Interventions since 2003.

Jonathan Sterne is Professor of Medical Statistics and Epidemiology at the University of Bristol, Director of the NIHR Bristol Biomedical Research Centre and co-Director of Health Data Research UK South-West. He leads a large team producing novel, high impact research on COVID-19 vaccination and long COVID, based on analyses of up to 55 million people. He has a longstanding interest in methodology for systematic reviews and meta-analysis. He co-led development of the RoB 2 tool for assessing risk of bias in randomised trials and the ROBINS-I and ROBINS-E tools for assessing risk of bias in non-randomised studies of interventions and exposures, respectively. Jonathan is a former co-convenor of the Cochrane Bias Methods Group and has published influential papers on reporting bias in meta-analysis, meta-epidemiology, causal inference and statistical methodology.