Systematic reviews of health interventions synthesise evidence from all RCTs addressing a particular research question, and are considered very high-quality evidence. Unfortunately, it has become clear that some RCTs included in systematic reviews are not authentic and may have been entirely fabricated. We call trials subject to data falsification, fabrication, or other serious research integrity issues “problematic studies”, and recent examples can be found amongst studies included in systematic reviews of ivermectin for the treatment of COVID-19. However, there is no consensus around how to identify these problematic studies when undertaking a systematic review, and often no checks are performed at all.

In the first half of this MSU web clinic, Jack presented the NIHR-funded INSPECT-SR project, which aims to develop a tool for identifying and excluding problematic RCTs from systematic reviews. The project will combine empirical evidence and consensus science approaches to develop a tool, which will then be refined through testing. He outlined the project, delivered interim results, and provided information about how to participate in the development of the tool.

In the second half, Jack described some of the analytic methods which are employed to assess the authenticity of RCT datasets.

Below you will find the videos from the July 2023 webinar. Recordings from other Methods Support Unit web clinics are available here. 

Part 1: INSPECT-SR project background and overview
Part 2: Some principles for investigating potentially problematic RCTs
Part 3: Questions and answers

Presenter bio

Jack Wilkinson is a Lecturer at the Centre for Biostatistics, University of Manchester. His research interests include clinical trials and systematic reviews of interventions, with a focus on reproductive medicine, and he is Statistical Editor for Cochrane Gynaecology and Fertility, BJOG, Fertility and Sterility, and Reproduction and Fertility. His interest in research misconduct began in 2019, when he was enlisted by a journal to investigate the integrity of several published clinical trials. He now regularly performs integrity investigations for journals and publishers of medical research. He is currently leading the NIHR-funded INSPECT-SR project, which aims to develop a tool for identifying and excluding problematic RCTs from systematic reviews.