Jacqueline Chandler, Miranda Cumpston, James Thomas, Julian PT Higgins, Jonathan J Deeks, Mike J Clarke

This chapter should be cited as: Chandler J, Cumpston M, Thomas J, Higgins JPT, Deeks JJ, Clarke MJ. Chapter I: Introduction. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook.

I.1 About Cochrane

Trusted evidence. Informed decisions. Better health.

Cochrane is a global network of health and social care practitioners, researchers, patient advocates and others, with a mission to promote evidence-informed decision making by producing high quality, relevant, accessible systematic reviews and other synthesized research evidence (www.cochrane.org). It is a not-for-profit organization whose members aim to produce credible information that is free from commercial sponsorship and other conflicts of interest (Bero 2018).

Cochrane works collaboratively with health professionals, policy makers and international organizations such as the World Health Organization (WHO) to support the development of evidence-informed guidelines and policy. Cochrane Reviews underpin many of these guidelines; for example, dozens of Cochrane Review were used as the evidence base for WHO guidelines on the critical public health issues of breastfeeding (WHO 2017) and malaria (WHO 2015), and for the WHO Essential Medicines List and Essential Diagnostics List.

There are many examples of the impact of Cochrane Reviews on health and social care. Influential reviews of corticosteroids for women at risk of giving birth prematurely (Roberts et al 2017), treatments for macular degeneration (Moja et al 2014) and tranexamic acid for trauma patients with bleeding (Ker et al 2015) have demonstrated the benefits of these life-changing interventions and influenced clinical practice around the world. Other reviews of anti-arrhythmic drugs for atrial fibrillation (Lafuente-Lafuente et al 2015) and neuraminidase inhibitors for influenza (Jefferson et al 2014) have raised important doubts about the effectiveness of interventions in common use.

The work of Cochrane is underpinned by ten key principles, listed in Box I.1.a.

Box I.1.a Cochrane’s principles

I.1.1 A brief history of Cochrane

The Cochrane Collaboration was founded in 1993, following a meeting of an initial group of 77 people from nine countries at the first Cochrane Colloquium in Oxford, UK.

The UK Cochrane Centre had been founded the year before, in 1992, arising from a ground-breaking programme of work by Iain Chalmers and colleagues in the area of pregnancy and childbirth. Inspired by Archie Cochrane’s statement that “It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials” (Cochrane 1979), Chalmers and colleagues developed the Oxford Database of Perinatal Trials and a series of systematic reviews published in Effective Care in Pregnancy and Childbirth (Chalmers et al 1989). The database became a regularly updated electronic publication in 1989, and developed into Cochrane Pregnancy and Childbirth Database in early 1993.

Now simply referred to as ‘Cochrane’, over 25 years this collaboration has grown to 13,000 members and 50,000 supporters from more than 130 countries by 2018, making it the largest organization involved in this kind of work. Cochrane continues to broaden its reach with networks across every continent and new country-level groups established each year, and Cochrane content has been translated into 14 languages.

I.1.2 Cochrane organization and structure

Cochrane’s community of contributors includes researchers, practitioners, people using the health system (consumers, carers and the general public), policy makers, editors, translators and more, all of whom share a common commitment to generating reliable, up-to-date evidence, and many of whom contribute on a voluntary basis. Editorial support and publication of Cochrane Reviews is co-ordinated by topic-related Cochrane Review Groups, organized into eight Networks. For most authors, this will be their primary point of contact with Cochrane (see Chapter II, Section II.1).

Cochrane Review Groups are supported in this work by a wide community of other groups working on methods development (Methods Groups), providing a local focus for Cochrane activities in different geographic areas (Geographic Groups), and focusing on different cross-cutting themes in health (Fields). All this activity is underpinned by a Central Executive team, which provides strategic support and direction, and leads initiatives to improve and assure the quality of review activity across Cochrane.

I.2 Cochrane Reviews

I.2.1 The evidence for Cochrane methodology

While Cochrane was one of the earliest organizations to produce and publish systematic reviews, many organizations and journals now do so. One of the key elements that sets Cochrane apart is its rigorous methods, and Cochrane has played a unique role in fostering the development of methodology for systematic reviews throughout its history. Cochrane Methods Groups are voluntary collaborations of some of the world’s leading methodological researchers in statistics, information retrieval, bias, qualitative methods, and many other specialist areas. These Methods Groups support and disseminate methods research that identifies the most effective and efficient methods for systematic reviews, minimizing bias and ensuring the appropriate analysis and interpretation of results (Chandler and Hopewell 2013, McKenzie et al 2015).

Decisions about the adoption of methods for use in Cochrane Reviews are supported by a Scientific Committee, comprising methodology experts and members of Cochrane’s editorial community.

The use of these rigorous methods is challenging and often time-consuming, but the work is not done for its own sake. As McKenzie and colleagues wrote, “Our confidence in the findings of systematic reviews rests on the evidence base underpinning the methods we use. Just as there are consequences arising from the choices we make about health and social care interventions, so too are there consequences when we choose the methods to use in systematic reviews.” (McKenzie et al 2015).

With this in mind, the guidance in this Handbook has been written by authors who are international leaders in their fields, many of whom are supported by the work of Cochrane Methods Groups, and where available it draws on research evidence including that brought together in Cochrane Methodology Reviews.

I.2.2 Types of Cochrane Review

While this Handbook focuses on systematic reviews of interventions, Cochrane publishes five main types of systematic reviews, and has developed a rigorous approach to the preparation of each.

I.2.2.1 Reviews of the effects of interventions

Most Cochrane Reviews consider evidence on the effects of health or social care interventions. These reviews focus primarily on randomized studies as the most robust research design for assessment of the relative effects of interventions. Where evidence is unlikely to be found in randomized studies, such as for rare adverse effects of interventions, or for the effects of large-scale public health interventions or organizational change, reviews include non-randomized studies. Intervention reviews may also address broader issues such as economic issues or patient experiences of the intervention.

I.2.2.2 Reviews of diagnostic test accuracy

Cochrane has published systematic reviews of diagnostic test accuracy (DTA) since 2008 (Leeflang et al 2013). These reviews evaluate how correctly a test detects the presence or absence of a target condition. Cochrane DTA reviews address conditions across health, including both pathologically defined diseases and more loosely defined indications for which treatments may be available. All types of tests are eligible, including: signs and symptoms from the patient history and examination; questionnaire-based tools, scores and decision rules; laboratory tests including biochemical, immunological, genetic, genomic and other ‘pan-omic’ technologies; imaging tests; and physiological measurements.

Evaluation of the accuracy of a test is one component of the assessment of whether test use could lead to improvement in patient outcomes. Direct evaluation of how a test (and consequent decision making and interventions) actually affects patient outcomes is best assessed by randomized studies that incorporate the effects of interventions that follow the test result. Such studies fit within the structure of Cochrane Intervention Reviews. However, randomized studies of test use are rare (especially outside the context of screening (Ferrante di Ruffano et al 2012)), whereas accuracy studies are relatively common and provide most of the available evidence to guide test use. This makes them worthy of detailed systematic review. Although the stages in a DTA review are the same as for reviews of interventions, specific methodological challenges are encountered at each step. These include formulation of review questions, searching for and locating studies, assessing study quality, meta-analysis and interpretation of findings. Full details of the methodology are described in a separate Cochrane Handbook for Diagnostic Test Accuracy Reviews.

I.2.2.3 Reviews of prognosis

Prognosis research provides information about the future health and well-being of individuals with specific diseases or conditions. Prognosis studies can provide information on the likelihood of a particular outcome or of disease recurrence; identify target groups for intervention; or identify factors associated with poor outcomes (Cochrane Methods Prognosis 2019).

Systematic review methods are increasingly used to synthesize findings from prognosis studies. However, this application is in its infancy, and systematic reviews of prognosis face challenges that have been overcome, or eased, for other reviews: lack of clarity in the indexing of these studies for bibliographic searches; low quality of primary studies; poor reporting; and difficulties in combining results across different research designs, analyses, and presentations of results (Cochrane Methods Prognosis 2019). The Cochrane Prognosis Methods Group is contributing to the development of tools and guidance to support reviews of prognosis studies.

I.2.2.4 Overviews of reviews

Cochrane Overviews of Reviews (Overviews) compile evidence from multiple systematic reviews into a single document, for example addressing a set of related interventions, diagnostic tests, populations, outcomes, or conditions. Overviews are aimed at decision makers, such as clinicians, policy makers, or informed consumers, and can address questions and sets of options that are often too broad for a single review. Further guidance on Overviews is available at handbook.cochrane.org.

I.2.2.5 Reviews of methodology

Cochrane Methodology Reviews seek to answer questions about various aspects of the methods for systematic reviews, randomized studies and other evaluations of health and social care. They provide an evidence base for the methods of these evaluations, as well as providing descriptive accounts of other relevant issues, for example, to show the scale of problems faced by researchers working on systematic reviews or making decisions about health and social care. Cochrane Methodology Reviews use a diverse range of evidence, including:

• experimental studies such as randomized studies (for example, to compare different strategies to increase response rates to surveys);
• comparative observational studies (for example, to examine the relationship between the use of reporting guidelines and the quality of research reports); and
• descriptive observational studies (for example showing the proportion of studies presented at conferences that go on to be published in full).

The Cochrane Methodology Review Group has editorial responsibility for all Methodology Reviews. Guidance on the contents of a Cochrane Methodology protocol and review is available at handbook.cochrane.org.

In addition to stand-alone research and reviews of published research, there are initiatives to promote the embedding of methodology research in primary studies and systematic reviews (Treweek et al 2018), so-called ‘Studies Within A Trial’ (SWAT) and ‘Studies Within A Review’ (SWAR). Further information on these initiatives is available at https://www.trialforge.org/.

I.2.3 Ongoing challenges for systematic reviews

The landscape in which systematic reviews are conducted continues to evolve. Old and emerging challenges continue to spark debate, research and innovation.

The time required to complete a full systematic review, which is often more than two years following the publication of a protocol, is a barrier both for author teams (representing a considerable commitment of often volunteer time) and for decision makers (who often require evidence within much shorter time frames). Methodology for undertaking reviews more rapidly is developing quickly. However, difficult choices are required in the trade-off between rigour and speed. The rise of technological solutions offers much potential, including collaboration tools, online crowd sourcing and automation of many aspects of the review process. Alongside consideration of appropriate ways to prioritize work, technology is also supporting more efficient approaches to keeping reviews up to date, with some reviews moving towards a ‘living’ systematic review model of very frequent, even continuous updates.

Cochrane Reviews have always encompassed complex questions of multi-component interventions, health systems and public health, and the challenging issues that arise from many of these reviews have prompted considerable thought and effort. Cochrane Reviews incorporate non-randomized studies where appropriate to the question, and a wider range of data sources is increasingly relevant to reviews, from unpublished clinical study reports produced by pharmaceutical companies, to novel challenges in appraising and interpreting ‘big data’ repositories. The use of systematic reviews is expanding, and new methods are developing, in areas such as environmental exposure.

These conversations will continue, and new questions will continue to arise. Cochrane will continue to contribute actively to the development and application of new methods, continually striving to improve both the validity and usefulness of reviews to decision makers.

I.2.4 Publication of Cochrane Reviews

Cochrane Protocols and Reviews are published in full online in the Cochrane Database Systematic Reviews (CDSR), a core component of the Cochrane Library. The CDSR is the leading journal for systematic reviews in health and social care. In addition to completed Cochrane Reviews, the CDSR includes protocols for Cochrane Reviews, editorials and supplements.

In addition to the CDSR, the Cochrane Library also provides access to:

• The Cochrane Central Register of Controlled Trials (CENTRAL) – a database of reports of randomized and quasi-randomized trials sourced from bibliographic databases (mainly MEDLINE and Embase) and other published and unpublished sources (see Chapter 4, Section 4.3.1.3).
• Cochrane Clinical Answers – brief, structured answers to clinically focused questions, designed to be actionable and to inform point-of-care decision making.
• Epistemonikos – systematic reviews from the Epistemonikos database of health evidence.

The Cochrane Library is available free at the point of use in many countries, thanks to national licences, and free access is provided in most low- and middle-income countries. All Cochrane Reviews published in full or updated since February 2013 are made open access 12 months after their publication. Mechanisms are also available to make individual reviews open access immediately on publication.

I.3 About this Handbook

Work on a handbook to support authors of Cochrane Reviews began in 1993, and the first version was published in May 1994. Since then, the document has evolved and grown, through the stewardship of several editorial teams, with regular updating of its contents being punctuated by major new editions. This is Version 6 of the Handbook, the first major revision since the first print edition of the Handbook was published in 2008.

This version of the Handbook is divided into four parts:

• About Cochrane Reviews specifically addresses the format, structure and expectations of systematic reviews produced with Cochrane, including planning, updating and reporting requirements. These chapters are likely to be of most interest to authors working with Cochrane. They are available online only and are not included in the print edition of this Handbook.
• Core methods provides the core methodology for undertaking systematic reviews on the effects of health interventions, with an emphasis on reviewing randomized trials. This section provides authors with a detailed account of the steps from starting a review to summarizing and interpreting the review findings.
• Specific perspectives in reviews outlines important considerations relating to particular populations (including issues of equity, types of interventions (particularly intervention complexity), and types of outcomes (such as adverse effects and patient-reported outcomes). It also addresses economic considerations and the important role of qualitative evidence.
• Further topics covers additional methodological issues, including reviewing non-randomized studies and variants of designs for randomized trials, prospective approaches to collecting and synthesizing evidence, and using individual participant data.

For this edition, each chapter that provides new or substantively updated guidance has been rigorously peer reviewed to ensure the guidance presented reflects the state of the science and is appropriate and efficient for use by Cochrane authors. The Handbook is updated regularly to reflect advances in systematic review methodology and in response to feedback from users. Please refer to handbook.cochrane.org for the most recent online version, interim updates to the guidance and details of previous versions of the Handbook. Feedback and corrections to the Handbook are also welcome via the contact details on the website.

I.3.1 What’s new in this edition

In this edition, every chapter of the Handbook has been extensively revised and new chapters added. Authors familiar with previous editions will find it valuable to re-read this extensively updated version and any chapter of interest.

In particular, this edition includes the following major new chapters and areas of guidance:

• Incorporation of Cochrane’s Methodological Expectations for Cochrane Intervention Reviews (MECIR) throughout the Handbook.
• Expanded guidance on reporting (Chapter III) and updating (Chapter IV) Cochrane Reviews.
• New guidance on developing eligibility criteria and setting up the PICO for each synthesis (Chapter 3).
• Expanded advice on assessing the risk of bias in included studies (Chapter 7), including Version 2 of the Cochrane Risk of Bias tool for randomized trials (Chapter 8) and the ROBINS-I tool for assessing risk of bias in non-randomized studies (Chapter 25).
• New guidance on summarizing study characteristics and preparing for synthesis (Chapter 3 and Chapter 9).
• New guidance on network meta-analysis (Chapter 11).
• New guidance on synthesizing results using methods other than meta-analysis (Chapter 12).
• Updated guidance on assessing the risk of bias due to missing results (reporting biases, Chapter 13).
• New guidance addressing intervention complexity (Chapter 17).

I.4 Chapter information

Authors: Jacqueline Chandler, Miranda Cumpston, James Thomas, Julian PT Higgins, Jonathan J Deeks, Mike J Clarke

Acknowledgements: We thank previous chapter authors Sally Green, Philip Alderson, Cynthia Mulrow and Andrew Oxman on whose text this version is based. We also thank Ruth Foxlee and Clare Davenport for their contributions.

I.5 References

Bero L. More journals should have conflict of interest policies as strict as Cochrane: BMJ Opinion; 2018. https://blogs.bmj.com/bmj/2018/11/12/lisa-bero-more-journals-should-have-conflict-of-interest-policies-as-strict-as-cochrane/.

Chalmers I, Enkin M, Keirse MJNC. Effective care in pregnancy and childbirth. Oxford (UK): Oxford University Press; 1989.

Chandler J, Hopewell S. Cochrane methods - twenty years experience in developing systematic review methods. Systematic Reviews 2013; 2: 76.

Cochrane AL. 1931-1971: a critical review, with particular reference to the medical profession. In: Teeling-Smith G, Wells N, editors. Medicines for the year 2000. London (UK): Office of Health Economics; 1979. p. 1-11.

Cochrane Methods Prognosis. About us: Cochrane; 2019. https://methods.cochrane.org/prognosis/about-us.

Ferrante di Ruffano L, Davenport C, Eisinga A, Hyde C, Deeks JJ. A capture-recapture analysis demonstrated that randomized controlled trials evaluating the impact of diagnostic tests on patient outcomes are rare. Journal of Clinical Epidemiology 2012; 65: 282-287.

Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, Spencer EA, Onakpoya I, Mahtani KR, Nunan D, Howick J, Heneghan CJ. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database of Systematic Reviews 2014; 4: CD008965.

Ker K, Roberts I, Shakur H, Coats TJ. Antifibrinolytic drugs for acute traumatic injury. Cochrane Database of Systematic Reviews 2015; 5: CD004896.

Lafuente-Lafuente C, Valembois L, Bergmann JF, Belmin J. Antiarrhythmics for maintaining sinus rhythm after cardioversion of atrial fibrillation. Cochrane Database of Systematic Reviews 2015; 3: CD005049.

Leeflang MMG, Deeks JJ, Takwoingi Y, Macaskill P. Cochrane diagnostic test accuracy reviews. Systematic Reviews 2013; 2: 82.

McKenzie JE, Clarke MJ, Chandler J. Why do we need evidence‐based methods in Cochrane? Cochrane Database of Systematic Reviews 2015; 7: ED000102.

Moja L, Lucenteforte E, Kwag KH, Bertele V, Campomori A, Chakravarthy U, D'Amico R, Dickersin K, Kodjikian L, Lindsley K, Loke Y, Maguire M, Martin DF, Mugelli A, Muhlbauer B, Puntmann I, Reeves B, Rogers C, Schmucker C, Subramanian ML, Virgili G. Systemic safety of bevacizumab versus ranibizumab for neovascular age-related macular degeneration. Cochrane Database of Systematic Reviews 2014; 9: CD011230.

Roberts D, Brown J, Medley N, Dalziel SR. Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth. Cochrane Database of Systematic Reviews 2017; 3: CD004454.

Treweek S, Bevan S, Bower P, Campbell M, Christie J, Clarke M, Collett C, Cotton S, Devane D, El Feky A, Flemyng E, Galvin S, Gardner H, Gillies K, Jansen J, Littleford R, Parker A, Ramsay C, Restrup L, Sullivan F, Torgerson D, Tremain L, Westmore M, Williamson PR. Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)? Trials 2018; 19: 139.

WHO. Guidelines for the treatment of malaria. 3rd ed. Geneva: World Health Organization; 2015. https://www.who.int/malaria/publications/atoz/9789241549127/en/.

WHO. Guideline: protecting, promoting and supporting breastfeeding in facilities providing maternity and newborn services Geneva: World Health Organization; 2017. https://www.who.int/nutrition/publications/guidelines/breastfeeding-facilities-maternity-newborn/en/.