Matthew J Page, Miranda Cumpston, Jacqueline Chandler, Toby Lasserson

This chapter should be cited as: Page MJ, Cumpston M, Chandler J, Lasserson T. Chapter III: Reporting the review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated August 2019). Cochrane, 2019. Available from www.training.cochrane.org/handbook.

III.1 Introduction

The effort of undertaking a systematic review is wasted if review authors do not report clearly what they did and what they found (Glasziou et al 2014). Clear reporting enables others to replicate the methods used in the review, which can facilitate attempts to verify or reproduce the results (Page et al 2018). Transparency can also make the review more usable for healthcare decision makers. For example, clearly describing the interventions assigned in the included studies can help users determine how best to deliver effective interventions in practice (Hoffmann et al 2017). Also, comprehensively describing the eligibility criteria applied, sources consulted, analyses conducted, and post-hoc decisions made, can reduce uncertainties in assessments of risk of bias in the review findings (Whiting et al 2016). For these reasons, transparent reporting is an essential component of all systematic reviews.

Surveys of the transparency of published systematic reviews suggest that many elements of systematic reviews could be reported better. For example, Page and colleagues evaluated a random sample of 300 systematic reviews of biomedical research indexed in MEDLINE in February 2014 (Page et al 2016). They found that in at least a third of the reviews there was no information on eligible publication types, the years of coverage of the search, the methods used to collect data and appraise studies, or the funding source of the review. However, Cochrane Reviews, which accounted for 15% of the sample, had more complete reporting than all other types of systematic reviews (Page et al 2016).

Possible reasons why more complete reporting of Cochrane Reviews has been observed include the use of software (RevMan, https://revman.cochrane.org/) and strategies in the editorial process that promote good reporting. RevMan includes many standard headings and subheadings which are designed to prompt Cochrane Review authors to document their methods and results clearly. In addition, the Methodological Expectations of Cochrane Intervention Reviews (MECIR) include recommendations to guide the reporting of these reviews.

The MECIR guidelines were developed in consultation with review authors, editors and methodologists from the Cochrane community, and form the basis of quality assurance work undertaken by the Cochrane Editorial and Methods Department. They cover both conduct and reporting for review protocols, new reviews and updates of reviews of interventions. The guidelines distinguish between conduct and reporting for good reason: good conduct does not necessarily lead to good reporting, good reporting cannot improve poor conduct, and poor reporting can obscure good or poor conduct of a review.

The MECIR reporting guidance is consistent with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2009 statement (Liberati et al 2009, Moher et al 2009). Review authors and Cochrane Review Groups are expected to follow the guidance developed specifically for Cochrane Reviews, in conjunction with PRISMA should that prove to be helpful. Reporting guidance is currently being developed for the methods and reporting of syntheses of quantitative data where meta-analysis was not possible or appropriate (the ICONS-Quant reporting guideline) (Campbell et al 2018). Review authors are advised to consult such guidance alongside MECIR once it is available.

Guidance on the composition of plain language summaries of Cochrane Reviews of interventions is also available. The guidance outlines the key messages from Cochrane Reviews that should be included in a plain language summary, in terms that are accessible to consumers and non-expert readers.

The structure of this chapter is built around the MECIR reporting guidance for Cochrane Review protocols (Section ‎III.2) and new Cochrane Reviews (Section ‎III.3) of interventions, and guidance for reporting plain language summaries (Section ‎III.4). The MECIR expectations of conduct are embedded in the relevant chapters of this Handbook. MECIR conduct and reporting guidance for updates of Cochrane Reviews of interventions are presented in Chapter IV. For the latest version of all MECIR conduct and reporting guidance, readers should consult the MECIR web pages, available at https://methods.cochrane.org/mecir.

Many of the standard headings recommended for use in Cochrane Reviews are referred to in this chapter, although the precise headings available in RevMan may be amended as new versions are released. New headings can be added and some standard headings can be deactivated; if the latter is done, review authors should ensure that all information expected (as outlined in the MECIR reporting guidelines) is still reported somewhere in the review.

III.2 Reporting of protocols of new Cochrane Reviews

Preparing a well-written review protocol is important for many reasons (see Chapter 1). The protocol is a public record of the question of interest and the intended methods before results of the studies are fully known. This helps readers to judge how the eligibility criteria of the review, stated outcomes and planned methods will address the intended question of interest. It also helps anyone who evaluates the completed review to judge how far it fulfilled its original objectives (Lasserson et al 2016). Investing effort in the development of the review question and planning of methods also stimulates review authors to anticipate methodological challenges that may arise, and helps minimize potential for non-reporting biases by encouraging review authors to publish their review and report results for all pre-specified outcomes (Shamseer et al 2015).

See the online MECIR Manual for the 44 MECIR reporting items for protocols of new Cochrane Reviews. They include guidance for reporting of the:

• Background;
• Objectives;
• Criteria for considering studies for inclusion in the review;
• Search methods for identification of studies (e.g. a list of all sources that will be searched, a complete search strategy to be implemented for at least one database);
• Data collection and analysis (e.g. types of information that will be sought from reports of included studies and methods for obtaining such information, how risk of bias in included studies will be assessed, and any intended statistical methods for combining results across studies); and
• Other information (e.g. acknowledgments, contributions of authors, declarations of interest, and sources of support).

These sections correspond to the same sections in a completed review, and further details are outlined in Sections ‎III.3.2, ‎III.3.3 and ‎III.3.7.

One key difference between a review protocol and a completed review is that the Methods section in a protocol should be written in the future tense. Because Cochrane Reviews are updated as new evidence accumulates, methods outlined in the protocol should generally be written as if a suitably large number of studies will be identified to allow the objectives to be met (even if this is assumed likely not to be the case at the time of writing).

The MECIR guidelines reflect the minimum expectations for good reporting of a review protocol. Further guidance on the level of planning required for each aspect of the review methods and the detailed information recommended for inclusion in the protocol is given in the relevant chapters of this Handbook.

III.3 Reporting of new Cochrane Reviews

The main text of a Cochrane Review should be succinct and readable. Although there is no formal word limit for Cochrane Reviews, review authors should consider 10,000 words a maximum for the main text of the review unless there is a special reason to write a longer review, such as when the question is unusually broad or complex. Most reviews should be substantially shorter.

People making decisions about health care are the target audience for Cochrane Reviews. This includes healthcare professionals, consumers and policy makers, and reviews should be accessible to these audiences. Cochrane Reviews should be written so that they are easy to read and understand by someone with a basic sense of the topic who is not necessarily an expert in the area. Some explanation of terms and concepts is likely to be helpful, and perhaps even essential. However, too much explanation can detract from the readability of a review. Simplicity and clarity are also vital to readability. The readability of Cochrane Reviews should compare to that of a well-written article in a general medical journal.

Review authors should ensure that reporting of objectives, important outcomes, results, caveats and conclusions is consistent across the main text, the tables and figures, the abstract, and any other summary versions of the review (e.g. ‘Summary of findings’ table and plain language summary). Although this sounds simple, it can be challenging in practice; authors should review their text carefully to ensure that readers of a summary version are likely to come away with the same overall understanding of the conclusions of the review as readers accessing the full text.

Plagiarism is not acceptable and all sources of information need to be cited (for more information see the Cochrane Editorial and Publishing Policy Resource on plagiarism). Also, the unattributed reproduction of text from other sources should be avoided. Quotes from other published or unpublished sources need to be indicated and attributed clearly, and permission may be required to reproduce any published figures.

See the online MECIR Manual for all MECIR reporting items for new Cochrane Reviews. In the remainder of this section we summarize the reporting guidance relating to different sections of a Cochrane Review.

III.3.1 Abstract

All reviews should include an abstract of not more than 1000 words, although in the interests of brevity, authors should aim to include no more than 700 words, without sacrificing important content. Abstracts should be targeted primarily at healthcare decision makers (clinicians, consumers and policy makers) rather than just to researchers. Terminology should be reasonably comprehensible to a general rather than a specialist healthcare audience. Abbreviations should be avoided, except where they are widely understood (e.g. HIV). Where essential, other abbreviations should be spelt out (with the abbreviations in brackets) on first use. Names of drugs and interventions that can be understood internationally should be used wherever possible. Trade or brand names should not be used and generic names are preferred.

Abstracts of Cochrane Reviews are made freely available on the internet and published in bibliographic databases that index the Cochrane Database of Systematic Reviews (e.g. MEDLINE, Embase). However, some readers may be unable to access the full review, or the full text may not have been translated into their language, so abstracts may be the only source they have to understand the review results (Beller et al 2013). It is important therefore that they can be read as stand-alone documents. The abstract should summarize the key methods, results and conclusions of the review. An abstract should not contain any information that is not in the main body of the review.

The content of a Cochrane Review abstract should include:

• Background (a summary of the rationale and context of the review);
• Objectives of the review;
• Search methods (including an indication of databases searched, and the date of the last search for which studies were fully incorporated);
• Selection criteria (including a summary of eligibility criteria for study designs, participants, interventions and comparators);
• Data collection and analysis (including a summary of any noteworthy methods for selecting studies, collecting data, evaluating risk of bias and synthesizing results; for many reviews it may be sufficient to state “We used standard methodological procedures expected by Cochrane”);
• Main results (including the findings of all important benefit and harm outcomes, irrespective of the statistical significance, magnitude or direction of the result, along with the GRADE assessment of the certainty of the evidence, which summarizes the impact of the risk of bias, inconsistency, imprecision, indirectness and publication bias on the results presented); and
• Author’s conclusions (including both implications for practice and research).

See the online MECIR Manual for reporting guidance for the abstract of a Cochrane Review.

III.3.2 Background and Objectives

Well-formulated review questions occur in the context of an already-formed body of knowledge. The Background section should address this context, including a description of the condition or problem of interest. It should help clarify the rationale for the review, and explain why the questions being addressed are important. It should be concise (generally around one page when typeset printed) and be understandable to the users of the intervention(s) under investigation.

It is important that the eligibility criteria and other aspects of the methods build on ideas that have been developed in the Background section. For example, if there are uncertainties in how variation in setting, dose of intervention or timing of outcome assessment influence the intervention effect, then it would be important to acknowledge them as a reason for doing the review and consider how the relevant aspects of the methods have been designed to identify relevant evidence and explore these uncertainties.

The following four standard subheadings in the Background section of a Cochrane Review are intended to facilitate a structured approach to the context and overall rationale for the review.

• Description of the condition: A brief description of the condition being addressed, and its significance, is a useful way to begin the review. It may include information about the biology, diagnosis, prognosis, prevalence, incidence and burden of the condition.
• Description of the intervention: A description of the experimental intervention(s) should place it in the context of any standard or alternative interventions, remembering that standard practice may vary widely according to context. The role of the comparator intervention(s) in standard practice should also be made clear. For drugs, basic information on clinical pharmacology should be presented where available, such as dose range, metabolism, selective effects, half-life, duration and any known interactions with other drugs. For more complex interventions, such as behavioural or service-level interventions, a description of the main components should be provided (see Chapter 17).
• How the intervention might work: This section should provide theoretical reasoning as to why the interventions under review may have an impact on potential recipients, for example, by relating a drug intervention to the biology of the condition. Authors may refer to a body of empirical evidence such as similar interventions having an impact on the target recipients or identical interventions having an impact on other populations. Authors may also refer to a body of literature that justifies the possibility of effectiveness. For reviews of complex interventions, a logic model (Kneale et al 2015) or conceptual framework may be useful to illustrate the proposed mechanism of action of the intervention and its components. This will also provide review authors with a framework for the methods and analyses undertaken throughout the review to ensure that the review question is clearly and appropriately addressed. More guidance on the conduct of reviews of complex interventions is presented in Chapter 17.
• Why it is important to do this review: Review authors should explain clearly why the questions being asked are important. Rather than justifying the review on the grounds that there are known eligible studies, it is more helpful to emphasize what aspects of, or uncertainties in, the accumulating evidence base now justify a systematic review. For example, it might be the case that studies have reached conflicting conclusions, that there is debate about the evidence to date, or that there are competing approaches to implementing the intervention.

Immediately following the Background section of the review, review authors should declare the review objectives. They should begin with a precise statement of the primary objective of the review, ideally in a single sentence. Where possible the style should be of the form “To assess the effects of [intervention or comparison] for [health problem] for/in [types of people, disease or problem and setting if specified]”. This might be followed by a series of secondary objectives relating to different participant groups, different comparisons of interventions or different outcome measures. If relevant, any objectives relating to the evaluation of economic or qualitative evidence should be stated. It is not necessary to state specific hypotheses.

See the online MECIR Manual for reporting guidance relevant to the Background and Objectives sections of a Cochrane Review.

III.3.3 Methods

The Methods section in a completed review should be written in the past tense, and should describe what was done to obtain the results and conclusions of the current review.

Review authors are expected to cite their protocol to make it clear that there was one. Often a review is unable to implement all of the methods outlined in the protocol. For example, planned investigations of heterogeneity (e.g. subgroup analyses) and small-study effects may not have been conducted because of an insufficient number of studies. In such circumstances, we recommend that the methods that were not implemented be removed from the main Methods section and outlined in the section headed ‘Differences between protocol and review’ or in an Appendix. A description of the methods not implemented can serve as a protocol for future updates of the review.

The Methods section of a Cochrane Review includes three main subsections, within which are a series of standard headings to guide authors in reporting all the relevant information. See Sections ‎III.3.3.1, ‎III.3.3.2 and ‎III.3.3.3 for a summary of content recommended for inclusion under each subheading.

III.3.3.1 Criteria for considering studies for this review

Review authors should declare all criteria used to decide which studies are included in the review. Doing so will help readers understand the scope of the review and recognize why particular studies they are aware of were not included. Eligible study designs should be described, with a focus on specific features of a study’s design rather than design labels (e.g. how groups were formed, whether the intervention was assigned to individuals or clusters of individuals) (Reeves et al 2017). Review authors should describe eligibility criteria for participants, including any restrictions based on age, diagnostic criteria, location and setting. If relevant, it is useful to describe how studies including a subset of relevant participants were addressed (e.g. when children up to the age of 16 years only were eligible but a study included children up to the age of 18 years). Eligibility criteria for interventions and comparators should be stated also, including any criteria around delivery, dose, duration, intensity, co-interventions and characteristics of complex interventions.

Review authors should specify the important outcomes of interest to the review, and define acceptable ways of measuring them. The review’s important outcomes should normally reflect at least one potential benefit and at least one potential harm. Additional information about outcomes of interest is helpful to include, including a description of how multiple variants of outcome measures (e.g. definitions, assessors, scales, time points) were addressed. Typically, studies should not be excluded from a review solely because no outcomes of interest were reported, because failure to report an outcome does not mean it was not assessed (Dwan et al 2017). However, on occasion it will be appropriate to include only studies that measured particular outcomes. For example, a review of a multi-component public health intervention promoting healthy lifestyle choices, focusing on reduction in smoking prevalence, might legitimately exclude studies that do not measure smoking rates. Review authors should specify if measurement of a particular outcome was used as an eligibility criterion for the review, and justify why this was done.

See the online MECIR Manual for reporting guidance relevant to the eligibility criteria for the review. Further guidance on planning eligibility criteria is presented in Chapter 3.

III.3.3.2 Search methods for identification of studies

It is essential that users of systematic reviews are given an opportunity to evaluate the methods used to identify studies for inclusion. Such an evaluation is possible when review authors report their search methods comprehensively. This involves specifying all sources consulted, including databases, trials registers, websites, and a list of individuals or organizations contacted. If particular journals were handsearched, this should be noted, but it is not necessary to describe handsearching done routinely to populate a Cochrane Specialized Register. Any specific methods used to develop the search strategy, such as automated text analysis or peer review, should also be noted. Specifying the dates of coverage of all databases searched and the date of the last search for which studies were fully incorporated can help users determine how up to date the review is. Review authors should also declare any limits placed on the search (e.g. by language, publication date or publication format).

To facilitate replication of a search, review authors should include in an Appendix the exact search strategy (or strategies) used for each database, including any limits and filters used. Search strategies can be exported from bibliographic databases, and these should be copied and pasted instead of re-typing each line, which can introduce errors.

See the online MECIR Manual for reporting guidance relevant to the search methods used to identify studies, and refer to Chapter 4 for guidance on search methods.

III.3.3.3 Data collection and analysis

Cochrane Reviews include several standard subheadings to enable a structured, detailed description of the methods used for data collection and analysis. Additional headings should be included where appropriate to describe additional methods implemented in the review, e.g. those specific to the analysis of qualitative or economic evidence. See the online MECIR Manual for guidance relevant to the reporting of data collection and analysis methods.

Selection of studies: There should be a description of how the eligibility criteria were applied, from screening of search results through to the final selection of studies for inclusion in the review. The number of people involved at each stage of the process should be stated, such as two authors working independently, along with an indication of how any disagreements were resolved. Any automated processes or software tools used to support selection should be noted. See Chapter 4 for guidance on the study selection process.

Data collection and management: Review authors should specify how data were collected for the included studies. This includes describing the number of people involved in data collection, whether they worked independently, how any disagreements were resolved, and whether standardized data collection forms were used (and if so, whether they were piloted in advance). A brief description of the data items (e.g. participant characteristics, intervention details) extracted from each report is recommended. If study authors or sponsors were contacted to obtain missing information or to clarify the information available, this should be stated. If methods for transforming or processing data in preparation for analysis were necessary (e.g. converting standard errors to standard deviations, extracting numeric data from graphs), these methods should be described. See Chapter 5 for guidance on data collection.

Assessment of risk of bias in included studies: There should be a description of the approach used to assess risk of bias in the included studies. This involves specifying the risk-of-bias tool(s) used, how many authors were involved in the assessment, and how the assessments were incorporated into the analysis or interpretation of the results. Cochrane Review authors are expected to use the tools recommended by Cochrane (described in Chapter 8 and Chapter 25). See Chapter 7 for guidance on study risk-of-bias assessment.

Measures of treatment effect: The effect measures used by the review authors to describe results in any included studies or meta-analyses (or both) should be stated. Examples of effect measures include the odds ratio (OR), risk ratio (RR) and risk difference (RD) for dichotomous data; the mean difference (MD) and standardized mean difference (SMD) for continuous data; and hazard ratio for time-to-event data. See Chapter 6 for more guidance on effect measures.

Unit of analysis issues: If the review includes study designs that can give rise to a unit-of-analysis error (when the number of observations in an analysis does not match the number of units randomized), the approaches taken to address these issues should be described. Studies that can give rise to unit-of-analysis errors include crossover trials, cluster-randomized trials, studies where interventions are assigned to multiple parts of the body of the same participant, and studies with multiple intervention groups where more than two groups are included in the same meta-analysis. See Chapter 23 for guidance on handling unit-of-analysis issues.

Dealing with missing data: Review authors may encounter various types of missing data in their review. For example, there may be missing information about the methods of the included studies (e.g. when the method of randomization is not reported), or missing statistics (e.g. when standard deviations of mean scores are not reported). Strategies to deal with such missing data should be reported. This may include attempts to obtain the missing data, and approaches to the analysis and interpretation of results in light of missing data (e.g. imputing missing standard deviations). See Chapter 10 for guidance on dealing with missing data.

Assessment of heterogeneity: Review authors should describe their approach to identifying statistical heterogeneity (e.g. non-quantitative assessment, I², Tau², or statistical test). See Chapter 10 for guidance on assessment of heterogeneity.

Assessment of non-reporting biases: Any methods used to assess the risk of non-reporting biases in a synthesis should be described. Such methods may include consideration of the number of studies missing from a synthesis due to selective non-reporting of results, or investigations to assess small-study effects (e.g. funnel plots), which can arise from the suppression of small studies with ‘negative’ results (amongst other reasons). See Chapter 13 for a description of methods for assessing risk of non-reporting biases in a synthesis.

Data synthesis: Review authors should describe any methods used for combining results across studies (e.g. meta-analysis, network meta-analysis). Where data have been combined in statistical software external to RevMan, authors should reference the software, commands and settings used to run the analysis. If relevant, other synthesis methods used when meta-analysis was not possible or appropriate should be described. See Chapter 10 for guidance on undertaking meta-analysis, Chapter 11 for guidance on undertaking network meta-analysis, and Chapter 12 for a description of other synthesis methods.

Subgroup analysis and investigations of heterogeneity: If subgroup analyses (or meta-regression) were performed, review authors should specify the potential effect modifiers explored, the rationale for each, whether they were identified before or after the results were known, and how they were compared (e.g. using a statistical test for interaction). See Chapter 10 for more information on investigating heterogeneity.

Sensitivity analyses: If any sensitivity analyses were performed to explore the robustness of meta-analysis results, review authors should specify the basis of each analysis (e.g. removal of studies at high risk of bias, imputing alternative estimates of missing standard deviations). See Chapter 10 for more information on sensitivity analyses.

Summarizing findings and assessing certainty of the evidence: Review authors should describe methods for summarizing the findings of the review, and assessing the certainty of the body of evidence for each main outcome (e.g. using the GRADE approach). If review authors used an alternative to the GRADE approach to assess certainty of the body of evidence, or deviated from standard GRADE methods, they should say so and provide a rationale. Review authors should also indicate which populations, interventions, comparisons and outcomes are addressed in ‘Summary of findings’ tables. For more details on completing ‘Summary of findings’ tables and using the GRADE approach, see Chapter 14.

III.3.4 Results

A narrative summary of the results of a Cochrane Review should be provided under the three standard subheadings in the Results section (see Sections ‎III.3.4.1, ‎III.3.4.2 and ‎III.3.4.3 for a summary of content recommended for inclusion under each subheading). Details about the effects of interventions (including summary statistics and effect estimates for each included study and for meta-analyses) can be presented in various tables and figures (see Section ‎III.3.4.4).

III.3.4.1 Description of studies

The results section should start with a summary of the results of the search (for example, how many references were retrieved by the electronic searches, how many were evaluated after duplicates were removed, how many were considered as potentially eligible after screening, and how many were included). Review authors are encouraged to include a PRISMA-type flow diagram demonstrating the flow of studies throughout the selection process (Moher et al 2009). Such flow diagrams can be created within RevMan.

To help readers determine the applicability of the review findings, review authors should describe the characteristics of the studies included in the review. In the Results section, a brief narrative summary of the included studies should be provided (by specifying the number of participants and summarizing characteristics of the study populations and settings, interventions, comparators, outcomes and funding sources). More details about each included study should be presented in the ‘Characteristics of included studies’ table. This table should include (at a minimum) the following information about each included study:

• basic study design or design features;
• baseline demographics of the study sample (e.g. age, sex/gender);
• sample size;
• details of all interventions (including what was delivered, by whom, in which setting, and how often; for more guidance see Hoffmann et al (2017));
• outcomes measured (with details on how and when they were measured);
• funding source; and
• declarations of interest among the primary researchers.

Studies that may appear to some readers to meet the eligibility criteria, but which were excluded, should be listed in the ‘Characteristics of excluded studies’ table, and an explicit reason for exclusion should be provided (one reason is usually sufficient). It is not necessary to include every study excluded at the full text screening stage in the table; rather, authors should use their judgement to identify those studies most likely to be considered eligible by readers, and hence most useful to include here. A succinct summary of the reasons why studies were excluded from the review should be provided in the Results section.

It is helpful to make readers aware of any completed studies that have been identified as potentially eligible but have not been incorporated into the review. This may occur when there is insufficient information to determine whether the study meets the eligibility criteria of the review, or when a top-up search is run immediately prior to publication and the review authors consider it unlikely that inclusion of the study would change the review conclusions substantially. A description of such studies can be provided in the ‘Characteristics of studies awaiting classification’ table. Readers should also be made aware of any studies that meet the eligibility criteria for the review, but which are still in progress and hence have no results available. This serves several purposes. It will help readers assess the stability of the review findings, alert research funders about ongoing research activity, and can serve as a useful basis for deciding when an update of the review may be needed. A description of such studies can be provided in the ‘Characteristics of ongoing studies’ table.

See the online MECIR Manual for reporting guidance relevant to the description of studies.

III.3.4.2 Risk of bias in included studies

To help readers determine the trustworthiness of the results of included studies, review authors should present and summarize their risk-of-bias assessments (see the online MECIR Manual for relevant reporting guidance). For each main result assessed, risk-of-bias judgements should be presented along with explicit support for these judgements. Forest plots created in RevMan can present the risk-of-bias judgements relating to each included study. The current subheadings in this section of the review are by domain, but review authors should also provide in the Results section a narrative summary of the risks of bias among results contributing to key outcomes of the review.

III.3.4.3 Effects of interventions

There are 24 MECIR items relevant to the reporting of effects of interventions (see the online MECIR Manual). We provide a summary of them in this and the following section.

Review authors should summarize in text form the results for all pre-specified review outcomes, regardless of the statistical significance, magnitude or direction of the effects, or whether evidence was found for those outcomes. The text should present the results in a logical and systematic way. This can be done by organizing results by population or comparison (e.g. by first describing results for the comparison of drug versus placebo, then describing results for the comparison of drug A versus drug B).

If meta-analysis was possible, synthesized results should always be accompanied by a measure of statistical uncertainty, such as a 95% confidence interval. It is also helpful to indicate the amount of information (numbers of studies and participants) contributing to each meta-analysis. If no data were available for particular review outcomes of interest, review authors should say so, so that all pre-specified outcomes are accounted for. Guidance on summarizing results from meta-analysis is provided in Chapter 10, guidance on results when meta-analysis was not possible or appropriate is provided in Chapter 12.

It is important that the results of the review are presented in a manner that ensures the reader can interpret the findings accurately. The direction of effect (increase or decrease, benefit or harm), should always be clear to the reader, and the minimal important difference in the outcome (if known) should be specified. Review authors should consider presenting results in formats that are easy to interpret. For example, standardized mean differences are difficult to interpret because they are in units of standard deviation, but can be re-expressed in more accessible formats (see Chapter 15). A common mistake to avoid is the confusion of ‘no evidence of an effect’ with ‘evidence of no effect’. When there is inconclusive evidence, it is wrong to claim that it shows that an intervention has ‘no effect’ or is ‘no different’ from the control intervention. In this situation, it is better to report the data, with a confidence interval, as being uncertain, for example when the confidence interval is compatible with either a reduction or an increase in the outcome, or with a negligible difference.

In addition to summarizing the effects of interventions, review authors should also summarize the results of any subgroup analyses (or meta-regression), sensitivity analyses, and assessments of the risk of non-reporting bias (if performed) that are relevant to each synthesis. A common issue in reporting the results of subgroup analyses that should be avoided is the misleading emphasis placed on the intervention effects within subgroups without reference to the between-subgroup difference (see Chapter 10).

A ‘Summary of findings’ table is a useful means of presenting findings for the most important comparisons and outcomes, whether or not evidence is available for them. A ‘Summary of findings’ table typically:

• includes results for one clearly defined population group;
• indicates the intervention and the comparator;
• includes seven or fewer patient-important outcomes;
• describes the characteristics of the outcomes (e.g. scale, scores, follow-up);
• indicates the number of participants and studies for each outcome;
• presents at least one baseline risk for each dichotomous outcome (e.g. study population or median/medium risk) and baseline scores for continuous outcomes (if appropriate);
• summarizes the intervention effect (if appropriate), and;
• includes an assessment of the certainty of the body of evidence for each outcome.

The assessment of the certainty of the body of evidence should follow the GRADE approach, which includes considerations of risk of bias, indirectness, inconsistency, imprecision and publication bias (see Chapter 14).

III.3.4.4 Presenting results of studies and syntheses in tables and figures

Simple summary data for each intervention group (such as means and standard deviations), as well as estimates of effect (such as mean differences), should be presented for each study for each outcome of interest to the review. This is achieved primarily by using the ‘Data and analyses’ section of the review. The ‘Data and analyses’ section has a hierarchical structure, presenting results in forest plots or other table formats, grouped first by comparison, and then for each outcome assessed within the comparison. Authors can also record in each table the source of all results presented, in particular, whether results were obtained from published literature, by correspondence, from a trials register, or from another source (e.g. clinical study report). Presenting such information facilitates attempts by others to verify or reproduce the results (Page et al 2018).

Forest plots display effect estimates and confidence intervals for each individual study and the meta-analysis (Lewis and Clarke 2001). Forest plots created in RevMan typically illustrate:

1. the summary statistics (e.g. number of events and sample size of each group for dichotomous outcomes) for each study;

2. point estimates and confidence intervals for each study, both in numeric and graphic format;

3. a point estimate and confidence interval for the meta-analytic effect, both in numeric and graphic format;

4. the total numbers of participants in the experimental and control groups;

5. labels indicating the interventions being compared and the direction of effect;

6. percentage weights assigned to each study;

7. estimates of heterogeneity (e.g. Tau²) and inconsistency (I²);

8. a statistical test for the meta-analytic effect.

Review authors should present the results of single studies in the review with the expectation that results of additional studies will be added in future when they become available.

If meta-analysis was not possible or appropriate, review authors might consider presenting alternative figures to present the results of included studies. These may include a harvest plot (Ogilvie et al 2008), effect direction plot (Thomson and Thomas 2013) or albatross plot (Harrison et al 2017) (see Chapter 12 for more details). Such plots may be produced in software other than RevMan and included as an ‘Additional figure’ in a Cochrane Review.

Review authors should ensure that all statistical results presented in the main review text are consistent between the text and tables or figures.

III.3.5 Discussion

A structured discussion can help readers consider the implications of the review findings. The Discussion subheadings in RevMan provide the structure for this section of the review.

Summary of main results: It is useful to provide a concise description of results for the main outcomes of the review, but this should not simply repeat text provided elsewhere. If the review has a number of comparisons this section should focus on those that are most prominent in the review, and that address the main review objectives.

Overall completeness and applicability: This section should present an assessment of how well the evidence identified in the review addressed the review question. It should indicate whether the studies identified were sufficient to address all of the objectives of the review, and whether all relevant types of participants, interventions and outcomes have been investigated.

Certainty of the evidence: Review authors should summarize the considerations that led to downgrading or upgrading the certainty of the evidence in their implementation of GRADE. This information can be based on explanations for downgrading decisions alongside the ‘Summary of findings’ tables in the review. Note that in the current version of RevMan this subheading defaults to ‘Quality of the evidence’.

Potential biases in the review process: It is important for review authors to reflect on and report any decisions they made that might have introduced bias into the review findings. For example, rather than emphasizing the comprehensiveness of the search for studies, review authors should consider which aspects of the design or execution of the search could have led to studies being missed. This might occur because of the complexity and low specificity of the search, because the indexing of studies in the area is poor, or because searches beyond bibliographic databases did not occur. If attempts to obtain relevant data were not successful, this should be stated. Additional limitations to consider include contestable decisions relating to the inclusion or exclusion of studies, or synthesis of study results. For example, review authors may have decided to exclude particular studies from a synthesis because of uncertainty about the precise details of the interventions delivered, or measurement instrument used. If data were imputed and alternative approaches to achieve this could have been undertaken, this might also be acknowledged. It may be helpful to consider tools that have been designed to assess the risk of bias in systematic reviews (such as the ROBIS tool (Whiting et al 2016)) when writing this section.

Agreements and disagreements with other studies or reviews: Review authors should also discuss the extent to which the findings of the current review agree or disagree with those of other reviews. Authors could briefly summarize the conclusions of previous reviews addressing the same question, and if the conclusions contrast with their own, discuss why this may have occurred (e.g. because of differences in eligibility criteria, search methods or synthesis approach).

See the online MECIR Manual for all reporting guidance relevant to the Discussion section. Further guidance on issues for consideration in the Discussion section is presented in Chapter 14 and Chapter 15.

III.3.6 Conclusions

There are two standard sections in Cochrane Reviews devoted to the authors’ conclusions.

Implications for practice: In this section, review authors should provide a general interpretation of the evidence so that it can inform healthcare or policy decisions. The implications for practice should be as practical and unambiguous as possible, should be supported by the data presented in the review and should not be based on additional data that were not systematically compiled and evaluated as part of the review. Recommendations for how interventions should be implemented and used in practice should not be given in Cochrane Reviews, as they may be inappropriate depending on the different settings and individual circumstances of readers. Authors may be helpful to readers by identifying factors that are likely to be relevant to their decision making, such as the relative value of the likely benefits and harms of the intervention, participants at different levels of risk, or resource issues.

Implications for research: This section of a Cochrane Review is often used by people making decisions about future research, and review authors should try to write something that will be useful for this purpose. Implications for how research might be done and reported (e.g. the need for randomized trials rather than other types of study, for better descriptions of interventions, or for the routine collection of patient-important outcomes) should be distinguished from what future research should be done (e.g. research in particular subgroups of people, on an as yet untested experimental intervention). In addition to important gaps in the completeness and applicability of the evidence noted in the Discussion, any factors that led to downgrading the evidence as part of a GRADE assessment may provide suggestions to be addressed by future research. This section could also usefully draw on what is known about any ongoing studies identified from trials register searches, and use any information about ongoing or recently completed studies to guide recommendations on whether new studies need to be initiated. It is important that this section is as clear and explicit as possible. General statements that contain little or no specific information, such as “Future research should be better conducted” or “More research is needed” are of little use to people making decisions, and should be avoided.

See the online MECIR Manual for reporting guidance relevant to the conclusions of a review.

III.3.7 Administrative information

A Cochrane Review should include several pieces of administrative information, many of which are standard in other journals. These include acknowledgements, contributions of authors, declarations of interest, differences between the protocol and review, and sources of support (see the online MECIR Manual for relevant reporting guidance).

Contributions of authors: The contributions of each author to the review should be described. It is helpful to specify which authors were involved in each of the following tasks: conception of the review; design of the review; co-ordination of the review; search and selection of studies for inclusion in the review; collection of data for the review; assessment of the risk of bias in the included studies; analysis of data; assessment of the certainty in the body of evidence; interpretation of data, and; writing of the review. Refer to the authorship and contributorship policy of the Cochrane Editorial and Publishing Policy Resource for criteria for authorship.

Declarations of interest: All authors should report any present or recent (three years prior to declaration) affiliations or other involvement in any organization or entity with an interest in the review’s findings that might lead to a real or perceived conflict of interest. They should also include the dates of the involvement. If there are no known conflicts of interest, this should be stated explicitly, for example, by writing “None known”. Authors should make themselves aware of the restrictions in place on authorship of Cochrane Reviews where conflicts of interest arise. The full policy on conflicts of interest is available in the Cochrane Editorial and Publishing Policy Resource.

Acknowledgements: Review authors should acknowledge the contribution of people not listed as authors of the review, including any assistance from the Cochrane Review Group responsible for handling the review, and any contributions to searching, data collection, study appraisal or statistical analysis performed by people not listed as authors. Written permission is required from those listed in this section.

Differences between protocol and review: Review authors may sometimes use different or additional methods from those described in the review protocol (e.g. making post-hoc changes to eligibility criteria, or adding subgroup analyses). This could occur because methods for dealing with a particular issue had not been specified in the protocol, pre-specified methods could not be applied due to insufficient data, or methods were changed because a preferable alternative arose or more recent guidance was identified. All changes of methods from protocol to review should be fully described and justified in this section of the review. When a review is updated, this section can also be used to describe changes between the methods in the previous and new versions of the review (see Chapter IV, Section IV.5).

Sources of support: Authors should acknowledge grants that supported the review, and other forms of support, such as support from their university or institution in the form of a salary. Sources of support are divided into ‘internal’ (provided by the institutions at which the review was produced) and ‘external’ (provided by other institutions or funding agencies). Each source, its country of origin and what it supported should be provided. Authors should make themselves aware of the restrictions in place on funding of Cochrane Reviews by commercial sources where conflicts of interest may arise. The full policy on conflicts of interest is available in the Cochrane Editorial and Publishing Policy Resource.

III.4 Reporting of plain language summaries in Cochrane Reviews

The plain language summary is a stand-alone summary of the systematic review. It should convey succinctly and clearly the key question and key findings of the review, in plain English that can be understood by consumers and non-expert readers. Authors writing a plain language summary should consider the target audience, which may include people with a health condition, carers, healthcare workers or policy makers. Some topics may need more explanation than others based on the likely familiarity the target audience has with the topic, and the same term may mean different things to different people.

Writing in plain language is a skill that is different from writing for a scientific audience. Review authors are encouraged to seek assistance to ensure that the summary is readily understood by a non-expert audience.

A complementary initiative to MECIR, the Plain Language Expectations for Authors of Cochrane Summaries (PLEACS), produced a set of specific reporting guidelines for plain language summaries . These expectations were developed collaboratively between consumers, representatives of Cochrane Review Groups, and methodologists. Subsequent to their release, Glenton and colleagues produced further guidance on writing a Cochrane Review plain language summary, designed to supplement PLEACS.

III.5 Chapter information

Authors: Matthew J Page, Miranda Cumpston, Jacqueline Chandler, Toby Lasserson

Acknowledgments: This chapter builds on an earlier version of the Handbook (Version 5, Chapter 4: Guide to the contents of a Cochrane protocol and review), edited by Julian Higgins and Sally Green. We thank them for their contributions to the earlier chapter. We thank Rachel Churchill, Robin Featherstone, Ruth Foxlee and Nuala Livingstone for their feedback on this chapter.

Declarations of interest: Toby Lasserson and Jacqueline Chandler are members of the core group who developed MECIR guidance, and were members of the PLEACS development group.

III.6 References

Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, Chalmers I, Gøtzsche PC, Lasserson T, Tovey D, for the PfAG. PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts. PLoS Medicine 2013; 10: e1001419.

Campbell M, Katikireddi SV, Sowden A, McKenzie JE, Thomson H. Improving Conduct and Reporting of Narrative Synthesis of Quantitative Data (ICONS-Quant): protocol for a mixed methods study to develop a reporting guideline. BMJ Open 2018; 8: e020064.

Dwan KM, Williamson PR, Kirkham JJ. Do systematic reviews still exclude studies with "no relevant outcome data"? BMJ 2017; 358: j3919.

Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 2014; 383: 267-276.

Harrison S, Jones HE, Martin RM, Lewis SJ, Higgins JPT. The albatross plot: A novel graphical tool for presenting results of diversely reported studies in a systematic review. Research Synthesis Methods 2017; 8: 281-289.

Hoffmann TC, Oxman AD, Ioannidis JP, Moher D, Lasserson TJ, Tovey DI, Stein K, Sutcliffe K, Ravaud P, Altman DG, Perera R, Glasziou P. Enhancing the usability of systematic reviews by improving the consideration and description of interventions. BMJ 2017; 358: j2998.

Kneale D, Thomas J, Harris K. Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews. PloS One 2015; 10: e0142187.

Lasserson T, Churchill R, Chandler J, Tovey D, Higgins JPT. Standards for the reporting of protocols of new Cochrane Intervention Reviews. In: Higgins JPT, Lasserson T, Chandler J, Tovey D, Churchill R, editors. Methodological Expectations of Cochrane Intervention Reviews. London: Cochrane; 2016.

Lewis S, Clarke M. Forest plots: trying to see the wood and the trees. BMJ 2001; 322: 1479-1480.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Medicine / Public Library of Science 2009; 6: e1000100.

Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine / Public Library of Science 2009; 6: e1000097.

Ogilvie D, Fayter D, Petticrew M, Sowden A, Thomas S, Whitehead M, Worthy G. The harvest plot: a method for synthesising evidence about the differential effects of interventions. BMC Medical Research Methodology 2008; 8: 8.

Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catalá-López F, Li L, Reid EK, Sarkis-Onofre R, Moher D. Epidemiology and reporting characteristics of systematic reviews of biomedical research: A cross-sectional study. PLoS Medicine 2016; 13: e1002028.

Page MJ, Altman DG, Shamseer L, McKenzie JE, Ahmadzai N, Wolfe D, Yazdi F, Catala-Lopez F, Tricco AC, Moher D. Reproducible research practices are underused in systematic reviews of biomedical interventions. Journal of Clinical Epidemiology 2018; 94: 8-18.

Reeves BC, Wells GA, Waddington H. Quasi-experimental study designs series-paper 5: a checklist for classifying studies evaluating the effects on health interventions-a taxonomy without labels. Journal of Clinical Epidemiology 2017; 89: 30-42.

Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, Group P-P. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015; 349: g7647.

Thomson HJ, Thomas S. The effect direction plot: visual display of non-standardised effects across multiple outcome domains. Research Synthesis Methods 2013; 4: 95-101.

Whiting P, Savovic J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. Journal of Clinical Epidemiology 2016; 69: 225-234.