Quality Assurance Editors in Cochrane Central Editorial Service use the questions below to assess the methods of submitted intervention reviews. We make these questions freely available so authors can check their own work before submission. 

This is the checklist for reviews. View the methods peer review checklist for protocols.

The assessment questions show the order in which checks are conducted. For further information on preparing your manuscript for submission, see our Author guidelines.

Implementation of protocol methods
 

Search date

• Is the search date < 12 months from date of submission? 
• Check date is consistent across the Dates and events, Abstract, and Methods sections 

Changes to methods

• Have the objectives, PICO and or synthesis methods remained the same since protocol was published?
  • Compare versions in RevMan to view changes between protocol and review
• If any changes were made, are they reasonable and clearly acknowledged/justified at the beginning of the Methods section?

Are all the methods still up to date and appropriate?

• See Methods peer review checklist: protocol for in-depth checks on planned methods

PRISMA diagram/flow chart

• Do the numbers presented in the PRISMA diagram match the Results of the search summary section and study tables?

Studies

• Are the Characteristics of included studies tables appropriately populated with relevant and consistent PICO detail?
  • Are the tables free of spelling/grammar errors that may affect readability?
  • Are quotations from study reports correctly cited within quotation marks?
• Check that the following tables/section are consistent with one another: 
  • Overview of Syntheses and Included Studies and table 
  • Characteristics of Included Studies tables
  • Description of studies within the main text of the review 
  • Check overall numbers reported in the review from these tables/sections are correct
•  Check the list of excluded studies – are all reasons for exclusion appropriate?
  • Inappropriate exclusions may be ‘excluded due to publication status’, or ‘no relevant outcomes reported’, etc)
• Characteristics of ongoing studies and studies awaiting classification tables: 
  • Check overall numbers reported in the review from these tables are correct
• Is the summary of included studies in the Results section grouped by PICO, not study ID?
• Does the summary of excluded studies in the Results section summarise the main reasons for exclusion?

Was the Risk of bias assessment appropriate?

• If using RoB 1:
  • Check for omission of standard domains; inclusion of any non-standard domains is explained and justified; domains appear well understood (fit between explanation and domain, appropriate judgments)
  • Check that the judgements presented are consistently reported in the main text. 
• If using RoB 2: 
  • Check against editorial checklist (available for download) 

Are the analyses consistent with the methods, appropriate and accurate?

• Do the comparisons presented in the Analyses section match the comparisons that were planned in the ‘Methods’ section, and that were feasible according to the Overview of Syntheses and Included Studies table(s)?
• Check each analysis to ensure statistical plans were adhered to:
  • Did authors use correct measures of treatment effects (MDs/SMD, OR/RR/RD)
  • Did authors adhere to their plan for using either fixed or random effects? If authors planned to conduct sensitivity analyses related to fixed/random effect models, did they complete these analyses?
  • Did authors adhere to their plans for addressing unit of analysis issues (including cross-over data, cluster RCT data, and multi-arm data – groups split/merged)
• Did authors adhere to their planned subgroup and sensitivity analyses?
  • Look for obvious outliers – check original study data if possible, to see if it is a data entry error.
• Check the forest plot labels are correct

Is the reporting of results in the Synthesis of results section accurate and appropriate?

• Order of results should be ‘comparison’ followed by ‘outcome’ (and should mirror order of results in Analyses section)
• All results from the 'Analyses' section should be summarised in this section
• All results interpreted accurately (e.g., focus on size, direction and confidence interval of result, and do not refer to ‘statistical significance’)
• If certainty assessed using GRADE, ensure it is reported in this section
• Ensure that ‘evidence of no effect’ is not confused for ‘no evidence of an effect’
• Ensure subgroup analyses presented appropriately:
  • Start with results of test for subgroup difference
  • Only go on to summarise individual subgroups if there is evidence of a difference

Is the Results > Equity assessment section appropriately populated?

• Does the section include a brief overview of the assessments (e.g. any overall comments or important differences and why that might be) and a link to an equity results additional supplementary material?
• If there was no formal equity assessment, this section should be left blank

Is the Results > Reporting bias section appropriately populated?

• Does the section include information relating to risk of bias due to missing results (arising from reporting biases) for each synthesis assessed?
  • If not applicable, this section should be left blank
• Check that authors have acknowledged that asymmetry observed in a funnel plot or test for asymmetry may be due to reasons other than publication bias
• Check that authors have not undertaken multiple test of publication bias and focused on results of one particular test without justification

Back to top

Summary of findings (SoF) tables
 

SoF table presents the main outcomes (benefits and harms) for the main comparison

• Are the comparisons and outcomes presented consistent with what was planned in the methods section?
• Are the same outcomes presented across all tables, regardless of type or availability of evidence?
• Are outcomes only presented once, at a single timepoint and single method of measurement? 

PICO (including Settings) are accurate and informative

• Population, Intervention and Comparison should be PICO ‘of interest’ to the review question
• Setting can be summary of settings covered by available evidence

Outcomes are fully defined

• Are timepoint(s) of measurement and method of measurement presented for each outcome?
  • Scale of measurement, range of scores specified can also be presented where applicable

Assumed and corresponding risks are included (where appropriate)

• Check the ‘assumed risk’ cell for continuous outcomes is populated with mean/median of control group
• Check for any dichotomous outcomes reporting “0 per 1000” in  both groups, as could be more helpful to replace this with raw data

GRADE ratings are justified are adequately explained

• Does each outcome have a GRADE rating?
• Are decisions made appropriate and consistent across outcomes?
• Does each GRADE rating have a footnote?
• Does each footnote state (a) how many levels outcome was downgraded, (b) for what GRADE domain, and (c) a brief reason why?

Clear and accurate summary of narrative results (where appropriate)

• Narrative summaries should appear as a merged cell below other results, and not in the ‘comments’ column

Quality ratings presented for narrative results (where appropriate)

• Narrative summaries should also be assessed using GRADE

Is the "Comments" column appropriately and consistently populated? 

• This does not have to be populated, as there are multiple different ways this column can be used. But if it is populated, ensure text is succinct and consistent. 

Back to top

Discussion and Conclusions
 

Summary of main results

• Does this section present a concise narrative description of results for the main outcomes of the review (SoF comparisons and outcomes)
  • Numerical results should not be repeated
• The priority is to ensure SoF table comparisons and outcomes are summarised in this section. Additional outcomes may be presented. Likewise, if non-SoF table comparisons and outcomes do not need to be summarised here.

Limitations of the evidence included in the review

• Do the authors discuss how well the evidence identified in the review addressed the review question? 
• Do the authors explain considerations that led to downgrading or upgrading the certainty of the evidence? 
• Do the authors discuss the applicability of the results for different populations and settings?

Limitations of the review processes

• Do the authors summarise limitations of the review processes used and comment on the potential impact of each limitation?

Agreements and disagreements with other studies or reviews

• Do the authors provide a general interpretation of their results in the context of other evidence? 

Authors' conclusions are appropriately worded and populated

• Implications for practice focuses on results from this direct synthesis only, and does not make recommendations
  • This section should provide a general interpretation of the evidence, and use language that accurately reflects the certainty of the results
• Equity-related implications for practice is either correctly populated or empty (will not publish)
• Implications for research draws on important gaps in the limitations of the evidence noted in the Discussion 
• Equity-related implications for research is either correctly populated or empty (will not publish)

Back to top

Abstract
 

Title

• Does the title reflect the review question and objective?

Rationale 

• Do authors provide a concise summary of the rationale for and context of the review (2-3 sentences)?
  • This could include a brief description of the evidence base, what is currently unknown or uncertain, and why it is important to resolve this uncertainty with this systematic review

Objective

• Does the objective include reference to the population(s), health conditions, and intervention comparison(s)?
  • The primary objective in the Abstract should match the objective in the main text exactly; secondary objectives can be paraphrased, or omitted

Eligibility criteria

• Have the inclusion and exclusion criteria for the review been clearly specified? 

Outcomes

• Have the outcomes been listed?
  • The main outcomes (i.e., those that were prioritised for SoF tables) must be listed in this section
  • Authors can decide whether they want to list all outcomes here. If they do, they should also clarify that these outcomes were not prioritised for SoF tables, hence why their results are not included in the section ‘Abstract > Synthesis of results’. 

Risk of bias

• Have the methods used to assess risk of bias in the included studies been specified?

Synthesis methods

• Have the methods used to present and synthesise results been specified?
  • E.g.,  statistical and analysis models used (if meta-analysis was conducted) or the synthesis without meta-analysis (SWiM) methods used. The certainty of the evidence is generally assessed with the GRADE approach 

Included studies

• Have the total number of included studies and participants been reported accurately? Have the characteristics of included studies been summarised?

Synthesis of results

• Have findings for all important outcomes been reported for main comparison(s), including information about harms?
  • Check consistency with the first SoF table and others as appropriate 
• Have the direction, magnitude and confidence intervals of effects been clearly described where appropriate?
  • Reporting for each main outcome should include the number of studies and participants for each outcome, a clear statement of the direction and magnitude of the effect, the effect estimate and confidence intervals if meta-analysis was used, and the GRADE assessment of the certainty of the evidence. 
• Have authors used confidence intervals and their width to determine presence or absence of an effect?
  • Authors have avoided describing results as ‘statistically significant’, or ‘non-significant
• Has a summary of the certainty or limitations of the evidence included in the review been reported?
  • E.g. study risk of bias, inconsistency, imprecision, publication bias)
• Have absolute effects been reported to illustrate the relative effects where appropriate?
  • Reporting may be constrained by the word count, but absolute effects may be easier to interpret than relative effects. 
  • Results showing very low certainty don’t always need absolute effects

Authors’ conclusions 

• Do the conclusions accurately reflect the evidence in the Synthesis of results section and SoF table(s)?
  • Authors should state the key conclusions and implications for practice and/or research including the magnitude and direction of effects, together with an indication of the certainty of the evidence
  • Suggested narrative statements as provided in the Cochrane Handbook, Chapter 15 Table 15.6.b 
• Have authors avoided making any direct recommendations for clinical practice?  

Plain language summary

• Does the PLS adhere to the structure and format of the Cochrane Handbook guidance for writing a plain language summary?
• Does the PLS avoid technical language and jargon?

Consistency

Are the key findings and conclusions consistent across all the summary versions of the review (Abstract, PLS, SoF tables, Authors Conclusions)?
 

Back to top