Guy Peryer, Su Golder, Daniela R Junqueira, Sunita Vohra, Yoon Kong Loke; on behalf of the Cochrane Adverse Effects Methods Group

Cite this chapter as: Peryer G, Golder S, Junqueira D, Vohra S, Loke YK. Chapter 19: Adverse effects. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook.

19.1 Introduction to issues in addressing adverse effects

Every healthcare intervention comes with the risk, great or small, of harmful or adverse effects. A Cochrane Review that considers only the favourable outcomes of the interventions that it examines, without also assessing the adverse effects, will lack balance and may make the intervention look more favourable than it should. All reviews should try to consider the adverse aspects of interventions.

This chapter addresses special issues about adverse effects in Cochrane Reviews. It focuses on methodological differences when assessing adverse effects compared with other outcomes.

19.1.1 Terminology and definitions

Poor standardization and usage of adverse effects terminology in study reports can produce challenges for review authors. Common, and closely related, terms include adverse event, adverse effect, serious adverse event, serious adverse effects, adverse drug reaction, side effect, complications and harms (Zorzela et al 2016). In this chapter we use the term adverse event for an unfavourable or harmful outcome that occurs during, or after, the use of a drug or other intervention, but is not necessarily caused by it, and an adverse effect (or harm) as an adverse event for which the causal relation between the intervention and the event is at least a reasonable possibility.

19.1.2 Special issues for addressing adverse effects

In this section we discuss some of the particular challenges when addressing adverse effects. First, there can be wide diversity across studies in how adverse events are defined, ascertained, analysed and reported. Second, adverse effects may not be known when studies were planned, so data collection processes and analytic strategies may not be in place. Third, many adverse events are too uncommon or too long-term to be observed within randomized trials.

19.1.2.1 Diversity in defining and monitoring of adverse events

A huge range of adverse events can occur in a research study, and there are multiple ways in which adverse effects can be ascertained and categorized by study investigators (Smith et al 2015). There are two broad strategies for collecting information on adverse events. Study investigators may use active monitoring or surveillance, which directs enquiry towards pre-defined adverse events of interest, usually following protocol-defined procedures for data collection, case definitions and adjudication. For example, if the event of interest is myocardial infarction, the study protocol might require collection of laboratory and electrocardiogram data for suspected events. These results might then be referred to an independent panel which adjudicates or ascertains the occurrence of an event. Such active monitoring usually relates to sets of potential adverse events that are either known or suspected to be associated with an intervention.

Although prospective collection of adverse event data is desirable, many adverse effects cannot be pre-specified because they are not yet known or suspected to be associated with an intervention. Thus, spontaneous report monitoring may occur, which involves recording all adverse events (pre-defined or not) throughout the duration of the study. Both participants and researchers recognizing any adverse event can file a report at any time. This may uncover new or unexpected adverse effects not previously associated with the intervention. For regulated products (e.g. drugs, biologics, vaccines), spontaneously reported adverse events are usually coded, grouped and categorized following established dictionaries for analysis and presentation.

Whichever monitoring method is used to collect information about adverse events, study investigators may combine adverse events into global or composite measures, which are often reported as total number of serious adverse events, or number of withdrawals due to adverse events, or total number of adverse events in an anatomic or organ system (e.g. gastrointestinal, cardiovascular). However, these composite measures do not give information on what exactly the events were, and so it is usually necessary to drill down for details of distinct or individual adverse events, such as nausea or rash.

Ideally, the definition and ascertainment of adverse events should be as uniform as possible across the included studies in the review. The lack of systematic monitoring or follow-up, coupled with divergent methods of seeking, verifying and classifying adverse events, can introduce heterogeneity in effect estimates among studies. Review authors will therefore need to pay close attention to outcome definition and method of monitoring when interpreting or comparing frequencies, rates and risk estimates for adverse effects.

19.1.2.2 Inconsistent and poor reporting of adverse effects

Inconsistent outcome definition and poor ascertainment are problematic for reviews that rely exclusively on published data. Information taken from published reports may be incomplete or lack specificity. Across multiple investigations of published versus unpublished studies, Golder and colleagues found a median of 43% of published studies reported adverse events data, compared with a median of 83% of unpublished studies (Golder et al 2016). A wider range of specific adverse events was found in sources other than published journal articles. In addition, when published and unpublished reports of the same study were compared, it was shown that the unpublished version was more likely to contain adverse effects data (median 95%) compared with the published version (median 46%). Similarly, a study of an obesity drug (orlistat or Xenical) by Schroll and colleagues compared study documents (protocol, clinical study report (CSR), and published report), and identified important inconsistencies (Schroll et al 2016). For example, adverse events in published studies were coded to appear less severe, with reduced incidence, compared with events reported in the unpublished CSRs. Of the total number of adverse events reported by trial investigators in CSRs, between 3% and 33% were subsequently reported in the corresponding published journal articles.

19.1.2.3 Different study designs to measure adverse events

Some adverse effects occur rarely or may only become apparent long after the start of intervention. This contrasts with adverse effects that have a higher incidence and occur soon after the intervention is delivered. A small randomized trial with only short-term follow-up may be able to capture common, immediately apparent adverse effects (e.g. skin reaction after injection) adequately. However, rare or long-term adverse effects may only be observed in non-randomized studies such as large cohort studies or case-control studies. Therefore, depending on the type of adverse outcome of interest, review authors may need to consider evidence extending beyond the time frame of randomized trials.

19.2 Formulation of the review

A starting point for assessing adverse effects of an intervention is to consider whether a review will evaluate both beneficial and adverse effects of an intervention, or just the adverse effects. Although most Cochrane Reviews look at both beneficial and adverse effects, review authors may decide to conduct a separate review of only the adverse effects of an intervention (see Box 19.2.a). Whichever strategy is taken, review authors will need to decide whether to focus only on a pre-specified set of adverse events (a ‘confirmatory’ approach), or analyse data on adverse events identified during the conduct of the review (an ‘exploratory’ approach). In practice, some review authors will use a hybrid of these two approaches. Consideration will also be needed of whether the same sources of evidence will be used to look at beneficial and adverse effects, or whether additional types of evidence will be sought to examine the adverse effects. Finally, the specific selection and definition of adverse effects will need to be considered. In this section we tackle these key considerations for formulating a review to look at adverse effects.

Box 19.2.a Reviews of adverse effects alone

19.2.1 Which adverse events to look at

19.2.1.1 Confirmatory approach

In a confirmatory approach, review authors list one or more adverse effects as outcomes of interest in their review protocol. Golder and colleagues found that approximately 80% of systematic reviews of adverse effects published between 1994 and 2011 used this approach, selecting particular events, or categories of events, as their main interest (Golder et al 2013).

When adopting the confirmatory approach, review authors should aim to pre-specify adverse effects that are anticipated or already recognized to be associated with the intervention, and assumed to be measured regularly and consistently in studies. Selection of adverse effects of interest can be based on biological, physiological or psychological plausibility. For example, in a review of a surgical intervention it is plausible to pre-specify ‘wound infection’ as an adverse outcome of interest. Similarly, a systematic review of drug therapy that affects platelets or clotting would be justified in pre-specifying bleeding as an adverse outcome of interest. In some cases, it may be reasonable to select adverse effects for review based on previously established observation or association, although the plausible mechanism of effect has not yet been established.

A key limitation of the confirmatory approach is the inability to handle unanticipated adverse effects that are reported in the included studies.

19.2.1.2 Exploratory approach

An exploratory approach to reviewing adverse effects does not include pre-specification of any particular adverse outcomes of interest. Rather, it typically involves extracting any, or all, of the adverse event data found within the included studies. Only about 20% of reviews of adverse effects specify this as their main approach (Golder et al 2013).

The exploratory approach can identify unanticipated and rare adverse effects of an intervention. This may inform which outcomes are investigated in future reviews of pre-specified adverse events that use the confirmatory approach. In addition, the exploratory approach may provide data on possible associations between an intervention and a list of observed adverse events, which can be used to generate new signals to add to existing safety profiles.

A limitation of the exploratory approach is that the specific adverse effects reported may have been selectively analysed and reported because of the nature of the findings (e.g. based on statistical significance rather than clinical importance). Also, post-hoc or arbitrary analytic decisions regarding data extraction and analysis are often required when review authors encounter long lists of adverse events. Processes for selection and synthesis of such data need consideration in the review protocol, even if the outcomes of interest are not fully specified.

19.2.1.3 Hybrid approach

The hybrid approach combines elements of both confirmatory and exploratory approaches to capture anticipated and previously unrecognized adverse effects of an intervention. Reviews based on this approach might list a small number of adverse outcomes of interest in the protocol, whilst allowing post-hoc exploratory analyses to capture adverse events data available from the studies identified. An example is provided in Box 19.2.b.

Regardless of the approach adopted, review authors should be mindful of the potential for problems related to definition and ascertainment of adverse events when reviews are based solely on published data.

Box 19.2.b Illustration of three approaches to reviewing the adverse effects of a particular intervention: acupuncture

19.2.2 Strategies for assessing beneficial and adverse effects in the same review

When conducting a review of both beneficial and adverse effects of interventions, review authors may:

1. use the same eligibility criteria to assess intended (beneficial) and unintended (adverse) effects, in terms of types of studies, types of participant and types of interventions; or
2. use different eligibility criteria for selecting studies that address unintended (adverse) effects compared with studies that address intended (beneficial) effects.

Using the same eligibility criteria to gather data on both types of outcome makes the review easier to conduct, not least because a single search can usually be undertaken if outcome terms are not stipulated in the search string. It also may allow for a direct comparison between beneficial and adverse effects, because the data are derived from the same types of studies (although it will not necessarily be the case that exactly the same studies report data on both beneficial and adverse effects). Two disadvantages of using the same eligibility criteria are (i) that the types of studies that are most appropriate to address the beneficial effects – typically randomized trials – may not be large enough or long enough to capture important adverse effects; and (ii) that it may lead to omission of relevant data on adverse effects if the adverse effects are also observed when the intervention is given for other conditions (see also Box 19.2.a).

Thus, review authors may apply different eligibility criteria when attempting to identify adverse effects data. The two main aspects of eligibility that may differ are the types of study design and the types of participants. It is also possible that studies performed for a different purpose may be eligible for the adverse effects component of the review.

• Different study designs: To address adverse effects it may be necessary to seek non-randomized studies, because the effects are unlikely to be seen in randomized trials due to their size, duration or restricted eligibility for participants: see Section 19.2.3.
• Different types of participants: Adverse effects data might be obtained from randomized trials evaluating the same or similar intervention but conducted in different populations or diseases (see also Box 19.2.a).
• Different purposes: There may be randomized trials with adverse effects data on participants of interest to the review, but which did not measure the beneficial outcomes relevant to the review (e.g. a pharmacokinetic study assessing drug concentrations in patients with the disease).

When different eligibility criteria are used to address beneficial and adverse effects, it will often be necessary to conduct a separate search for the two (or more) sets of studies (see Section 19.3), and it may be necessary to plan different methods in other aspects such as assessing risk of bias (see Section 19.4).

19.2.3 Selecting types of study design

Cochrane Reviews typically include randomized trials because randomization should distribute both known and unknown confounding variables equally across intervention groups (see Chapter 3, Section 3.3.1). However, the duration of follow-up in a randomized trial may not be sufficient to capture long-term adverse effects, and criteria for selecting participants into randomized trials may exclude participants at increased risk of harm (such as people with comorbidities or older adults living with frailty). Also, randomized crossover trials (see Chapter 23, Section 23.2) may not be appropriate for investigating some adverse effects, particularly if exposure to an intervention in one period results in an adverse event occurring in a later period. Non-randomized studies of interventions such as cohort studies (assembled from disease or drug/device registries) and case-control studies may be more likely than randomized trials to provide data on some types of adverse effects. However, non-randomized studies tend to be at greater risk of bias (see Chapter 24).

Spontaneous case reports or case series may assist in signalling rare and previously unknown events. However, for most Cochrane Reviews, these data sources should be used for scoping purposes only (particularly as they do not have denominator data to allow estimation of risks or rates). These spontaneous reports may guide drafting of the protocol when there is a need to choose relevant or important adverse effects as outcomes of interest.

19.2.4 Selecting adverse effects of interest

Review authors may define outcomes of interest based on severity, timing or the type of adverse effects that could occur based on the known mode of action of the intervention. Different sources may be used to inform pre-specification of adverse effects of interest. These sources include clinicians’ observations in case reports, patients’ reports on internet forums, scoping reviews, regulatory approved product information leaflets (e.g. from the US Food and Drug Administration) or other sources (e.g. British National Formulary, Meyler’s Side Effects of Drugs).

Composite adverse outcomes are often reported by trials. Common examples include ‘total number of participants with adverse events’, or ‘numbers of withdrawals due to adverse events’. Review authors should recognize major difficulties in interpreting composite adverse outcomes that are potentially constructed from hundreds of diverse events, because an important signal of rare serious adverse events could be masked by common, trivial adverse events. Also, review authors should hesitate to interpret data on withdrawals as surrogate markers for safety or tolerability, for the following reasons.

• The attribution of reason(s) for discontinuation is complex and may be due to mild but irritating side effects, toxicity, lack of efficacy, non-medical reasons, or a combination of causes.
• The pressures on patients and investigators under trial conditions to reduce the number of withdrawals and dropouts can result in rates that do not reflect the experience of adverse events within the wider population.
• Unblinding of intervention assignment often precedes the decision to withdraw. This can lead to an over-estimate of the intervention’s effect on patient withdrawal. For example, symptoms of patients in the placebo arm are less likely to lead to discontinuation. Conversely, patients in the active intervention group who complained of symptoms suggesting adverse effects may have been more readily withdrawn.

19.3 Identification of evidence

19.3.1 Search methods for adverse effects data

When considering the search process, review authors may decide to perform a single search to retrieve studies evaluating both benefits and harms. If so, the search strategy should be designed to take account of the selected approach, either confirmatory, exploratory or hybrid, and any differences in eligibility criteria for addressing beneficial and adverse effects. A single search may be reasonable if it is sufficiently broad (e.g. if it captures all studies containing a specific drug name or intervention) without being limited to specific study designs or types of participants.

In general, we recommend consideration of a separate bespoke search for data on adverse effects, particularly if the study designs that evaluate adverse effects of interest are different from those that report efficacy. It is unlikely that a single search that is focused on efficacy or effectiveness studies will be sufficient to identify evidence on all adverse effects in a comprehensive manner.

19.3.2 Allocating resources for the search

Despite significant improvements in reporting of adverse effects in primary studies, specific terms relating to adverse effects may not feature in the title, abstract, keywords or bibliographic database indexing systems. To determine the necessary work and resources involved, careful scoping when drafting the review question is recommended. This may need to account for the inclusion of unpublished data (see Section 19.3.4 and Chapter 4) and non-randomized studies (see Chapter 24).

19.3.3 Sources to search

Due to the variable content and indexing techniques of healthcare databases, it is important not to restrict adverse effect review searches to a single source, nor to a limited combination of the primary clinical research databases. Performing a search in MEDLINE alone is not recommended.

A case study reviewing adverse effects of thiazolidinedione use in patients diagnosed with type 2 diabetes mellitus tested over 60 sources (Golder and Loke 2012).The results indicated that the minimum combination of sources required to identify all relevant references included 11 sources: the pharmaceutical company website, Science Citation Index, Embase, BIOSIS Previews, British Library Direct, Medscape DrugInfo, American Hospital Formulary Service (AHFS First), Thomson Reuters Integrity, Conference Papers Index, hand searching and reference checking. In this specific example, just searching MEDLINE failed to retrieve 66% of relevant references. A search strategy conducted in MEDLINE, Embase and the Cochrane Central Register of Controlled Trials (CENTRAL) failed to retrieve 57% of relevant references. This example illustrates the breadth of sources needed to ensure identification of relevant data. Authors will need to consider sources most relevant to their clinical question; the list above is an illustration only.

Identifying adverse effects of pharmacological interventions often requires search methods that are different from those required for reviews of non-pharmacological interventions, or medical devices. Further guidance for sourcing adverse effects data is given in the online Technical Supplement to Chapter 4.

19.3.4 Including unpublished sources

Review authors should search for unpublished sources of data on adverse effects. We consider unpublished sources to be those outside of a peer-reviewed journal. This includes: clinical study reports (CSR), trials registers and regulatory agency websites. Tang and colleagues showed the value of searching ClinicalTrials.gov for data on serious adverse events (Tang et al 2015). Among 300 trials with serious adverse events mentioned in ClinicalTrials.gov, 78 (26%) did not have a corresponding publication, and for the remaining 202 trials, 26 (13%) published articles did not mention serious adverse events. Limiting search strategies to published reports may therefore not produce a balanced review, leading to underestimates of harm.

Mandatory changes applied to trials regulated by the Food and Drug Administration (FDA) regarding the submission of adverse events data to ClinicalTrials.gov, and the legislated publication of clinical data by the European Medicines Agency (EMA), means that previous accessibility limitations are steadily improving. Although accessibility is likely to continue to improve, the logistics and feasibility of routinely using such data sources for adverse effects reviews has yet to be established. Review authors should therefore report on the number of unpublished studies identified and instances where data on adverse effects were inaccessible.

19.3.5 Search methods: specific and generic outcome terms

Searching for specific adverse effects outcomes is similar to searching for specific benefit outcomes, so that search terms for the particular adverse effects outcome(s) are included in the search string. Examples of specific adverse effects terms are: ‘headache’, ‘blood loss’ or ‘dysphagia’. However, it is likely that this method will lack sensitivity due to variation in reporting and indexing.

A possible option for the larger databases is to use a broad search involving two components at the same time: generic index terms combined with specific free-text searches using the ‘OR’ Boolean function. Both specific and generic search techniques have strengths and limitations, but the strengths are increased and limitations reduced when they are combined. It is therefore advisable to combine index terms and free-text searching (where possible) to increase search sensitivity and reduce the possibility of missing relevant material. More details are provided in the online Technical Supplement to Chapter 4.

19.4 Appraisal of evidence

19.4.1 Challenges in assessing risk of bias for adverse effects data

Assessing risk of bias for pre-specified adverse effects that are actively monitored in included studies is generally the same as for the pre-specified beneficial effects. However, adverse effects are seldom specified as primary outcomes, and often are not pre-specified at all, so there is often lack of clarity in the methods used to obtain adverse effects data. Thus, different susceptibilities to bias can arise for adverse effects due to the way in which they are measured, recorded and reported. It is important that the outcome measure is appropriate for detection of the adverse effect, and that the outcomes are measured or ascertained using a method that is comparable across intervention groups (see Chapter 8, Section 8.6). Study participants prematurely stopping assigned intervention or withdrawing from the study (due to adverse events) can result in dissimilar observation times for ascertaining future adverse events. When assessing the risk of bias for missing outcome data, it is important to consider the possibility of differential follow-up and informative censoring. A particular challenge when assessing risk of bias for adverse effects data is that of selective reporting. Results based on spontaneously reported adverse outcomes may lead to concerns that these were selected post hoc based on the finding being noteworthy. Similarly, unusual composite outcomes may be reported to hide or emphasize particular findings.

19.4.2 Recommended tools for assessing risk of bias in adverse effects data

Review authors should use the currently recommended risk-of-bias tools, the RoB 2 tool for randomized trials (see Chapter 8), and the ROBINS-I tool for non-randomized studies (see Chapter 25). Although these tools are most easily directed at outcomes that have been pre-specified by the review team, they are suitable for any type of quantitative outcome analysed in a review. Where adverse effects are extracted post hoc from included trials in an exploratory approach, it may not be possible to list important co-interventions or confounding variables in the review protocol, as would usually be expected for using the ROBINS-I tool.

Particular issues in assessing risk of bias for adverse effects data include outcome definition and methods of monitoring adverse effects. These warrant special attention when there are significant concerns over bias towards the null stemming from poor definition, ascertainment or reporting of harms. This is particularly important for new or unexpected adverse events that have not been pre-specified as outcomes of interest in the trials, and where monitoring and reporting may be potentially inadequate. Additional resources such as the McHarm tool (Chou et al 2010) and the Agency for Healthcare Research and Quality (AHRQ) assessment tool (Chou et al 2007, Viswanathan and Berkman 2012) provide further discussion of these issues.

19.4.3 Selective outcome reporting bias of adverse effects data

Selective outcome reporting refers to authors reporting a subset of variables, based on the results, from among all the outcomes originally analysed (see Chapter 7). Selective outcome reporting distorts the body of available evidence on which to conduct data synthesis and can lead to a high risk of bias (Kicinski et al 2015). Missing or partially reported adverse effects data are common in systematic reviews evaluating adverse effects (Saini et al 2014).

There is evidence that Cochrane Reviews may suffer from reporting bias. Kicinski and colleagues explored the potential impact of reporting bias on meta-analyses in Cochrane Reviews published between 1990 and 2005 (Kicinski et al 2015). They applied hypothesized mechanisms of reporting bias to 802 meta-analyses of efficacy and 304 meta-analyses of safety that each combined at least 10 individual estimates. The results from their model indicated that statistically significant results favouring treatment were more likely to be included in meta-analyses of efficacy than non-significant results. In contrast, results showing no evidence of adverse effect had greater probability of inclusion in a meta-analysis of safety than statistically significant results of adverse effects. Reporting bias therefore, may lead to the erroneous conclusion that an intervention is safe or relatively free from adverse effects.

19.5 Synthesis and interpretation of evidence

19.5.1 Estimating intervention effects from adverse effects

Review authors can have greater confidence in their interpretation of adverse effects data when outcomes are defined, monitored and reported as pre-specified outcomes in the research studies. In contrast, where the adverse effects are unexpected or ascertained ad hoc through spontaneous reporting, review authors will have to make more cautious interpretations regarding perceived safety or lack of harm, unless there is evidence that monitoring and reporting were sufficiently robust to have accurately captured any events of concern (Loke and Mattishent 2015).

It is important to evaluate the consistency and similarity of case definitions and methods of ascertainment for harms outcomes from the various included studies before comparing or synthesizing adverse effects data across studies. An important source of potential heterogeneity in effect estimates for adverse effects is variation in outcome definition and measurement. Review authors should ask study authors to resolve any ambiguity by providing additional data, or disaggregated data, which can be reanalysed more consistently.

Important analytical challenges relating to imprecision of estimates and rare events are covered in Chapter 10 (Section 10.4.4); see also Section 19.5.2 for particular challenges of determining whether there were zero adverse events.

Grouping adverse effects together in a composite measure (e.g. total number of adverse effects) can only give a broad impression, and may lead to genuine differences between the interventions in individual adverse effects being obscured. Owing to differences in coding and categorization of adverse effects between studies, review authors should avoid trying to increase numbers of events available for analysis by constructing composite categories that have not been reported in the primary studies. Conversely, review authors should be alert to situations in which the coding of adverse effects splits data unnecessarily (e.g. pain in leg, pain in arm), which may dilute the signal of a more global effect (e.g. all patients affected by pain).

Review authors should include at least one adverse effect outcome in the ‘Summary of findings’ table. If the review did not focus on detailed evaluation of any adverse effects, then the review authors should make an explicit statement that harms were not assessed, rather than say (or imply) the intervention appears to be safe.

19.5.2 Synthesizing and interpreting ‘zero events’

It can be difficult, or unwise, to determine that there were no adverse events of a specific type. Although trial reports may provide tables detailing withdrawals (and reasons) or serious adverse effects, they will not necessarily include all events of interest to the review authors. New or unexpected adverse events may have been missed if ascertainment relied solely on spontaneous reporting. Furthermore, trials may report statements such as “no serious harms were found” without specifying their definition of serious harms, or that “there was no evidence of significant adverse effects”, without giving the numbers of events on which such a conclusion is based.

If a serious adverse event of interest, such as heart failure, was not explicitly mentioned in the text or the serious adverse effects tables, the question then arises as to whether it is reasonable to interpret this as zero heart failure events. We generally recommend against extracting data as ‘zero’ unless it is clearly listed as such in the study report. Even where heart failure is explicitly reported as ‘zero’, we suggest that review authors carefully check the methods section of the included study for details on the rigour of monitoring for the adverse outcome (e.g. specific active surveillance for heart failure, versus reliance only on spontaneous reports that are prone to under-reporting). Ambiguity frequently crops up in the extraction and interpretation of absence of harms, so review authors should record how they reached a decision of ‘zero events’.

19.6 Chapter information

Authors: Guy Peryer, Su Golder, Daniela Junqueira, Sunita Vohra, Yoon Kong Loke; on behalf of the Cochrane Adverse Effects Methods Group

Acknowledgements: We thank Julian Higgins, Jamie Kirkham and Barbara Jennings.

Funding: This work was supported by Cochrane Methods Innovation Fund.

19.7 References

Chou R, Fu R, Carson S, Saha S, Helfand M. Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions. Journal of Clinical Epidemiology 2007; 60: 18-28.

Chou R, Aronson N, Atkins D, Ismaila AS, Santaguida P, Smith DH, Whitlock E, Wilt TJ, Moher D. AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. Journal of Clinical Epidemiology 2010; 63: 502-512.

Golder S, Loke YK. The contribution of different information sources for adverse effects data. International Journal of Technology Assessment in Health Care 2012; 28: 133-137.

Golder S, Loke YK, Zorzela L. Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011. Journal of Clinical Epidemiology 2013; 66: 253-260.

Golder S, Loke YK, Wright K, Norman G. Reporting of adverse events in published and unpublished studies of health care interventions: a systematic review. PLoS Medicine 2016; 13: e1002127.

Kicinski M, Springate DA, Kontopantelis E. Publication bias in meta-analyses from the Cochrane Database of Systematic Reviews. Statistics in Medicine 2015; 34: 2781-2793.

Loke YK, Mattishent K. If nothing happens, is everything all right? Distinguishing genuine reassurance from a false sense of security. CMAJ: Canadian Medical Association Journal 2015; 187: 15-16.

Saini P, Loke YK, Gamble C, Altman DG, Williamson PR, Kirkham JJ. Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ 2014; 349: g6501.

Schroll JB, Penninga EI, Gøtzsche PC. Assessment of Adverse Events in Protocols, Clinical Study Reports, and Published Papers of Trials of Orlistat: A Document Analysis. PLoS Medicine 2016; 13: e1002101.

Smith PG, Morrow RH, Ross DA. Outcome measures and case definition in: Field Trials of Health Interventions: A Toolbox. Smith PG, Morrow RH, Ross DA, editors. Oxford (UK): Oxford University Press; 2015.

Tang E, Ravaud P, Riveros C, Perrodeau E, Dechartres A. Comparison of serious adverse events posted at ClinicalTrials.gov and published in corresponding journal articles. BMC Medicine 2015; 13: 189.

Viswanathan M, Berkman ND. Development of the RTI item bank on risk of bias and precision of observational studies. Journal of Clinical Epidemiology 2012; 65: 163-178.

Zorzela L, Loke YK, Ioannidis JP, Golder S, Santaguida P, Altman DG, Moher D, Vohra S, PRISMA Harms Group. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ 2016; 352: i157.