Patricia Aluko (corresponding author), Erin Graybill, Dawn Craig, Catherine Henderson, Michael Drummond, Edward CF Wilson, Shannon Robalino, Luke Vale; on behalf of the Campbell and Cochrane Economics Methods Group

Cite this chapter as: Aluko P, Graybill E, Craig D, Henderson C, Drummond M, Wilson ECF, Robalino S, Vale L; on behalf of the Campbell and Cochrane Economics Methods Group. Chapter 20: Economic evidence. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook.

20.1 Introduction

Economics is the study of the optimal allocation of limited resources for the production of benefit to society. Resources include human time and skills, equipment, buildings, energy and any other inputs used to achieve a specified course of action. These courses of action might relate, for example, to a clinical decision to refer a patient for a healthcare intervention (including management of complications and follow-up care), or a policy decision to implement a public health intervention.

In the face of limited resource availability, decision makers often need to consider not only the beneficial and adverse health effects of interventions, but the impacts on the use of healthcare resources, costs associated with use of those resources, and ultimately their value – decision makers also need information on efficiency. The need for evidence on both effectiveness and efficiency are closely aligned in healthcare decision making. For these reasons, incorporating economic perspectives and evidence into Cochrane Reviews – alongside (and informed by) the evidence for beneficial and adverse effects – can make the findings of the review more useful for decision making (MacLehose et al 2012, Niessen et al 2012).

The focus of this this chapter is on methods to incorporate a health economics perspective into a Cochrane Review. Decisions about whether to include an economic perspective in a Cochrane Review should be included in the planning stage. Further support with this stage is available from the Economics Methods Group and can be found in other chapters of this Handbook.

A number of economics terms are used in this chapter but it is not expected that the reader will be familiar with economics terminology. Where a brief definition is possible it is provided but where a fuller definition is needed please see the glossary and supplementary material, available on the Campbell and Cochrane Economics Methods Group website.

20.1.1 Economic perspectives and economic evidence

Incorporating an economic perspective into a Cochrane Review involves the relatively straightforward task of placing an ‘economics lens’ on the health condition (population), intervention(s) and effectiveness question(s) under investigation, in order to highlight economic issues of potential importance to end-users such as the importance of a particular research question or the burden of a health condition on a society or specific group. An economic perspective might provide information about whether a more costly intervention is worth any additional benefits and whether the information could change a policy decision. In comparison, incorporating economic evidence into a Cochrane Review requires the application of specialized methods and procedures to include estimates of the cost or other economic effects of the interventions in the review.

In this chapter we restrict the term economic evidence to information on resource use, or costs or cost-effectiveness data taken from studies that draw comparisons for patient populations that match those of the Cochrane Review. The type of studies that we are interested in are economic evaluations. These are full economic evaluations that compare the costs and effects of two or more interventions. Partial economic evaluations are also possible and these compare only costs or effects but not both. Relevant partial economic evaluations that compare only effects would already be included in the review (under this definition a trial comparing the effects and harms of an intervention is a form of a partial economic evaluation). Partial economic evaluations that consider costs only are called cost-analyses. It is not currently recommended to include these and methodological research is needed to assess the value of including them. Further information describing how full and partial economic evaluations are defined is provided in the glossary and supplementary material, which are available on the Campbell and Cochrane Economics Methods Group website.

Two optional methodological frameworks have therefore been developed for incorporating economic evidence into reviews. The methodological and practical implications of each approach should be considered carefully at an early stage of planning the protocol for a systematic review. The two methodological frameworks are:

1. integrated full systematic review of economic evidence; and
2. brief economic commentary.

The integrated full systematic review of economic evidence is covered only briefly in this chapter. A detailed definition and description can be found on the Campbell and Cochrane Economics Methods Group website. This approach is substantially more resource intensive when implemented in full than the brief economic commentary. This is because it requires additional ‘economic’ methods procedures to be integrated into each stage of the main systematic review of intervention effects. Conducting an integrated full systematic review of economic evidence will also require specialist input to the author team from a health economist, with experience (or support from someone with experience) of applying the framework, at all stages of the process.

The brief economic commentary framework is less intensive but also less rigorous, and most of this chapter focuses on this approach. This framework is specifically designed to support the inclusion of economic evidence in Cochrane Reviews without requiring specialist input from health economists (beyond initial guidance and training in the method and procedures), and without placing a major additional workload burden on author teams or editorial bases. This framework can be viewed as a ‘minimal framework’ for incorporating economic evidence, with inherent limitations that will require appropriate caveats in the commentary.

20.1.2 Core principles for the methods for the review of economic evidence

Three core principles underpin both frameworks.

(1) Economics evidence should not be presented alone

Full reviews or brief economic commentaries developed with the aim of summarizing evidence on the costs and/or cost-effectiveness of interventions should not in general be conducted as a standalone exercise. They must place the relevant economic evidence (in this case the impacts on resource use, costs and/or cost-effectiveness) into the context of reliable evidence for intervention effects on health and related outcomes. Failure to do so can lead to a biased summary of the evidence and a distorted assembly of data from primary studies, because data on the evidence of effects used in identified economic evaluations are highly likely to be (at best) only a subset of the data used to provide the summary of evidence of effects (including assessment of the quality of that evidence). The evidence of effects produced by a Cochrane Review will be the most up-to-date synthesis and any published economic evaluation can, at best, be based on only a subset of the data that were available at some earlier time point.

Furthermore, economic evaluations may be susceptible to a specific source of publication bias (or indeed conduct bias). For example, audits of some clinical areas have shown that clinical effect sizes in randomized trials published with a concurrent economic evaluation are systematically larger than those in randomized trials without. This may reflect the difficulty in publishing planned economic evaluations conducted alongside ‘inconclusive’ trials. Also, decisions made whilst planning a trial may mean that an economic evaluation is excluded (e.g. because it is felt implausible that an effective intervention could be anything other than cost-saving). However, such reasoning may not be reflected in published trial protocols or final study reports. Both of these issues compound the issue of reporting biases in randomized trials (see Chapter 13).

(2) Consider contributors to economic outcomes rather than specific resources or settings

Given the international audience of end-users of Cochrane Reviews, any assumptions in the review about the setting for decision making (such as the availability of resources or the structure of the health system), and any specific resource estimates may not be appropriate. The primary aim of economics components of reviews should be to explain how interventions affected incremental resource use, costs, health outcomes and cost-effectiveness when implemented at specific times in specific settings (i.e. a focus on ‘what happens?’ (Petticrew 2015)) and what drives variation in estimates of economic and health outcomes between studies and settings. This will help end-users understand key economic trade-offs between alternatives that could be used in practice in their own setting.

(3) Consider how economics evidence may inform future research

A key secondary aim of economics components of reviews should be to present health and economic outcome data outputs from Cochrane Reviews in formats that facilitate the reuse of these data as inputs to the subsequent, or parallel, development of new model-based economic evaluations.

20.1.3 Criteria for prioritizing inclusion of economic evidence in a Cochrane Review

20.1.3.1 Rationale and principles

Whilst all reviews could have an economic component, an economic component might not always be necessary. In general, it is more likely to be important to incorporate economic evidence into a review when important differences are expected between the intervention(s) and comparator(s) being compared in terms of their impacts on resource use and associated costs. In addition, pragmatic factors, such as the availability of specialist expertise and research resources available, may also impact on the final decision.

Some commissioners of systematic reviews have found it useful to develop decision algorithms, such as the one shown in Table 20.1.a, to help prioritize systematic reviews of the effects of health interventions for inclusion of economic evidence (Frick et al 2012).

Table 20.1.a provides three criteria to help prioritize reviews for inclusion of economic evidence:

1. the expected incremental effect of an intervention (i.e. how large is the difference in effect between intervention options likely to be? The smallest meaningful effect might correspond to the minimally important difference, or the difference in effect likely to be meaningful to patients);
2. the expected incremental cost of the intervention (i.e. what are the key elements of resource use likely to be affected, and how large is the difference likely to be in cost between intervention options? How important might this difference be to decision makers?); and
3. the likelihood that economic evidence could change potential decisions about use of an intervention (this may take into consideration other contextual factors, such as prevalence of a condition or health system factors).

20.1.3.2 Making judgements about the criteria

Each of these criteria is dichotomized for simplicity: large or small incremental effect, high or low incremental cost, and a high or low probability that economic evidence will affect potential decisions concerning the adoption of the intervention.

It can be challenging to judge the likely size of incremental effects and costs in these broad, dichotomized terms, in advance of conducting the research. However, this is an essential first step in planning any study of intervention effects or economic evaluation, just as it is in planning systematic reviews of such studies. In practice, it may be easier to apply this algorithm when planning an update of an already published Cochrane Review. This is because the results of the current, published version may indicate potential sources of important differences in resource use and costs between the intervention(s) and comparator(s). For example, a summary effect size that shows an increased/decreased risk of a revisional procedure being required following a surgical intervention implies a difference in resource use and costs associated with performing additional/fewer revisional procedures (including those associated with management of any complications and follow-up care).

Prior to conducting the review the expected probability that economic evidence could change adoption decisions is largely a subjective judgement. This judgement is again challenging to make given the intended international audience of end-users of Cochrane Reviews. Authors are therefore encouraged to consult a health economist who can provide specialist advice to about what factors would be worth considering when making a judgement.

20.1.3.3 Using the criteria for prioritizing inclusion of economic evidence in a Cochrane Review

There are two rows in Table 20.1.a for which the decision to de-prioritize or prioritize incorporation of economic evidence is relatively clear. The first scenario is characterized by a large incremental beneficial effect, a low incremental cost, and a low probability of the economic evidence changing the decision. In this scenario, a very low priority is placed on the incorporation of economic evidence into review. This is because with a large beneficial effect on health (which is likely to translate into lower subsequent use of health services and lower associated healthcare costs) and small input costs, the intervention is likely to be cost-effective (possibly cost-saving) overall. It would, however, be important to state this reasoning in the Background section of a protocol and review.

Conversely, if the expected incremental beneficial effect is small, the expected incremental costs are high, and the economic evidence has a high probability of changing the decision, then this algorithm places a high priority on the incorporation of economic evidence.

The other rows of Table 20.1.a represent six further scenarios that fall between these two extremes. For example, the second row represents a scenario in which the incremental beneficial effect is small, the incremental cost is low, and the economic evidence has a high probability of changing the decision. This scenario may occur when, for example, the expected cost impact of the intervention is small but the health condition targeted by the intervention has a very high prevalence, such that the cumulative impact of small changes in costs across a large number of treated patients adds up to a large overall change in costs at the level of a region or a country, so affordability may be very important to a decision maker.

The decision algorithm in Table 20.1.a excludes scenarios in which the intervention is expected to be associated with negative incremental cost (i.e. net savings) and a positive incremental effect relative to the comparator (and vice versa); in other words, situations in which decisions to adopt or reject are expected to be straightforward because the intervention is clearly better or clearly worse than the comparator (i.e. it dominates, or is dominated by the comparator).

It is important to understand that if the decision algorithm shown in Table 20.1.a suggests that low (or very low) priority should be placed on incorporating economic evidence, this does not necessarily imply that doing so would provide no useful information for decision makers. Rather, it implies that a low (or very low) priority might be assigned to devoting limited research time and resources to conducting the economics component of a review.

Table 20.1.a Decision algorithm to help prioritize reviews for inclusion of economic evidence (reproduced from Frick et al (2012))

20.2 Formulation of the review

20.2.1 Planning the economic component of the review

Regardless of which of the two methodological frameworks will be applied, authors of Cochrane Reviews aiming to incorporate economic evidence will need to plan the economics component from the very first stages. Further guidance and information on the planning can be accessed through the Campbell and Cochrane Economics Methods Group website.

The concise details of methods and procedures that will be used to develop the brief economic commentary should be planned at the protocol stage, and can be described in the ‘Methods’ section under a separate subheading, ‘Incorporating economic evidence’.

Once a decision to include economic evidence has been taken, it is advisable to consult with a health economist with experience of Cochrane Review methods as soon as possible.

20.2.2 Formulating the objective

The economic question can be formulated with close reference to the question(s) that frame the systematic review of intervention effects. The research questions to be addressed by Cochrane Reviews of intervention effects are conventionally formulated as objectives, for example:

To assess the effects of aspirin [intervention] versus placebo [comparator] for primary prevention of heart attacks [condition and primary health outcome] among adults aged >50 years [population].

The questions for a brief economic commentary need to be expressed in the form of an objective, usually a secondary objective for the review. However, the most important objective in this case is to summarize the availability and principal findings in terms of costs and cost-effectiveness of eligible economic evaluations.

20.2.3 Introducing the economic perspective on the decision problem in the Background section

20.2.3.1 Purpose of introducing the economic perspective in the Background section

The aim of incorporating an economic perspective into the review is to place an ‘economic lens’ on the health condition (population) being addressed and the interventions being investigated in the review. This should be discussed in the Background to the review, to highlight the relevance of economic issues and context to the questions that the review will address.

Three distinct economic issues to consider highlighting in the Background section of a review are:

1. the economic burden of the health condition (i.e. the ‘cost of illness’);
2. potential impacts of intervention(s) on resource use (costs); and
3. general issues of intervention costs and cost-effectiveness that are relevant for the readers of the review to consider.

To address the first point, the ‘Description of the condition’ section of the Background can be expanded to include a discussion of the economic burden, or cost of illness of the condition being addressed. A brief literature search will be required to identify source material for this section, and guidance for this is presented in Section 20.2.3.2. The second and third points should be reported in the Background section on ‘How the intervention might work’ and ‘Why is it important to do this review’. For the second and third points supplementary searches to identify source material are not required. Instead, the review should consider of the potential impacts of the intervention on resource use and their importance to decision making (as considered in the early planning stages and framing of the question, described in Section 20.1.3).

Depending on the scope of the cost-of-illness studies found, the commentary in the ‘Description of the condition’ section should include:

• a brief, general statement of the scale of economic burden/cost-of-illness to healthcare systems, patients and/or their families and/or society as a whole; and
• monetized estimates of the economic burden of disease to healthcare systems, patients and/or their families and/or to societies.

We further recommend that any monetized estimates presented should include details of the country, currency and price year, if reported, in which the source studies were conducted.

An example commentary of how to summarize information on the economic burden of disease is presented in Box 20.2.a using example text extracted from a published Cochrane Review of surgery for faecal incontinence in adults (Brown et al 2013). Box 20.2.b and Box 20.2.c provide example text for potential impacts of intervention(s) on resource use (costs); and cost-effectiveness, which are taken from a published Cochrane Review of bone morphogenetic protein (BMP) for fracture healing in adults (Garrison et al 2010).

Box 20.2.a Example commentary on economic burden of the health condition (cost of illness)

Box 20.2.b Example commentary on potential impacts of intervention(s) on resource use (costs)

Box 20.2.c Example commentary on the general issue of intervention costs and cost-effectiveness

20.2.3.2 Identifying cost-of-illness studies for the Background section

The target type of health economics study (source material) needed to inform this brief commentary in the ‘Description of the condition’ section of the Background is the cost-of-illness study. A cost-of-illness study is a form of economic analysis that aims to describe, measure and value the total resources used in the management of a specific health condition, or within a specific patient population (Abdelhamid and Shemilt 2010) (see also the training resources on the Campbell and Cochrane Economics Methods Group website).

The objective of this search is to locate the few most useful articles that report information on the economic burden of the condition being addressed (cost-of-illness). It is not to conduct a comprehensive search of the literature and identify all relevant studies. Rather, the focus might be searching two or more databases (see below) where it is most likely a cost-of-illness study may be found. As noted above, the most useful sources of this information are likely to be found in the one or two articles that report a recently conducted cost-of-illness study, or a recently conducted review of cost-of-illness studies, focused on international comparisons, and which includes estimates of the wider economic burden not just in terms of the costs of management but also in terms of the costs of ill-health itself to an individual and to a society. In common with other material used in the Background section, a formal assessment of the quality and risk of bias of the cost-of–illness study is not conducted. However, it is still useful to know the key features that affect the validity of cost-of-illness studies (Larg and Moss 2011).

This search should be conducted when preparing the protocol for the review or when conducting an update of the review. Targeted search strategies to identify relevant cost-of-illness studies should be based on keyword search terms designed to capture ‘Population’ concepts, adapted from those ‘Population’ keyword terms used in strategies designed to search for eligible studies of effects for the main review. This set of keyword terms should be coupled (using the ‘AND’ operator) with a filter designed to retrieve cost-of-illness studies and run in general biomedical electronic literature databases, such as MEDLINE, EMBASE, CINAHL, PsycINFO or PubMed. We recommend a search of at least MEDLINE and EMBASE, with further databases searched if deemed relevant for the specific review topic. There are no specialist tertiary health economics electronic literature databases that currently tag records of cost-of-illness studies specifically, and no search filters designed specifically for cost-of-illness studies have been evaluated and validated (Jenkins 2004). We suggest using the search filters provided here. The search filters themselves have been piloted in the development of brief economic commentaries to successfully identify relevant cost of illness studies (Box 20.2.d, Box 20.2.e and Box 20.2.f shows the filter for MEDLINE (OvidSP), EMBASE (OvidSP) and PsycINFO, respectively).

Box 20.2.d MEDLINE (OvidSP) filter for cost-of-illness studies

Box 20.2.e EMBASE (OvidSP) filter for cost-of-illness studies

Box 20.2.f PsycINFO filter for cost-of-illness studies

20.2.4 Formulating eligibility criteria

For a brief economic commentary it is not necessary to include separate eligibility criteria describing the population, intervention(s), comparator(s) and outcomes (PICO) for economics studies that will be sought to inform the review. The eligibility criteria for studies that will be used to develop the commentary are the same as those set for the main systematic review of intervention effects with respect to the PICO elements.

To reflect this it is recommended to add a section to the Methods called ‘Incorporating economic evidence’, to state this clearly. This section should then go on to state supplementary criteria with respect to the type of economic evaluation study designs. For example:

We will develop a brief economic commentary based on current methods guidelines (http://methods.cochrane.org/economics/) to summarize the availability and principal findings of [trial-based and model-based] full economic evaluations (cost-effectiveness analyses, cost-utility analyses, cost-benefit analyses)* that compare the use of aspirin versus placebo for primary prevention of heart attacks among adults aged >50 years. This commentary will focus on the extent to which principal findings of eligible economic evaluations indicate that an intervention might be judged favourably (or unfavourably) from an economic perspective, when implemented in different settings.

* a definition of these terms can be found in the Glossary and a fuller explanation is provided in the supplementary material on the Campbell and Cochrane Economics Methods Group website.

20.3 Identification of evidence

Alongside the main search for studies for inclusion in the review, a separate search strategy should be planned (at the protocol stage for a new review or when planning an update of an existing review) and conducted during the review stage for eligible health economic evaluations to inform development of a brief economic commentary. The following elements are recommended for this supplementary search:

1. checking reference lists and conduct forward citation tracking from eligible studies of effects identified for inclusion in the main review;
2. conducting a search of NHS Economic Evaluation Database (NHS EED) using keyword terms based on intervention (and possibly comparator) concepts; and
3. applying specialist search filters to sets of records retrieved by searches of one or two selected general electronic biomedical literature databases searched for the main review of intervention effects. Examples of relevant search filters can be obtained from the Economics Methods Group.

The primary rationale for incorporating using specialist search filters is the need to identify reports of eligible full economic evaluations published since NHS EED stopped being updated at the end of 2014. If a brief economic commentary is restricted to full economic evaluations only, then we recommend using specialist searches from 1 January 2014 as the NHS EED was based on rigorous and comprehensive searches for full economic evaluations before that date.

20.3.1 Selecting studies and collecting data

For a brief economic commentary, procedures for selecting eligible full economic evaluations for inclusion are less onerous than required for an integrated full review. This reflects both the intention to minimize the workload for author teams and caveats for the discussion of the findings of identified economic evaluations (see Section 20.5.1).

Identified economic evaluations will still need to be screened against eligibility criteria relating to study population, intervention and comparator already defined for the main systematic review of intervention effects. It is recommended that this task needs to be undertaken by one review author only. One author will also need to classify each economic evaluation using the general procedure described below (including establishing any links with eligible trials included in the main review of intervention effects).

Collecting data for a brief economic commentary requires the extraction of two types of data: basic details of the characteristics of each identified economic evaluation; and brief text extracts that summarize their principal findings.

Basic data collected on the characteristics of each economic evaluation should include:

• the analytic framework (trial- or model-based) and type (cost analysis, cost-effectiveness analysis, cost-utility analysis, cost-benefit analysis) of economic evaluation (to be summarized as a count of each type identified as part of the commentary (see also Section 20.5.1);
• the analytic perspective (whose costs and benefits a decision maker views as important) and time horizon (the duration over which costs and effects are assessed) adopted for costs and (if applicable) effects in each analysis;
• the main cost items included in each analysis (e.g. costs that fall under the following categories of health sector costs, other sector costs, patient and family costs and productivity impacts hospital care costs, direct health care costs; indirect non-health care costs); and
• the setting (i.e. country in which the study was performed), currency and price year used in each analysis.

It is helpful to classify cost items into four categories: health sector costs, other sector costs, patient and family costs, and productivity impacts (Drummond et al 2015) (although not all economic evaluations will follow this structure). The categories included will be driven primarily by the analytic perspective of the study. Health sector costs include the cost to the system or insurers of care provided (excluding costs directly paid by patients) and can include items such as primary care physician contacts (e.g. face-to-face visits or formal contacts via phone or via the internet, etc), prescribed medications, inpatient and outpatient hospital contacts, as well as any specialist tertiary care contacts. Other sector costs include costs borne by social services, education, local authorities, or police and criminal justice services. Patient and family costs could include any direct payment or co-payments for medications or care, or out of pocket expenses such as travel or arranging child or adult care while attending appointments. Productivity losses are the loss of output to the economy, and are usually measured in terms of time off work due to accessing care as well as morbidity or premature mortality.

For principal findings, the following data should be collected:

• verbatim text on conclusions drawn by the authors of each economic evaluation (with respect to what the study authors report as their main (base case) analysis; and
• text that summarizes uncertainty surrounding authors’ principal conclusions (i.e. based on the results of any sensitivity analyses conducted).

For example, the following verbatim text was extracted from a report of a model-based cost-utility analysis that compared two interventions for preventing heart attacks and death in patients with non-ST-elevation myocardial infarction. This extract was used in the development of an exemplar brief economic commentary based on a Cochrane Review of factor Xa inhibitors for acute coronary syndromes (ACS) as part of a pilot study (Shemilt et al 2011):

Our results suggest that the use of fondaparinux together with triple antiplatelet therapy in NSTE-ACS patients submitted to early (non-urgent) invasive therapy is cost saving. The strategy of fondaparinux was found to be dominant in almost all the scenarios considered, and the highest cost-effectiveness of fondaparinux was found in younger patients, patients at high risk of a cardiac event (high TIMI score) and patients at the highest risk of bleeding. (Latour-Perez and de Miguel Balsa 2009)

20.4 Appraisal of evidence

A brief economic commentary need not include (or report) assessments of methodological quality of included economic evaluations. This guidance reflects both the intention to minimize the additional workload burden placed on author teams and the limiting caveats that will be placed on discussion of the principal findings of identified economic evaluations in the review (see text at the end of Section 20.3.1). However, it is mandatory for this limitation to be explicitly described in the text of a brief economic commentary, for example:

It is important to highlight that we did not subject any of the [N] identified economic evaluations to critical appraisal and we do not attempt to draw any firm or general conclusions regarding the relative costs or efficiency of [‘Intervention X’] compared with [‘Comparator Y’].

20.5 Synthesis and interpretation of evidence

20.5.1 Analysing and presenting results

An exemplar brief economic commentary is shown in Box 20.5.a (Shemilt et al 2011) and further examples can be found in supplementary material and training materials on the Campbell and Cochrane Economics Methods Group website.

The findings of the brief economic commentary should be incorporated into the Discussion (and not the Results) section of a Cochrane Review. The most appropriate place for this material is where the results of the systematic review of effects are put into context of other information and other reviews.

The overall aim of this element of the commentary is to summarize the availability and principal findings of identified eligible economic evaluations, with appropriate caveats, rather than to present the detailed results of a systematic search for evidence.

This commentary should include a brief narrative summary of:

• the electronic health economics literature databases searched;
• the number of relevant economic evaluations identified for each eligible comparison (each eligible intervention/comparison combination);
• the descriptive information collected from each study;
• principal conclusions as reported by the authors of each analysis (with respect to the base case analysis); and
• principal sources of uncertainty regarding authors’ principal conclusions (based on the results of any sensitivity analyses conducted).

In a Cochrane Review, all published reports of economic analyses and/or economic evaluations used to inform the brief economic commentary should be cited as ‘Additional references’, not as ‘Included studies’, unless they are also eligible and included as part of the main review of effects.

Box 20.5.a Example brief economic commentary

20.5.2 Interpreting results and drawing conclusions

Discussion points in a brief economic commentary can be concise and over-interpretation of the results of this relatively modest exercise must be avoided. Interpretation and discussion points should focus on the extent to which it is judged clear, based on consistency in principal findings between identified economic evaluations, that the intervention(s) could be considered promising from an economic perspective (with appropriate caveats). In the example brief economic commentary shown in Box 20.5.a, the discussion points gave a qualified statement that one intervention (fondaparinux) appeared to be cost-saving while not inferior in terms of effects compared to other interventions measured. In this specific example, the basis for this qualified inference was evidence for consistent results favouring use of fondaparinux among full economic evaluations identified for inclusion in the brief economic commentary.

Example standard forms of words for potential use in different scenarios, depending on the profile of included economic evaluations, are shown in Box 20.5.b. “End users of this review will need to assess the extent to which methods and results of identified economic evaluations may be applicable (or transferable) to their own setting” is a recommended standard caveat for all brief economic commentaries.

Box 20.5.b Example forms of words for concise discussion points in a brief economic commentary

20.6 Chapter information

Authors: Patricia Aluko, Erin Graybill, Dawn Craig, Catherine Henderson, Michael Drummond, Edward CF Wilson, Shannon Robalino, Luke Vale; on behalf of the Campbell and Cochrane Economics Methods Group

Acknowledgements: The authors wish to thank Economics Methods Group Administrator, Jan Legge, for support whilst preparing this chapter, attendees at health economics group workshops that have provided valuable comments on the approaches to incorporate economics into Cochrane Reviews. We would also like to thank the peer reviewers of this chapter and the editorial team of the Handbook for comments and advice.

20.7 References

Abdelhamid A, Shemilt I. Glossary of terms. In: Shemilt I MM, Vale L, Marsh K, Donaldson C editor. Evidence-based decisions and economics: health care, social welfare, education and criminal justice. Oxford: Wiley-Blackwell; 2010.

Brown SR, Wadhawan H, Nelson RL. Surgery for faecal incontinence in adults. Cochrane Database of Systematic Reviews 2013; 7: CD001757.

Drummond M, Sculpher M, Claxton K, Stoddart G, Torrance G. Methods for Economic Evaluation of Health Care Programmes. Fourth ed. USA: Oxford University Press; 2015.

Frick K, Neissen L, Bridges J, Walker D, Wilson R, Bass E. Usefulness of Economic Evaluation Data in Systematic Reviews of Evidence. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012. 12(13)-EHC114-EF https://www.ncbi.nlm.nih.gov/books/NBK114533/.

Garrison KR, Shemilt I, Donell S, Ryder JJ, Mugford M, Harvey I, Song F, Alt V. Bone morphogenetic protein (BMP) for fracture healing in adults. Cochrane Database of Systematic Reviews 2010; 6: CD006950.

Gilbody S, Bower P, Sutton AJ. Randomized trials with concurrent economic evaluations reported unrepresentatively large clinical effect sizes. Journal of Clinical Epidemiology 2007; 60: 781-e710.

Jenkins M. Evaluation of methodological search filters--a review. Health Information and Libraries Journal 2004; 21: 148-163.

Larg A, Moss JR. Cost-of-illness studies: a guide to critical evaluation. Pharmacoeconomics 2011; 29: 653-671.

Latour-Perez J, de Miguel Balsa E. Cost Effectiveness of Fondaparinux in Non-ST-Elevation Acute Coronary Syndrome Pharmacoeconomics 2009; 27: 585-595.

MacLehose H, Hilton J, Tovey D. The Cochrane Library: Revolution or evolution? Shaping the future of Cochrane content (background paper). The Cochrane Collaboration’s Strategic Session; 2012; Paris, France.

Maxwell CB, Holdford DA, Crouch MA, Patel DA. Cost-effectiveness analysis of anticoagulation strategies in non-ST-elevation acute coronary syndromes. Annals of Pharmacotherapy 2009; 43: 586-595.

Mellgren A, Jensen LL, Zetterstrom JP, Wong WD, Hofmeister JH, Lowry AC. Long-term cost of fecal incontinence secondary to obstetric injuries. Diseases of the Colon and Rectum 1999; 42: 857-865; discussion 865-857.

Niessen L, Bridges J, Lau B, Wilson R, Sharma R, Walker D, Frick K, Bass E. Assessing the Impact of Economic Evidence on Policymakers in Health Care - A Systematic Review. Agency for Healthcare Research and Quality (US); 2012. Contract No.: No. 12(13)-EHC133-EF https://effectivehealthcare.ahrq.gov/topics/economic-evidence/research.

Petticrew M. Time to rethink the systematic review catechism? Moving from ‘what works’ to ‘what happens’. Systematic Reviews 2015; 4.

Sculpher MJ, Lozano-Ortega G, Sambrook J, Palmer S, Ormanidhi O, Bakhai A, Flather M, Steg PG, Mehta SR, Weintraub W. Fondaparinux versus Enoxaparin in non-ST-elevation acute coronary syndromes: short-term cost and long-term cost-effectiveness using data from the Fifth Organization to Assess Strategies in Acute Ischemic Syndromes Investigators (OASIS-5) trial. American Heart Journal 2009; 157: 845-852.

Shemilt I, Mugford M, Vale L, Craig D, on behalf of the Campbell and Cochrane Economics Methods Group. Searching NHS EED and HEED to inform development of economic commentary for Cochrane intervention reviews. 2011. http://methods.cochrane.org/economics/sites/methods.cochrane.org.economics/files/public/uploads/brief_economic_commentaries_study_report.pdf.

Yusuf S, Mehta SR, Chrolavicius S, Afzal R, Pogue J, Granger CB, Budaj A, Peters RJ, Bassand JP, Wallentin L, Joyner C, Fox KA. Comparison of fondaparinux and enoxaparin in acute coronary syndromes. New England Journal of Medicine 2006; 354: 1464-1476.