Finding out whether Cochrane is making a difference

Finding out whether Cochrane is making a difference
Finding out whether Cochrane is making a difference


Simple evaluation tools

Cochrane teams are passionate about producing, sharing and encouraging people to use Cochrane evidence – but are we succeeding? Are people using Cochrane evidence when they make decisions? This page has tools to help you evaluate how you spread, share and advocate for evidence.

  • The tips and tools on this page are designed for people without experience of evaluation but they may also be useful for those who already know about evaluation.
  • Pick and choose which tools are most relevant to you. You don’t need to use them all and you can use the tools in any order.
  • Ideally start thinking about how you will know whether you are successful when you begin planning your activity. However, if you have already started your activity - don't worry! It's never too late to evaluate. 
  • If you need help applying the tools or have suggestions for improvements, please contact the KT department

You can scroll down the page to browse through all the tools, or use the links below to jump to what you’re most interested in: 

What are the main steps in planning an evaluation?  

The diagram below shows the 5 steps for planning an evaluation. Each of the steps is described in more detail further down this page.
 

Other resources for this section:

Back to top

Step 1: Who to involve in planning your evaluation

First think about why you are evaluating your work.

Who will use your evaluation, what do they want to know and how can you involve them in what you do?

You can use a Stakeholder Map to help identify the most important people to involve in planning your evaluation.

Maybe your Cochrane Group wants to find out what people think of Cochrane evidence so you can keep improving. Perhaps a funder wants to know whether to keep investing in Cochrane. Or maybe a consumer group wants to know who uses Cochrane evidence so they can decide whether to get involved.

Involving the right people in planning your evaluation is important so:-

  • everyone agrees about what your knowledge translation activities are trying to achieve
  • you collect evaluation information that people want and will use
  • you get fresh ideas about what to evaluate and how to do it
  • you get the help or resources you need

Back to top

Step 2: Be clear about what you want to achieve

The process of thinking through what you are going to do and involving others is what’s important, not the document you create. Don't spend a lot of time making a 'perfect' plan.

A good place to start is an Outcomes Map. This is a simple tool to visualize what you want to achieve (outcomes) and how your activities will lead to the outcomes you want.

Sometimes people jump straight into planning an activity or an evaluation without being clear about what they’re trying to achieve. If you spend a bit of time mapping what you’re doing and why first, it will be easier to decide what to measure and how.

Here you can see:-

  • what an Outcomes Map is
     
  • a template to create your own Outcomes Map 
     
  • a 10 minute video describing some of the principles of an Outcomes Map,
    which is sometimes called a theory of change 

 

Back to top

Step 3: What could you measure?

This section provides examples of WHAT to evaluate.

Your Outcomes Map will help you think about the difference between:-

  • Outputs: things we create or do (like a translated blog, training workshop or meeting with policy makers)
  • Outcomes: what we want to achieve (like more people knowing about or using Cochrane evidence)

Sometimes we focus a lot on keeping track of our outputs, like how many Plain Language Summaries we’re creating, or how many people attend our training. To understand whether we’re really making a difference, we need to think about measuring some of our outcomes too.

You might want to achieve outcomes like these:-

  • increasing people’s awareness of Cochrane evidence
  • making Cochrane evidence easy to understand
  • increasing people’s skills to use our evidence
  • helping people use evidence to inform their decisions

Here you can see:-

Back to top

Step 4: What evaluation methods can you use?

This section provides examples of HOW to evaluate.

Evaluation does not have to be a large time-consuming research project where you need special skills. Evaluation can be small and simple.

Here are some ideas about ways to collect evaluation information:-

  • A good start is keeping a record of all your knowledge translation activities. Many Cochrane Groups use a spreadsheet or table to keep records of what they’re doing. Here is a simple example of monitoring template with a 7 minute video about how to adapt something like this for your work. 

  • There are many ways to collect information. Here are some tips to help you choose data collection methods.

  • If you use social media to disseminate your evidence, you can use statistics from the social media platforms. You can watch this 20 minute video about how to use Twitter analytics. 
     
  • Questionnaires are a common way to collect information. Here are some tips to help you design better questionnaires. 
     
  • Listen to hear the experience of using surveys for evaluation. 
     
  • You might be able to adapt these examples of survey questions that other Cochrane Groups have used. 
     
  • There are many other resources online about collecting information. For example, you can find information here about conducting interviews (Part 1 and Part 2). 

Back to top

Step 5: Putting it all together in an evaluation plan

The other sections of this webpage have shown you how to:-

  • clarify what you are doing and what you want to achieve (Outcomes Map)
  • identify who to work with to plan your evaluation (Stakeholder Map)
  • consider what to measure (indicators)
  • decide how to collect evaluation data (methods) 

An Evaluation Plan brings all of these elements together in one table or document. It provides more detail about what you will do to evaluate your work, when, how and who will do it.

Your evaluation plan does not need to be long or complex. It can be a simple table listing what to measure and when.

Here you can see:-

  • a 5 minute video introduction to developing an Evaluation Plan

  • a simple template to create an Evaluation Plan table or short document
     
  •  an example of an evaluation plan table and an example of a more detailed evaluation plan from the Cochrane Community
     
  • a basic template for planning your KT activities which prompts you about evaluation
     
  • a webinar about how to develop and evaluate a KT Plan 

Other organizations also have useful tips about how to plan your evaluation. Some you might find useful are:

Back to top

Examples: learn from what others are doing

Lots of Cochrane Groups are evaluating the difference they’re making or how to keep improving their work.

Here are some examples:-

There are also examples of the difference Cochrane evidence can make here.

You can sign up to the free Comms Digest to receive up to date examples of how Cochrane Groups are sharing and evaluating their work. 


Back to top

Get in touch

We’ll add more stories and tools to this webpage over time.

Get in touch to share your story about evaluating your KT work, to help create more resources or to ask questions:

     Karen Head
     Knowledge Translation Project Manager
     Cochrane Central Executive
     khead@cochrane.org

These resources were produced by the KT Evaluation Working Group in 2020:-

  • Alex Mignone, Cochrane Nursing, Australia
  • Andrea Cervera-Alepuz, Cochrane Iberoamerica, Spain
  • Bey-Marrie Schmidt, Cochrane South Africa
  • Celeste Naude, Cochrane Nutrition, South Africa
  • Claudia Bolling, Cochrane Germany
  • Debra de Silva, The Evidence Centre, UK
  • Elena Kostova, Cochrane Gynaecology and Fertility, The Netherlands
  • Eugenie Johnson, Cochrane Incontinence, UK
  • Gonzalo Casino, Cochrane Iberoamerica, Spain
  • Jo Anthony, Knowledge Translation Department, UK
  • Juliane Ried, Governance and Strategy Unit, Germany
  • Karen Head, Knowledge Translation Department, Switzerland
  • Lauren Gerlach, Cochrane US Network
  • Lisa Pfadenhauer, Germany
  • Richard Morley, Cochrane People Services, UK
  • Sarah Elliott, Cochrane Child Health, Canada
  • Tamara Kredo, Cochrane South Africa
  • Victoria Leo Rosas, Cochrane Iberoamerica, Spain

Back to top