jump to content

Preparing a Research or Evaluation Plan

Preparing a research or evaluation plan

If you are collecting your own evidence, this page contains the resources that will help you build a robust research or evaluation plan. These resources will also help you to carry out the research or evaluation effectively. There are multiple decisions involved in creating an evaluation or research plan. This page provides resources to support each of the following decision points:

  1. Introduction to research and evaluation planning
  2. Developing a good question
  3. Creating a logic model to decide what evidence to collect
  4. Deciding what needs to be in the research or evaluation plan
  5. Evidence collection
  6. Good practice in conducting research and evaluations

There is also a section on an alternative approach to research and evaluation.

1: Introduction to research and evaluation planning

The following resources give a general overview of key considerations and steps to follow in planning an evaluation. The points made also apply to research planning. These resources are a good place to start before going on to detailed research or evaluation planning.

 

Service evaluation: an introductory guide Age UK

This guide provides a straightforward introduction to planning an evaluation. It defines two types of evaluation (process and impact evaluations) and provides 7 questions to consider in planning and designing an evaluation. It has a useful glossary and link to further information.

 

Manage an evaluation or evaluation planning Better Evaluation

This publication covers engaging stakeholders, overseeing evaluations, decision-making, determining what resources are needed, document management planning and development of an evaluation plan. There is a small section on how to develop evaluation capacity.

 

Guidelines for good practice in evaluation UK Evaluation Society

This document provides brief guidance written by the UK Evaluation Society on different aspects of evaluation. These guidelines have been written “…for use by the novice and the experienced alike” and set out principles to inform evaluation practice, practical guidance for evaluation practice and ethical considerations.

 

Introductory guide to evaluation Data Cymru

This is an accessible and comprehensive web guide to planning and carrying out evaluations. It clearly sets out the evaluation process, describes types of evaluation, evaluation frameworks and the role of logic models and theories of change. It has a useful glossary. There is a direct link to the HM Treasury Magenta Book, which is a key resource for planning evaluations. This guide is available in Welsh.

2: Developing a good question

 

Choosing the right questions: tips for conducting program evaluation Wilder Research

This four-page resource provides information on how to determine the key evaluation questions to explore. It covers outcome, process and satisfaction evaluations and provides practical guidance.

 

Evaluation questions checklist for program evaluation Western Michigan University

This resource provides an accessible and practical aid to developing relevant evaluation questions. It covers how to determine evaluation questions that are ‘pertinent’, ‘reasonable’, ‘specific’ and ‘answerable’. It also includes a list of other useful resources.

 

Good evaluation questions: a checklist to help focus your evaluation Centre for Disease Control and Prevention

“The most serious mistakes are not being made as a result of wrong answers. The truly dangerous thing is asking the wrong question.” Starting with this quote from Peter Drucker, this two-page resource has been developed from practice wisdom and knowledge of the evaluation literature. It includes a concise checklist to assist in the development of a suitable evaluation question. Checklist items cover aspects including stakeholder engagement and the fit, relevance and feasibility of the evaluation.

 

Measuring what matters Co-Production Network for Wales

This publication emerged from a series of seminars in Wales called ‘Tents around the Campfire’ which explored different perspectives on evaluation. This publication is a comprehensive tool to assist you to get to the heart of your evaluation needs and identify the right questions and data collection methods to use. It is based on 4 steps.

STEP 1: Select your evaluation question

STEP 2: Consider your people, purpose, and process

STEP 3: Identify your data collection options (selecting from 52 named methods)

STEP 4: Check and record your decision

The tool will assist you to move from what you want to do, to the data collection methods that will enable this to happen.

 

Practical guide for engaging stakeholders in developing evaluation questions Robert Wood Johnson Foundation Evaluation Series

This resource has an accessible five-step framework for involving stakeholders in setting evaluation questions. It covers different engagement methods to use in planning as well as strategies for managing challenges. It includes template sheets to apply the resources used in the framework

3: Create a logic model to decide what evidence to collect

 

Using logic models in evaluation The Strategy Unit

This briefing paper gives a short introduction to logic models and their benefits. It also provides guidance on how to use logic models.

 

Developing a logic model Evaluation Support Scotland

This resource provides an accessible guide to logic models. It takes the reader through 2 types of logic models ‘The Weavers Triangle’ and the ‘Wisconsin Model’, giving examples of each type and how to produce them.

 

Logic mapping: hints and tips The Tavistock Institute

Although this was developed for the Department of Transport, the content is applicable to other contexts. It provides an overview of how logic models are used in evaluations. Chapter 2 provides a step-by-step guide to logic mapping.

4: Deciding what needs to be in the research or evaluation plan

 

Evaluation Support Scotland (ESS)

This website has evaluation guides to help plan an evaluation. These guides include:

 

Rainbow framework Better Evaluation

Better Evaluation is an international collaboration working to ‘improve evaluation theory and practice by sharing information about evaluation options (methods, strategies, processes) and approaches (collections of methods)’. The Rainbow Framework organises 34 evaluation options (methods, strategies, and processes) by the 7 colours of the rainbow. Its aim is to assist in decision making about all aspects of evaluation planning.

 

Evaluation practice handbook World Health Organisation

Chapter 2 in this handbook provides a useful step-by-step guide to planning an evaluation. Although the guidance is written for WHO staff, the principles covered are relevant to other audiences.

 

Evaluation toolkit Life Changes Trust

This toolkit is a web resource. It provides comprehensive coverage of all aspects of evaluation including an overview of what is evidence, planning activities and outcomes, evaluation design, data collection methods, logic models and types of data and analysis. There is a section on matching evaluation questions to data collection methods. The toolkit can also be downloaded as a pdf. There is a valuable search engine on the home page and a glossary section.

 

Evaluation plan workbook Innovation Network

This has detailed guidance for writing an evaluation plan, including clear guidance on indicators. It includes templates for an evaluation plan.

5: Evidence collection

You may find it helpful to refer to guidance about matching data collection methods to evaluation and research questions.

 

Decision tree Age UK

This decision tree helps you select an appropriate method for collecting evidence.

 

Evaluation toolkit Life Changes Trust

The toolkit has a section on matching data collection methods to evaluation questions.

 

Other resources

Once you have decided what data collection methods you are going to use you might want to find more guidance about how to use these methods. You can access further resources by visiting these other pages of the Using Evidence Resource Guide:

  1. Good practice in conducting research and evaluations

These questions will test whether the evaluation or research plan is in line with good practice:

 

Have you involved all the relevant people in planning?

Who wants to know? Tips for involving stakeholders in your program evaluation Wilder Research

This contains four pages of practical guidance about involving stakeholders (i.e., relevant people) in planning the research.

 

Have you considered if your plan meets the relevant ethical requirements?

Social Care Wales Code of professional practice for social care

The standards in the Code of Practice need to be applied when planning an evaluation to ensure that all ethical requirements are met.

 

Research ethics and data protection Inspiring Impact

This is an accessible guide to some of the research ethics and data protection requirements which apply when carrying out an evaluation. This will help you determine if you have, as far as possible, planned a reliable, valid evaluation or research study that will have useful information for people outside the service.

 

Essentials of survey research and analysis Ronald J Polland

This has a short chapter on reliability and validity. It explains what things to consider when planning a valid and reliable study.

 

Have you ensured your evidence collection approach is suitable for all relevant members of the local society- i.e., is your approach inclusive?

Measuring what matters Co-Production Network for Wales

This guide provides information about how you can select methods that enable the people you are most interested in to participate, including how to adapt methods to make them more inclusive.

 

An alternative approach

An alternative to conducting a research project or evaluation with people using the service is to engage in a peer review. In a peer review, professionals from another service visit and provide feedback on the work you are doing.  Engaging in a peer review can offer challenge, shared learning and support from people who are facing the same problems.

 

A guide for boroughs being reviewed and reviewers Association of Directors of Adult Social Services in London

There are limited resources available on how to conduct/ engage in a Peer Review but this guide, written for the needs of services in London, contains helpful information on planning and conducting a peer review.