jump to content

What constitutes effective evidence

‘What counts as knowledge and whose knowledge counts?’

Hodgson and Canvin, 2005

What do we mean by effective evidence?

Using evidence to inform practice is at the heart of the performance and improvement framework for social services. A longstanding debate in health and social care revolves around questions of how ‘evidence’ is defined and whose knowledge, or evidence, counts. For the purposes of this Resources Guide, we are using the following definition of effective evidence, which has been developed in discussions with local authority staff in Wales. 

Effective evidence: a consensus definition

Practitioners involved in the development of the Using Evidence Resources Guide developed a definition of what effective evidence means to people in practice:

“Effective evidence helps develop an understanding of phenomena and answers questions. It is contextual, relevant, and timely, drawing on different sources of knowledge (e.g., research, lived experience, practitioner, and organisational knowledge). Effective evidence is accessible, and all relevant individuals can engage with it. It is useful and can lead to improvement, but it is also open (transparent) about its limitations.”

This definition identifies some key considerations when planning an evaluation or research project, e.g., transparent, and inclusive approaches.

Taking a whole system approach

Sandra Nutley and her colleagues at the Research Unit for Research Utilisation, University of St Andrews, suggest that developing effective evidence is a complex task. It requires a whole system approach to evidence that enables:

  • Practitioners to access evidence that is engaging and can be applied in practice.
  • Organisational systems that allow practitioners to use and apply evidence.
  • Organisational cultures that provide learning environments in which practitioners can experience and try things out.

 

The Developing Evidence Enriched Practice (DEEP) programme, developed in Wales, has a range of tools and approaches to support taking a whole system approach to the generation and use of evidence.

Case studies of evidence informed performance improvement

Refreshing Respite: West Wales Care Partnership

In early 2019, the West Wales Care Partnership decided to undertake a review of respite care services across the region. Working with the Wales School for Social Care Research, they began by exploring research evidence and organisational evidence about respite from other parts of the UK and then used this as a catalyst to gather local contextual knowledge from carers, practitioners, and the people they support. They achieved this through Exploratory Talk discussions in each of the three counties. These discussions were audio-recorded, transcribed, and then analysed for key themes. These themes and many of the associated stories and quotes were then shared in a regional event involving a researcher, senior managers, practitioners, carers, and the people they support. Participants at this event worked collectively to identify a set of principles and priorities for development, which were then incorporated into the Refreshing Respite report and associated recommendations.

Key themes raised during the day were also woven into a poem that was shared at the end of the event. This poem supported a connection between head and heart in what matters most. Members of the Partnership are currently taking some of these themes forward, including an exploration of how local hospitality services can contribute to providing short breaks for carers. Their dialogic approach to exploring and using diverse types of evidence had the additional benefit of building relationships and trust between carers, practitioners, and managers.

 COVID-19 innovation: North Wales Research, Innovation, and Improvement Coordination Hub

The North Wales Research, Innovation and Improvement Coordination Hub carried out a review of innovations in health and social care that took place due to the Covid-19 pandemic. The work used Most Significant Change techniques to capture people’s stories about the changes that took place and to learn from them. The findings of the work are being used to identify which changes we should keep and where we need to improve, particularly around integrated health and social care and the use of digital technologies. The report is available here: Population Needs Assessment Rapid Review

App development: Here2there

In 2019 a small group of people in North Wales identified challenges services were facing in relation to evidencing the impact of their interventions and making their planning processes as person-centred and meaningful as possible. This was the catalyst for the development of the Here2there.me (H2t.me) tool. H2t.me allows an organisation to support individuals, via a web-based control panel and smartphone app, to develop a person-centred profile and resultant set of goals and actions. Individuals then collect evidence of their achievements via words and pictures on the smartphone app, similar to many social media apps. This ‘timeline’ of words and pictures is called the story.

 

Each individual also agrees a circle of support, key people who will support them to achieve these goals. The circle of support uses the app to see what the individual is achieving, provide encouragement, and post their own evidence to support this achievement. Regular reviews are then held, and progress is rated on a 1-10 scale against each goal. This rating results from a dialogue between a named mentor and the individual, underpinned by the rich evidence provided within their story.

 

H2t.me has been piloted within a range of services over a two-year period, this has included learning disability, children’s homes, and work support programmes. This has all been supported through a grant from the North Wales Transformation Project for LD services and a Welsh Government SBRI competition called ‘Better Lives Closer to home’ (H2t.me was one of 3 winners from across Wales and England).

 

Feedback from users is being used to improve the tool on an ongoing basis. This has included adding actions to goals, the ability to map the individual’s goals (expressed in their words) against external KPIs such as the National Outcomes Framework and having planned strategies for individuals based on their level of understanding and capacity.

 

More information about the project can be found at www.here2there.me.uk