Evaluating personal outcomes focused ways of working
Here we bring together learning from evaluation of personal outcome focused interventions to provide some practical guidance for those planning, commissioning and delivering personal outcomes focused approaches and seeking to monitor and evaluate this work. Links to relevant resources and reports are presented throughout.
What do we mean by a personal outcomes approach?
There is a growing realisation across health and social care that if we are to be effective we need to work with people, tailoring support to their circumstances, needs and hopes for the future. This requires health and social care organisations, planners and commissioners to shift from being ‘service led’ to ‘outcome focused’. Scotland has pioneered a range of approaches that focus organisations on what matters to the person and why through:
- Engagement with the person in one or more ‘good conversations’ through which the practitioner supports the person to identify what matters to them and co-produce a plan for support, building on their assets and strengths
- Recording of the outcomes important to the person, the plan for achieving these, including the actions the person will take, and progress towards these
- Use of that data and information to plan and develop support for the person and to gain understanding at an organisational level of the systems of services and supports that need to be in place to meet needs and improve outcomes for people.
Read more about personal outcomes approaches in practice in this example from the Midlothian health and wellbeing practitioner service.
Evaluating personal outcomes focused approaches
Internationally, there has been relatively little work to evaluate the effectiveness of personal outcomes focused approaches (see Miller, 2017 for a review). Qualitative research has shown that these ways of working do deliver benefits for people, however there has been little progress in developing evaluation approaches that are effective for these kinds of interventions.
In the drop down below, three reasons are listed for why little progress in developing effective evaluation approaches has been made.
Focusing on contribution as a tool to evaluate personal outcomes approaches
This concept of contribution forms the basis of an approach to evaluation that is widely promoted in Scotland called Contribution Analysis (CA). CA is a theory-based approach to evaluation that involves developing a ‘theory of change’ for the project and using this to structure the data collection and analysis. A theory of change simply refers to the way the project team understand how what they do makes a difference. How the service operationalises this theory of change can then be captured diagrammatically in what can be called a ‘results chain’, ‘logic model’ or ‘outcome map’ The conceptual development and application of CA has been influenced by individuals such as John Mayne and Steve Montague who have described the process as ‘results-based management’ involving the gathering of a range of forms of evidence in order to tell the story about programmes have contributed to outcomes in the short-term, medium-term and long-term.
Over the past four years Dr Sarah Morton and Dr Ailsa Cook of Outcome Focus have been adapting and refining CA to develop an approach that can be used by organisations to learn, improve and tell the story of the difference they make. At the heart of this approach is a seven step process (pictured right).
As organisation, you can produce an outcome map that shows what needs to be in place for your approach to improve outcomes for people. A starting point would be for the outcome map to show what the experience is like from the perspective of the person.
You can audit the information you already gather to see how well you evidence that people were having the experience expressed in the outcome map. You can use or adapt existing information from case notes, activity logs and patient story proformas. For example adding in a field into case notes so that practitioners could reflect specifically on how people felt about the process.
Through focusing on contribution and bringing together qualitative and quantitative data, it is possible for organisations using personal outcomes approaches to robustly and meaningfully evaluate their work. Ensuring that practitioners are closely involved in this process and that there is time for analysis and learning and important strategies to maximise the benefit of this work.