There is cause for celebration in 2017 for some realist researchers and evaluators as it is the 20th anniversary of the publication of Realistic Evaluation by Pawson and Tilley.¹1. R. Pawson, N. Tilley. Realistic evaluation. Sage, 1997. Whilst this may be the case, there is still a high chance that realistic evaluation (or realist evaluation as it is now more commonly referred to) will be unfamiliar to many public health practitioners and researchers. You are probably thinking ‘so what?’ and I would not blame you, as there are many research methods and approaches and realist research approaches are but one of many. But I would urge you to read on as why it would matter to you would depend on the type of problems you deal with and knowledge you need.

I am happy to be challenged on this, but I suspect that you have noticed that researchers and evaluators keep mentioning that interventions in public health are ‘complex’ and that outcomes that occur in these interventions depends on ‘context’. Some would argue that most (if not all) public health problems are complex and that many researchers, rather than use methodologies that are able to deal with complexity and context dependent outcomes, prefer to ignore them or take a more reductionist approach. Knowing how to make sense of complex interventions with context dependent outcomes is not easy, but nor is it impossible. One possible option is to use realist approaches.

Until I had learnt more about realist research approaches, I used to wonder how and why outcomes were influenced by context and also what to do about the issue of complexity. Whilst I am certain that there will be research approaches out there that have explanations of how context influences outcomes and/or how to undertake research or evaluation of complex interventions, to me, the appeal of realist research approaches are their explicit and coherent approaches to these issues. In addition, they have the added bonus of being approaches that focus on producing knowledge that is potentially transferable to other contexts. So, what are realist approaches to research or evaluation?

In this editorial, at best I will only be able to provide a very brief overview and explanation of realist approaches. There are two realist research approaches that I will cover in this editorial -realist evaluation and realist review. Realist evaluation is a form of theory drive evaluation approach -that is it uses primary data (data you have to go out and collect) to confirm, refute and refine realist theory or theories about the phenomenon of interest.¹Realist evaluations often start with a programme theory -that is an explanation of how, why, for whom, in what contexts and to what extent an intervention is meant to ‘work’. At the start of an evaluation, it may not be possible to cover all these aspects of this initial programme theory. This might be because of knowledge gaps about the intervention of interest. However, as the evaluation progresses, data are gathered to develop this theory and confirm, refute and refine (or ‘test’) aspects of it. To enable such ‘testing’ to take place, programme theories need to be middle-range in nature -in other words expressed at the level of abstraction that is close enough to the observable data to permit empirical testing.²2. R. Merton. On theoretical sociology. Five essays, old and new. The Free Press, 1967. An example might be a realist evaluation of a smoking cessation service, where you collect primary data to (for example) understand how, why, for whom and in what contexts ex-smokers who have attended the service relapse. To start off such an evaluation, you would develop a programme theory, which might consist of middle-range explanations. For example, ex-smokers who have attended the service relapse because they feel ill prepared to deal with social and environmental cues to smoking as this is not covered in the service provided to them. The expectation should be that a programme theory of any complex intervention would consist of multiple middle-range explanations that need to be developed and ‘tested’.

Realist review (or synthesis as it is also known as) is a form of theory-driven evidence synthesis -in this case documentary evidence (e.g., published studies, policy documents, etc.) are used to confirm, refute and refine realist theory or theories about the phenomenon of interest.³3. R. Pawson. Evidence-based policy: a realist perspective. Sage, 2006. To go back to the example of a relapse of attendees at a smoking cessation service, in a realist review documentary data is searched for, analysed and synthesised instead.

To explain the outcomes (intended and unintended) observed within any phenomenon, realist evaluation and review have a particular approach to analysis, succinctly captured in the heuristic context+mechanism=outcome (or C+M=O). This heuristic sets out a realist explanation of the links between context and outcome, ‘through’ mechanisms. An outcome within a phenomenon is caused to happen by a mechanism, which is only triggered when certain context is present. Put simply, a mechanism is a ‘causal force’ that makes an outcome happen. To illustrate using a fictional example, within the smoking cessation service we may find that some smokers even when attending the service don’t tell the health care professional running the service that they are still smoking. A realist explanation for this might be as follows -when a smoker is unfamiliar with the health care professional providing the smoking cessation service (context), they are less likely to tell her/him that they are continuing to smoke (outcome) because they fear they will be scolded (mechanism). Fear of being scolded causes non-disclosure of continued smoking. This illustrative explanation of the influence of context on a mechanism to cause an outcome is called a context-mechanism-outcome configuration (CMOC). In realist evaluation and reviews, CMOCs should be expressed in the middle-range so that it is possible to use data to develop and ‘test’ them. Within any programme theory for a complex intervention, we would expect to find multiple CMOCs. What programmes or interventions do is to try change the contexts in such a way that the appropriate mechanisms are triggered to give the desired outcomes. Here then is the explicit and coherent realist explanation of the links between context and outcomes.

When it comes to complexity and the stance taken in realist evaluation and realist reviews, Pawson has set out an explanation.⁴4. R. Pawson. The science of evaluation: a realist manifesto. Sage, 2013. In summary programmes and interventions are complex because of: volitions -people make choices-; implementation chains -these are often long and tortuous-; context -there are many from the micro to macro levels-; time -programmes originate from somewhere-; outcomes -these are multiple, intended, unintended, more proximal and so on-; rivalry -programmes compete with each other in the real world and-; and emergence -programmes change with time and when they are implemented. Any approach to evidence synthesis or evaluation needs to be able to understand and account for these issues, and a well conducted piece of realist research sets out to do just this.

A concern often voiced about realist evaluation and realist reviews are that they are hard to undertake and more time consuming than other approaches. However, my personal experience in using these approaches is that this is not necessarily the case. Why perhaps this misconception has occurred is because realist evaluation and realist reviews are new to many researchers and evaluators. As such, for them, there is much to learn and this causes both delays as well as challenges. On top of this, there is a paucity of high quality examples in the literature and insufficient resources and training materials available to learn from. As with learning anything new, there is no substitute for practice and learning from more experienced practitioners.

In summary, realist evaluation and realist reviews are research approaches that explicitly and coherently link context to outcomes and set out to tackle the issues of complexity. They do so by having a simple analytic logic, context+mechanism=outcome and when this is used with other processes within a realist evaluation or review, has a means of making sense of complex programmes or interventions. At best, I have only been able to provide a very brief introduction into realist evaluation and review. For those interested in finding out more, then the three books (by Pawson and Tilley) listed in the reference list of this editorial form the most detailed resources. Freely available resources and training materials for realist evaluation and review may also be found on the RAMESES Projects website (www.ramesesproject.org). The RAMESES Projects have started to address the issues of producing the resources needed to help in the delivery of high quality realist evaluations and realist reviews, but more are still needed. Thus, finally, to provide a space for realist researchers to ask questions, share ideas, debate and discuss or advertise courses or jobs and generally advance realist research approaches, there is the RAMESES JISCMail email listserv (www.jiscmail.ac.uk/RAMESES) which is open to all. I hope you will join us!

Acknowledgements

I want to thank the two anonymous reviewers for their suggestions on how to improve this editorial.

References

Publication Dates