As an outcomes nerd from way back, I am loving the current attention to what we measure and how we do it. CEDA’s recent report, ‘Disrupting disadvantage’, is the latest example.
It calls for more and better evaluation of community services to measure the impact they’re having on reducing poverty and disadvantage. Yes! It argues that better evaluation requires capacity and capability to be built. Yes! It recognises that governments must play a key role, but academics and other researchers can also make a contribution. Yes!
Then there are other assumptions that show the continuing need to grapple with the nature of entrenched disadvantage and what is required to address it. One is the call for randomised control trials as a method of evaluating programs and services in response to disadvantage.
Randomised control trials are recognisable from their use in medicine, for instance to understand the impact of a particular medicine on a certain illness. There is a clear transaction (taking medication) and effect to measure (the difference between those who take the medication and those who don’t).
But most of the social policy challenges of our time are deep-seated and complex. These kinds of problems don’t have simple solutions. They require collaborative and system-level responses.
The rapidly evolving landscape of collaborative service models demonstrates how organisations are increasingly working together in response to complex and intersecting need in the communities they serve. Health justice partnership is one example, where community lawyers work through health and other service settings to help people address the underlying legal problems that affect their health and wellbeing. From mouldy public or rental housing that drives respiratory problems, to family breakdown and family violence, health justice partnerships enable services to respond to the multiple, intersecting health, legal and social problems that hold people in disadvantage. There are often different motives to work together for legal partners compared to health partners. There are likely to be multiple objectives across justice, health and wellbeing. Perhaps most significantly, there are a range of different outcomes that can be achieved at the level of individuals (clients or patients), practitioners providing these services, the organisations they work for and the system they are part of.
The precision of a randomised control trial is a blunt tool when we need to identify what difference has been made by a range of actors working together, recognising that each may have contributed to outcomes but none is solely responsible for them.
Another assumption to build on is that we only need skills and capability in government to improve policy evaluation. We need that, but not only in government. Many community services that are funded by governments are delivered by not-for-profit organisations. Many of these are charities: independent of government, with their own purpose and strategy. Health Justice Australia works with organisations like this every day, supporting them to innovate and collaborate in response to changing and diverse needs among the communities they serve. The skills and capability to identify and then measure their outcomes are just as important among these organisations as among their government funders. So too for the non-government policy actors and intermediaries who are influencing system level change.
We need collaborative solutions in response to the complex problems of today. The good news is, this work is well underway. Let’s make sure, then, that we are building the skills and capability to identify what difference these responses make, across different approaches to measuring outcomes and among the different actors striving to achieve them.
