From Popper to Poundland

A few days ago, I blogged about the Society for Evidence Based Policing. In response, another blogger (TonyOX3) wrote to me highlighting this report that came out a while back (June 2012):

Test, Learn, Adapt:Developing Public Policy with Randomised Controlled Trials(by Laura Haynes, Owain Service, Ben Goldacre & David Torgerson)

It is a good report and I commend it to you. As the introduction to this report states:

Randomised controlled trials (RCTs) are the best way of determining whether a policy is working. They are now used extensively in international development, medicine, and business to identify which policy, drug or sales method is most effective. They are also at the heart of the Behavioural Insights Teamʼs methodology. However, RCTs are not routinely used to test the effectiveness of public policy interventions in the UK. We think that they should be.

I think they should be too. I did a degree in psychology and as many people know, a huge focus of this degree is on scientific method. So when I arrived in my first 'proper' job in a local housing department, I was genuinely shocked that there seemed (to me) to be an almost complete absence of established methods or knowledge of best practice from around the country and beyond. I still have those concerns.

I have always been puzzled as to why politicians and senior public servants seem so reticent at using science to determine whether a new policy works or not? At a cynical level, I wonder if it is about ego, pomposity, fear of being found to be less than effective, worry about how the media will see such an experiment or even a knowledge that while the scientific understanding of many (most?) politicians is very low, it is even lower amongst most journalists... I don't know and I am cautious about speculating without some proper research. It is also the case that there are many academics in the social science field who are very uncomfortable with applying the rigour of RCTs to many socio-political interventions. I respect their positions, but I would hope that a middle ground can be found that means we get just a little bit more 'hard' science into the policies devised by politicians and senior public servants.

Which brings me round to highlighting the decision yesterday that nobody can be compelled (under current regulations) to participate in working 'for free' for companies while they are on benefits. The story can be understood here. For me the most telling comment came from one the people involved who said that her placement in Poundland was a "complete waste of time" and "the experience did not help" her get a job. From my limited knowledge of the case, it would seem that the regulations were designed in such a way that meant the court had no option but to award the case in favour of people who had been forced to take on a placement or face losing their benefits.

So here is an example of where an RCT could easily have been used to test and refine those regulations. If the intention by DWP was to assist participants in gaining new skills and confidence whereby they were more able to get a job (rather than merely massaging unemployment figures or punishing those on benefits or pandering to the Daily Mail etc..) then this could easily have been done through a randomised controlled trial. Had that been done, I would put money on there being no such court case decisions like the one from yesterday.

Meanwhile plans to roll out Payment by Results and privatisation of the probation services continues apace even when the pilots (by no means RCTs but an attempt to validate the policy) have been abandoned.

May I remind this Government that one of the people that Margaret Thatcher admired greatly was Karl Popper who was a strong advocate of good scientific method. To quote Wikipedia, Karl Popper was "known for his attempt to repudiate the classical observationalist / inductivist form of scientific method in favour of empirical falsification". Mrs Thatcher herself was a chemist and understood the importance of science.

So please can we have a little less populist policy making from this government and a bit more use of good science to determine whether various policies actually work or not!