Crime trends: asking the right questions

The news is awash this morning with headline news that upwards of a fifth of crime reported to the police is not being recorded. You can see various stories here from the HMIC summary & copy of interim report (yes it's not the final one), the BBC headline story (Police not recording a fifth of crimes, watchdog report suggests) and other stories: Daily Telegraph, & Daily Mail.

But hang on, have we not been here before? There is this headline in the Mirror: Police forces fail to probe millions of offences fully, this one in the Guardian: Police crime figures being manipulated, admits chief inspector and even the Morning Star: Critics claim crime statistics revamp will mislead the public. These last three stories are dated 23/1/14, 18/12/13 & 31/12/13 respectively.

And then of course there is the two years of near hell that whistleblower James Patrick has been through over his blogging about the same matter.

And of course, this is really not a new phenomenon when Huw Evans is able to dig up the Audit Commission's report on the same matter from ten years ago. (Link is here). And if you follow Bernard Rix's retweet, you will see a whole discussion about when exactly the police began (as some would call it) "fiddling the stats".

So where do we go from here (again)? Bernard poses the challenge that the HMIC may not be asking the right questions. Instead of asking "to what extent can police recorded crime information be trusted?", Bernard suggests that the question ought to be "What are the cost-effective ways of improving the recording of police crime information?" as this question will lead to establishing what should now be done. (I tend to agree that the HMIC has probably already addressed its own question since the answer, quoting Paul Daniels, is 'not a lot'!)

I would say, let's start this from a different angle. I received an email from the Office of National Statistics the other day (23 April) in reply to my last email to them. I must say, that I am seriously impressed since my last email was not one requiring an answer but nevertheless a dedicated civil servant at the ONS felt moved to reply, 3 months later. (Thank you Meghan Elkin!) I have blogged about this before where I explain that I am investigating what resources would be required to use the National Crime Survey at the level of local police forces (so that PCCs could be held to proper account among other objectives). Here is her reply in full:

Firstly, I apologise for the delay in sending a reply to you. 

We’ve considered how we might respond to your query that would be of most help to you. I hope the examples below are helpful in not only demonstrating the size of the sample required, but also how this varies across crime types. England and Wales figures used in the examples can be found in the published appendix tables: www.ons.gov.uk/ons/publications/re-reference-tables.html?edition=tcm%3A77-314526 

Vehicle related theft has a relatively high prevalence rate. Vehicle related theft – England and Wales Year ending March 2012 5.4% Year ending March 2013 4.6%

The 0.9 percentage point decrease was statistically significant for England and Wales. To observe this 0.9 percentage point change as being statistically significant requires a sample size of over 8,000. The current sample size for a police force area is around 650. 

Or alternatively, with a sample size of 650, we would need to observe a 2.6 percentage point decrease (from 5.4%) for the change to be significant. This is similar to the decrease observed between 2006/07 and 2012/13 for England and Wales as a whole. 

Burglary has a much lower prevalence rate. Burglary – England and Wales Year ending March 2012 2.4% Year ending March 2013 2.1%

The 0.3 percentage point decrease was statistically significant for England and Wales. To observe this 0.3 percentage point change as being statistically significant requires a sample size of around 27,000. 

I hope these examples are helpful in demonstrating that large sample sizes are required in order to trust changes in prevalence rates of crime as being true changes. The required increase would require more resources that would be available for ONS to produce figures with the same level of certainty at Police Force Area level as our current England and Wales figures. 

Kind regards, Meghan

I am no statistician, but I think there is more to debate here about what other alternative methods their might be to have reliable data on experienced crime (as opposed to recorded crime) in a locality. But that is for another email and blog (or two!)

So, we seem stuck between a rock and hard place: where we cannot really trust the calibration of recorded crime statistics (locally or nationally) and the more reliable ONS National Crime Survey can only operate at a national level which is a very long way away from operational and indeed strategic decision making at the sharp end of policing. The National Crime Survey may be fine for national politicians to claim that crime is going down (although there are problems with what crime even the national survey doesn't count such as business crime) but it is of no use at the local area level. 

So how will the public judge whether a PCC has done well or not on the single most important measure of what the police do (according to the Home Secretary)? Put simply: they cannot. Although I proposed some metrics by which PCCs can be assessed a while back. 

This is such a convoluted topic: I have even touched upon the whole issue of performance measurement and how targets twist & deform policing (and all other public services), even when the reasons for such targets are completely understandable (see this old article by Bob Jones, West Midlands PCC)

So, I will finish here (as other jobs are calling this May morning) with a response to Bernard's challenge to come up with a better question than his on what the HMIC ought to be investigating. Here is my suggestion that not only the HMIC should be looking into, but also the College of Policing, the Association of PCCs and indeed every police management team in the country:

How can we more reliably measure the impact of policing activities on the levels of crime in our communities so that we increasingly know what works and what does not?

And there is far more to say, of course...! When is this Groundhog Day due to end?!