But hang on, have we not been here before? There is this headline in the Mirror: Police forces fail to probe millions of offences fully, this one in the Guardian: Police crime figures being manipulated, admits chief inspector and even the Morning Star: Critics claim crime statistics revamp will mislead the public. These last three stories are dated 23/1/14, 18/12/13 & 31/12/13 respectively.
And then of course there is the two years of near hell that whistleblower James Patrick has been through over his blogging about the same matter.
And of course, this is really not a new phenomenon when Huw Evans is able to dig up the Audit Commission's report on the same matter from ten years ago. (Link is here). And if you follow Bernard Rix's retweet, you will see a whole discussion about when exactly the police began (as some would call it) "fiddling the stats".
So where do we go from here (again)? Bernard poses the challenge that the HMIC may not be asking the right questions. Instead of asking "to what extent can police recorded crime information be trusted?", Bernard suggests that the question ought to be "What are the cost-effective ways of improving the recording of police crime information?" as this question will lead to establishing what should now be done. (I tend to agree that the HMIC has probably already addressed its own question since the answer, quoting Paul Daniels, is 'not a lot'!)
I would say, let's start this from a different angle. I received an email from the Office of National Statistics the other day (23 April) in reply to my last email to them. I must say, that I am seriously impressed since my last email was not one requiring an answer but nevertheless a dedicated civil servant at the ONS felt moved to reply, 3 months later. (Thank you Meghan Elkin!) I have blogged about this before where I explain that I am investigating what resources would be required to use the National Crime Survey at the level of local police forces (so that PCCs could be held to proper account among other objectives). Here is her reply in full:
Firstly, I apologise for the delay in sending a reply to you.
We’ve considered how we might respond to your query that would be of most help to you. I hope the examples below are helpful in not only demonstrating the size of the sample required, but also how this varies across crime types. England and Wales figures used in the examples can be found in the published appendix tables: www.ons.gov.uk/ons/publications/re-reference-tables.html?edition=tcm%3A77-314526
Vehicle related theft has a relatively high prevalence rate. Vehicle related theft – England and Wales Year ending March 2012 5.4% Year ending March 2013 4.6%
The 0.9 percentage point decrease was statistically significant for England and Wales. To observe this 0.9 percentage point change as being statistically significant requires a sample size of over 8,000. The current sample size for a police force area is around 650.
Or alternatively, with a sample size of 650, we would need to observe a 2.6 percentage point decrease (from 5.4%) for the change to be significant. This is similar to the decrease observed between 2006/07 and 2012/13 for England and Wales as a whole.
Burglary has a much lower prevalence rate. Burglary – England and Wales Year ending March 2012 2.4% Year ending March 2013 2.1%
The 0.3 percentage point decrease was statistically significant for England and Wales. To observe this 0.3 percentage point change as being statistically significant requires a sample size of around 27,000.
I hope these examples are helpful in demonstrating that large sample sizes are required in order to trust changes in prevalence rates of crime as being true changes. The required increase would require more resources that would be available for ONS to produce figures with the same level of certainty at Police Force Area level as our current England and Wales figures.
Kind regards, Meghan
I am no statistician, but I think there is more to debate here about what other alternative methods their might be to have reliable data on experienced crime (as opposed to recorded crime) in a locality. But that is for another email and blog (or two!)
So, we seem stuck between a rock and hard place: where we cannot really trust the calibration of recorded crime statistics (locally or nationally) and the more reliable ONS National Crime Survey can only operate at a national level which is a very long way away from operational and indeed strategic decision making at the sharp end of policing. The National Crime Survey may be fine for national politicians to claim that crime is going down (although there are problems with what crime even the national survey doesn't count such as business crime) but it is of no use at the local area level.
So how will the public judge whether a PCC has done well or not on the single most important measure of what the police do (according to the Home Secretary)? Put simply: they cannot. Although I proposed some metrics by which PCCs can be assessed a while back.
This is such a convoluted topic: I have even touched upon the whole issue of performance measurement and how targets twist & deform policing (and all other public services), even when the reasons for such targets are completely understandable (see this old article by Bob Jones, West Midlands PCC)
So, I will finish here (as other jobs are calling this May morning) with a response to Bernard's challenge to come up with a better question than his on what the HMIC ought to be investigating. Here is my suggestion that not only the HMIC should be looking into, but also the College of Policing, the Association of PCCs and indeed every police management team in the country:
How can we more reliably measure the impact of policing activities on the levels of crime in our communities so that we increasingly know what works and what does not?
And there is far more to say, of course...! When is this Groundhog Day due to end?!
An excellent blog as always.ReplyDelete
A point that has been overlooked when it comes to comparing performance, year on year, is that currently it is not like for like.
Since May 2012 most fraud offences have not been recorded locally but via Action Fraud.
These were previously classified as Other Crime, offences in this category have fallen by upwards of 90% nationally and represent 4% of All Crime.
This is compensated by the ONS nationally by adding back in offences reported to Action Fraud but this cannot be done locally.
So when PCC's publish performance figures for All Crime, 4% of any decrease in overall crime is down to a change in reporting procedure not performance.
This will soon not be the case 2nd Q of 2014 will see for the first time since the election of PCC's a like for like comparison. How will they explain the sudden rise in crime?
Useful information - thank you.Delete
Action Fraud's procedures are so time-intensive a number of financial institutions simply do not report fraud to it. One institution asked can we have a simple method of dumping the data online and then Action Fraud can do what it wants. Answer came none.ReplyDelete
At least one large urban force has published it receives nine hundred reports from Action Fraud per month, they have a team to assess if any warrant investigation. I thought they were being recorded locally or in force. Interesting.
One PCC stated that targets had NO impact on recording practices. He must have missed his own Deputy Chief Constable's statement to a meeting upon arrival - 'This force's problem is that it over records crime'. The same PCC allows his force to use 'milestones' instead of targets. Anyone who reads the papers presented to the PCC would notice the emphasis placed on 'milestones' and the difficulties in reaching them.
The same PCC sits there and is told by his crime 'lead' we now better understand crime and are responding better. Oddly he didn't ask for "heads to roll" when the local paper reported for the second time city centre retail crime was not being recorded when reported.
Ever since there's been crime figures there has been gaming of them by various organizations. Police recorded figures are best seen as a measure of police activity rather than the level of crime in society but even then are very imperfect for that.ReplyDelete
There's no magic bullet solution; local crime surveys might help but a culture change away from concentrating on the abstract figures would help.