Tag: stats

I’ve written before about their foolish and inaccurate claim that the black-to-white racial disparity among those shot by police is 21 to 1. I said, given the group they look at, the number is 9 to 1. But without any slight-of-hand or misleading highlighting of statistical outliers, the actually black-to-white racial disparity, the take-home stat, is 4 to 1.

More than two months passed. The inaccurate 21-to-1 figure was bandied about by the NPR, the New York Times, and The Economist.

Then, on a quiet Christmas Eve, ProPublica’s Ryan Gabrielson and Ryann Grochowski Jones posted an article to address criticism (mainly brought by me and David Klinger) of their initial study.

I don’t want to waste much more time on this; I’ve wasted too much already (see 1, 2, 3, 4). But I do find it funny, in their piece, after many paragraphs focusing on the red herring non-issue of hispanic undercount, there it is — buried in the 11th paragraph — they kind of admit I’m right: the ratio might be 9 to 1!

Maybe I should just stop there and say, “you’re welcome.”

But, but, I can’t! Because then there it is — a revisionist gem — they say the actual number doesn’t really matter: “And whether 9 times as great, 17 times or 21 times, the racial disparity remains vast, and demands deeper investigation.”

What the fuck?!

The 21-times ratio is the only real point of your original article (which is still up and unapologetic)! And the only real point of my bitching was that 21-times is wrong. Now even 4:1 or 9:1 may be too large. And it does demand deeper investigation. So why not investigate deeper (Or at least crib from those who have)? According to ProPublica: “the data is far too limited to point to a cause for the disparity.” Actually, no. The disparity can be explained pretty well, without too much “deep” digging. What I’m about to tell isn’t the “deepest” investigation, mind you, but it’s a start. And it’s on me, guys. Gratis.

The black to white racial disparity (all ages) of those killed by cops since 2000 (and reported to the UCR, which is big caveat) is 4 to 1. The racial disparity among those who kill cops is 5 to 1 (the rate is per capita, mind you, not the absolute number). I’d bet $20 it holds for teens, too.

Now one could say, as does Prof. Klinger, that the data on police-involved homicides are simply too limited to make any point at all. But if one is willing to play with bad data (and I’m game, if they’re the best we got), then you can’t say your conclusion is fine but… other conclusions? …well, “the data is far too limited.”

Finally — and it goes back to my point about outliers and cherry-picked bullshit data — ProPublica has the chutzpah to say they can’t go back further in time — thus including more data, increasing statistical validity, and decreasing the magnitude of their conclusion — because, get this: they can’t get accurate population numbers.

So let me get this right: they’re fine using fucked-up UCR data on justified police-involved homicide, they’re fine cherry picking an outlier three-year sample with an “n” (total cases) of 62, but they wouldn’t dare look at more years because we can’t estimate the US population between 2001 and 2007? Are they on crack? Are they stupid? Or are they simply blinded by ideologically bias. I honestly do not know. But it’s a nonsensical line of statistical integrity for them to draw.

Here is it in their words:

Using Census 2000 and Census 2010 data for baselines assumes that the ratio of populations remain static, and that a snapshot of population rates for a subset of time can be assumed to be accurate for an entire period. We know that’s not true…. To test the critics’ argument, we calculated risk ratios for as far back as the American Community Survey data goes (2008) [ed note: the ACS actually goes back to 2005, but whatever]. From 2006 to 2008, the risk ratio was 9.1 to 1 (with a 95 percent confidence interval 6.19, 13.39).

First of all, stop the fancy talk about “risk ratio” and “confidence interval.” You either don’t know what you’re talking about or you’re knowingly trying to mislead.

Speaking above your reader’s head is a dirty rhetorical trick to hoodwink gentle reades into trusting your statistical acumen (which is pretty crappy). As my grand pappy used to say, “Ain’t no need to use a 25-cent word when a 5-cent one will do.” (See, now I’m usin’ the reverse rhetorical trick by affectin’ an aww-sucks-I’m-just-a-common-guy style of speech here.) For what it’s worth, my papou was an immigrant who spoke with a Greek accent.

“Risk ratio” here means nothing more than “more likely.” “Confidence interval,” well, if you’re going to use it, explain it. Better yet, explain it accurately* or at least point out that it supports 9:1 more than 21:1.

More to the point, it’s pointless to discuss statistical nuances of irrelevancy! Of all the problems in your analysis, you’re going to draw the line at estimating population in Census off-years? Really?! It’s like we’re sitting in your rusted jalopy and you tell me you can’t drive me home because the windshield wipers aren’t working. But you failed to mention the fact that the whole thing is up on cinder blocks!

Of course we can estimate population figures, you fools! The US population grew 9.7% between 2000 and 2010. Talk about easy math! Go on, be bold, you dirty devil: assume a linear population growth for all categories. Divide 10% by 10. It comes out to 1% a year. I know it’s not perfect, but it’ll be close enough; trust me. (Actually population growth of 9.7% over 10-years comes out 0.925% compounded continuously.)

Will this population estimate be perfect? No. Is it good enough? Yes. Will it tell you far more about what you claim to show? Of course. Is that why you won’t do it? Probably. Would this population estimate be the single most accurate number in your entire analysis? Abso-fucking-lutely.

Trying to set the record straight is a bit like pissing into the wind. The substantively wrong pro-publica story has now been repeated by every news source I can find.

I suspect that over time the idea that from 2010-2012, blacks males 15-19 years-old were 21 times more likely than non-hispanic-whites males to be killed by police will simply become remembered as: police are 21 times more likely to shoot black people. But it’s not true! (There I am again, getting spattered by my own pee.)

The real figure they’re talking about — not just the numbers from 2010 to 2012 — the real figure is not 21 to 1 but 9 to 1. And when one includes hispanics in the count, the black-to-white ratio goes down to 5.5 to 1. If one looks at black and white men of all ages killed by police, the ratio is (just?) 4 to 1. (Ed note: based on later better data, the ratio is actually closer to 3 to 1.)

Now you may wonder why I’m quibbling. What’s my point? Well, it’s important to base opinions and public policy on fact. And for starters, 4 to 1 versus 21 to 1 is a huge difference.

One could also argue that even a disparity of 4:1 is unacceptable. And it is, on some level. But in the population examined by ProPublica — the same subset in which blacks are 9 times (not 21 times) as likely as whites to be killed by police — the black-to-white homicide ratio is 15:1. We know police-involved homicides correlate with homicide and violence in the community they police. So what rate of disparity would one expect in police-involved homicides? Certainly not 1 to 1.

If you’re going to honestly talk about racial disparities in police-involved shootings, you need to discuss levels of violence among those with whom police interact. If one thinks police shootings are primarily an issue of racist police — if one thinks police only shoot black people, if one thinks white people are never stopped by police for minor offenses — one is not only wrong, but one won’t come up with any effective solutions. The vast majority of police-involved shootings are justified. That said, there are bad shootings. But this is more a police problem more than a race problem.

If one wishes — as one should — to reduce the racial disparity of police-involved shootings, one needs to focus on racial disparities in crime and violence in general. If one wishes — as one should — to reduce the incidences of unjustified police shootings and improper police use-of-force, one needs to improve police training and reduce police militarization.

To replicate the pro-publica study, here are the numbers for the past 15 years (15-19 year-old black and non-hispanic-white men, shot and killed by police and reported to the Uniform Crime Reports). This is the black-to-white ratio for police-involved homicides. All are based on population rates per 100,000 (using constant 2010 census figures, not adjusted for year):

Past 1 year (2012, n = 24): 13 to 1

Past 2 years (2011-2012, n = 45): 16 to 1

Past 3 years (2010-2012, n = 62): 21 to 1

Past 4 years (2009-2012, n = 92): 17 to 1

Past 5 years (2008-2012, n = 110): 17 to 1

Past 6 years (2007-2012, n = 140): 15 to 1

Past 7 years (2006-2012, n = 162): 12 to 1

Past 8 years (2005-2012, n = 183): 10 to 1

Past 9 years (2004-2012, n = 209): 9 to 1

Past 10 years (2003-2012, n = 226): 10 to 1

Past 11 years (2002-2012, n = 249): 9 to 1

Past 12 years (2001-2012, n = 262): 9 to 1

Past 13 years (2000-2012, n = 286): 9 to 1

Past 14 years (1999-2012, n = 312): 9 to 1

Past 15 years (1998-2012, n = 339): 9 to 1

With the above data, you can’t say anything conclusive from just the first few years of data. Certainly the group that I would least want to pick and highlight is the three-year (2010-2012) statistical outlier. Cherry-picking the highest number would be dishonest, but even assuming it’s just accidental is still shoddy research. One would expect the results to bounce around for the first few years and then settle down. Only then can one find validity — the idea that the number has any meaning.

Why pick the past three years instead of the past 2, 4, or 15 years? One key to analyzing statistics is skepticism of “amazing” anomalies, especially from a small group. Something can be (in fact, will be 1 in 20 times) statistically significant but substantively irrelevant.

But why is the 3-year cumulative number so high? Because only one non-hispanic white teen got shot and killed by police in 2010. Since the sample is so small, one strange year can screw up the data. But over more years the numbers settle down. Here one needs to go back maybe 8 to 10 years to find any substantive meaning. (And even then all this UCR data on police-involved homicides should be taken with a gigantic grain of salt.)

[Also, there’s a bit more rambling detail, in less coherent form, in one and two previous posts. Here’s a follow-up post.]

[Update: Cut to the chase. You might just want to read my summary post.]

One of my liberal de Blasio-loving not-so-fond-of-cops friend send me an email with the subject “you gotta check yo facts” and a link to ProPublica: “Young black males in recent years were at a far greater risk of being shot dead by police than their white counterparts – 21 times greater.”

“Well, that’s interesting,” I thought, “It also can’t be true.” Since I kind of know these numbers (and had discussed them with my friend). So I guess I do have to check my facts. I then wasted a half day running the numbers myself (when I could have been giving my undivided attention to the Orioles’ loss).

Now it’s always dangerous to say my numbers are right and theirs are wrong. But I trust my numbers, because I just ran them. And I’m good at this. And then I ran them again. I’d like to see their numbers because, well, I think they’re wrong. But clearly one of us is wrong. I hope it’s not me.

In the past three years (2010-2012) among those 15-19 year old, 54 blacks and 36 have been shot and killed by police. This is according to the UCR stats that are not perfect. But while the data here are not complete, they’re OK in many ways. And the black-white ratio should hold-up just fine.

If my data are wrong, please do correct me.

In the 15-19 population population, there are 8,728,271 white males. (Click through to: “Annual Estimates … by Sex, Age, Race, and Hispanic Origin”) There are 1,978,081 black males, 15-19 years-old (2010 census).

Per year, for the past 3 years, this is a police-involved homicide rate of 0.14 per 100,000 for whites and 0.99 for blacks. 0.91 divided by 0.14 is 6.5, not 21. For the past three years black males 15-19 are 6 or 7 times more likely than white males to be shot and killed by police, not 21 times.

From ProPublica:

The 1,217 deadly police shootings from 2010 to 2012 captured in the federal data show that blacks, age 15 to 19, were killed at a rate of 31.17 per million, while just 1.47 per million white males in that age range died at the hands of police.

Now even if one takes a 3-year rate per million (which is statistically odd for two italicized reasons), the rate for blacks is 30 (close to 31 but not replicated). Where I think the error lies is that the rate for whites is not 1.47 but rather 4.3. That’s a big difference.

My numbers are based on the years 2010-2012: 36 whites shot and killed. 8.7 million white males 15-19.

[Their 95% confidence interval is vast: “between 10 and 40 times greater risk.” This, leaving aside the wrong number, seems to me to be a gross misuderstanding of confidence interval. The overall number (the “n,” in stat terminology) of young people killed by police over the past three years is not large. But there’s a difference between a small “population” and a small “sample” size.

A confidence interval tells you the odds your sample reflects the total population. Say you ask 100 potential voters if they would vote for Obama. Four or 40% say yes. So what are the odds that Obama would win 40% of the vote? Well you don’t know for sure because you didn’t ask everybody. But based on those 100 you did ask, you can come up with a range, say 35-45 percent, at which you can say there is 19 in 20 chance that if we did ask everybody, it would be in this range. That’s a confidence interval.

Again, if I’m wrong here, correct me! It’s been 18 years since I took a statistics class in graduate school. And I wasn’t even good at it.

If you poll everybody — if you have an election — you don’t have a confidence interval. You have a result! Even with its flaws, the UCR is pretty complete. If blacks are X-times more likely to be killed, that’s that! There is not a sample but a population. You don’t have a confidence interval if you sample everybody in a population. You have a number. But it is a small population.

I also wonder why they only picked people shot and killed, rather than all persons killed. It’s a minor difference, but why make more work when you don’t have to? 99.2 percent of people killed by cops are killed with a gun.)]

Well conveniently you can just add more years to get a larger population. I don’t know why they didn’t. (Well, I suspect because it’s work. It’s a bit of a pain to download and select from each year’s UCR sample. But that is what researchers do. I mean, I just happen to have the last 15 years compiled and ready to use because, well, that’s what researchers do. On a Saturday night. While watching baseball.)

So instead of looking at the past three years, let’s increase the population by looking at the past 15 years. From 1998-2012, 210 white and 242 black male 15-19 year-olds have been shot and killed by police. This comes out to an annual rate of 0.16 (per 100,000) for white males and 0.82 for black males.

So over the past 15 years black male teens are 5.1 times more likely — five times more likely — than whites to be shot and killed by police. Five times; not 21.

Now maybe 21 and 7 and 5 are close enough for you. Or maybe you think 5 times more is 5 times too many. But what number would be OK? Given ration disparities in violent crime, one shouldn’t expect 1:1. One might expect police to be more likely to shoot and kill people who shoot and kill other people. (Remember that we’re using rates here, which take into account the population difference, that there are 7 whites for every black in America.)

The homicide rate for black men 15-19 is 9 times the rate for white men. (From 2010 to 2012, looking at men 15-19, 2,382 blacks and 1,209 whites have been murdered by criminals. The homicide rate for these young white men is 4.6 per 100,000. For these young black men, the homicide rate is 40.7.)

So given the 9:1 racial disparity in the homicide rate among young men, what racial disparity would one expect in police-involved shootings? There’s no right answer to this question. But I don’t think it’s unreasonable for the racial disparity of those young men shot and killed by police to be reflective of the racial disparity in violence and homicides among young men. And in fact, the police-involved ratio, at 5:1 (not 21:1 or even 9:1), is much less.

[Updated to reflect population data from 2010 census rather than ACS estimate. It doesn’t change much. Also, see next post and my summary.]