Confidence Intervals

* This is a footnote to the above post. (and the third in a series on basic math concepts)

The ProPublica people don’t explain confidence intervals at all in this piece, but in their original they say, “a 95 percent confidence interval indicates that black teenagers are at between 10 and 40 times greater risk of being killed by a police officer.” Er… actually, no.

A “95 percent confidence interval” doesn’t indicate anything about the real word. What a 95 percent confidence interval “indicates” is that there is statistically a 19 in 20 chance that the “real” number they’re looking for is somewhere between 10 and 40. (And a not-insignificant 1 in 20 chance that it’s not!) A wide confidence interval may sound dramatic, but it’s a red flag which means there isn’t enough data.

Compare these two statements:

A) “a 95% confidence interval indicates that black teenagers are at between 5 and 80 times greater risk of being killed by a police officer.”

B) “a 95% confidence interval indicates that black teenagers are 10 to 11 times greater risk of being killed by a police officer.”

The first may sound more damning, but the large number (80 times!) just comes from ambiguity because there’s not enough data). The second statement actually tells us much more, and with much greater accuracy.