It's all about the numbers

Statistics on unemployment, body counts from Iraq, the size of the national debt  - it's all about the numbers. There's no getting around the fact that numbers matter, it's the way we make sense of the news and the world around us. But as Ellen Thompson, an analyst at the U.S Census Bureau told me recently, "Numbers are never that straightforward."

As a researcher at Nightly, it's my job to track down numbers and verify their accuracy. For Monday's story by correspondent Kevin Tibbles about the pension crisis, I started with a statistic, printed in a leading business magazine, regarding the amount that state and local governments contribute to public pension funds. The magazine attributed the statistic to the U.S. Census Bureau, so off I went. It seemed pretty straightforward, except neither I nor the public information officer could confirm it. 


It turned out that it was derived from adding up the total payments column from the census's quarterly report on public pension funds. The catch is, there's a completely different set of statistics that the census issues annually on public pension payouts. Why are the two numbers different? One follows a fiscal year, the other a calendar year; one uses data from the 100 largest pension funds, the other tries to include all of them; one is compiled for the Federal Reserve, the other is just there.

So now it appears as a simple choice between two numbers, right? Not quite. One can make the argument that it's inaccurate to simply report total payments, without providing a breakdown of what employees contribute versus the government. This can either change the picture, or in this case strengthen the argument that governments cannot afford to pay for employee pension plans. But can we report the numbers that way? Not safely.

As one census official warned me, "You you can't reliably "impute" the breakdowns to specific categories. The Federal Reserve just uses these numbers to see if things are going up or down." Does that mean we can't trust the totals? The answer from the census official: "These are strong estimates."

It begs the question, when can we trust the numbers? At Nightly, as we report the good and the bad about lots of things, we confirm, verify and try to give you our strongest estimate. But it's always important, as viewers and journalists, to put it in context and figure out the breakdown, if we can. Sometimes you just have to admit that the number doesn't exist or will never be accurate like I did Monday on another research request. This time the question was how many civilians had been killed by sectarian violence in Iraq. We will probably never know.

As for making the decision on which statistic to use on the pension funds - our decision was made for us. The Census Bureau only started to issue annual statistics on public pensions in 2004. But if it's good enough for the Fed, then I think it's good enough for us.

Discuss this post