Why Governments should publish raw data

This raises an obvious question: why should we have to exercise arcane technical skills in order to get at public data? As one US expert put it, "converting PDF to XML [ie web format] is a bit like converting hamburgers back into cows". We'd like the cows, please. After all, we paid for their upkeep. Why can't all official numerical data be published in internet-friendly formats? That's what the Obama administration is now doing. And it's what the UK government would be doing if Gordon Brown's commitment to "transparency" was anything other than an expedient gimmick.

This isn't just a matter for techies; it's about the health of our democracy in a networked age. In the open source movement, there's a saying that "given enough eyeballs, all bugs are shallow" - which, being translated, says that the speed with which bugs are discovered and fixed is directly proportional to the numbers of people using or examining the code. The same is true for public information.

We've got the internet and analytical tools such as Benford's law (http://bit.ly/xtMDo) for detecting fraudulent patterns. Now all we need is the raw data. Let's have it.