Is Scraperwiki the future of journalism? by Chris Bradley for The Nattering Nabob

According to Wikipedia, a Wiki is “a website that allows the easy creation and editing of any number of interlinked web pages via a website”. Put 'leaks' on the end of 'Wiki' and Uncle Sam, who isn’t so kind to internet freedom and free speech anymore, labels founder Julian Assange Public Enemy No.1, as the world learns far more about America's international relations than they would ever have liked the public to know.

Is that 'bad news'? Well, not from a journalist’s point of view or anyone who thinks the game is rigged. As Alan Rusbridger, editor of The Guardian, said in defence of publication of the US embassy cables: “It’s not the job of journalists to keep secrets.” But what if those secrets are locked in a huge vault of complex data? Opening Pandora’s Box is not that easy. It is if you’re Julian Todd, a computer programming super-brain - and owner of a business card given to him by one Julien D’Assange in 2007. Todd has the key to unlock some of those vaults... ScraperWiki. 

ScraperWiki 'mines' websites using computer programmes that look for information and present it in a user-friendly way. 

Let's take a question:  'Is there a class divide in Parliament?' The data scraped from the Telegraph's directory of politicians proved unfounded and, most importantly, data was collated to dispel any argument that it is full of toffs who  have had a private education. Our data found that Parliament is represented (roughly) by the same amount of privately educated as state educated MPs.  

In fact, most of the candidates and elected MPs came from the education sector, which was surprising.  A news story could potentially be: 'How many MPs who voted to raise tuition fees had a free education and how many of them previously worked in education?'

Julian Todd warns not to limit the scope to specifics. Wait until all the data is scraped, analyse it and put those journalism skills into practice. There may be even more stories to come from that data. ScraperWiki is truly groundbreaking; a new method of investigative journalism that champions the old virtues of truth and accuracy.

With the internet being a vast silo of data just waiting to be scraped anything is possible, journalists just need to be looking. Politicians, businessmen, policemen, oil magnates, tax dodgers, bankers or criminals may be nervous about being exposed as corrupt, incompetent, or even caught. They should be. The truth will be scraped out. 

Governments are scared of losing control of the flow of information. ScraperWiki gives that control to the user, and if it makes journalists unpopular it means they are doing their jobs properly. A new underground press could be gathering momentum; a band of data-scraping truth-seekers.