In the world of database programming, a framework is like an offset press: hard to build — Django 1.0 required three years of open-source development — but once it’s set up, there’s no faster way to churn out content. Hand-coding an application like the Guardian’s would have been like publishing a daily newspaper with movable type.Trust me, the four lessons at the click are really worth the time.
Four crowdsourcing lessons from the Guardian’s (spectacular) expenses-scandal experiment
@Nieman Journalism Lab:
Okay, question time: Imagine you’re a major national newspaper whose crosstown archrival has somehow obtained two million pages of explosive documents that outed your country’s biggest political scandal of the decade. They’ve had a team of professional journalists on the job for a month, slamming out a string of blockbuster stories as they find them in their huge stack of secrets.
How do you catch up?
If you’re the Guardian of London, you wait for the associated public-records dump, shovel it all on your Web site next to a simple feedback interface and enlist more than 20,000 volunteers to help you find the needles in the haystack.
Your cost for the operation? One full week from a software developer, a few days’ help from others in his department, and £50 to rent temporary servers.
Journalism has been crowdsourced before, but it’s the scale of the Guardian’s project — 170,000 documents reviewed in the first 80 hours, thanks to a visitor participation rate of 56 percent — that’s breathtaking. We wanted the details, so I rang up the developer, Simon Willison, for his tips about deadline-driven software, the future of public records requests, and how a well-placed mugshot can make a blacked-out PDF feel like a detective story.