|on 20 Mar Posted by Admin Category: Search Engines|
by Gord Hotchkiss
Jimmy Wales, the co-founder of Wikipedia, is placing a fairly large bet that people can trump technology in the search engine game. According to a recent report in Yahoo, He's putting $4 million (of other people's money) plus an undisclosed 'large amount' from Amazon on the line, betting that he can steal 5% of the total search market away from Google with his new project, Wikia. The Problem with People
Wales has called both Google and Yahoo the 'black boxes' of the internet, criticizing them for the secrecy maintained around their ranking algorithms, but details on exactly how Wikia will work have been equally scarce. All we've heard so far is that an online community with a distinct and clear purpose -- a moral purpose -- that unites people and brings them together to do something useful, will work to make web search a better experience for us all. The 'how' of how Wikia will work has been lacking to this point. But it's likely to follow a similar path to Wikipedia. The online community will act as an army of human editors, ensuring the quality of the results by collectively agreeing on them in some fashion. The theory here is that there is no better filter for results aimed at humans than those same humans.
Human 'Signal Noise'
But the minute you put people into the equation, you introduce 'signal noise'. In engineering parlance, you add friction between the end user and the desired content. Automated algorithms are relatively friction free. Results are ranked with mathematical objectivity, based on universally applicable principles. Queries flow through this channel to connect with the content as determined by the algorithms.
People are smarter and more intuitive than the smartest algorithm, but they're also political. And the reality is, the very segment that Wikia (and Wikipedia) depends most on are those most prone to politics.
Anytime you depend on people to do things out of the 'goodness of their hearts' you attract a certain kind of person. They're community minded, true, but it's very much their definition of community. They can also be elitist, obstinate, territorial and dismissive of those 'outside the circle'. These people tend to show up in the same places, condo strata councils, non profit organizations, PTA's, Church groups, and, online, in forums and on wikis. They have the time to contribute, probably because no one can stand them, so they don't have an active social life outside their chosen cause. I'm not saying everyone that contributes falls into this category, but come on, admit it, everyone reading this now has someone firmly in mind that fits the above description. They get possessive about their online community, which is both a good and a bad thing. With possessiveness comes politics, and signal noise.
Good Intentions, Bad Results
If you need more evidence, look at what is currently happening in the best known communities that depend on online 'Good Samaritans'. On Digg, the Bury Brigade has been publicly acknowledged by Digg founder Kevin Rose. Any story that doesn't meet their criteria for what is interesting gets quickly buried, never to rise to the surface again. That's censorship, and it's just some of the signal noise you can expect when you introduce people.
Wikipedia has come under frequent criticism for the same issue, a handful of community elite (with a decidedly left wing bent) dictating what should and shouldn't be included as entries.
A Growth Bottleneck
But perhaps the biggest challenge for Wikia is scalability. If you put your faith in people as your competitive advantage, you have to be prepared to accept the restrictions that come with that. If Wales is counting on people to help compile the index and rank it, that introduces a potentially significant bottleneck.
Search engines are different than encyclopedias. Encyclopedias are much less dynamic, even when you have an encyclopedia as fluid and ever-growing as Wikipedia. Search engines have to be much more sensitive to new content. A lower traffic entry on Wikipedia could probably go untouched for months at a time and it wouldn't significantly impact the value of that