Search is Broken

If you’ve done a search at your favorite search engine, you’ve probably come across your share of search engine spam. It’s almost impossible to avoid, especially on certain searches. Recently I was looking up information about a rare medical condition on Google. It was impossible to find credible information via the search engine. Well, after fumbling through different keyword combos, I just went to Wikipedia and immediately found what I was looking for.

Status Quo

Where is Google’s innovation in search in the last 2 or 3 years? I know they plan on implementing what they call “universal search” by combining Google Base, Google News and other properties in their search results. In my opinion, this is just going to add to the problem of spam. It may not be in Google’s best interest to fix the search spam problems because most of the spammers use Adsense to monetize there SEO spamming efforts.

Google should add refinement options to there results by Open Directory Project categories, page rank, publish date and related keywords. Having refinement options would do a lot to combat the problem of users trying to figure out the proper keyword combo that will get them relevant non-spammed results.

Humans are Smarter

Along the lines of my previous use of Wikipedia, I’m using del.icio.us now to find stuff on general topics. The great thing about del.icio.us is that every url entry is human generated. Yahoo would be wise to leverage the collective wisdom of the del.icio.us users to help its own search engine. It has a huge opportunity to supplant the search algorithm with its own human-filtered results, and with the user-entered tags, create a great refinement system.

It seems that there will always be problems when companies build products that have no human intervention to discern meaning or intent. It’s a cat and mouse game, and the humans usually figure out a way to out-maneuver the algorithm.

What do think? Is search engine spam unstoppable, is there a better way, or do you think it’s not a problem at all?