Human vs. Algorithm
As a web designer, I have to deal with SEO (Search Engine Optimization) a lot. (between you and me, a lot more then I would like to). My opinion on SEO is that it is 99% best practices, good marketing and good content. The other 1% percent is the gray area that takes up a larger percentage of my time. SEO is important, but it’s not some magic elixir that is the answer to all your site’s problems. I try to design for human users and not for Googlebot.
Search is Broken
If you’ve done a search at your favorite search engine, you’ve probably come across your share of search engine spam. It’s almost impossible to avoid, especially on certain searches. Recently I was looking up information about a rare medical condition on Google. It was impossible to find credible information via the search engine. Well, after fumbling through different keyword combos, I just went to Wikipedia and immediately found what I was looking for.
Status Quo
Where is Google’s innovation in search in the last 2 or 3 years? I know they plan on implementing what they call “universal search” by combining Google Base, Google News and other properties in their search results. In my opinion, this is just going to add to the problem of spam. It may not be in Google’s best interest to fix the search spam problems because most of the spammers use Adsense to monetize there SEO spamming efforts.
Google should add refinement options to there results by Open Directory Project categories, page rank, publish date and related keywords. Having refinement options would do a lot to combat the problem of users trying to figure out the proper keyword combo that will get them relevant non-spammed results.
Humans are Smarter
Along the lines of my previous use of Wikipedia, I’m using del.icio.us now to find stuff on general topics. The great thing about del.icio.us is that every url entry is human generated. Yahoo would be wise to leverage the collective wisdom of the del.icio.us users to help its own search engine. It has a huge opportunity to supplant the search algorithm with its own human-filtered results, and with the user-entered tags, create a great refinement system.
It seems that there will always be problems when companies build products that have no human intervention to discern meaning or intent. It’s a cat and mouse game, and the humans usually figure out a way to out-maneuver the algorithm.
What do think? Is search engine spam unstoppable, is there a better way, or do you think it’s not a problem at all?
→ Jeff @ August 2nd, 2007 at 3:44 pm
You’re right and you’re wrong.
The balance is the ultimate. What we need is algorithms that interpret human input without the human knowing. Digg is what you speak of, but it’s ridiculously flawed in the opposite way.
I’ve been working on such algorithms on one of my more popular social sites, and I’m pleased to say that without a math degree I’ve made significant headway, and Google search results prove it - as my site comes up second for a 3-letter word. Zero SEO — it’s all because of the users. :)
→ Rik Catlow @ August 6th, 2007 at 6:11 am
Yes, there needs to be a balance. All human driven products won’t be spam free. There are ways for humans to work as a unit to disrupt systems and produce spam. There needs to be a combo, but the core idea here is computers should not be used to discern meaning or intent of a human input. Computers are good at picking up patterns and should be used as a first line of defense, not the be all end all.
→ Joshua @ August 21st, 2007 at 4:31 am
I agree with you on using human sites for searching.
My queries in google have evolved from ‘camera I would like to buy’ to:
site:digg.com ‘camera I would like to buy’
site:del.icio.us ‘camera I would like to buy’
site:search.yahoo.com ‘camera I would like to buy’ ;-) ;-)
well. ok. you get my point.
→ mike pan @ September 20th, 2007 at 11:21 pm
Site without user moderation will always be prone to spam, one way or another, this is why wikipedia, digg, and del.icio.us are always a better source of information than a simple google search.