Update 30 June 2009: Added a couple of links at the end of this post for additional information from some recent research/experimenting.
Update 19 July 2014: Translated into Swedish by Valeria Aleksandrova. Many thanks!
Web standards is generally a good thing. When done well, it can help to create lean pages, improve markup quality, provide better accessibility and can be easier to maintain (even while supporting IE6!)
One claim often made in the past to promote web standards is that following web standards (using CSS-based layouts, proper use of heading elements in markup, etc) will improve SEO.
Though this might be desirable, it is not the case; most sites on the web don’t follow web standards and most rank very high in their areas. Search engines are not going to penalize the 90+ % of sites that don’t follow web standards; that’s like shooting themselves in the foot!
As I describe in an earlier post on search engine ranking vs indexing while there are technical things one can do to ensure the site gets indexed well (e.g. URL canonicalization, good use of the <title /> element, etc), for ranking it is mostly about having good content that people will link to.
But as Matt Cutts, head of Google’s web spam team notes in this very short video, using CSS-based layouts or table-based layouts will have no bearing on search engine friendliness:
There are many good reasons to follow web standards principles, to use CSS-based layout and progressive enhancement but SEO, unfortunately, is not one of them.
Update 30 June 2009: A couple of decent articles recently talk about similar things:
Pingback: Mobile SERPs - Mobile Search Engine Results Page » Web Standards – a good thing, but won't help with SEO — onenaught.com
You’re right. If search engines start penalizing all those sites which are in the grey area, the internet will suddenly become empty!!
If your code has a lot of HTML or CSS errors then it most likely will have browser rendering issues. The pages can look just horrible.
Nothing will scare off a visitor quicker than a website that looks bad and even worse, is completely discombobulated.
I agree, once search engines starts penilizing webmasters because of css will be bad
The whole search engine optimization industry has gone mad in the last few years with people and companies offering all sorts of underhand tactics to try and play the system. Fortunately these kind of techniques never produce lasting results.