A recent episode of the Big Web Show, 'Designing with Data' touched on this topic and it got me thinking.
In the episode Zeldman mentions Doug Bowman's famous '41 shades of blue' experience at Google. For those unfamiliar with the story, Bowman was designer at Google, and on one occasion, the engineers challenged Bowman's colour choice by testing 41 shades of blue, suggesting that the data would yield a better blue than his.
So is testing 41 shades of blue really a bad idea? My opinion is no. In fact, I think testing 41 shades of blue is exactly what the internet does best.
Optimization, analytics, A/B testing are the internets' biggest advantages over traditional software platforms (iOS/Android/Mac/Windows). When you build a native desktop/mobile application, you can't A/B test your software. Most of the time, you can't even tell when your software crashes someone's computer, whereas with internet-based applications, you not only know when your software crashes, but also why -- just look at the log files.
Internet-based applications turn every HTTP request into an opportunity to improve your software. On the web, we use analytics constantly. We have information about everything from page-views, click-through rates, heat-maps, search terms, keywords, load-times etc. Traditional software platforms have none of that. So why ignore this advantage?
I believe it's a good idea to use all this data to test and optimize our applications. However, notice that I haven't used the word 'design' in my argument yet. I've used the words 'test' and 'optimize'. I believe optimizing your application will make it better, but, as with polishing a turd, it still won't make it great.