11 ms·

Google Instant Proves Google's Design Process is Broken

85 points by cornelln 16y ago

bpodgursky 16y ago
The Google News changes introduced earlier this year proved this to me. You'd be challenged to find a single person on the internet who prefers the new Google News format. It took months for the design team to grant even the slightest hint of a concession, letting people switch back to an (albeit uglier and less efficient) two-column format.
- uuilly 16y ago
  I also think their new image search is a disaster. I like the infinite scroll bar (which bing did before them) but I hate how the images pop in and out. The whole experience is jarring.
  - elai 16y ago
    I love the new image search, and find the image popout quite useful when I want to see a image in more detail without the jarring experience of actually going to the page, and not having to click next again to see more images. I don't hover around images unless i want to see detail, i press down on my keyboard to see more.
- txxxxd 16y ago
  Ironically, the type of sweeping changes made to News are exactly the kind of "brilliant design solutions convert people over time" that this article is proposing. If your assertion that "you'd be challenged to find a single person on the internet who prefers the new Google News" is true, than more testing is exactly the right approach - which again is the opposite of the advice in this article.
statictype 16y ago
It's broken only in the sense that they approach design as an engineering problem instead of 'hire a designer and do whatever he says'.
- michael_dorfman 16y ago
  That's a nice looking strawman you've got there. A better approach would be "hire a professional designer, and generate some significant ideas, and then take an engineering approach to testing those ideas." In other words: who should come up with the B's for those A/B tests-- engineers, or designers? (Or both?)
melvinram 16y ago
"A second might matter tremendously to an engineer... that's kind of a silly way of thinking about it." I remember a study done by Google that showed that by shaving off something like 1 second from their load times, they were able to increase the number searches people did substantially, which for Google = millions of dollars in extra revenue from ad clicks. Seconds still matter.
- hanibash 16y ago
  If revenue took precedence over user experience for Google in the past, Google would be very different today.
- txxxxd 16y ago
  Totally agree, this author is missing a critical point: shaving a second of a search doesn't just mean that Google feels faster - it also means that everything else suddenly feels slower in comparison.
  - mbreese 16y ago
    I'm less interested in how fast Google can return a result. (That should be as fast as possible...) What really matters to me is how quickly can I find what I'm looking for and go on my way. This is the only metric that really matters. Total time = search result + visual scanning/deciding Right now, for me, the total time has increased because the time it takes for me to scan the page and figure out if I want to click a link has increased. Also, because you're being bombarded with extra links as you type, I'm more apprehensive about clicking a link, because I'm wondering if there was a better choice using a different variation of the query. I loved the older method of as you type updating the query box, but this is information overload.
    - Kadrith 16y ago
      I just open the links in a new tab, then once I have several open I check to see if they are what I want. If they are not I close them and go back to the search tab to refine what I am looking for. Slightly different topic, but I like Wonder Wheel for searching as well. I start with something intentionally vague if I don't know the exact phrase I want for describing something. Then I select the best choice from the Wonder Wheel.
beej71 16y ago
Fortunately Google Instant has Google Classic built in. You just have to type in a search term and then hit RETURN. I use it this way most of the time out of habit. Personally I think Google Instant is great, and this guy's design suggestions for making it more readable pretty much gut it. But hey, I'm an engineer, so I like efficiency.
- mbreese 16y ago
  It would be Google Classic if the bottom half of the screen would stop changing as I'm typing. Right now, I hate using Google proper... it hurts my eyes. Then again, I pretty much just use the location bar in Chrome to do Google searches, so I'm not affected much, but Instant is a big difference over Classic. I'm all for efficiency, but I prefer clarity.
thomas11 16y ago
I found the article a good read and would have found it somewhat convincing, if it wasn't on a page loaded with everything from a giant photograph to different horizontal bars to all rainbow colors at the bottom, directly under a pink bar.
rxin 16y ago
This is just very different mentality between thinking as a designer vs thinking as an engineer. Google in particular is a data driven company. Most decisions are made by results from experiments. Google is able to do this because the economies of scale. They have tons of data. As a matter of fact, the "experimental traffic" is higher than 100%, indicating multiple levels of experiments are conducted in every search (on average). I remember the quote from a session "A/B testing is for marketers. Engineers run multivariate experiments." In terms of traditional designers: their process is more of an art than science. They think, and trust their instincts. They traditionally operate in a setting that can't afford this kind of experiments. The reality is people are weird and unpredictable. It's statistics and hard science when it's backed by data. The challenge, of course, is that it is easy to iterate incrementally using measurement data. But for disruptive changes, although you could measure it, the change itself (the alternatives) is often not obvious.
- msy 16y ago
  You cannot measure everything. You cannot A/B test frustration, or confusion, or discomfort. You cannot measure slight changes in the way your brand or site is perceived. You can vaguely measure of proxies but they're just that, proxies. The danger of a culture built around worshiping data to the point of ignoring everything else is that you forget that you cannot measure everything, including many of the things that matter most.
  - sokoloff 16y ago
    Partially true. While you can't directly test for frustration or confusion, what you can test for is the important (to your business) effects of that frustration or confusion. Higher bounce rates, lower conversion, changes in dominant navigation path, lower average order value, etc. can be determined from A/B or multi-variate testing, and a lot of companies are much less interested in whether a user is confused than they are in whether any potential confusion is causing a reduction in CR% or AOV. If the article's rant had argued "A/B or MV testing is a hill-climb and incremental changes backed only by test data virtually assures you'll find a local maxima, but only a daring design decision is capable of moving you to a different, higher, hill," I'd be more inclined to agree. As it stands, it reads to me like a designer who is frustrated by not being able to scratch his own personal itches, without regard to the underlying business results: http://www.artlebedev.com/mandership/140/ http://www.artlebedev.com/mandership/140/
hopeless 16y ago
"has your G-mail or Google Reader gotten any easier to use, or less stressful on your eyes? ... No. Just look at the music listings in new, redesigned iTunes." Actually, yes I do find gmail and reader easier to use - actually quote preferable. In contrast, iTunes is one of the most confusing pieces of software I have to use on a daily basis - second only to Lotus Notes. All that is Apple is not great.
- boyakasha 16y ago
  I agree entirely. I'm very happy with Gmail, and switching from Winamp to iTunes just left me wishing that a Mac version of Winamp existed, even after using iTunes for about two years.
- derefr 16y ago
  I've wanted to expel this rant for a while: when you pick it apart, iTunes is basically as great as it can be while still running on Windows as well as OSX (not to say that it's great in any absolute sense.) It has to be monolithic to be multi-platform, and it has to be multi-platform to serve as a life-support system for all the iConsumerElectronics on Windows. If iTunes was OSX-only, it could spray itself all around OSX as a bunch of cute one-screen utilities with clever integration hooks: 1. The App Store would become part of Software Update (which would thus become a general Mac App Store and App Update manager—hopefully buying out AppFresh and giving MacPorts a GUI); 2. Contacts, Bookmarks, Notes, etc., and the transfer of media to the iConsumerElectronics in a friendly, GUIful manner, would all be a part of the iSync utility (yes, that exists—it's the ghetto for synching phones that aren't made by Apple); 3. Podcasts would just be a client program that relies on the same background-downloading daemon that System Updates do, with a modification to read arbitrary feeds, and extract enclosed media files (or torrents!); once downloaded, iSync would just see them and sync them; 4. iTMS would just be a website, which would expose special content types that Safari would know what to do with (audio/x-apple-ringtone = save to the Music/Ringtones folder, etc.); 5. and iTunes would be left to be a music library, consisting in its entirety of Playlists, Genius, and perhaps the Radio (and hooks to send events to Ping, if it likes.) If iTunes was OSX-only, it wouldn't need to know how to burn CDs; it could just allow you to export a playlist as a folder of MP3s, and then integrate audio-CD burning as an option in the OSX Burn Folder menu. If iTunes was OSX-only, it wouldn't need to have sections for TV Shows, Movies, Books, Ringtones; those would just be folders on your hard drive, which iTMS (through Safari) would write to, and iSync would read from. There's a thousand other ways it could be better and slimmer—but, if you'll notice, none of these things could work given the restriction that they have to work on Windows as well.
awongh 16y ago
I guess I'm in the minority, but I totally agree w/ the OP. I think instant works for me about half the time, the other half it's just confusing and messy. I'm hoping that future iterations of instant can bring some polish to the UX, because right now it seems messy and unnecessary. If I was to continue to work as rapidly as it does now I want some more context to the results it's spitting out, like in the 1st design iteration they show.... I feel like they could have hit it out of the park with some real magic, but instead they (b)punted and put out the ugly untested beta. This is the search design for the people who use google to get to their facebook login, and not for those who are truly searching for information.
- panacea 16y ago
  "This is the search design for the people who use google to get to their facebook login" Do you personally even encounter 'Google Instant'? If I'm not using my browser's search bar, I've got my personalized iGoogle page open. They've made changes to the vanilla Google homepage which is most likely, predominantly visited by the sort of people who use Google to get to their facebook login anyway.
  - awongh 16y ago
    I do this sometimes too, which is usually when google instant works for me, like when I've visited a page a bunch of times, but I can't be bothered to bookmark it or save the search in some other way, and I know it will come up easily in the first page of results- I just want to get to a page super fast, and it's not a url per se, but something specific I searched for and need again. I'd say most times I search through my chrome bar, but sometimes I do end up on the google page for whatever reason.
- orangecat 16y ago
  This is the search design for the people who use google to get to their facebook login Even if that's true, there's more of them than there are of us.
boyakasha 16y ago
The central point of the article is that certain people (designers, for lack of a precise term) can use their intuition to get better results than actually testing alternatives; to quote the article: "Testing can only tell you so much -- and it often only reveals that people only like things that are similar to what they've had before. But brilliant design solutions convert people over time, because they're both subtle and ground breaking." The article provides no evidence to support its claim. For all the anecdotes, not a single one describes a situation where the intuitions of "designers" outperformed a solution based on testing alternatives. However, the pro-designer propaganda flies thick and fast, repeatedly implying designers would have made a better solution, or seen the solution as obvious without the need for testing: - "While that solution seems obvious and not particularly elegant" - "Is it just us who find our eyeballs spinning in their sockets...?" - "Obviously, none of these (prototypes) were going to work" - "A little design know-how would have made that obvious" - "(They eventually settled on a blue that is basically the average of all the blues used in hyperlinks across the web. Duh.)" - "Google's 'solution' to providing instant results still seems so primitive and ugly" - "But brilliant design solutions convert people over time, because they're both subtle and ground breaking." - "testing artificially limits the worldview of the people" - "has your G-mail or Google Reader gotten any easier to use, or less stressful on your eyes? Have either of them become a pleasure to look at or play with? No." The author isn't interested in making a coherent argument. The author's interest lies in unashamedly gratifying the readers' sense of importance. It is a website for designers, after all.
- mattmanser 16y ago
  I felt that the central point the article was trying to make, and I thought made it well, is that design is not about testing a set of obvious solutions and then declaring X the winner because it was tested well. Firstly that often results in horrid UX experiences, as Google Instant is. It is also, weak for the following reasons: 1. No innovation happens (which is why Apple came up with the smartphone touch UI and Google just copies it) 2. People don't like change, so will favour things close to what they had before You have fallen straight into the same odd thinking dominating Google, that UI is incremental, testable and easily measurable. You want precise and measurable. That's not design. Jonathan Ives is worth his weight in gold for his design skills as much as Linus Torvalds is worth his weight in gold as a programmer. And we as engineers have to accept that. Google will never come up with a great design with the way it approaches the problem. While I love their products, none of them have ever blown me away because of their elegance or coherence. They only know how to do simplicity and are beginning to forget how to do that too.
zaidf 16y ago
Your entire post seems to be a rant on google testing designs that would be "obvious" for most designers. I am a designer I don't think I agree that some of the things you say are obvious are that obvious. I also believe that line of attack completely ignores all the gold google runs into in the form of non-obvious optimal solutions. So while a majority of tests might return the obvious solution as the optimal one, a minority may return non-obvious optimal solutions. You completely ignore that possibility. With that said, I agree, I hate google Instant. It's a shiny object that may have limited shelf-life.
deleted 16y ago
[deleted]
points 16y ago
What a boring rant. Almost every single point I look at, he's wrong on.
cliffkuang 16y ago
Hey all---I'm the original author of the post, and I wanted to thank you for starting a feisty debate. Really thrilled to see both supporters and critics of my view. One thing I do want to point out tho: The post was really meant to show one, alternative way of thinking. I actually believe that the best results come from some kind of blend of the two approaches. That is what I think Apple is really good at -- and not just making pretty boxes for circuit boards. Also, in the comments I made about a second mattering, it was exactly my point that while seconds are important to engineers, they may or may not be to users. And a focus on actual user stresses is the essence of design. Anyway, thanks for reading and commenting!
- vessenes 16y ago
  Cliff, I totally disagree with your premise. If I put it in the nicest possible way, you are suggesting that engineers can hit some sort of local maxima for their design decisions with testing, but that 'professional' designers can get them out of that with bold, intuitive changes. It's nice to think so. It's certainly a possible difficulty in design: many designers have made their names reworking a tired design into something iconic and revolutionary. (This article about Tag Heuer's digital stopwatch is a great example: http://lenovoblogs.com/designmatters/?p=3748 http://lenovoblogs.com/designmatters/?p=3748). In this case though, I think you're missing it. If you believe design's God is usability, that is that design is about providing function to actual users of the design, then you're on the wrong side of this conversation: the side with only your opinion, arguing against the side with billions of datapoints. Since search is the only way Google makes money, let me ask you a rhetorical question: how long do you think Google Instant will stay turned on if it worsens the user experience? By worsens, I mean that empirically users demonstrate less advertising engagement, or show that they are getting worse search results? So on the seconds conversation: seconds matter to USERS. Seconds only matter to google's ass-kicking advertising engineering team because they matter to users. Google is accidentally leaking some proprietary search information to you in those videos -- they're telling you: ultra-ultra-fast search makes Google more money because users like it better. Where good design is less easy to statistically evaluate, say with Google Buzz or News, this approach falters, but I think picking on Google Instant/Homepage search is choosing exactly the wrong section of the company to complain about: these guys are the absolute best in the business in the search results / advertising world, it's their bread and butter, and they test the hell out of stuff like this before it launches. As to the matter of personal speed preferences, a counterpoint: speed matters to me a lot, and I want to use a search engine like it's an extension of my brain. Case in point: one of the computers on my desktop uses duckduckgo for its search bar. It's noticeably slower than google for searches. Maybe .2 or .3 seconds vs .05 seconds. That's annoying. I notice it every time I search with DDG; it takes me out of the flow of whatever I'm doing that required some searchable information. I would guess that I'm not alone.
acdha 16y ago
A great example of begging the question. The basic assumption is that Google's UIs are unpopular without any evidence that this is true; given the wide popularity among the people I know (even the designers) I think it'd be more accurate to say that Google is successfully producing clean, functional designs and not even attempting web design contest entries. UI is more than looking pretty - gmail is popular because it stays out of the way and lets you focus on content, which it handles quite well; in contrast, iTunes increasingly feels like a train-wreck because they swap icons without fixing any of the quirky behaviours and non-standard interface conventions which users must memorize. It was also a confusing comparison because a UI for managing a tightly-curated, highly-structured list containing on average a few thousand items pre-selected by the user isn't obviously relevant to arbitrarily searching across billions of unknown items.
- cliffkuang 16y ago
  Hi there---Thanks for reading. Actually wasn't begging the question at all, or even contesting the results of the testing. Obviously UI is about more than looking pretty, but I was arguing the the Google Instant experience is a good example of testing producing a result that's far from optimal (and which could be a lot better). As for iTunes, obviously, if you read the piece, I wasn't comparing it to Google. I was comparing the process that produced it -- and how Apple has always gotten better in UI, with each generation, while adding complexity. I don't think Google can really say that for themselves, as the profusion of Google products shows.
  - eds 16y ago
    I just read the article. In it, you assume the product could be better, as you just repeated here ("could be a lot better"). Please explain how that's not begging the question.
    - cliffkuang 16y ago
      That was a premise of the article, that the Instant experience was a failure. Obvs, you can disagree, but that's a starting point, for an article that's mostly concerned with answering "Why"
randallsquared 16y ago
While Google Instant is somewhat annoying to me, and therefore turned off, on my Mac (because of the interaction between it and Safari), I think that the process of throwing any old idea against the wall and seeing if it sticks is a sound one in the usual case. The argument the article is making seems to be that theory is better than experiment; I think this would only be true in the case where experiment has shown over and over to be in line with theory, but UI design is such a young field that I'm strongly skeptical of that claim. Surely there have been enough recent accidental or surprising improvements that experiment with unusual ideas is still worthwhile?
some1else 16y ago
"Before Google Instant, probably the most infamous example of Google's design-by-testing approach was the "41 Blues" --- Google's engineers apparently couldn't decide on two shades of blue for showing search results, so they tested 41 of them to see which attracted the most clicks. (They eventually settled on a blue that is basically the average of all the blues used in hyperlinks across the web. Duh.)" I don't see why this would be subject of cricism. User testing is the one and only validator for good design decisions. Every designer knows this, especially when the decisions closely pertain to User Experience.
uuoc 16y ago
Google instant is the reason why my NoScript whitelist no longer contains an allow javascript rule for google. No javascript, no instant. No instant, no distractions while I'm typing in my search string. The single most important factor Google should be concentrating upon is the quality of the search. If the search returns exactly what I'm looking for at the top of the first page, then I'm happy. I don't care if it took Google 3 seconds longer to produce that quality result. In fact, if by taking 3 seconds longer, the search result quality were to jump by 2x or 3x, that would be much better than trying to flash results in front of me as I type.
- hyperbovine 16y ago
  This approach does not massage one's nerd G-spot as effectively, but you can disable it like a normal person by clicking the link to the right of the search bar.
  - uuoc 16y ago
    Except that I also don't allow google to set cookies either.
aneth 16y ago
If Apple tested a version of the iPhone that had four buttons on the front, it probably would have tested better because one button was unfamiliar to users and would have seemed inadequate. The point here is that sometimes good design requires challenging and educating users - doing the right thing instead of the easy thing. Apple's delight of their users is often when the user realizes he didnt actually need something he thought he did. Of course, I am describing a process involving risk, which can be mitigated through some testing, but will always be somewhat correlated with reward.
- slowpoison 16y ago
  I'm going to wager that the choice of a single button on the iPhone is bad design, esp having experienced both single button and multi-button designs (Android phones). I've designed apps for iPhone and Android and what ends up happening on iPhone apps is that you have to put additional buttons in your app to compensate for the lack of buttons on the iPhone. Take the virtually omnipresent back button in iPhone apps. Most applications need a back button. Hence on the iPhone, they draw this button on the top left corner. I much prefer the Android phones' hard back button, not only because it makes single-handed operation possible, but also because sometimes soft back buttons don't cut it - what do you do when you want to switch to the last app or what if your app lost focus to another app because of some unsolicited event and you want to go back? Same for search button. Now I'm not saying Android got all things right. In fact Android UI, UX is nowhere as pleasant as iPhone's. But, I don't agree that a single button on iPhone was a good decision. I'd like to know if there have been any real-life studies or empirical data on this.
  - aneth 16y ago
    Not every application needs a back button. Virtual interfaces allow the interface to tailor to the need of the application, instead of the other way around. I agree sometimes a back button seems like it would help, but after using iOS products for a long time (this is on an iPad, I don't miss hardware buttons much.
moultano 16y ago
It seems like his complaint is really that Google Instant isn't incremental enough for him. It "tests poorly" with him in exactly the way that a really innovative design should: it requires him to adjust how he uses it.
albertzeyer 16y ago
Funny that the website where this is published has such a crappy design. * Vertical scrollbar at width of 800. * The dotted gray border right to the text is annoying (it flickers on my TFT). * At lot of vertical space wasted on the left. * I don't find it very pretty at a whole. (But that is of course subjective.)

85 points by cornelln 16y ago

The Google News changes introduced earlier this year proved this to me. You'd be challenged to find a single person on the internet who prefers the new Google News format. It took months for the design team to grant even the slightest hint of a concession, letting people switch back to an (albeit uglier and less efficient) two-column format.

uuilly 16y ago
I also think their new image search is a disaster. I like the infinite scroll bar (which bing did before them) but I hate how the images pop in and out. The whole experience is jarring.
- elai 16y ago
  I love the new image search, and find the image popout quite useful when I want to see a image in more detail without the jarring experience of actually going to the page, and not having to click next again to see more images. I don't hover around images unless i want to see detail, i press down on my keyboard to see more.
txxxxd 16y ago
Ironically, the type of sweeping changes made to News are exactly the kind of "brilliant design solutions convert people over time" that this article is proposing. If your assertion that "you'd be challenged to find a single person on the internet who prefers the new Google News" is true, than more testing is exactly the right approach - which again is the opposite of the advice in this article.

statictype 16y ago

It's broken only in the sense that they approach design as an engineering problem instead of 'hire a designer and do whatever he says'.

michael_dorfman 16y ago
That's a nice looking strawman you've got there. A better approach would be "hire a professional designer, and generate some significant ideas, and then take an engineering approach to testing those ideas." In other words: who should come up with the B's for those A/B tests-- engineers, or designers? (Or both?)

melvinram 16y ago

"A second might matter tremendously to an engineer... that's kind of a silly way of thinking about it." I remember a study done by Google that showed that by shaving off something like 1 second from their load times, they were able to increase the number searches people did substantially, which for Google = millions of dollars in extra revenue from ad clicks. Seconds still matter.

hanibash 16y ago
If revenue took precedence over user experience for Google in the past, Google would be very different today.
txxxxd 16y ago
Totally agree, this author is missing a critical point: shaving a second of a search doesn't just mean that Google feels faster - it also means that everything else suddenly feels slower in comparison.
- mbreese 16y ago
  I'm less interested in how fast Google can return a result. (That should be as fast as possible...) What really matters to me is how quickly can I find what I'm looking for and go on my way. This is the only metric that really matters. Total time = search result + visual scanning/deciding Right now, for me, the total time has increased because the time it takes for me to scan the page and figure out if I want to click a link has increased. Also, because you're being bombarded with extra links as you type, I'm more apprehensive about clicking a link, because I'm wondering if there was a better choice using a different variation of the query. I loved the older method of as you type updating the query box, but this is information overload.
  - Kadrith 16y ago
    I just open the links in a new tab, then once I have several open I check to see if they are what I want. If they are not I close them and go back to the search tab to refine what I am looking for. Slightly different topic, but I like Wonder Wheel for searching as well. I start with something intentionally vague if I don't know the exact phrase I want for describing something. Then I select the best choice from the Wonder Wheel.

beej71 16y ago

Fortunately Google Instant has Google Classic built in. You just have to type in a search term and then hit RETURN. I use it this way most of the time out of habit. Personally I think Google Instant is great, and this guy's design suggestions for making it more readable pretty much gut it. But hey, I'm an engineer, so I like efficiency.

mbreese 16y ago
It would be Google Classic if the bottom half of the screen would stop changing as I'm typing. Right now, I hate using Google proper... it hurts my eyes. Then again, I pretty much just use the location bar in Chrome to do Google searches, so I'm not affected much, but Instant is a big difference over Classic. I'm all for efficiency, but I prefer clarity.

thomas11 16y ago

I found the article a good read and would have found it somewhat convincing, if it wasn't on a page loaded with everything from a giant photograph to different horizontal bars to all rainbow colors at the bottom, directly under a pink bar.

rxin 16y ago

This is just very different mentality between thinking as a designer vs thinking as an engineer. Google in particular is a data driven company. Most decisions are made by results from experiments. Google is able to do this because the economies of scale. They have tons of data. As a matter of fact, the "experimental traffic" is higher than 100%, indicating multiple levels of experiments are conducted in every search (on average). I remember the quote from a session "A/B testing is for marketers. Engineers run multivariate experiments." In terms of traditional designers: their process is more of an art than science. They think, and trust their instincts. They traditionally operate in a setting that can't afford this kind of experiments. The reality is people are weird and unpredictable. It's statistics and hard science when it's backed by data. The challenge, of course, is that it is easy to iterate incrementally using measurement data. But for disruptive changes, although you could measure it, the change itself (the alternatives) is often not obvious.

msy 16y ago
You cannot measure everything. You cannot A/B test frustration, or confusion, or discomfort. You cannot measure slight changes in the way your brand or site is perceived. You can vaguely measure of proxies but they're just that, proxies. The danger of a culture built around worshiping data to the point of ignoring everything else is that you forget that you cannot measure everything, including many of the things that matter most.
- sokoloff 16y ago
  Partially true. While you can't directly test for frustration or confusion, what you can test for is the important (to your business) effects of that frustration or confusion. Higher bounce rates, lower conversion, changes in dominant navigation path, lower average order value, etc. can be determined from A/B or multi-variate testing, and a lot of companies are much less interested in whether a user is confused than they are in whether any potential confusion is causing a reduction in CR% or AOV. If the article's rant had argued "A/B or MV testing is a hill-climb and incremental changes backed only by test data virtually assures you'll find a local maxima, but only a daring design decision is capable of moving you to a different, higher, hill," I'd be more inclined to agree. As it stands, it reads to me like a designer who is frustrated by not being able to scratch his own personal itches, without regard to the underlying business results: http://www.artlebedev.com/mandership/140/ http://www.artlebedev.com/mandership/140/

hopeless 16y ago

"has your G-mail or Google Reader gotten any easier to use, or less stressful on your eyes? ... No. Just look at the music listings in new, redesigned iTunes." Actually, yes I do find gmail and reader easier to use - actually quote preferable. In contrast, iTunes is one of the most confusing pieces of software I have to use on a daily basis - second only to Lotus Notes. All that is Apple is not great.

boyakasha 16y ago
I agree entirely. I'm very happy with Gmail, and switching from Winamp to iTunes just left me wishing that a Mac version of Winamp existed, even after using iTunes for about two years.
derefr 16y ago
I've wanted to expel this rant for a while: when you pick it apart, iTunes is basically as great as it can be while still running on Windows as well as OSX (not to say that it's great in any absolute sense.) It has to be monolithic to be multi-platform, and it has to be multi-platform to serve as a life-support system for all the iConsumerElectronics on Windows. If iTunes was OSX-only, it could spray itself all around OSX as a bunch of cute one-screen utilities with clever integration hooks: 1. The App Store would become part of Software Update (which would thus become a general Mac App Store and App Update manager—hopefully buying out AppFresh and giving MacPorts a GUI); 2. Contacts, Bookmarks, Notes, etc., and the transfer of media to the iConsumerElectronics in a friendly, GUIful manner, would all be a part of the iSync utility (yes, that exists—it's the ghetto for synching phones that aren't made by Apple); 3. Podcasts would just be a client program that relies on the same background-downloading daemon that System Updates do, with a modification to read arbitrary feeds, and extract enclosed media files (or torrents!); once downloaded, iSync would just see them and sync them; 4. iTMS would just be a website, which would expose special content types that Safari would know what to do with (audio/x-apple-ringtone = save to the Music/Ringtones folder, etc.); 5. and iTunes would be left to be a music library, consisting in its entirety of Playlists, Genius, and perhaps the Radio (and hooks to send events to Ping, if it likes.) If iTunes was OSX-only, it wouldn't need to know how to burn CDs; it could just allow you to export a playlist as a folder of MP3s, and then integrate audio-CD burning as an option in the OSX Burn Folder menu. If iTunes was OSX-only, it wouldn't need to have sections for TV Shows, Movies, Books, Ringtones; those would just be folders on your hard drive, which iTMS (through Safari) would write to, and iSync would read from. There's a thousand other ways it could be better and slimmer—but, if you'll notice, none of these things could work given the restriction that they have to work on Windows as well.

awongh 16y ago

I guess I'm in the minority, but I totally agree w/ the OP. I think instant works for me about half the time, the other half it's just confusing and messy. I'm hoping that future iterations of instant can bring some polish to the UX, because right now it seems messy and unnecessary. If I was to continue to work as rapidly as it does now I want some more context to the results it's spitting out, like in the 1st design iteration they show.... I feel like they could have hit it out of the park with some real magic, but instead they (b)punted and put out the ugly untested beta. This is the search design for the people who use google to get to their facebook login, and not for those who are truly searching for information.

panacea 16y ago
"This is the search design for the people who use google to get to their facebook login" Do you personally even encounter 'Google Instant'? If I'm not using my browser's search bar, I've got my personalized iGoogle page open. They've made changes to the vanilla Google homepage which is most likely, predominantly visited by the sort of people who use Google to get to their facebook login anyway.
- awongh 16y ago
  I do this sometimes too, which is usually when google instant works for me, like when I've visited a page a bunch of times, but I can't be bothered to bookmark it or save the search in some other way, and I know it will come up easily in the first page of results- I just want to get to a page super fast, and it's not a url per se, but something specific I searched for and need again. I'd say most times I search through my chrome bar, but sometimes I do end up on the google page for whatever reason.
orangecat 16y ago
This is the search design for the people who use google to get to their facebook login Even if that's true, there's more of them than there are of us.

boyakasha 16y ago

The central point of the article is that certain people (designers, for lack of a precise term) can use their intuition to get better results than actually testing alternatives; to quote the article: "Testing can only tell you so much -- and it often only reveals that people only like things that are similar to what they've had before. But brilliant design solutions convert people over time, because they're both subtle and ground breaking." The article provides no evidence to support its claim. For all the anecdotes, not a single one describes a situation where the intuitions of "designers" outperformed a solution based on testing alternatives. However, the pro-designer propaganda flies thick and fast, repeatedly implying designers would have made a better solution, or seen the solution as obvious without the need for testing: - "While that solution seems obvious and not particularly elegant" - "Is it just us who find our eyeballs spinning in their sockets...?" - "Obviously, none of these (prototypes) were going to work" - "A little design know-how would have made that obvious" - "(They eventually settled on a blue that is basically the average of all the blues used in hyperlinks across the web. Duh.)" - "Google's 'solution' to providing instant results still seems so primitive and ugly" - "But brilliant design solutions convert people over time, because they're both subtle and ground breaking." - "testing artificially limits the worldview of the people" - "has your G-mail or Google Reader gotten any easier to use, or less stressful on your eyes? Have either of them become a pleasure to look at or play with? No." The author isn't interested in making a coherent argument. The author's interest lies in unashamedly gratifying the readers' sense of importance. It is a website for designers, after all.

mattmanser 16y ago
I felt that the central point the article was trying to make, and I thought made it well, is that design is not about testing a set of obvious solutions and then declaring X the winner because it was tested well. Firstly that often results in horrid UX experiences, as Google Instant is. It is also, weak for the following reasons: 1. No innovation happens (which is why Apple came up with the smartphone touch UI and Google just copies it) 2. People don't like change, so will favour things close to what they had before You have fallen straight into the same odd thinking dominating Google, that UI is incremental, testable and easily measurable. You want precise and measurable. That's not design. Jonathan Ives is worth his weight in gold for his design skills as much as Linus Torvalds is worth his weight in gold as a programmer. And we as engineers have to accept that. Google will never come up with a great design with the way it approaches the problem. While I love their products, none of them have ever blown me away because of their elegance or coherence. They only know how to do simplicity and are beginning to forget how to do that too.

zaidf 16y ago

Your entire post seems to be a rant on google testing designs that would be "obvious" for most designers. I am a designer I don't think I agree that some of the things you say are obvious are that obvious. I also believe that line of attack completely ignores all the gold google runs into in the form of non-obvious optimal solutions. So while a majority of tests might return the obvious solution as the optimal one, a minority may return non-obvious optimal solutions. You completely ignore that possibility. With that said, I agree, I hate google Instant. It's a shiny object that may have limited shelf-life.

deleted 16y ago

[deleted]

points 16y ago

What a boring rant. Almost every single point I look at, he's wrong on.

cliffkuang 16y ago

Hey all---I'm the original author of the post, and I wanted to thank you for starting a feisty debate. Really thrilled to see both supporters and critics of my view. One thing I do want to point out tho: The post was really meant to show one, alternative way of thinking. I actually believe that the best results come from some kind of blend of the two approaches. That is what I think Apple is really good at -- and not just making pretty boxes for circuit boards. Also, in the comments I made about a second mattering, it was exactly my point that while seconds are important to engineers, they may or may not be to users. And a focus on actual user stresses is the essence of design. Anyway, thanks for reading and commenting!

vessenes 16y ago
Cliff, I totally disagree with your premise. If I put it in the nicest possible way, you are suggesting that engineers can hit some sort of local maxima for their design decisions with testing, but that 'professional' designers can get them out of that with bold, intuitive changes. It's nice to think so. It's certainly a possible difficulty in design: many designers have made their names reworking a tired design into something iconic and revolutionary. (This article about Tag Heuer's digital stopwatch is a great example: http://lenovoblogs.com/designmatters/?p=3748 http://lenovoblogs.com/designmatters/?p=3748). In this case though, I think you're missing it. If you believe design's God is usability, that is that design is about providing function to actual users of the design, then you're on the wrong side of this conversation: the side with only your opinion, arguing against the side with billions of datapoints. Since search is the only way Google makes money, let me ask you a rhetorical question: how long do you think Google Instant will stay turned on if it worsens the user experience? By worsens, I mean that empirically users demonstrate less advertising engagement, or show that they are getting worse search results? So on the seconds conversation: seconds matter to USERS. Seconds only matter to google's ass-kicking advertising engineering team because they matter to users. Google is accidentally leaking some proprietary search information to you in those videos -- they're telling you: ultra-ultra-fast search makes Google more money because users like it better. Where good design is less easy to statistically evaluate, say with Google Buzz or News, this approach falters, but I think picking on Google Instant/Homepage search is choosing exactly the wrong section of the company to complain about: these guys are the absolute best in the business in the search results / advertising world, it's their bread and butter, and they test the hell out of stuff like this before it launches. As to the matter of personal speed preferences, a counterpoint: speed matters to me a lot, and I want to use a search engine like it's an extension of my brain. Case in point: one of the computers on my desktop uses duckduckgo for its search bar. It's noticeably slower than google for searches. Maybe .2 or .3 seconds vs .05 seconds. That's annoying. I notice it every time I search with DDG; it takes me out of the flow of whatever I'm doing that required some searchable information. I would guess that I'm not alone.

acdha 16y ago

A great example of begging the question. The basic assumption is that Google's UIs are unpopular without any evidence that this is true; given the wide popularity among the people I know (even the designers) I think it'd be more accurate to say that Google is successfully producing clean, functional designs and not even attempting web design contest entries. UI is more than looking pretty - gmail is popular because it stays out of the way and lets you focus on content, which it handles quite well; in contrast, iTunes increasingly feels like a train-wreck because they swap icons without fixing any of the quirky behaviours and non-standard interface conventions which users must memorize. It was also a confusing comparison because a UI for managing a tightly-curated, highly-structured list containing on average a few thousand items pre-selected by the user isn't obviously relevant to arbitrarily searching across billions of unknown items.

cliffkuang 16y ago
Hi there---Thanks for reading. Actually wasn't begging the question at all, or even contesting the results of the testing. Obviously UI is about more than looking pretty, but I was arguing the the Google Instant experience is a good example of testing producing a result that's far from optimal (and which could be a lot better). As for iTunes, obviously, if you read the piece, I wasn't comparing it to Google. I was comparing the process that produced it -- and how Apple has always gotten better in UI, with each generation, while adding complexity. I don't think Google can really say that for themselves, as the profusion of Google products shows.
- eds 16y ago
  I just read the article. In it, you assume the product could be better, as you just repeated here ("could be a lot better"). Please explain how that's not begging the question.
  - cliffkuang 16y ago
    That was a premise of the article, that the Instant experience was a failure. Obvs, you can disagree, but that's a starting point, for an article that's mostly concerned with answering "Why"

randallsquared 16y ago

While Google Instant is somewhat annoying to me, and therefore turned off, on my Mac (because of the interaction between it and Safari), I think that the process of throwing any old idea against the wall and seeing if it sticks is a sound one in the usual case. The argument the article is making seems to be that theory is better than experiment; I think this would only be true in the case where experiment has shown over and over to be in line with theory, but UI design is such a young field that I'm strongly skeptical of that claim. Surely there have been enough recent accidental or surprising improvements that experiment with unusual ideas is still worthwhile?

some1else 16y ago

"Before Google Instant, probably the most infamous example of Google's design-by-testing approach was the "41 Blues" --- Google's engineers apparently couldn't decide on two shades of blue for showing search results, so they tested 41 of them to see which attracted the most clicks. (They eventually settled on a blue that is basically the average of all the blues used in hyperlinks across the web. Duh.)" I don't see why this would be subject of cricism. User testing is the one and only validator for good design decisions. Every designer knows this, especially when the decisions closely pertain to User Experience.

uuoc 16y ago

Google instant is the reason why my NoScript whitelist no longer contains an allow javascript rule for google. No javascript, no instant. No instant, no distractions while I'm typing in my search string. The single most important factor Google should be concentrating upon is the quality of the search. If the search returns exactly what I'm looking for at the top of the first page, then I'm happy. I don't care if it took Google 3 seconds longer to produce that quality result. In fact, if by taking 3 seconds longer, the search result quality were to jump by 2x or 3x, that would be much better than trying to flash results in front of me as I type.

hyperbovine 16y ago
This approach does not massage one's nerd G-spot as effectively, but you can disable it like a normal person by clicking the link to the right of the search bar.
- uuoc 16y ago
  Except that I also don't allow google to set cookies either.

aneth 16y ago

If Apple tested a version of the iPhone that had four buttons on the front, it probably would have tested better because one button was unfamiliar to users and would have seemed inadequate. The point here is that sometimes good design requires challenging and educating users - doing the right thing instead of the easy thing. Apple's delight of their users is often when the user realizes he didnt actually need something he thought he did. Of course, I am describing a process involving risk, which can be mitigated through some testing, but will always be somewhat correlated with reward.

slowpoison 16y ago
I'm going to wager that the choice of a single button on the iPhone is bad design, esp having experienced both single button and multi-button designs (Android phones). I've designed apps for iPhone and Android and what ends up happening on iPhone apps is that you have to put additional buttons in your app to compensate for the lack of buttons on the iPhone. Take the virtually omnipresent back button in iPhone apps. Most applications need a back button. Hence on the iPhone, they draw this button on the top left corner. I much prefer the Android phones' hard back button, not only because it makes single-handed operation possible, but also because sometimes soft back buttons don't cut it - what do you do when you want to switch to the last app or what if your app lost focus to another app because of some unsolicited event and you want to go back? Same for search button. Now I'm not saying Android got all things right. In fact Android UI, UX is nowhere as pleasant as iPhone's. But, I don't agree that a single button on iPhone was a good decision. I'd like to know if there have been any real-life studies or empirical data on this.
- aneth 16y ago
  Not every application needs a back button. Virtual interfaces allow the interface to tailor to the need of the application, instead of the other way around. I agree sometimes a back button seems like it would help, but after using iOS products for a long time (this is on an iPad, I don't miss hardware buttons much.

moultano 16y ago

It seems like his complaint is really that Google Instant isn't incremental enough for him. It "tests poorly" with him in exactly the way that a really innovative design should: it requires him to adjust how he uses it.

albertzeyer 16y ago

Funny that the website where this is published has such a crappy design. * Vertical scrollbar at width of 800. * The dotted gray border right to the text is annoying (it flickers on my TFT). * At lot of vertical space wasted on the left. * I don't find it very pretty at a whole. (But that is of course subjective.)