6 ms·

"real data" That's a stretch. Look how many people are in some of these studies. Well, you could look if he bothered to tell you, which he rarely does, becau

"real data" That's a stretch. Look how many people are in some of these studies. Well, you could look if he bothered to tell you, which he rarely does, because I'm guessing these aren't actually "studies" but are actually interviews with some soundbytes extracted to back up preconceived notions. I'm not saying he's always wrong, that would entail gathering and presenting actual data, but he's never given me any good reason to think he's right rather than just a pundit with first-mover advantage.

SoftwareMaven 15y ago
Do you have sites/people you prefer to useit? I'm always looking for good UX info, and would love to find gems I'm missing.
- efsavage 15y ago
  Not really, I'm not sure what I would even classify as "good UX info". There are things that work, and things that don't. Things that look good and things that don't. Things that feel good and things that don't. Things that communicate themselves well and things that don't. UX is supposedly looking for the intersection of all of those axes but they're all very subjective. If you want to be good at UX, never ever ever go to sites like useit, you won't learn anything helpful, and you'll probably learn something harmful. You just need to get out there and experience what's available and think very hard about what's right and wrong with it. There are no rules, no laws, and the lessons you learn are difficult to translate to anything universal. The moment you start thinking about "85% of ipad users like blue buttons" or "People can only remember 7 things" you lose. Tomorrow's successful UI is going to break today's rules. EDIT: I should clarify, I don't mean to say that I don't peruse portfolio sites and have a number of blog subscriptions, but none of them I'd feel confident recommending. The quality of such sites varies widely over time so you kind of have to just let it all meld together and draw your own conclusions.
  - daleharvey 15y ago
    I have never said this on hacker news but it definitely still follows the "to your face" rule, This is terrible terrible advice. A lot of neilsens advice certainly isnt universal, however the main thing to take away from neilsen is not the conclusions but the methodology in coming to those conclusions. UX is not voodoo, there are certainly parts of it that are more subjective than others however if 5 out of 10 people have a problem performing an actions in your app then you have a problem. Neilsen provides good methodologies for which you can start performing these tests yourself.
    - efsavage 15y ago
      Not sure what the "to your face" rule is? His advice is almost _all_ conclusions phrased as if to be applied universally. Example, from the PDF: "If you have a lot of content (such as product information) to display, use a separate page rather than a modal view". That is not a friendly tip or even a very constrained piece of advice. It doesn't say "consider using" or "try", it says "use". If you think I'm being nitpicky you haven't been subjected to project owners sending you emails with quips like that, expecting you defend yourself against the laws set down by the experts. It may actually make sense in many cases, but there are certainly cases where it's not true. As for methodology, he offers very little insight and no new ideas. You can spend an hour reading Steve Krug's book and understand the mechanics of usability testing. The rest is common sense and experience. UX is not voodoo, it's an art. Usability is a holistic quality, not an aggregation like uptime or code coverage or conversion rates. It's pretty widely accepted that the Tivo is a hallmark of usability. Why? Can you answer that only in statistics? Can you then justify why you picked those particular statistics and why they determine "usability"? Are your results reproducible? I'm not saying you actually need to run through the full scientific method on usability, I'm just saying that running through only parts of it to sound fancy benefits nobody. You need to learn when things are just better, and you need to admit that it's ultimately an opinion, an idea, an artform.
      - SoftwareMaven 15y ago
        The "to your face" rule is "don't put anything in a comment you wouldn't say to a person's face".
ugh 15y ago
Not many people are needed to identify usability problems. If you want to quantify the effects you indeed need many, many people but Nielsen doesn’t actually do that. Three, four, five people are plenty for those kinds of studies.
- efsavage 15y ago
  Not many people are needed to test _specific_ problems with _specific_ environments/programs/features. 24 of 26 people not being able to figure out how to sign up for hacker news would be a clearly identified problem. Drawing overarching conclusions about the entire ipad ecosystem from 16 out of 50 million users on 26 apps out of ~100,000 and providing no methodology, no data? That's not enough to distill into "ipad usability" for an entire year. He'd have far more credibility if he simply said "I don't like this feature, and after asking several people, I know I'm not the only one. Here's why I don't like it, and here's what I think can be done about it."
  - ugh 15y ago
    He sits them in front of computer and looks what they are doing. That would be a terrible way of finding specific problems. It’s a great way of searching for problems. I suggest you look into the method Nielsen actually uses.
    - efsavage 15y ago
      I know how he does it, that's my point. Would you be able to make useful, high-level recommendations to the auto industry after riding in the back seat of 26 drivers for an hour or so? Do you think it matters what kind of car you're driving? What kind of roads you're driving on? What time of day it is? What state or country you're in? Of course it does. If your conclusion is that "driving on the PCH in a convertible is fun" then, no, you don't need many participants or any scientific rigor. But if you're say things like "the roads aren't wide enough" or "radios are too loud" or "signs aren't big enough" then you're going to need to back that up with some context and some data. My beef here is that people who don't want to really think about this stuff but want to seem like they do will read articles like his, which draw conclusions of the latter type described above, and spout them off as authoritative science, because that's how it's presented by Jakob Nielsen, Ph. D. This confuses clients and teams and ends up having people defend their decisions against a ghost of a misinterpretion of a supposed rule, and ultimately yields an inferior product. I've seen his articles quoted verbatim by designers and clients alike, almost always solely to justify their own opinion, but with the weight of his authority.
ColinDabritz 15y ago
In this case he has been very explicit about the number and types of people in the study. From the article: "In total, 16 iPad users participated in the new study. Half were men, half were women. The age distribution was fairly even for 14 users between the ages of 21–50 years; we also had 2 users older than 50. Occupations spanned the gamut, from personal chef to realtor to vice president of human resources. " I think the key is not to think in terms of 'statistically significant' as in a formal scientific study of a single fact, but in terms of 'useful feedback from real users.'

SoftwareMaven 15y ago

Do you have sites/people you prefer to useit? I'm always looking for good UX info, and would love to find gems I'm missing.

efsavage 15y ago
Not really, I'm not sure what I would even classify as "good UX info". There are things that work, and things that don't. Things that look good and things that don't. Things that feel good and things that don't. Things that communicate themselves well and things that don't. UX is supposedly looking for the intersection of all of those axes but they're all very subjective. If you want to be good at UX, never ever ever go to sites like useit, you won't learn anything helpful, and you'll probably learn something harmful. You just need to get out there and experience what's available and think very hard about what's right and wrong with it. There are no rules, no laws, and the lessons you learn are difficult to translate to anything universal. The moment you start thinking about "85% of ipad users like blue buttons" or "People can only remember 7 things" you lose. Tomorrow's successful UI is going to break today's rules. EDIT: I should clarify, I don't mean to say that I don't peruse portfolio sites and have a number of blog subscriptions, but none of them I'd feel confident recommending. The quality of such sites varies widely over time so you kind of have to just let it all meld together and draw your own conclusions.
- daleharvey 15y ago
  I have never said this on hacker news but it definitely still follows the "to your face" rule, This is terrible terrible advice. A lot of neilsens advice certainly isnt universal, however the main thing to take away from neilsen is not the conclusions but the methodology in coming to those conclusions. UX is not voodoo, there are certainly parts of it that are more subjective than others however if 5 out of 10 people have a problem performing an actions in your app then you have a problem. Neilsen provides good methodologies for which you can start performing these tests yourself.
  - efsavage 15y ago
    Not sure what the "to your face" rule is? His advice is almost _all_ conclusions phrased as if to be applied universally. Example, from the PDF: "If you have a lot of content (such as product information) to display, use a separate page rather than a modal view". That is not a friendly tip or even a very constrained piece of advice. It doesn't say "consider using" or "try", it says "use". If you think I'm being nitpicky you haven't been subjected to project owners sending you emails with quips like that, expecting you defend yourself against the laws set down by the experts. It may actually make sense in many cases, but there are certainly cases where it's not true. As for methodology, he offers very little insight and no new ideas. You can spend an hour reading Steve Krug's book and understand the mechanics of usability testing. The rest is common sense and experience. UX is not voodoo, it's an art. Usability is a holistic quality, not an aggregation like uptime or code coverage or conversion rates. It's pretty widely accepted that the Tivo is a hallmark of usability. Why? Can you answer that only in statistics? Can you then justify why you picked those particular statistics and why they determine "usability"? Are your results reproducible? I'm not saying you actually need to run through the full scientific method on usability, I'm just saying that running through only parts of it to sound fancy benefits nobody. You need to learn when things are just better, and you need to admit that it's ultimately an opinion, an idea, an artform.
    - SoftwareMaven 15y ago
      The "to your face" rule is "don't put anything in a comment you wouldn't say to a person's face".

ugh 15y ago

Not many people are needed to identify usability problems. If you want to quantify the effects you indeed need many, many people but Nielsen doesn’t actually do that. Three, four, five people are plenty for those kinds of studies.

efsavage 15y ago
Not many people are needed to test _specific_ problems with _specific_ environments/programs/features. 24 of 26 people not being able to figure out how to sign up for hacker news would be a clearly identified problem. Drawing overarching conclusions about the entire ipad ecosystem from 16 out of 50 million users on 26 apps out of ~100,000 and providing no methodology, no data? That's not enough to distill into "ipad usability" for an entire year. He'd have far more credibility if he simply said "I don't like this feature, and after asking several people, I know I'm not the only one. Here's why I don't like it, and here's what I think can be done about it."
- ugh 15y ago
  He sits them in front of computer and looks what they are doing. That would be a terrible way of finding specific problems. It’s a great way of searching for problems. I suggest you look into the method Nielsen actually uses.
  - efsavage 15y ago
    I know how he does it, that's my point. Would you be able to make useful, high-level recommendations to the auto industry after riding in the back seat of 26 drivers for an hour or so? Do you think it matters what kind of car you're driving? What kind of roads you're driving on? What time of day it is? What state or country you're in? Of course it does. If your conclusion is that "driving on the PCH in a convertible is fun" then, no, you don't need many participants or any scientific rigor. But if you're say things like "the roads aren't wide enough" or "radios are too loud" or "signs aren't big enough" then you're going to need to back that up with some context and some data. My beef here is that people who don't want to really think about this stuff but want to seem like they do will read articles like his, which draw conclusions of the latter type described above, and spout them off as authoritative science, because that's how it's presented by Jakob Nielsen, Ph. D. This confuses clients and teams and ends up having people defend their decisions against a ghost of a misinterpretion of a supposed rule, and ultimately yields an inferior product. I've seen his articles quoted verbatim by designers and clients alike, almost always solely to justify their own opinion, but with the weight of his authority.

ColinDabritz 15y ago

In this case he has been very explicit about the number and types of people in the study. From the article: "In total, 16 iPad users participated in the new study. Half were men, half were women. The age distribution was fairly even for 14 users between the ages of 21–50 years; we also had 2 users older than 50. Occupations spanned the gamut, from personal chef to realtor to vice president of human resources. " I think the key is not to think in terms of 'statistically significant' as in a formal scientific study of a single fact, but in terms of 'useful feedback from real users.'