4 ms·

I wonder if one couldn't put together a software package that Mechanical Turk participants could use to generate similar (if lower-quality) data?

by jcl 18y ago

run4yourlives 18y ago
Why? The main deliverable in user testing is quality. It's much better to have 5 detailed pieces of data than 100 that don't say anything at all.
- jcl 18y ago
  A smaller sample means a greater chance that it's not representative. If 10% of the population is colorblind (or left-handed, etc), you stand a greater-than-50% chance that a random group of five people won't identify a problem for this segment. A larger number of available testers means more fresh eyeballs, hence a better chance of repeat queries yielding accurate first-impression data. Feedback from people whose job it is to evaluate websites may not be representative of the average site visitor. They may be more familiar with common site layouts or have specific pet peeves.
  - iamwil 18y ago
    Probably depends on what you're looking for. When you're just starting out a site, just a couple users will point out all sorts of user interface weaknesses that'll keep you busy fixing for a long while.
  - deleted 18y ago
    [deleted]
- tlrobinson 18y ago
  I think there's room for both. Turkers might simulate the casual browsers who happen upon your site. There could be valuable data buried in what they do in a short period of time.
deleted 18y ago
[deleted]
raffi 18y ago
FYI, I've already done this. I started the project last week and am still testing it (it works though). http://www.feedbackarmy.com/ http://www.feedbackarmy.com/ Visit the site, give it your URL and some specifics to focus on, dump a credit card number into paypal, and subscribe to an RSS feed for the results.
- jcl 18y ago
  Ha, that's great. I was thinking more along the lines of a piece of software that runs alongside the Turker's browsing session and keeps track of what they see and what they do with the mouse cursor, perhaps augmented by a video of the user's face taken by webcam, if available. But even simple feedback -- as you have built -- is extremely useful.

I wonder if one couldn't put together a software package that Mechanical Turk participants could use to generate similar (if lower-quality) data?

by jcl 18y ago

I wonder if one couldn't put together a software package that Mechanical Turk participants could use to generate similar (if lower-quality) data?

run4yourlives 18y ago

Why? The main deliverable in user testing is quality. It's much better to have 5 detailed pieces of data than 100 that don't say anything at all.

jcl 18y ago
A smaller sample means a greater chance that it's not representative. If 10% of the population is colorblind (or left-handed, etc), you stand a greater-than-50% chance that a random group of five people won't identify a problem for this segment. A larger number of available testers means more fresh eyeballs, hence a better chance of repeat queries yielding accurate first-impression data. Feedback from people whose job it is to evaluate websites may not be representative of the average site visitor. They may be more familiar with common site layouts or have specific pet peeves.
- iamwil 18y ago
  Probably depends on what you're looking for. When you're just starting out a site, just a couple users will point out all sorts of user interface weaknesses that'll keep you busy fixing for a long while.
- deleted 18y ago
  [deleted]
tlrobinson 18y ago
I think there's room for both. Turkers might simulate the casual browsers who happen upon your site. There could be valuable data buried in what they do in a short period of time.

deleted 18y ago

[deleted]

raffi 18y ago

FYI, I've already done this. I started the project last week and am still testing it (it works though). http://www.feedbackarmy.com/ http://www.feedbackarmy.com/ Visit the site, give it your URL and some specifics to focus on, dump a credit card number into paypal, and subscribe to an RSS feed for the results.

jcl 18y ago
Ha, that's great. I was thinking more along the lines of a piece of software that runs alongside the Turker's browsing session and keeps track of what they see and what they do with the mouse cursor, perhaps augmented by a video of the user's face taken by webcam, if available. But even simple feedback -- as you have built -- is extremely useful.