4 ms·

UserTesting.com - 15 minute video of a user testing your site for $19

90 points by pchristensen 18y ago

run4yourlives 18y ago
I really like this idea, and I'll probably use this for my own site currently in development. Have you considered selling an embeddable recorder that people can use right on their site? Why or why not? i.e. I pay you $xx for a google analytics style plugin that I can embed and have any user that wants to provide feedback... Just trying to understand why you chose to implement as it currently stands. Once again, really nice.
picnichouse 18y ago
I've used it - really helpful. Wish they would send their testers eye tracking monitors - that would rock.
- robg 18y ago
  Agreed, but do they not show the cursor? Also, do they have a "more users, fewer minutes" option? I know I'd rather have 15 users for one minute each, or seven for two minutes each. After that, most users will have decided to stay or flee any ways. It would be good to get a sense of that early decision-making, especially across a statistically-reliable sample.
  - kevintwohy 18y ago
    Tried it out -- shows the user's cursor fine in my experience.
    - robg 18y ago
      Cool. They could just give their testers instructions to use the cursor as a pointer to where they're looking. Or you could put that in your instructions to them. It's a decent enough proxy for eye movements any ways.
      - laut 18y ago
        Have you seen footage from an eye tracking session? A user can't explain or point his mouse to where he is looking. The eyes move quickly and somewhat unconsciously.
staunch 18y ago
I've used it and it is good. Highly recommended. The videos you get aren't super great quality but they're useful and the price is right. The only complaint I have is that I'm a tad jealous I didn't start this site.
- jcl 18y ago
  I wonder if one couldn't put together a software package that Mechanical Turk participants could use to generate similar (if lower-quality) data?
  - run4yourlives 18y ago
    Why? The main deliverable in user testing is quality. It's much better to have 5 detailed pieces of data than 100 that don't say anything at all.
    - jcl 18y ago
      A smaller sample means a greater chance that it's not representative. If 10% of the population is colorblind (or left-handed, etc), you stand a greater-than-50% chance that a random group of five people won't identify a problem for this segment. A larger number of available testers means more fresh eyeballs, hence a better chance of repeat queries yielding accurate first-impression data. Feedback from people whose job it is to evaluate websites may not be representative of the average site visitor. They may be more familiar with common site layouts or have specific pet peeves.
      - iamwil 18y ago
        Probably depends on what you're looking for. When you're just starting out a site, just a couple users will point out all sorts of user interface weaknesses that'll keep you busy fixing for a long while.
      - deleted 18y ago
        [deleted]
    - 18y ago
cabalamat 18y ago
I like the idea. I'll probably use it at some point.
ericb 18y ago
It's a cool idea. I'm not sure that selling mainly on price is a good move business-wise, though.
- Prrometheus 18y ago
  There would probably be demand for a more expensive, higher-quality service. Someone else in this thread mentioned adding eye tracking. It also might be useful to add multiple trials from random individuals and create some sort of aggregate statistics. These are both things that you could charge a large premium for. What is cool about this site from a business perspective is that it is a low-capital, low-cost way to do things with just one person, much like girlinyourshirt.com
wesley 18y ago
I love it, will definitely use this. But please add an affiliate program.
- ckinnan 18y ago
  They offer a white label program: Can our usability firm get a white label version of your service? Yes, you can resell it to your clients with your logo (not UserTesting.com) so you can use your brand and set your pricing. Email us at resellerusertesting.com and we'll get you started.
bprater 18y ago
I was curious how much the reviewers got. A little digging reveals that they get $10 per review.
plaggypig 18y ago
Can anybody share a video session that they themselves commissioned? I'd like to see how the quality of feedback compares with the "official" demo of tripadvisor. P.S. Somebody else mentioned eye tracking - I think commercial solutions for this are pretty expensive, but I've seen a few hobbyists roll their own using low cost webcams, though I'm not sure how they fair in terms of accuracy. P.P.S. And somebody else mentioned just embedding a screen grabber directly onto your website - I'm sure it's achievable but with a browser extension, i.e. Mozilla. I'll have a play around and see what I can come up with.
- bemmu 18y ago
  It's possible from Javascript without extensions. I did a prototype of this, and while streaming the mouse x&y coords to a server was trivial, the hard part was making sure that what the user is seeing is the same layout I'm seeing myself. For example if the user has a larger font size, then the layout might be different, so those coordinates would not mean what I think they mean. Faced with this I abandoned my own project on this, but I noticed that sites like http://robotreplay.com/ http://robotreplay.com/ have appeared, not sure how they dealt with that issue.
jasonlbaptiste 18y ago
How do you guys think they find testers? It's specific to demographic and such, so i think it's beyond mechanical turk?
sachinag 18y ago
We've used it in the past, and will in the future - BUT this is not rigorous usability testing of the sort that you'll get in the lab. They're self-selected and they don't do well with loosely defined objectives (even though they tend to be North American native English speakers). If you have access to a Mac and can recruit your own volunteers (college campus, etc), I'd recommend getting Silverback - http://silverbackapp.com/ http://silverbackapp.com/ . Not only do you get the screen capture, you also get to see frustration/delight on users' faces. Plus, it's cheaper if you plan on doing four or more tests.
nazgulnarsil 18y ago
I'll do this for $10. :p
davidw 18y ago
"Don't Make Me Think" ( http://www.squeezedbooks.com/book/show/9/dont-make-me-think-a-common-sense-approach-to-web-usability-2nd-edition http://www.squeezedbooks.com/book/show/9/dont-make-me-think-... ) has a good section on cheap usability testing.
simplegeek 18y ago
It's a cool idea, but I will be reluctant to use it. Sole reason being, the method they select personnas to do usability testing isn't transparent and I'm not sure how much useful feedback will be from those personnas.

90 points by pchristensen 18y ago

I really like this idea, and I'll probably use this for my own site currently in development. Have you considered selling an embeddable recorder that people can use right on their site? Why or why not? i.e. I pay you $xx for a google analytics style plugin that I can embed and have any user that wants to provide feedback... Just trying to understand why you chose to implement as it currently stands. Once again, really nice.

picnichouse 18y ago

I've used it - really helpful. Wish they would send their testers eye tracking monitors - that would rock.

robg 18y ago
Agreed, but do they not show the cursor? Also, do they have a "more users, fewer minutes" option? I know I'd rather have 15 users for one minute each, or seven for two minutes each. After that, most users will have decided to stay or flee any ways. It would be good to get a sense of that early decision-making, especially across a statistically-reliable sample.
- kevintwohy 18y ago
  Tried it out -- shows the user's cursor fine in my experience.
  - robg 18y ago
    Cool. They could just give their testers instructions to use the cursor as a pointer to where they're looking. Or you could put that in your instructions to them. It's a decent enough proxy for eye movements any ways.
    - laut 18y ago
      Have you seen footage from an eye tracking session? A user can't explain or point his mouse to where he is looking. The eyes move quickly and somewhat unconsciously.

staunch 18y ago

I've used it and it is good. Highly recommended. The videos you get aren't super great quality but they're useful and the price is right. The only complaint I have is that I'm a tad jealous I didn't start this site.

jcl 18y ago
I wonder if one couldn't put together a software package that Mechanical Turk participants could use to generate similar (if lower-quality) data?
- run4yourlives 18y ago
  Why? The main deliverable in user testing is quality. It's much better to have 5 detailed pieces of data than 100 that don't say anything at all.
  - jcl 18y ago
    A smaller sample means a greater chance that it's not representative. If 10% of the population is colorblind (or left-handed, etc), you stand a greater-than-50% chance that a random group of five people won't identify a problem for this segment. A larger number of available testers means more fresh eyeballs, hence a better chance of repeat queries yielding accurate first-impression data. Feedback from people whose job it is to evaluate websites may not be representative of the average site visitor. They may be more familiar with common site layouts or have specific pet peeves.
    - iamwil 18y ago
      Probably depends on what you're looking for. When you're just starting out a site, just a couple users will point out all sorts of user interface weaknesses that'll keep you busy fixing for a long while.
    - deleted 18y ago
      [deleted]
  - 18y ago

cabalamat 18y ago

I like the idea. I'll probably use it at some point.

ericb 18y ago

It's a cool idea. I'm not sure that selling mainly on price is a good move business-wise, though.

Prrometheus 18y ago
There would probably be demand for a more expensive, higher-quality service. Someone else in this thread mentioned adding eye tracking. It also might be useful to add multiple trials from random individuals and create some sort of aggregate statistics. These are both things that you could charge a large premium for. What is cool about this site from a business perspective is that it is a low-capital, low-cost way to do things with just one person, much like girlinyourshirt.com

wesley 18y ago

I love it, will definitely use this. But please add an affiliate program.

ckinnan 18y ago
They offer a white label program: Can our usability firm get a white label version of your service? Yes, you can resell it to your clients with your logo (not UserTesting.com) so you can use your brand and set your pricing. Email us at resellerusertesting.com and we'll get you started.

bprater 18y ago

I was curious how much the reviewers got. A little digging reveals that they get $10 per review.

plaggypig 18y ago

Can anybody share a video session that they themselves commissioned? I'd like to see how the quality of feedback compares with the "official" demo of tripadvisor. P.S. Somebody else mentioned eye tracking - I think commercial solutions for this are pretty expensive, but I've seen a few hobbyists roll their own using low cost webcams, though I'm not sure how they fair in terms of accuracy. P.P.S. And somebody else mentioned just embedding a screen grabber directly onto your website - I'm sure it's achievable but with a browser extension, i.e. Mozilla. I'll have a play around and see what I can come up with.

bemmu 18y ago
It's possible from Javascript without extensions. I did a prototype of this, and while streaming the mouse x&y coords to a server was trivial, the hard part was making sure that what the user is seeing is the same layout I'm seeing myself. For example if the user has a larger font size, then the layout might be different, so those coordinates would not mean what I think they mean. Faced with this I abandoned my own project on this, but I noticed that sites like http://robotreplay.com/ http://robotreplay.com/ have appeared, not sure how they dealt with that issue.

jasonlbaptiste 18y ago

How do you guys think they find testers? It's specific to demographic and such, so i think it's beyond mechanical turk?

sachinag 18y ago

We've used it in the past, and will in the future - BUT this is not rigorous usability testing of the sort that you'll get in the lab. They're self-selected and they don't do well with loosely defined objectives (even though they tend to be North American native English speakers). If you have access to a Mac and can recruit your own volunteers (college campus, etc), I'd recommend getting Silverback - http://silverbackapp.com/ http://silverbackapp.com/ . Not only do you get the screen capture, you also get to see frustration/delight on users' faces. Plus, it's cheaper if you plan on doing four or more tests.

nazgulnarsil 18y ago

I'll do this for $10. :p

davidw 18y ago

"Don't Make Me Think" ( http://www.squeezedbooks.com/book/show/9/dont-make-me-think-a-common-sense-approach-to-web-usability-2nd-edition http://www.squeezedbooks.com/book/show/9/dont-make-me-think-... ) has a good section on cheap usability testing.

simplegeek 18y ago

It's a cool idea, but I will be reluctant to use it. Sole reason being, the method they select personnas to do usability testing isn't transparent and I'm not sure how much useful feedback will be from those personnas.