12 ms·

What other types of guerrilla testing have people tried before? This sounds like a fairly witty idea for gathering user interaction information, but I wonder i

What other types of guerrilla testing have people tried before? This sounds like a fairly witty idea for gathering user interaction information, but I wonder if it's actually functional.

jcl 17y ago
I believe there's at least one website review service that is build on top of Mechanical Turk, if that helps any.
- megamark16 17y ago
  Thanks, that's good to know. I know someone once mentioned using Mechanical Turk and that those sometimes ended up being their most loyal users, but I cannot recall the context. Has anyone used these services and what results did you see?
  - daleharvey 17y ago
    kartme posted a good post about changes they made with usability testing. they mentioned uservoice which is on turk if I remember, they mentioned doing a followup post explaining more about what the did during testing. http://kartme.com/blog/8-usability-lessons-learned-members http://kartme.com/blog/8-usability-lessons-learned-members
    - raffi 17y ago
      I like their approach using a variety of tools at different stages of product development to get the job done. I think something our field may benefit from is a guide that explains how to use the different usability services and at what point in the project cycle they should be used.
      - phil_KartMe 17y ago
        Thanks for the mention and compliments. I can only say test, test, test. Another hack to is to do testing in student centers (as opposed to bars) so you can get people on their own computers.
  - jcl 17y ago
    Some related FeedbackArmy-related posts, including some reviews: http://news.ycombinator.com/item?id=785489 http://news.ycombinator.com/item?id=785489 http://news.ycombinator.com/item?id=376956 http://news.ycombinator.com/item?id=376956 http://news.ycombinator.com/item?id=373900 http://news.ycombinator.com/item?id=373900
  - pw 17y ago
    I've tried Feedback Army and was disappointed w/ the results. Not that it isn't a cool idea, but I think the give and take of a good usability test requires that it be done in person. If you watch usability expert Steve Krug doing a demo test (http://network.businessofsoftware.org/video/steve-krug-on-the-least-you http://network.businessofsoftware.org/video/steve-krug-on-th...), you'll see what I mean.
    - raffi 17y ago
      I recently saw a blog post where someone wrote about the feedback they received and they weren't happy. The same person then revised their questions and tried a more task oriented approach and they were much happier. http://www.i-proving.ca/space/David+Jones/blog/2010-01-27_1?showComments=true#startComments http://www.i-proving.ca/space/David+Jones/blog/2010-01-27_1?... Then again, Feedback Army is just a usability testing tool. The right one for the job. I think it's best when a breadth of ideas are needed quickly or you need some ideas about where to start improving your site. You should use other methods to do testing that requires a lot of depth from one or two people. Here are a few more posts where folks have written up their experiences: http://visualwebsiteoptimizer.com/split-testing-blog/how-an-investment-of-10-can-give-you-website-improvement-ideas-worth-thousands-of-dollars/ http://visualwebsiteoptimizer.com/split-testing-blog/how-an-... http://www.mattmazur.com/2010/02/reflections-on-precedens-hackernews-launch/ http://www.mattmazur.com/2010/02/reflections-on-precedens-ha... (p.s. I run Feedback Army :))
Groxx 17y ago
Anything run by a 3rd party strikes me as immediately questionable for accurate testing. They necessarily use people more geared towards using applications, as otherwise they wouldn't be there in the first place, getting paid to do something that most people wouldn't touch. They can certainly provide extensively useful insight, so running through a UI-review company is still a good idea, but can there really be a guerrilla testing company? The idea of guerrilla testing is to get totally untainted results, isn't it?

What other types of guerrilla testing have people tried before? This sounds like a fairly witty idea for gathering user interaction information, but I wonder if it's actually functional.

jcl 17y ago

I believe there's at least one website review service that is build on top of Mechanical Turk, if that helps any.

megamark16 17y ago
Thanks, that's good to know. I know someone once mentioned using Mechanical Turk and that those sometimes ended up being their most loyal users, but I cannot recall the context. Has anyone used these services and what results did you see?
- daleharvey 17y ago
  kartme posted a good post about changes they made with usability testing. they mentioned uservoice which is on turk if I remember, they mentioned doing a followup post explaining more about what the did during testing. http://kartme.com/blog/8-usability-lessons-learned-members http://kartme.com/blog/8-usability-lessons-learned-members
  - raffi 17y ago
    I like their approach using a variety of tools at different stages of product development to get the job done. I think something our field may benefit from is a guide that explains how to use the different usability services and at what point in the project cycle they should be used.
    - phil_KartMe 17y ago
      Thanks for the mention and compliments. I can only say test, test, test. Another hack to is to do testing in student centers (as opposed to bars) so you can get people on their own computers.
- jcl 17y ago
  Some related FeedbackArmy-related posts, including some reviews: http://news.ycombinator.com/item?id=785489 http://news.ycombinator.com/item?id=785489 http://news.ycombinator.com/item?id=376956 http://news.ycombinator.com/item?id=376956 http://news.ycombinator.com/item?id=373900 http://news.ycombinator.com/item?id=373900
- pw 17y ago
  I've tried Feedback Army and was disappointed w/ the results. Not that it isn't a cool idea, but I think the give and take of a good usability test requires that it be done in person. If you watch usability expert Steve Krug doing a demo test (http://network.businessofsoftware.org/video/steve-krug-on-the-least-you http://network.businessofsoftware.org/video/steve-krug-on-th...), you'll see what I mean.
  - raffi 17y ago
    I recently saw a blog post where someone wrote about the feedback they received and they weren't happy. The same person then revised their questions and tried a more task oriented approach and they were much happier. http://www.i-proving.ca/space/David+Jones/blog/2010-01-27_1?showComments=true#startComments http://www.i-proving.ca/space/David+Jones/blog/2010-01-27_1?... Then again, Feedback Army is just a usability testing tool. The right one for the job. I think it's best when a breadth of ideas are needed quickly or you need some ideas about where to start improving your site. You should use other methods to do testing that requires a lot of depth from one or two people. Here are a few more posts where folks have written up their experiences: http://visualwebsiteoptimizer.com/split-testing-blog/how-an-investment-of-10-can-give-you-website-improvement-ideas-worth-thousands-of-dollars/ http://visualwebsiteoptimizer.com/split-testing-blog/how-an-... http://www.mattmazur.com/2010/02/reflections-on-precedens-hackernews-launch/ http://www.mattmazur.com/2010/02/reflections-on-precedens-ha... (p.s. I run Feedback Army :))

Groxx 17y ago

Anything run by a 3rd party strikes me as immediately questionable for accurate testing. They necessarily use people more geared towards using applications, as otherwise they wouldn't be there in the first place, getting paid to do something that most people wouldn't touch. They can certainly provide extensively useful insight, so running through a UI-review company is still a good idea, but can there really be a guerrilla testing company? The idea of guerrilla testing is to get totally untainted results, isn't it?