6 ms·
The $5 Guerrilla User Test
- megamark16 17y agoWhat other types of guerrilla testing have people tried before? This sounds like a fairly witty idea for gathering user interaction information, but I wonder if it's actually functional.
- jcl 17y agoI believe there's at least one website review service that is build on top of Mechanical Turk, if that helps any.
- megamark16 17y agoThanks, that's good to know. I know someone once mentioned using Mechanical Turk and that those sometimes ended up being their most loyal users, but I cannot recall the context. Has anyone used these services and what results did you see?
- daleharvey 17y agokartme posted a good post about changes they made with usability testing. they mentioned uservoice which is on turk if I remember, they mentioned doing a followup post explaining more about what the did during testing. http://kartme.com/blog/8-usability-lessons-learned-members http://kartme.com/blog/8-usability-lessons-learned-members
- raffi 17y agoI like their approach using a variety of tools at different stages of product development to get the job done. I think something our field may benefit from is a guide that explains how to use the different usability services and at what point in the project cycle they should be used.
- phil_KartMe 17y agoThanks for the mention and compliments. I can only say test, test, test. Another hack to is to do testing in student centers (as opposed to bars) so you can get people on their own computers.
- jcl 17y agoSome related FeedbackArmy-related posts, including some reviews: http://news.ycombinator.com/item?id=785489 http://news.ycombinator.com/item?id=785489 http://news.ycombinator.com/item?id=376956 http://news.ycombinator.com/item?id=376956 http://news.ycombinator.com/item?id=373900 http://news.ycombinator.com/item?id=373900
- pw 17y agoI've tried Feedback Army and was disappointed w/ the results. Not that it isn't a cool idea, but I think the give and take of a good usability test requires that it be done in person. If you watch usability expert Steve Krug doing a demo test (http://network.businessofsoftware.org/video/steve-krug-on-the-least-you http://network.businessofsoftware.org/video/steve-krug-on-th...), you'll see what I mean.
- raffi 17y agoI recently saw a blog post where someone wrote about the feedback they received and they weren't happy. The same person then revised their questions and tried a more task oriented approach and they were much happier. http://www.i-proving.ca/space/David+Jones/blog/2010-01-27_1?showComments=true#startComments http://www.i-proving.ca/space/David+Jones/blog/2010-01-27_1?... Then again, Feedback Army is just a usability testing tool. The right one for the job. I think it's best when a breadth of ideas are needed quickly or you need some ideas about where to start improving your site. You should use other methods to do testing that requires a lot of depth from one or two people. Here are a few more posts where folks have written up their experiences: http://visualwebsiteoptimizer.com/split-testing-blog/how-an-investment-of-10-can-give-you-website-improvement-ideas-worth-thousands-of-dollars/ http://visualwebsiteoptimizer.com/split-testing-blog/how-an-... http://www.mattmazur.com/2010/02/reflections-on-precedens-hackernews-launch/ http://www.mattmazur.com/2010/02/reflections-on-precedens-ha... (p.s. I run Feedback Army :))
- Groxx 17y agoAnything run by a 3rd party strikes me as immediately questionable for accurate testing. They necessarily use people more geared towards using applications, as otherwise they wouldn't be there in the first place, getting paid to do something that most people wouldn't touch. They can certainly provide extensively useful insight, so running through a UI-review company is still a good idea, but can there really be a guerrilla testing company? The idea of guerrilla testing is to get totally untainted results, isn't it?
- ThomPete 17y agoI call BS on this. It has no usage what so ever. You can't test for success you can only monitor it.
- forcer 17y agoWhy no usage what so ever? Any process of getting user's feedback is better than doing nothing. We are so skewed of using our own apps that we think all is perfect. Nothing ever is. I think its a great idea how to get more usage data. If I try it I will let you know how it went. I am thinking of using some screenrecording software like SilverBackApp to archive and analyze results later when not drunk :) Janusz Founder of http://www.UsabilityTest.com http://www.UsabilityTest.com (free usability testing for web apps)
- Groxx 17y agoInteresting site... if it works like it appears to, I wouldn't be one of the survey testers, but I'm sure it'd nab a lot of people, enough to at least get some ideas. Good luck with it!
- ThomPete 17y agoBecause success is not measured by the usability of your product. There are countless examples of way too complex products that are still successful and almost any successful company you can think of DIDN'T do usability testing. Companies like Amazon do A/B testing, Apple don't do usability testing at all. Usability starts to matter later on, when you can measure on actual customer behavior rather than user opinion. Down vote me all you want. I challenge anyone to prove that usability tests are actually making more successful products.
- shalmanese 17y agoBoth Amazon & Apple do usability testing. Apple doesn't do focus groups which are an entirely different thing.
- ThomPete 17y agoshalmanese Amazon do A/B tests. Apple doing usability tests? Not according to those I know there. Don't you mean testing?
- CoryMathews 17y agoThis is awesome. I am going to start doing this all the time. Its extremely good also that my target market is college students :) even better.
- c_allison 17y agoNot only a good way to test cheaply, but a good way to network! Win-win.
- run4yourlives 17y agoThis is popular in some design circles, particularly with evaluating reactions to 2/3 designs prior to launch. I'm not sure you'd get exactly the same quality of results with a user test, and demographics is key - after all, the blue-collar workers at the pub aren't the people to ask about your b2b conversion tool - but it's certainly the type of thing that we should be doing.
- shpxnvz 17y agoI'd imagine that for just about any demographic you care to target, there's a bar you can find that caters to them. Heck, I've seen bars that have clientele heavily biased to a specific industry, and in some extreme cases, even a single employer.
- jdrock 17y agoOne additional note that should be added: results will only be useful if your desired customer base is represented well by people at a bar.
- Spikefu 17y agoFTA: "Bars are wonderful at segmenting by demographic. Match the bar you’re going to with the user population you want to target. Different bars will produce slightly different results but the variation is not huge."
- jdrock 17y agoYeah, I don't know a bar frequented by people interested in the Semantic Web or web-crawling.
- ynniv 17y agoDoesn't sound like a consumer application. Given that your profile mentions 80legs, I think you're in the B2B space.
- jdrock 17y agoWell, that was sort of the point of my original comment. The test will work if bar-people represent your target customer base. This won't be the case for B2B products and may not be the case for some Consumer products.
- shalmanese 17y agoSophisticated users will reveal sophisticated usability flaws in your application but even unsophisticated users can reveal the unsophisticated flaws. One really great tip (for sober testing, not drunk) is to use role playing. If you were soliciting feedback for a blog analytics package for example, you could tell users "Imagine you're the CEO of a lollipop manufacturer". Most people end up getting really into the role you're asking them to play and ask surprisingly insightful questions. That being said, you still need to test with sophisticated users but they're much harder to get and their time is expensive so it's always a good idea to test with a few unsophisticated users first and fix the obvious usability flaws so you can maximize the amount of relevant feedback you do get from your sophisticated users.
- alexyim 17y agoGreat. Now I have this to try in addition to the Parents Test
- kolya3 17y agoBring a laptop that you don't care about. Drunk people spill.
- rodyancy 17y agoI've done the sober equivalent to this (coffee shops). Not only did it point out some usability issues, like, "oh, he thinks THAT'S a button," it also proved to be a great way to generate ideas. Most of the input we received was dead wrong, but we had to think about why it was wrong, and that got us thinking in ways we hadn't thought before, which led to good ideas. I suspect the drunken version would be more fun and just as useful, assuming you are producing a consumer product. As for B2B, if your domain is specific, this might not be the most valuable use of your time.
- olegk 17y agoI don't see myself doing this.
- coriander 17y agoSo you want programmers to talk to people in real life. Interesting.
- AFerenci 17y agoI experienced the same effectiveness (95%-99%) of getting the "guy/gal reading the newspaper" to volunteer for the study first hand. Although, this was more focused in cafe environments. Our team was trying to get beta users signed up for a location-based iphone app in the Boston/Cambridge area and most people reading newspapers in coffee shops or cafes are frequently willing participants. I even noticed that 7/10 times these people will engage with you in conversation regarding feedback/UX improvements etc. In parallel with Rodyancy's experience, some of the user feedback was arbitrary (one guy started comparing our application to time/space continuum theories...etc) but nonetheless, I would advocate approaching the newspaper guy/gal as they are already on the knowledge quest and usually are more open-minded. Oh, and avoid the book/textbook readers. These people are either really engaged in novel (imagine tapping on someone's shoulder during a movie) or are stressed-out students.
- dkimball 17y agoI'm kind of disappointed that Steve Krug didn't think up something like this for _Rocket Surgery Made Easy_ (http://www.sensible.com/rocketsurgery/index.html http://www.sensible.com/rocketsurgery/index.html); this would be a very convenient source of manpower for his model of "hallway usability testing," and would allow even larger numbers of even more easily set-up tests than the model he describes. It's good to see how usability testing is becoming de-mythologized these days; all of us should keep in mind, for current and future program development, that you really don't need a very sophisticated apparatus.
- Groxx 17y agoStart off approaching a group of the same sex as you so that the encounter isn’t sexualized. Next, move on to a mixed sex group and then finish the night with an opposite gender interaction so you get a nice demographic spread. I wonder if that last bit is intentionally a double-entendre. Especially since the proposed environment involves a bar. It's an interesting technique, and I'll definitely keep the strategy in mind for future endeavors.
- shalmanese 17y agoThe double entendre was completely unintentional. Testing against women is important but one of the big hangups a lot of people have is the sexual component. This is one technique I've discovered to help ease out of that mindset. As a strategy for picking up women, this is somewhere down the bottom in terms of effectiveness. As it turns out, asking someone to participate in a user study is about the least sexiest thing imaginable.