4 ms·

It doesn't matter how big you are, the same cheep method works for informal but fairly accurate quick and dirty testing. Beer. You think I'm kidding but if yo

It doesn't matter how big you are, the same cheep method works for informal but fairly accurate quick and dirty testing. Beer. You think I'm kidding but if you go to a bar, there are lots of people who will test whatever you put in front of them for a free drink. If you want to keep your costs down even further then go during happy hour. Is that "user testing" no... not in the sense we think of it today. Is it testing of the usability of your product: yes it is. Im guessing that most apps issues that are more likely the latter and not the former. Basic things that you think are clear that really aren't (your too close to the problem and the solution). If you explain, point and guide through the same aspect of your feature 10 times in 10 tests people might not get it! These are the problems you might want to solve, dumb it down, change it up, make it better. Does this work? Yes, as long as you can moderate. If you can you be neutral and friendly and avoid "traps" that will alter your results. Saying things like "MY app" or reacting personally will alter your results, and learning when to "guide them" through a frustrating point and when to let them puzzle through it is a bit of an art (this too offers clues if your guiding people at the same place). For me personally a drink or two helps ME in the process... I have also discovered that this process works very well with pairs of people. Two friends already have a rapport and will happily talk to each other to try to get through something problematic. One can encourage or discourage the other and their dialog can be enlightening. Letting them play together with your app and drink free drinks can give you a LOT more insight than the next three individuals testing. Anything more than two people becomes problematic so look to avoid large groups.

lucasverra 8y ago
great advice, will be implemented :)
daleco 8y ago
Guerilla testing is a great starting point. Do you usualy give one or multiple scenarios to complete? Be aware that you are not controlling the environment and the users may be distracted by outside factors. Which may affect some of your metrics (time to perform tasks, success rate...)
- pests 8y ago
  > not controlling the environment and the users may be distracted by outside factors So probably how they will be when actually using your app? I do agree though in a reproduce-ability/data-integrity standpoint.
- hluska 8y ago
  I'm not OP, but I've used similar styles of pub/coffee shop testing. When I do it, I don't give scenarios or bother measuring things like success rate. It's very freeform, unscripted and raw. It's kind of like hallway testing for people who work alone. And, instead of coworkers, you're working with complete strangers who might not have any idea what's happening. If I had to pick one thing to measure, I'd be more inclined to measure positive versus negative fucks (I'm not joking), but honestly, it's more about the experience and the totally raw, sometimes terrifyingly honest feedback I hear. Numbers aren't useful because the most useful sessions go way off the rails.
- zer00eyz 8y ago
  > Do you usualy give one or multiple scenarios to complete? I am OP, and it depends. Im old enough that I have done this with paper prototypes, and it is shockingly effective. Now most of the code I write is compartmentalized enough that with a bit of effort I can come up with a one off digital artifact. Just rip out the chunk of your app that you want to test and present ONLY that. Spend a day building ugly shims early enough in the process and you might make some dramatic changes! > Be aware that you are not controlling the environment and the users may be distracted by outside factors. Which may affect some of your metrics (time to perform tasks, success rate...) Even with those things going on your going to be shocked at not only what issues come up but how frequently. That thing you did that you thought was simple and sane, no one gets it. Do more than half of your test subjects comment on the same thing (good, bad, both) then you have a problem. The key isn't controlling the key is listening... I also want to state that a slighty drunk, distracted bar patron, getting through your process without quitting, telling you your product is shit, or giving up is a positive sign that in the real world people who are more motivated than "beer" are going to have a fairly positive and expedient experience.
Exuma 8y ago
Interesting idea about "Guerrilla testing"
alexbeloi 8y ago
This is a great idea to gather problems/questions. There's no substitute for scaled A/B testing, and finding worthwhile things for testing can be the hardest part.
- zer00eyz 8y ago
  There are lots of risks in A/B testing. Is B a complete failure for some basic UI issue, and you just lost real customers or harmed a relationship? Doing this step first is a good way to avoid some of the worst case issues.
cpeterso 8y ago
Appcues has hosted Drunk User Testing events with the slogan "Design Sober. Test Drunk." :) https://www.drunkusertesting.com/ https://www.drunkusertesting.com/

lucasverra 8y ago

great advice, will be implemented :)

daleco 8y ago

Guerilla testing is a great starting point. Do you usualy give one or multiple scenarios to complete? Be aware that you are not controlling the environment and the users may be distracted by outside factors. Which may affect some of your metrics (time to perform tasks, success rate...)

pests 8y ago
> not controlling the environment and the users may be distracted by outside factors So probably how they will be when actually using your app? I do agree though in a reproduce-ability/data-integrity standpoint.
hluska 8y ago
I'm not OP, but I've used similar styles of pub/coffee shop testing. When I do it, I don't give scenarios or bother measuring things like success rate. It's very freeform, unscripted and raw. It's kind of like hallway testing for people who work alone. And, instead of coworkers, you're working with complete strangers who might not have any idea what's happening. If I had to pick one thing to measure, I'd be more inclined to measure positive versus negative fucks (I'm not joking), but honestly, it's more about the experience and the totally raw, sometimes terrifyingly honest feedback I hear. Numbers aren't useful because the most useful sessions go way off the rails.
zer00eyz 8y ago
> Do you usualy give one or multiple scenarios to complete? I am OP, and it depends. Im old enough that I have done this with paper prototypes, and it is shockingly effective. Now most of the code I write is compartmentalized enough that with a bit of effort I can come up with a one off digital artifact. Just rip out the chunk of your app that you want to test and present ONLY that. Spend a day building ugly shims early enough in the process and you might make some dramatic changes! > Be aware that you are not controlling the environment and the users may be distracted by outside factors. Which may affect some of your metrics (time to perform tasks, success rate...) Even with those things going on your going to be shocked at not only what issues come up but how frequently. That thing you did that you thought was simple and sane, no one gets it. Do more than half of your test subjects comment on the same thing (good, bad, both) then you have a problem. The key isn't controlling the key is listening... I also want to state that a slighty drunk, distracted bar patron, getting through your process without quitting, telling you your product is shit, or giving up is a positive sign that in the real world people who are more motivated than "beer" are going to have a fairly positive and expedient experience.

Exuma 8y ago

Interesting idea about "Guerrilla testing"

alexbeloi 8y ago

This is a great idea to gather problems/questions. There's no substitute for scaled A/B testing, and finding worthwhile things for testing can be the hardest part.

zer00eyz 8y ago
There are lots of risks in A/B testing. Is B a complete failure for some basic UI issue, and you just lost real customers or harmed a relationship? Doing this step first is a good way to avoid some of the worst case issues.

cpeterso 8y ago

Appcues has hosted Drunk User Testing events with the slogan "Design Sober. Test Drunk." :) https://www.drunkusertesting.com/ https://www.drunkusertesting.com/