8 ms·

I wish I’d thought of that. Operation Starbucks

325 points by westonh 13y ago

vishl 13y ago
I've considered standing outside on a hot day with a cooler of icecream sandwiches. At $1 each it's even more cost efficient, though people may not be as willing to stop and chat as they are in starbucks.
guylhem 13y ago
You may enjoy taking a marketing class :-) From what I understand, market analysis can always be done on the cheap if you want to take some accuracy risk - there's a clear self selection bias here, along with a geographic one (there may a common socioeconomic background for the coffee shop patrons) Also, if you care about kids, it may have worked better at a coffee shop in front of a toy store in a mall, or at the food court if it's near say a pet store, to get actual parents - there might be fewer parents at a typical starbuck. But that is important if you believe they are you client - ie the persons who'll make the purchase decision, but maybe it will be the kids themselves in this example? Critic aside, it's a great idea for a generalist app, but it may be less than ideal for say a specialized app. Also, you should consider the alternatives - how much feedback do you think the same $50 may have got you using the mechanical turk? There is a sincerity risk, but if you get say 100x more feedback, with the law of large numbers, I think that at least the obvious flaws could be noticed.
- learc83 13y ago
  >Also, if you care about kids, it may have worked better at a coffee shop in front of a toy store in a mall Also if you're actually trying to get kids to interact with the app, you'll get a much better response if you have a woman with you. Parents are more likely to be comfortable letting their children interact with a woman.
chasing 13y ago
Yeah, I like this. A founder I worked with did something similar but just gave out gift cards to talk to people about their needs in the space she was working in. Before having a product to show off. It seems like a great way to get out of the echo-chamber a bit.
unreal37 13y ago
Interesting that sitting in a cafe offering people $5 cash for their opinion would be a terrible idea, but offering them a free coffee is brilliant. To @vishl, I don't take food from people on the street. I don't trust it. "What if it's poisoned?" What you're thinking does happen - some people offer me free candies, Reese Peanut Butter cups etc as an incentive to talk to me on the street. I avoid them. The coffee coming from Starbucks directly makes it safer.
- mcintyre1994 13y ago
  I think psychologically the voucher is worth more because it's something they were already going to spend, it's a tangible saving. I'm not a social scientist or whatever, but logically I think that's how people may look at this offer. I definitely agree that if you're not producing the consumable that's a benefit too (although of course they'd probably use $5 cash to buy a coffee).
  - albemuth 13y ago
    Dan Ariely has written about research around this topic: http://danariely.com/2009/08/05/the-trouble-with-cold-hard-cash/ http://danariely.com/2009/08/05/the-trouble-with-cold-hard-c...
    - mcintyre1994 13y ago
      Good read, thanks!
  - deleted 13y ago
    [deleted]
- learc83 13y ago
  > "What if it's poisoned?" You won't take sealed candy from people on the street because it might be poisoned? If the candy is sealed they could have just as easily snuck it onto a store shelf.
  - 13y ago
deleted_account 13y ago
I imagine a future where instead of running the gauntlet of clipboard wielding public researchers, it'll be pseudo-entrepreneurs hawking their herp-derp OS app.
megablast 13y ago
Asking if someone will pay is probably as valuable as not asking. Most people aren't going to say something bad about your work, when you are right in front of them. And talk is cheap.
- learc83 13y ago
  You could ask them to fill write their responses on a piece of paper with no name and drop it in a box. Make it clear that it's anonymous.
hawkharris 13y ago
Too many early-stage companies focus on mobilizing people through — if you'll forgive the dreadful buzzword — social media. If you can connect directly with your audience face-to-face, as Jim did, that's often a smarter strategy. I like the way Jim was able to segment his audience in a low-tech way (chatting with the parent-kid teams who are his core users/customers). Yeah, you might be able to achieve a greater # of impressions online, but it's much harder to make those impressions meaningful and to make sure that they translate into action. ("Raising awareness" is complete bullshit. If people are aware but they don't take action, no one cares.)
simonholroyd 13y ago
My team has done similar user testing in Starbucks quite a few times. Here's my $0.02: If you're willing to be bold, walk right up to people and ask, you don't need to offer free coffee or gift cards. We've found this is a better strategy for a few reasons: 1. It's cheaper. 2. It does not dramatically change the number of test users you can get in a given afternoon. And 3. You're more likely to get critical negative feedback if you dont put the users in the position of owing you for the coffee.
- shardling 13y ago
  > If you're willing to be bold, walk right up to people and ask, you don't need to offer free coffee or gift cards. If many people did this, it would quickly become an incredible annoyance. Have some respect for the commons!
  - soofaloofa 13y ago
    They can always say no.
    - kenbellows 13y ago
      The option to say no doesn't make it any less annoying.
    - chris_mahan 13y ago
      They can also say "Fuck you" loud enough for everyone in the starbux to hear and for the manager to call the police and escort you out. I guess in a way that can get your site some publicity, but I gather that's not exactly what you're after.
  - kenbellows 13y ago
    This. As much as it may work right now, because basically no one is doing it, as soon as it becomes a popular method, it seem to me it will stop working pretty quickly. Everyone doing it will become equivalent to a Jehova's Witness knocking on your door. That said, I imagine I'd be more okay with it if every time I said yes I got a free coffee. Aside: I wonder how different reactions would be in different countries/cultures.
aashaykumar92 13y ago
Along the same lines: I launched a social calendar app with my roommate for students at UMichigan and to initially gain users, we bought $55 worth of pizzas and to get a slice, all students had to do was download our app and register as a user--we setup in one of the busiest halls on campus and needless to say, the pizza ran out pretty quickly. Looking back, it was great for the numbers and a was a good marketing strategy, but we didn't gain much else from it. I wish we had setup a demo or something of the sort so that we could have also received feedback. Asking vital questions about the product should be geared toward customers, not as much founders, and we learned that the hard way.
- weisser 13y ago
  Some guys with an app paid for all food orders during a 4 hour block (10pm-2am) at a popular foodtruck at SXSW. All you had to do to get a free ~$7 burrito (some people got a bunch of stuff) was to download their app and create a photo album. While this boosted downloads during that time period I have not looked at the app (I don't even remember what it was called) since. It would have been more memorable if they had discussed the app with me for the free burrito.
  - darkstar999 13y ago
    > I don't even remember what it was called Don't let that happen, people! If you are paying $7+ just to get someone to install your app, make it worth it!
    - weisser 13y ago
      Exactly. Sating my apetite for a burrito at 1:50am will make me happy but there are better ways of generating more interest and in a way that people will remember for less money.
MCompeau 13y ago
We tried this just yesterday at a couple Starbucks locations in Toronto for our new lines of 3D printed jewelry (http://www.hotpopfactory.com/ http://www.hotpopfactory.com/). It was great, because we could hone in on specific demographics that interested us based on the location and time of day. For the most part we found people weren't actually interested in taking any money for coffee (we didn't go the gift card route, we just approached people in line directly). When we offered to pay some people actually found it a bit off-putting. That said though, when you approach people candidly looking for feedback on a project you're personally invested in, people seemed pretty generous with their time. We ended up getting great feedback on everything from the usability of our website, to the desirability of our product and pricing. We even ended up finding a couple new customers along the way. I'd highly recommend this experiment for any consumer facing product, it can be a bit daunting at first, but when used strategically it can be far more productive then having your head down coding in isolation all day.
- munificent 13y ago
  I love the bacon necklaces: http://www.hotpopfactory.com/product_boreal-necklace.html http://www.hotpopfactory.com/product_boreal-necklace.html ;)
- theIV 13y ago
  I'll sidetrack for a minute and say I think the jewelry looks awesome. Shooting-from-the-hip site feedback: it is a bit weird that some (maybe all?) of the product pages recommend–"you might also like"–the same product you are already looking at.
  - mtrimpe 13y ago
    Love the product as well! My from-the-hip feedback: * From the opening screen I couldn't figure out if you were Hot Pop factory or Platonix. After I got that you sell Platonix and Boreal I still don't know how you two are related. * My #1 fear: How easily does this stuff break? * Where's the story? You probably have a good one (3D-printing) but I don't see it here. * Where's the gold? Annoying question I know, but people want metals so have you considered a printing moulds and smelting process?
nostromo 13y ago
If possible, say that you're doing it as research for an agency. People are nice and don't want to hurt your feelings. ("Yeah, I'd totally buy that! Oh, I didn't bring my credit card, sorry...") Also, if you have competitors, try getting feedback on their apps as well. It can give you hints about what they are doing right and wrong, and how you can differentiate.
- Felix21 13y ago
  Woow Feedback on competitors apps. What a great idea!
  - micheljansen 13y ago
    I've done this a few times. It's also a good idea to hide which product is yours, so you get less biased feedback.
- Felix21 13y ago
  Woow Feedback on competitors apps. What a great idea!
- JoshLedgard 13y ago
  I like the idea of getting feedback on competitors apps. :)
will_brown 13y ago
Does this remind anyone else of the South Park episode "Free Hat", where Cartman writes "Free Hat" on the advertising poster in the belief that freebies are necessary to attract people?
- mseebach 13y ago
  Well, we clearly learned from the movie that freebies are, indeed, required. http://www.youtube.com/watch?v=lrNrLly1vvM http://www.youtube.com/watch?v=lrNrLly1vvM
snowwrestler 13y ago
> He never asked if I’d give him money today for it. He got feedback from someone he didn’t know would be a paying customer or not. That's because verbal pricing feedback from random people tends to be terribly inaccurate. Most people actually have a pretty poor conscious idea of what they will buy and how much they would pay for a future product. Many times I've seen people in focus groups say they would pay $hundreds for an electronics product, then reveal that (for instance) they just had to sell their stereo on Craigslist for rent money. (Translation: no way in hell they would actually pay $hundreds for the product.) This sort of guerilla action is great for usability testing of a mass-market product because the utility of the feedback increases with the ignorance of the test subject. If your product can be understood and used by tech luddites, it will work fine for tech savvies too. But pricing research needs much more rigorous look at valid data on potential markets and customers.
aantix 13y ago
I've done this exact thing for images for a site I was building.. Most will say know but others are very willing and will even sign a release. It's much cheaper than purchasing photos from a stock photography site and much more fun.
mkilling 13y ago
The main investment here is time, especially compared to online usability testing services (usertesting.com etc.). That said, early in product development in-person feedback is invaluable and you can get so much more out of it than of a simple video of a person using your product.
adrianhoward 13y ago
This sort of approach is a classic way of doing cheap usability testing. If you've not done this sort of work before I'd recommend getting hold of the (short and entertaining) "Rocket Surgery Made Easy: The Do-it-yourself Guide to Finding and Fixing Usability Problems" by Steve Krug. It'll give you a bunch of useful tips on how to get the best out of our time.
laurentoget 13y ago
I wonder how the manager of that starbucks feels about that. On one hand you are buying some coffee. On the other hand you are probably driving away the customers who can afford to pay for their own coffee and do not feel like being your guinea pig.
- burntsushi 13y ago
  > On the other hand you are probably driving away the customers who can afford to pay for their own coffee and do not feel like being your guinea pig. Eh? Seems like a simple "No thanks" would due just fine.
- thenomad 13y ago
  On the other, other hand, you're giving some of his/her customers an interesting, novel experience where they get to feel that their opinion is valuable. Oh, and ensuring that they'll remember that Starbucks far more than most others.

325 points by westonh 13y ago

vishl 13y ago

I've considered standing outside on a hot day with a cooler of icecream sandwiches. At $1 each it's even more cost efficient, though people may not be as willing to stop and chat as they are in starbucks.

guylhem 13y ago

You may enjoy taking a marketing class :-) From what I understand, market analysis can always be done on the cheap if you want to take some accuracy risk - there's a clear self selection bias here, along with a geographic one (there may a common socioeconomic background for the coffee shop patrons) Also, if you care about kids, it may have worked better at a coffee shop in front of a toy store in a mall, or at the food court if it's near say a pet store, to get actual parents - there might be fewer parents at a typical starbuck. But that is important if you believe they are you client - ie the persons who'll make the purchase decision, but maybe it will be the kids themselves in this example? Critic aside, it's a great idea for a generalist app, but it may be less than ideal for say a specialized app. Also, you should consider the alternatives - how much feedback do you think the same $50 may have got you using the mechanical turk? There is a sincerity risk, but if you get say 100x more feedback, with the law of large numbers, I think that at least the obvious flaws could be noticed.

learc83 13y ago
>Also, if you care about kids, it may have worked better at a coffee shop in front of a toy store in a mall Also if you're actually trying to get kids to interact with the app, you'll get a much better response if you have a woman with you. Parents are more likely to be comfortable letting their children interact with a woman.

chasing 13y ago

Yeah, I like this. A founder I worked with did something similar but just gave out gift cards to talk to people about their needs in the space she was working in. Before having a product to show off. It seems like a great way to get out of the echo-chamber a bit.

unreal37 13y ago

Interesting that sitting in a cafe offering people $5 cash for their opinion would be a terrible idea, but offering them a free coffee is brilliant. To @vishl, I don't take food from people on the street. I don't trust it. "What if it's poisoned?" What you're thinking does happen - some people offer me free candies, Reese Peanut Butter cups etc as an incentive to talk to me on the street. I avoid them. The coffee coming from Starbucks directly makes it safer.

mcintyre1994 13y ago
I think psychologically the voucher is worth more because it's something they were already going to spend, it's a tangible saving. I'm not a social scientist or whatever, but logically I think that's how people may look at this offer. I definitely agree that if you're not producing the consumable that's a benefit too (although of course they'd probably use $5 cash to buy a coffee).
- albemuth 13y ago
  Dan Ariely has written about research around this topic: http://danariely.com/2009/08/05/the-trouble-with-cold-hard-cash/ http://danariely.com/2009/08/05/the-trouble-with-cold-hard-c...
  - mcintyre1994 13y ago
    Good read, thanks!
- deleted 13y ago
  [deleted]
learc83 13y ago
> "What if it's poisoned?" You won't take sealed candy from people on the street because it might be poisoned? If the candy is sealed they could have just as easily snuck it onto a store shelf.
- 13y ago

deleted_account 13y ago

I imagine a future where instead of running the gauntlet of clipboard wielding public researchers, it'll be pseudo-entrepreneurs hawking their herp-derp OS app.

megablast 13y ago

Asking if someone will pay is probably as valuable as not asking. Most people aren't going to say something bad about your work, when you are right in front of them. And talk is cheap.

learc83 13y ago
You could ask them to fill write their responses on a piece of paper with no name and drop it in a box. Make it clear that it's anonymous.

hawkharris 13y ago

Too many early-stage companies focus on mobilizing people through — if you'll forgive the dreadful buzzword — social media. If you can connect directly with your audience face-to-face, as Jim did, that's often a smarter strategy. I like the way Jim was able to segment his audience in a low-tech way (chatting with the parent-kid teams who are his core users/customers). Yeah, you might be able to achieve a greater # of impressions online, but it's much harder to make those impressions meaningful and to make sure that they translate into action. ("Raising awareness" is complete bullshit. If people are aware but they don't take action, no one cares.)

simonholroyd 13y ago

My team has done similar user testing in Starbucks quite a few times. Here's my $0.02: If you're willing to be bold, walk right up to people and ask, you don't need to offer free coffee or gift cards. We've found this is a better strategy for a few reasons: 1. It's cheaper. 2. It does not dramatically change the number of test users you can get in a given afternoon. And 3. You're more likely to get critical negative feedback if you dont put the users in the position of owing you for the coffee.

shardling 13y ago
> If you're willing to be bold, walk right up to people and ask, you don't need to offer free coffee or gift cards. If many people did this, it would quickly become an incredible annoyance. Have some respect for the commons!
- soofaloofa 13y ago
  They can always say no.
  - kenbellows 13y ago
    The option to say no doesn't make it any less annoying.
  - chris_mahan 13y ago
    They can also say "Fuck you" loud enough for everyone in the starbux to hear and for the manager to call the police and escort you out. I guess in a way that can get your site some publicity, but I gather that's not exactly what you're after.
- kenbellows 13y ago
  This. As much as it may work right now, because basically no one is doing it, as soon as it becomes a popular method, it seem to me it will stop working pretty quickly. Everyone doing it will become equivalent to a Jehova's Witness knocking on your door. That said, I imagine I'd be more okay with it if every time I said yes I got a free coffee. Aside: I wonder how different reactions would be in different countries/cultures.

aashaykumar92 13y ago

Along the same lines: I launched a social calendar app with my roommate for students at UMichigan and to initially gain users, we bought $55 worth of pizzas and to get a slice, all students had to do was download our app and register as a user--we setup in one of the busiest halls on campus and needless to say, the pizza ran out pretty quickly. Looking back, it was great for the numbers and a was a good marketing strategy, but we didn't gain much else from it. I wish we had setup a demo or something of the sort so that we could have also received feedback. Asking vital questions about the product should be geared toward customers, not as much founders, and we learned that the hard way.

weisser 13y ago
Some guys with an app paid for all food orders during a 4 hour block (10pm-2am) at a popular foodtruck at SXSW. All you had to do to get a free ~$7 burrito (some people got a bunch of stuff) was to download their app and create a photo album. While this boosted downloads during that time period I have not looked at the app (I don't even remember what it was called) since. It would have been more memorable if they had discussed the app with me for the free burrito.
- darkstar999 13y ago
  > I don't even remember what it was called Don't let that happen, people! If you are paying $7+ just to get someone to install your app, make it worth it!
  - weisser 13y ago
    Exactly. Sating my apetite for a burrito at 1:50am will make me happy but there are better ways of generating more interest and in a way that people will remember for less money.

MCompeau 13y ago

We tried this just yesterday at a couple Starbucks locations in Toronto for our new lines of 3D printed jewelry (http://www.hotpopfactory.com/ http://www.hotpopfactory.com/). It was great, because we could hone in on specific demographics that interested us based on the location and time of day. For the most part we found people weren't actually interested in taking any money for coffee (we didn't go the gift card route, we just approached people in line directly). When we offered to pay some people actually found it a bit off-putting. That said though, when you approach people candidly looking for feedback on a project you're personally invested in, people seemed pretty generous with their time. We ended up getting great feedback on everything from the usability of our website, to the desirability of our product and pricing. We even ended up finding a couple new customers along the way. I'd highly recommend this experiment for any consumer facing product, it can be a bit daunting at first, but when used strategically it can be far more productive then having your head down coding in isolation all day.

munificent 13y ago
I love the bacon necklaces: http://www.hotpopfactory.com/product_boreal-necklace.html http://www.hotpopfactory.com/product_boreal-necklace.html ;)
theIV 13y ago
I'll sidetrack for a minute and say I think the jewelry looks awesome. Shooting-from-the-hip site feedback: it is a bit weird that some (maybe all?) of the product pages recommend–"you might also like"–the same product you are already looking at.
- mtrimpe 13y ago
  Love the product as well! My from-the-hip feedback: * From the opening screen I couldn't figure out if you were Hot Pop factory or Platonix. After I got that you sell Platonix and Boreal I still don't know how you two are related. * My #1 fear: How easily does this stuff break? * Where's the story? You probably have a good one (3D-printing) but I don't see it here. * Where's the gold? Annoying question I know, but people want metals so have you considered a printing moulds and smelting process?

nostromo 13y ago

If possible, say that you're doing it as research for an agency. People are nice and don't want to hurt your feelings. ("Yeah, I'd totally buy that! Oh, I didn't bring my credit card, sorry...") Also, if you have competitors, try getting feedback on their apps as well. It can give you hints about what they are doing right and wrong, and how you can differentiate.

Felix21 13y ago
Woow Feedback on competitors apps. What a great idea!
- micheljansen 13y ago
  I've done this a few times. It's also a good idea to hide which product is yours, so you get less biased feedback.
Felix21 13y ago
Woow Feedback on competitors apps. What a great idea!
JoshLedgard 13y ago
I like the idea of getting feedback on competitors apps. :)

will_brown 13y ago

Does this remind anyone else of the South Park episode "Free Hat", where Cartman writes "Free Hat" on the advertising poster in the belief that freebies are necessary to attract people?

mseebach 13y ago
Well, we clearly learned from the movie that freebies are, indeed, required. http://www.youtube.com/watch?v=lrNrLly1vvM http://www.youtube.com/watch?v=lrNrLly1vvM

snowwrestler 13y ago

> He never asked if I’d give him money today for it. He got feedback from someone he didn’t know would be a paying customer or not. That's because verbal pricing feedback from random people tends to be terribly inaccurate. Most people actually have a pretty poor conscious idea of what they will buy and how much they would pay for a future product. Many times I've seen people in focus groups say they would pay $hundreds for an electronics product, then reveal that (for instance) they just had to sell their stereo on Craigslist for rent money. (Translation: no way in hell they would actually pay $hundreds for the product.) This sort of guerilla action is great for usability testing of a mass-market product because the utility of the feedback increases with the ignorance of the test subject. If your product can be understood and used by tech luddites, it will work fine for tech savvies too. But pricing research needs much more rigorous look at valid data on potential markets and customers.

aantix 13y ago

I've done this exact thing for images for a site I was building.. Most will say know but others are very willing and will even sign a release. It's much cheaper than purchasing photos from a stock photography site and much more fun.

mkilling 13y ago

The main investment here is time, especially compared to online usability testing services (usertesting.com etc.). That said, early in product development in-person feedback is invaluable and you can get so much more out of it than of a simple video of a person using your product.

adrianhoward 13y ago

This sort of approach is a classic way of doing cheap usability testing. If you've not done this sort of work before I'd recommend getting hold of the (short and entertaining) "Rocket Surgery Made Easy: The Do-it-yourself Guide to Finding and Fixing Usability Problems" by Steve Krug. It'll give you a bunch of useful tips on how to get the best out of our time.

laurentoget 13y ago

I wonder how the manager of that starbucks feels about that. On one hand you are buying some coffee. On the other hand you are probably driving away the customers who can afford to pay for their own coffee and do not feel like being your guinea pig.

burntsushi 13y ago
> On the other hand you are probably driving away the customers who can afford to pay for their own coffee and do not feel like being your guinea pig. Eh? Seems like a simple "No thanks" would due just fine.
thenomad 13y ago
On the other, other hand, you're giving some of his/her customers an interesting, novel experience where they get to feel that their opinion is valuable. Oh, and ensuring that they'll remember that Starbucks far more than most others.