7 ms·

FeedbackArmy.com - simple, cheap usability testing for your site

37 points by raffi 18y ago

dmix 18y ago
Hmmm purchasing usability testing from a site using a dated design and bright green font on a black background? No thanks.
- deleted 18y ago
  [deleted]
- joseakle 18y ago
  Here's my free usability review of that site: The font is too small.
  - ivankirigin 18y ago
    I thought the same thing. I closed the tab a moment after coming to the page, despite being in the market for such testing.
    - deleted 18y ago
      [deleted]
    - raffi 18y ago
      <3 I can't disagree with what you're saying. My design skills are weak but I wanted to validate the concept first. Today I've seen enough interest to validate moving forward with the project. I'm going to invest in a site facelift to bring the presentation to Web 2.0 standards. Thank you for taking the time to look and post your thoughts here. -- Raphael
      - ivankirigin 18y ago
        Checkout the account creation or sign in form on Tumblr. http://www.tumblr.com/login http://www.tumblr.com/login It's huge.
- bayareaguy 18y ago
  Perhaps they should submit their site to http://www.feedbackarmy.com http://www.feedbackarmy.com ? Oh wait...
- astrec 18y ago
  It's not usability testing at all: in a usability test we're interested in how people use the product, not how they say they use the product (although that can be interesting too).
grag 18y ago
Anyone know if this uses Amazon Mechanical Turk?
- deleted 18y ago
  [deleted]
- raffi 18y ago
  it does (see the questions page)
brandnewlow 18y ago
What are the odds that the idea came from reading comments here: http://news.ycombinator.com/item?id=372305 http://news.ycombinator.com/item?id=372305
- deleted 18y ago
  [deleted]
- raffi 18y ago
  Actually the idea came from http://news.ycombinator.com/item?id=362459 http://news.ycombinator.com/item?id=362459
  - dmix 18y ago
    What value to add to the process? Why wouldn't I just post up the job on Amazon?
    - deleted 18y ago
      [deleted]
    - raffi 18y ago
      Lets assume you've never posted a job to Mechanical Turk before. Here is what you need to do: 1. Create an Amazon Web Services account 2. Figure out how much you want to spend (and don't forget to calculate Amazon's share) 3. Prepay the exact amount or more 4. Create a Human Intelligence Task template 5. Publish the Human Intelligence Task 6. Login to Amazon's portal regularly and navigate several pages to find your results. Have fun clicking refresh. 7. Review the results and promptly pay your workers (or they may ignore future requests) How much is your time worth? Feedback Army spends the majority of the fee towards paying the mechanical turk workers. The Feedback Army software saves you from many of these steps and conveniently transforms the results into an RSS feed.
      - dmix 18y ago
        I have never used Mechanical Turk before and I see that you are adding some value. If you improve the look of your own site you might have something here. Another idea might be to offer larger scale tests with graphical reporting on the results, basically using the low cost/test advantage that Turks offer.
iamdave 18y ago
This is going to sound incredibly assholish, but it's incredibly true. FeedbackArmy.com is going to tank. http://www.feedbackarmy.com/feedback.slp http://www.feedbackarmy.com/feedback.slp 1)That page is why. If someone has the capacity to produce (or have produced) a website, they have the capacity to create a form with two elements and garner feedback. It is incredibly lackluster, easy to replicate and certainly not worth $7 for 10 responses. 2) Based on the comments here people aren't sold on Usability Feedback on websites when the website you've created doesn't suit the needs people are coming to the site for in the first place. 3)The process lends itself quite easily for scammers and cons. There's almost no security measures in place, the design is inherently broken and doesn't provide any peace of mind from the process that you have to perform to get "feedback". I'm not sold. This can be done about 50 ways better than what it is now; I really do not see myself paying any amount of money for the presentation you've put in front of me. It's an effort, I wont say good effort, I wont say bad effort. It's an effort made, and at this point I'm going to say no to this.
- thorax 18y ago
  It's $7, lighten-up. I'm about to spend $15/person/hour for deeper usability tests and now I can ask less in-depth questions but as often as I want for much less? Pure gold.
aditya 18y ago
Wow. There's a lot of negativity and criticism on here, don't be disillusioned, you got something up and out in 2 weeks that says more about you than most other people spending time "commenting" on here :-) Best of luck, and get on that usability thing these people are talking about...
- raffi 18y ago
  Thanks for the encouragement. I've been meaning to play with mechanical turk for awhile and this seemed like a good idea. I'm going to hire a professional and get a snappy design going as quick as possible.
patio11 18y ago
I went ahead and bought a review. I just spent a few hundred dollars launching a mini-site, what is an extra $7? Rounding error -- if they catch a typo I'll consider it money well spent. I'll let you know about the quality of results. One request: my results should not be available on a publicly accessible URL. It doesn't bother me, personally, but it will bother some people.
- raffi 18y ago
  We'll see what happens. I'm watching everyones results like a hawk right now too. You may see a few responses at night (someone who just did a request already has 2) but I find its during the day when they come the quickest. If you're interested in catching typos you should specify that to the reviewers. The reviewers are good at doing what they are told. Do you have any suggestions on the publicly accessible URL? I don't want to force people to create usernames/passwords. I think its a hassle for a quick-use service. Or at least its a pet peeve of mine. How about specifying a password with the request and requiring that password in the URL to access the feedback later on. This way you can subscribe to the results in any feed reader but still have some protection in case people try to guess your URL to see the feedback.
- patio11 18y ago
  Here are the results of my experiment, as promised. Here's my little mini-site: http://www.christmasbingocards.com http://www.christmasbingocards.com This is a holiday promotion aimed at my customers. They're overwhelmingly non-technical folks, and not too sophisticated at the Internet. (I am well-aware that most people here would not purchase my software. I'm totally OK with that: you are not the audience.) I asked the Turkers three questions (and will tell you my rationales): 1) Is it IMMEDIATELY clear what the site offers when you open it? [Rationale: This site is a sledgehammer, not a scalpel. I want them to understand what it offers before they hit their back button. If thought is required, I failed at design.] 2) Try to print out Christmas bingo cards. Tell me how long it takes you from me giving you that instruction to something coming out from your printer. [Rationale: My target is that fourty-something mother with kids, with no special computer skills, can accomplish this in 90 seconds. There is actually a bit of an overlap between Turkers and my market.] 3) If you thought your friends would find Christmas bingo to be a fun activity, would you recommend this site for it to them? Why or why not? [Rationale: The Turking population has a significant overlap with my core audience. Why not ask a quick thumbs up/down?] What it taught me: 1) The site achieves its singular purpose fairly well. 2) It is attractive enough to move several testers into mentioning it to their friends. (As in, not hypothetical.) The feedback lets me know "OK, the site is basically OK and needs to be built-out more than rethought from scratch". Knowing that, and the peace of mind from knowing that, are worth $7 to me easily. I hope this service continues getting developed, because I can see a lot of uses for it in my business. (Yeah, I could roll my own with the Turk API... but I value my time at $100 an hour. This means I never want to be playing the sit-at-computer-refreshing-Amazon-so-I-can-authorize-fourty-cent-payment game.)
sonink 18y ago
I tried feedbackarmy - from a purely ROI perspective I think the usability methodology is flawed. It seems that the folks who give the feedback are actually trying to be a bit too helpful - to the extent that they think about questions as a challenge and try their best to answer them. For example - If I pose a question that "What does this site do?" in a usbility experiment, I am expecting to figure out if the audience immidiately grasp the utility of a site. However in this case it seems that the audience takes that as a challenge modifying the question to mean "Can you figure out what this side does?" and try out very hard to find the utility of a site and in the process totally defeating the purpose of a usability review. It might be an artifact of the way I posed the question, but then I was just following the sites examples. I think using the site in the way it is setup (messaging, mturk) the concept does not give a good ROI w.r.t usability. However, one thing that I did like was that the turkers gave some suggestions to improve the site - some of which was really valuable. So maybe if you change your positioning to something like an active feedback/testing tool it might work better.

37 points by raffi 18y ago

dmix 18y ago

Hmmm purchasing usability testing from a site using a dated design and bright green font on a black background? No thanks.

deleted 18y ago
[deleted]
joseakle 18y ago
Here's my free usability review of that site: The font is too small.
- ivankirigin 18y ago
  I thought the same thing. I closed the tab a moment after coming to the page, despite being in the market for such testing.
  - deleted 18y ago
    [deleted]
  - raffi 18y ago
    <3 I can't disagree with what you're saying. My design skills are weak but I wanted to validate the concept first. Today I've seen enough interest to validate moving forward with the project. I'm going to invest in a site facelift to bring the presentation to Web 2.0 standards. Thank you for taking the time to look and post your thoughts here. -- Raphael
    - ivankirigin 18y ago
      Checkout the account creation or sign in form on Tumblr. http://www.tumblr.com/login http://www.tumblr.com/login It's huge.
bayareaguy 18y ago
Perhaps they should submit their site to http://www.feedbackarmy.com http://www.feedbackarmy.com ? Oh wait...
astrec 18y ago
It's not usability testing at all: in a usability test we're interested in how people use the product, not how they say they use the product (although that can be interesting too).

grag 18y ago

Anyone know if this uses Amazon Mechanical Turk?

deleted 18y ago
[deleted]
raffi 18y ago
it does (see the questions page)

brandnewlow 18y ago

What are the odds that the idea came from reading comments here: http://news.ycombinator.com/item?id=372305 http://news.ycombinator.com/item?id=372305

deleted 18y ago
[deleted]
raffi 18y ago
Actually the idea came from http://news.ycombinator.com/item?id=362459 http://news.ycombinator.com/item?id=362459
- dmix 18y ago
  What value to add to the process? Why wouldn't I just post up the job on Amazon?
  - deleted 18y ago
    [deleted]
  - raffi 18y ago
    Lets assume you've never posted a job to Mechanical Turk before. Here is what you need to do: 1. Create an Amazon Web Services account 2. Figure out how much you want to spend (and don't forget to calculate Amazon's share) 3. Prepay the exact amount or more 4. Create a Human Intelligence Task template 5. Publish the Human Intelligence Task 6. Login to Amazon's portal regularly and navigate several pages to find your results. Have fun clicking refresh. 7. Review the results and promptly pay your workers (or they may ignore future requests) How much is your time worth? Feedback Army spends the majority of the fee towards paying the mechanical turk workers. The Feedback Army software saves you from many of these steps and conveniently transforms the results into an RSS feed.
    - dmix 18y ago
      I have never used Mechanical Turk before and I see that you are adding some value. If you improve the look of your own site you might have something here. Another idea might be to offer larger scale tests with graphical reporting on the results, basically using the low cost/test advantage that Turks offer.

iamdave 18y ago

This is going to sound incredibly assholish, but it's incredibly true. FeedbackArmy.com is going to tank. http://www.feedbackarmy.com/feedback.slp http://www.feedbackarmy.com/feedback.slp 1)That page is why. If someone has the capacity to produce (or have produced) a website, they have the capacity to create a form with two elements and garner feedback. It is incredibly lackluster, easy to replicate and certainly not worth $7 for 10 responses. 2) Based on the comments here people aren't sold on Usability Feedback on websites when the website you've created doesn't suit the needs people are coming to the site for in the first place. 3)The process lends itself quite easily for scammers and cons. There's almost no security measures in place, the design is inherently broken and doesn't provide any peace of mind from the process that you have to perform to get "feedback". I'm not sold. This can be done about 50 ways better than what it is now; I really do not see myself paying any amount of money for the presentation you've put in front of me. It's an effort, I wont say good effort, I wont say bad effort. It's an effort made, and at this point I'm going to say no to this.

thorax 18y ago
It's $7, lighten-up. I'm about to spend $15/person/hour for deeper usability tests and now I can ask less in-depth questions but as often as I want for much less? Pure gold.

aditya 18y ago

Wow. There's a lot of negativity and criticism on here, don't be disillusioned, you got something up and out in 2 weeks that says more about you than most other people spending time "commenting" on here :-) Best of luck, and get on that usability thing these people are talking about...

raffi 18y ago
Thanks for the encouragement. I've been meaning to play with mechanical turk for awhile and this seemed like a good idea. I'm going to hire a professional and get a snappy design going as quick as possible.

patio11 18y ago

I went ahead and bought a review. I just spent a few hundred dollars launching a mini-site, what is an extra $7? Rounding error -- if they catch a typo I'll consider it money well spent. I'll let you know about the quality of results. One request: my results should not be available on a publicly accessible URL. It doesn't bother me, personally, but it will bother some people.

raffi 18y ago
We'll see what happens. I'm watching everyones results like a hawk right now too. You may see a few responses at night (someone who just did a request already has 2) but I find its during the day when they come the quickest. If you're interested in catching typos you should specify that to the reviewers. The reviewers are good at doing what they are told. Do you have any suggestions on the publicly accessible URL? I don't want to force people to create usernames/passwords. I think its a hassle for a quick-use service. Or at least its a pet peeve of mine. How about specifying a password with the request and requiring that password in the URL to access the feedback later on. This way you can subscribe to the results in any feed reader but still have some protection in case people try to guess your URL to see the feedback.
patio11 18y ago
Here are the results of my experiment, as promised. Here's my little mini-site: http://www.christmasbingocards.com http://www.christmasbingocards.com This is a holiday promotion aimed at my customers. They're overwhelmingly non-technical folks, and not too sophisticated at the Internet. (I am well-aware that most people here would not purchase my software. I'm totally OK with that: you are not the audience.) I asked the Turkers three questions (and will tell you my rationales): 1) Is it IMMEDIATELY clear what the site offers when you open it? [Rationale: This site is a sledgehammer, not a scalpel. I want them to understand what it offers before they hit their back button. If thought is required, I failed at design.] 2) Try to print out Christmas bingo cards. Tell me how long it takes you from me giving you that instruction to something coming out from your printer. [Rationale: My target is that fourty-something mother with kids, with no special computer skills, can accomplish this in 90 seconds. There is actually a bit of an overlap between Turkers and my market.] 3) If you thought your friends would find Christmas bingo to be a fun activity, would you recommend this site for it to them? Why or why not? [Rationale: The Turking population has a significant overlap with my core audience. Why not ask a quick thumbs up/down?] What it taught me: 1) The site achieves its singular purpose fairly well. 2) It is attractive enough to move several testers into mentioning it to their friends. (As in, not hypothetical.) The feedback lets me know "OK, the site is basically OK and needs to be built-out more than rethought from scratch". Knowing that, and the peace of mind from knowing that, are worth $7 to me easily. I hope this service continues getting developed, because I can see a lot of uses for it in my business. (Yeah, I could roll my own with the Turk API... but I value my time at $100 an hour. This means I never want to be playing the sit-at-computer-refreshing-Amazon-so-I-can-authorize-fourty-cent-payment game.)

sonink 18y ago

I tried feedbackarmy - from a purely ROI perspective I think the usability methodology is flawed. It seems that the folks who give the feedback are actually trying to be a bit too helpful - to the extent that they think about questions as a challenge and try their best to answer them. For example - If I pose a question that "What does this site do?" in a usbility experiment, I am expecting to figure out if the audience immidiately grasp the utility of a site. However in this case it seems that the audience takes that as a challenge modifying the question to mean "Can you figure out what this side does?" and try out very hard to find the utility of a site and in the process totally defeating the purpose of a usability review. It might be an artifact of the way I posed the question, but then I was just following the sites examples. I think using the site in the way it is setup (messaging, mturk) the concept does not give a good ROI w.r.t usability. However, one thing that I did like was that the turkers gave some suggestions to improve the site - some of which was really valuable. So maybe if you change your positioning to something like an active feedback/testing tool it might work better.