12 ms·

Beta Family: 50k Users to Test Your App

150 points by mcone 9y ago

Operyl 9y ago
Interesting that it's a device enrollment, but also a tad scary. There seems to be a lot of trust that you want your testers to give, and there was basically no heads up to the user. EDIT: I see what they're doing now, crafty, it doesn't actually install any profiles in the end they're just capturing your UDID instead of entering it manually. EDIT 2: Nice, I was expecting survey spam: https://betafamily.com/app/12089/feed-item/28330 https://betafamily.com/app/12089/feed-item/28330
- Raphmedia 9y ago
  I hate to be that person but in the requirements of that link, is "Requirements [...] Male & Female" really needed? How is this a requirement? This would be better written as "No gender requirement" or simply removing that line unless only one gender is specified.
  - arkitaip 9y ago
    There's also the 13-99 age requirement for, you know, when you are ignorant to believe that your market really is everyone between 13 and 99.
    - Operyl 9y ago
      Seems to be going well for the people asking testers to fill out their paid surveys lol.
zerkten 9y ago
This looks like an interesting site. The immediate focus is towards the developer of the apps. I wonder if it'd be more successful if the initial site focus was on the apps (read: opportunities) listed under Browse Tests. Getting a bigger base of people who like to test stuff apps would seem like a pretty valuable thing and I had to think for a few moments about how I can be on the other side. Maybe a less persistent person would have browsed away.
abol3z 9y ago
My app is in Arabic language, do you have testers for Arabic?
mabermoske 9y ago
From the page: "6 000 tests completed, trusted by 6 000 companies..." They've had no repeat customers?
- jermaustin1 9y ago
  I want this answered. How have only 6000 tests been completed with 6000 customers? Even if a single customer ran just two tests, the numbers wouldn't match. This leads me to believe both numbers are fake.
  - lugg 9y ago
    Or accidental variable reuse in a template. Could be either.
    - jageen 9y ago
      In this case they forget to use their own service for test.
      - betafamily 9y ago
        yeah! We do need to test our service better ;)
  - betafamily 9y ago
    Hi! The numbers are out of date, we need to update them. The real number is about 3x bigger...
- mygo 9y ago
  Maybe they rounded?
mongodude 9y ago
I have used this service, does not work as advertised. We hardly got 2-3 testers to test our App.
- fastball 9y ago
  What was your reward value?
- mephitix 9y ago
  I'm guessing you didn't put up a reward. I put just a $5 reward per tester and got ~100 testers who applied within 2 weeks.
  - giarc 9y ago
    Do you have to provide the reward to every tester, or do they apply and you pick the ones you want?
    - mephitix 9y ago
      Once you create the test, you can open it up for applications. You can also search based on the demographic you want (age range, geographic, etc.) and invite users yourself. If people apply, you can pick from those. I invited about 60 testers and had ~180 others apply. I was able to pick from that 180 set - and the reward goes only to them. Also, before you start the test, you buy 'Beta' credits. You can choose to use these however you want, for example if you buy 50 beta credits, you can give a $5 reward to 10 testers, or a $10 reward to 5 testers. You can buy more credits as you go if you want to offer rewards to more testers.
greenwalls 9y ago
I tried this with an Android app and it went well for us. The only problem was that the UI was super slow in Firefox, but it worked well in Chrome. That was several months ago so hopefully they fixed it by now.
le-mark 9y ago
I would definitely use a service like this. The last time a launched a side project I found it extremely difficult to engage users, even users that had signed up with an email. They simply wouldn't respond to anything; feedback requests, surveys, nothing. I took this as a strong signal there was no market or that I'd done something horribly wrong in the implementation. If I could've just paid $X to get some honest feedback, would have saved me a lot of time.
- rom16384 9y ago
  With https://www.usertesting.com/ https://www.usertesting.com/ you can get videos of users using your app and answer some questions. It's useful for usability testing.
duren 9y ago
> 50,000 testers representing 207 nationalities Wait, how many nationalities are there?
- unkown-unknowns 9y ago
  https://www.quora.com/How-many-nationalities-are-there-in-the-world https://www.quora.com/How-many-nationalities-are-there-in-th...
- nfriedly 9y ago
  There's more than 200 countries listed on https://en.wikipedia.org/wiki/List_of_sovereign_states https://en.wikipedia.org/wiki/List_of_sovereign_states (I didn't actually count them, but jQuery says the table has 238 rows; a few of these are headers/footers, but there's still enough to have more than 207.) That isn't exactly equal to a nationality, but it's probably a good minimum - I could easily believe that there are more than 207 nationalities.
- merb 9y ago
  the un has 193 members and there are still 54 countries that aren't a member of the un, so, it might be possible that they have that many nationalities, but I doubt that this is untrue.
0x0 9y ago
Interesting that they list "iTunes Connect internal testflight" as a supported distribution method (in addition to external testflight). That must surely be against some kind of terms of service and developer agreement. Also, it's a pretty bad idea either way because you would have to grant the testers access to your iTunesConnect. :O
- warent 9y ago
  That's not true. Testflight is clunky but secure and gets the job done. You just enter in the email addresses of people you want to receive a test version, and those people need the Testflight app to use it. IIRC You can even control which users are distributed which version Out of curiosity, have you personally used Testflight? Maybe it's changed since I last did
  - bananaboy 9y ago
    That's how TF works right now. Not sure where the parent's complaints come from.
  - 0x0 9y ago
    Note the difference between testflight internal and testflight external.
nkkollaw 9y ago
Really cool design. There are a few of these services, but I always wonder how engaged are the testers. I would assume most of them don't really care about trying your app. I'd then much rather post it on https://www.reddit.com/r/alphaandbetausers/ https://www.reddit.com/r/alphaandbetausers/, although it's not amazingly active.
- adventured 9y ago
  In the end of course, nothing will ever properly replace real users that actually care about your product. There's nothing Beta Family can ever do to match that. A service that is serious about improving, will focus on actual users that want to use the product because it meets a need. Any attempt to automate away that hard, required process of business improvement, must necessarily result in a hollow outcome.
  - nkkollaw 9y ago
    I think one way to get people who are genuinely interested would be for you to build a mailing list. I always got good users from Beta List.
  - betafamily 9y ago
    We do agree that your own "fans" are highly important. That is why we encourage app companies to combine their own fans with the testers of our community.
- betafamily 9y ago
  Thanks! We do have a lot of highly engaged testers: https://betafamily.com/toplist https://betafamily.com/toplist
covercash 9y ago
I've had a similar idea to this in the past, not only to test but also to seed apps with some initial users. My incentive to get users signed up is reserved usernames. I want covercash as my first choice username for EVERY new app/service. I'm still pissed I slept on Instagram for too long. Give me first crack at my username on all new apps and I'll kick the tires for them.
- goldenkey 9y ago
  You should probably treat the mental disease first and foremost....
draugadrotten 9y ago
Clicked on available tests, found a listing for "funimate pro lip sync" Lip sync sounds like fun so I clicked. Was immediately presented with a test but not what I expected. Instead of a lip sync video maker, it was for a "Bitcoin Price Tracker ACrypto - Test is looking for testers. Part of the Funimate Pro Version lip sync music video maker testing family" My trust evaporated right there.
- betafamily 9y ago
  Hi, we are lookig in to this.
mephitix 9y ago
Lot of comments putting this service down - but I love it. I used them for a beta launch. Got ~180 people who applied for it. Got a ton of honest feedback that helped me plan for my next beta round. The actual website is a bit slow and not so intuitive, but has a bunch of features. Support is great. Having people put in their device UDIDs makes the process faster to register in the Apple dev portal. There are other services I'm planning to try out, like http://erlibird.com http://erlibird.com and http://playtestcloud.com http://playtestcloud.com. PlayTestCloud will record audio (maybe video?) of people trying out your app/game. I imagine it depends on the capabilities of the device OS though.
- mkilling 9y ago
  (PlaytestCloud co-founder) We record the players' screens (with their permission) in iOS, Android and browser games. We offer a free trial - would love for you to try it out and let us know what you think!
  - mephitix 9y ago
    Thanks - I already reached out. I'm excited to try it out for another game in Sept! Your support team has already been very helpful.

150 points by mcone 9y ago

Interesting that it's a device enrollment, but also a tad scary. There seems to be a lot of trust that you want your testers to give, and there was basically no heads up to the user. EDIT: I see what they're doing now, crafty, it doesn't actually install any profiles in the end they're just capturing your UDID instead of entering it manually. EDIT 2: Nice, I was expecting survey spam: https://betafamily.com/app/12089/feed-item/28330 https://betafamily.com/app/12089/feed-item/28330

Raphmedia 9y ago
I hate to be that person but in the requirements of that link, is "Requirements [...] Male & Female" really needed? How is this a requirement? This would be better written as "No gender requirement" or simply removing that line unless only one gender is specified.
- arkitaip 9y ago
  There's also the 13-99 age requirement for, you know, when you are ignorant to believe that your market really is everyone between 13 and 99.
  - Operyl 9y ago
    Seems to be going well for the people asking testers to fill out their paid surveys lol.

zerkten 9y ago

This looks like an interesting site. The immediate focus is towards the developer of the apps. I wonder if it'd be more successful if the initial site focus was on the apps (read: opportunities) listed under Browse Tests. Getting a bigger base of people who like to test stuff apps would seem like a pretty valuable thing and I had to think for a few moments about how I can be on the other side. Maybe a less persistent person would have browsed away.

abol3z 9y ago

My app is in Arabic language, do you have testers for Arabic?

mabermoske 9y ago

From the page: "6 000 tests completed, trusted by 6 000 companies..." They've had no repeat customers?

jermaustin1 9y ago
I want this answered. How have only 6000 tests been completed with 6000 customers? Even if a single customer ran just two tests, the numbers wouldn't match. This leads me to believe both numbers are fake.
- lugg 9y ago
  Or accidental variable reuse in a template. Could be either.
  - jageen 9y ago
    In this case they forget to use their own service for test.
    - betafamily 9y ago
      yeah! We do need to test our service better ;)
- betafamily 9y ago
  Hi! The numbers are out of date, we need to update them. The real number is about 3x bigger...
mygo 9y ago
Maybe they rounded?

mongodude 9y ago

I have used this service, does not work as advertised. We hardly got 2-3 testers to test our App.

fastball 9y ago
What was your reward value?
mephitix 9y ago
I'm guessing you didn't put up a reward. I put just a $5 reward per tester and got ~100 testers who applied within 2 weeks.
- giarc 9y ago
  Do you have to provide the reward to every tester, or do they apply and you pick the ones you want?
  - mephitix 9y ago
    Once you create the test, you can open it up for applications. You can also search based on the demographic you want (age range, geographic, etc.) and invite users yourself. If people apply, you can pick from those. I invited about 60 testers and had ~180 others apply. I was able to pick from that 180 set - and the reward goes only to them. Also, before you start the test, you buy 'Beta' credits. You can choose to use these however you want, for example if you buy 50 beta credits, you can give a $5 reward to 10 testers, or a $10 reward to 5 testers. You can buy more credits as you go if you want to offer rewards to more testers.

greenwalls 9y ago

I tried this with an Android app and it went well for us. The only problem was that the UI was super slow in Firefox, but it worked well in Chrome. That was several months ago so hopefully they fixed it by now.

le-mark 9y ago

I would definitely use a service like this. The last time a launched a side project I found it extremely difficult to engage users, even users that had signed up with an email. They simply wouldn't respond to anything; feedback requests, surveys, nothing. I took this as a strong signal there was no market or that I'd done something horribly wrong in the implementation. If I could've just paid $X to get some honest feedback, would have saved me a lot of time.

rom16384 9y ago
With https://www.usertesting.com/ https://www.usertesting.com/ you can get videos of users using your app and answer some questions. It's useful for usability testing.

duren 9y ago

> 50,000 testers representing 207 nationalities Wait, how many nationalities are there?

unkown-unknowns 9y ago
https://www.quora.com/How-many-nationalities-are-there-in-the-world https://www.quora.com/How-many-nationalities-are-there-in-th...
nfriedly 9y ago
There's more than 200 countries listed on https://en.wikipedia.org/wiki/List_of_sovereign_states https://en.wikipedia.org/wiki/List_of_sovereign_states (I didn't actually count them, but jQuery says the table has 238 rows; a few of these are headers/footers, but there's still enough to have more than 207.) That isn't exactly equal to a nationality, but it's probably a good minimum - I could easily believe that there are more than 207 nationalities.
merb 9y ago
the un has 193 members and there are still 54 countries that aren't a member of the un, so, it might be possible that they have that many nationalities, but I doubt that this is untrue.

0x0 9y ago

Interesting that they list "iTunes Connect internal testflight" as a supported distribution method (in addition to external testflight). That must surely be against some kind of terms of service and developer agreement. Also, it's a pretty bad idea either way because you would have to grant the testers access to your iTunesConnect. :O

warent 9y ago
That's not true. Testflight is clunky but secure and gets the job done. You just enter in the email addresses of people you want to receive a test version, and those people need the Testflight app to use it. IIRC You can even control which users are distributed which version Out of curiosity, have you personally used Testflight? Maybe it's changed since I last did
- bananaboy 9y ago
  That's how TF works right now. Not sure where the parent's complaints come from.
- 0x0 9y ago
  Note the difference between testflight internal and testflight external.

nkkollaw 9y ago

Really cool design. There are a few of these services, but I always wonder how engaged are the testers. I would assume most of them don't really care about trying your app. I'd then much rather post it on https://www.reddit.com/r/alphaandbetausers/ https://www.reddit.com/r/alphaandbetausers/, although it's not amazingly active.

adventured 9y ago
In the end of course, nothing will ever properly replace real users that actually care about your product. There's nothing Beta Family can ever do to match that. A service that is serious about improving, will focus on actual users that want to use the product because it meets a need. Any attempt to automate away that hard, required process of business improvement, must necessarily result in a hollow outcome.
- nkkollaw 9y ago
  I think one way to get people who are genuinely interested would be for you to build a mailing list. I always got good users from Beta List.
- betafamily 9y ago
  We do agree that your own "fans" are highly important. That is why we encourage app companies to combine their own fans with the testers of our community.
betafamily 9y ago
Thanks! We do have a lot of highly engaged testers: https://betafamily.com/toplist https://betafamily.com/toplist

covercash 9y ago

I've had a similar idea to this in the past, not only to test but also to seed apps with some initial users. My incentive to get users signed up is reserved usernames. I want covercash as my first choice username for EVERY new app/service. I'm still pissed I slept on Instagram for too long. Give me first crack at my username on all new apps and I'll kick the tires for them.

goldenkey 9y ago
You should probably treat the mental disease first and foremost....

draugadrotten 9y ago

Clicked on available tests, found a listing for "funimate pro lip sync" Lip sync sounds like fun so I clicked. Was immediately presented with a test but not what I expected. Instead of a lip sync video maker, it was for a "Bitcoin Price Tracker ACrypto - Test is looking for testers. Part of the Funimate Pro Version lip sync music video maker testing family" My trust evaporated right there.

betafamily 9y ago
Hi, we are lookig in to this.

mephitix 9y ago

Lot of comments putting this service down - but I love it. I used them for a beta launch. Got ~180 people who applied for it. Got a ton of honest feedback that helped me plan for my next beta round. The actual website is a bit slow and not so intuitive, but has a bunch of features. Support is great. Having people put in their device UDIDs makes the process faster to register in the Apple dev portal. There are other services I'm planning to try out, like http://erlibird.com http://erlibird.com and http://playtestcloud.com http://playtestcloud.com. PlayTestCloud will record audio (maybe video?) of people trying out your app/game. I imagine it depends on the capabilities of the device OS though.

mkilling 9y ago
(PlaytestCloud co-founder) We record the players' screens (with their permission) in iOS, Android and browser games. We offer a free trial - would love for you to try it out and let us know what you think!
- mephitix 9y ago
  Thanks - I already reached out. I'm excited to try it out for another game in Sept! Your support team has already been very helpful.