6 ms·

TestFlight Beta Testing

72 points by npollock 12y ago

nixarn 12y ago
It's about time! It's been such a hassle always needing the device id, creating the provisioning profiles, having to recreate the profiles and rebuild the project for every user you add. And also being limited to only 100 devices.
- soup10 12y ago
  Testflight has been around for a while. was using them for beta tests a couple years ago. looks like apple acquired them and made it official.
  - dblacc 12y ago
    Yeah I think it was apparent that was the plan when they acquired them. It just took them soo long to get it integrated. Using testflight as it was, was good but it was a little painful. This in comparison should be a lot more simplified and smooth.
- jdhawk 12y ago
  Seriously, this was one of the most painful pieces of iOS devel...
atlbeer 12y ago
Kudo's to them on branding. The name was so good even Apple decided to keep it!
div 12y ago
I'm using both Apple TestFlight and the original TestFlight, and the original is just such a smoother experience, even with having to add UDID's to provisioning profiles. Things that are annoying about Apple TestFlight compared to the original: - itunesconnect, while better than before, pales in comparison to the simplicity of testflightapp - builds submitted to Apple "take a while" before they're available, testflightapp is instant once you've uploaded. - managing users in groups seems to be absent
- colinbartlett 12y ago
  Last week I tested 7 builds with my team in the span of an hour. This all worked great because I was using original TestFlight which allows me to distribute immediately. Waiting on Apple for each one of those builds? No, thank you.
- jobu 12y ago
  The "Activity" screen on the original TestFlight was also a key feature when working with outside beta testers. It made it easy to see where users got stuck in the process and coach/prod them through it.
Someone1234 12y ago
> Apps made available to external testers require a Beta App Review and must comply with the full App Store Review Guidelines before testing can begin. A review is required for new versions of your app that contain significant changes. Up to 10 apps can be tested at a time, internally or externally. Eww. Still doesn't hold a candle to an APK.
- jc4p 12y ago
  Or using an enterprise certificate.
  - chimeracoder 12y ago
    > Or using an enterprise certificate. Be careful with this! I see a lot of companies using this as a way to avoid the pains of obtaining UUIDs (for testing) or of distribution on the App Store, if their production product is not intended for public use. But if you read the fine print for enterprise certificates on iOS, they're not actually supposed to be used for distributing applications for test-use on-site, or non-test use off-site ("site" in this case referring to your physical office), or for use by non-employees (this includes contractors!). It's completely ridiculous, and I was shocked to discover this, but as far as I can tell, it's true (I even had a phone call with an Apple rep who explained the various certificates to me). Of course, if you're operating on a small scale they're unlikely to find out if you're using the enterprise certs in this way. But it's still a risk to be aware of.
    - eropple 12y ago
      One of my prior employers was using an enterprise certificate to deploy an application to clients, themselves in an enterprise environment. Our Apple rep told us, in no uncertain terms, that we were fine. So maybe not everybody's on the same page...?
      - christoph 12y ago
        I think the rule of thumb here is common sense. I know for certain a large number of med comms agencies in the pharmaceutical industry use enterprise certs for "internal" app distribution (think one off events, internal meetings etc. maybe needing to be run on 500 devices for 1 day only), where content may not be legally signed off/approved until 2am for an 8am start the next day. There's simply no way (other than jailbreaking) to get around this at the moment. If Apple were to start enforcing this with an iron fist, a lot of this industry (which is, believe it or not, responsible for a lot of device sales) would quickly migrate over to Android, now the devices look and function as well as their iOS counterparts. This doesn't hurt Apple's ecosystem - everyone is running Macbooks, iPads, iPhones, paying enterprise fees, etc. Most of these apps have no place clogging up the Appstore or consuming Apple reviewers time. They just don't want you buying an enterprise cert to push out your emulator bundled with illegal ROMs or illegal movie streaming app, easily downloaded by all & sundry off a public web server. This is why they put up the barriers of DUNS numbers, brief phone interviews, etc., before they will issue you an enterprise account.
marpalmin 12y ago
Is it only for ios8?
- Scorponok 12y ago
  Yep.
drewcrawford 12y ago
Tester? Seeking tester? I'm running a list to connect developers to people who want to test their apps. All feedback appreciated. http://sealedabstract.com/betalist http://sealedabstract.com/betalist
- chrisBob 12y ago
  >This list connects people who want testers, with people who need them. I get what you mean, but this doesn't quite make sense.
  - drewcrawford 12y ago
    Good point. Better now?
- winslow 12y ago
  No android love? Also no linux option under operating system?
chrisBob 12y ago
Does anyone have experience yet with the timeline for the beta app review required for external testers? This is significant because I feel like I would just have internal users bring their device over, or install it themselves from Xcode. Its the external users that seem the most important with this. My regular submissions have ranged from 7 days to 20 hours, but I would hope they will keep the beta review to the shorter end of that.
- Scorponok 12y ago
  I'm very curious too! If it takes anything more than a day to get an app review, this tool is going to be pretty useless for us. :( I've submitted an update to our app to external app review, so hopefully I'll have more information soon. [Update: Now "In Review", for what that's worth.]
  - Scorponok 12y ago
    And we just got approved! Took ~2 hours.
    - chrisBob 12y ago
      Wow. Thats not bad at all. It must be a pretty cursory look, or maybe just a static analysis looking for something specific. That's good news
    - marpalmin 12y ago
      One a half hour for a new app
    - davier26 12y ago
      ~4 hours and still in review...
    - mrborgen 12y ago
      Took about half a day for me. The waiting time is well worth the 900 extra beta testers you can add. At least for us.
deleted 12y ago
[deleted]
Rafert 12y ago
Sorry to see that the older TestFlight seems to be losing functionality. Sessions show up as anonymous and remote logging doesn't work for me anymore. I'm thinking of switching to http://hockeyapp.net/ http://hockeyapp.net/
mrborgen 12y ago
I read that the beta app review only applies for the 1.0 version, or if you make significant changes. Not sure about the time frame though.
iheart2code 12y ago
It's disappointing seeing how TestFlight has been integrated into iTunes Connect. What mostly upsets me is the seemingly arbitrary review before the app is released to testers. It's exactly this kind of a delay ("several hours") that keeps me from using Google Play for Android app beta & alpha testing. I end up with tight deadlines and my testers include clients & superiors. I can't afford a variable chunk of time between app compilation and release to testers. I've had really good experiences with Crashlytics and have heard good things about Hockeyapp. I'll stick with 3rd party distribution for the forseeable future.
mcmatterson 12y ago
I'm going to do a bit of shameless self-promotion for my CLI tool for ad-hoc deployments, hawk: https://github.com/mtrudel/hawk https://github.com/mtrudel/hawk Using a bit of XML glue, it uses simple building blocks of xcodebuild, S3, and email to give you one command building and distribution of your app, all driven by a plaintext Hawkfile. I (and a few colleagues) have been using it for a couple of years now, and it's been a real boon. Would love feedback.

72 points by npollock 12y ago

It's about time! It's been such a hassle always needing the device id, creating the provisioning profiles, having to recreate the profiles and rebuild the project for every user you add. And also being limited to only 100 devices.

soup10 12y ago
Testflight has been around for a while. was using them for beta tests a couple years ago. looks like apple acquired them and made it official.
- dblacc 12y ago
  Yeah I think it was apparent that was the plan when they acquired them. It just took them soo long to get it integrated. Using testflight as it was, was good but it was a little painful. This in comparison should be a lot more simplified and smooth.
jdhawk 12y ago
Seriously, this was one of the most painful pieces of iOS devel...

atlbeer 12y ago

Kudo's to them on branding. The name was so good even Apple decided to keep it!

div 12y ago

I'm using both Apple TestFlight and the original TestFlight, and the original is just such a smoother experience, even with having to add UDID's to provisioning profiles. Things that are annoying about Apple TestFlight compared to the original: - itunesconnect, while better than before, pales in comparison to the simplicity of testflightapp - builds submitted to Apple "take a while" before they're available, testflightapp is instant once you've uploaded. - managing users in groups seems to be absent

colinbartlett 12y ago
Last week I tested 7 builds with my team in the span of an hour. This all worked great because I was using original TestFlight which allows me to distribute immediately. Waiting on Apple for each one of those builds? No, thank you.
jobu 12y ago
The "Activity" screen on the original TestFlight was also a key feature when working with outside beta testers. It made it easy to see where users got stuck in the process and coach/prod them through it.

Someone1234 12y ago

> Apps made available to external testers require a Beta App Review and must comply with the full App Store Review Guidelines before testing can begin. A review is required for new versions of your app that contain significant changes. Up to 10 apps can be tested at a time, internally or externally. Eww. Still doesn't hold a candle to an APK.

jc4p 12y ago
Or using an enterprise certificate.
- chimeracoder 12y ago
  > Or using an enterprise certificate. Be careful with this! I see a lot of companies using this as a way to avoid the pains of obtaining UUIDs (for testing) or of distribution on the App Store, if their production product is not intended for public use. But if you read the fine print for enterprise certificates on iOS, they're not actually supposed to be used for distributing applications for test-use on-site, or non-test use off-site ("site" in this case referring to your physical office), or for use by non-employees (this includes contractors!). It's completely ridiculous, and I was shocked to discover this, but as far as I can tell, it's true (I even had a phone call with an Apple rep who explained the various certificates to me). Of course, if you're operating on a small scale they're unlikely to find out if you're using the enterprise certs in this way. But it's still a risk to be aware of.
  - eropple 12y ago
    One of my prior employers was using an enterprise certificate to deploy an application to clients, themselves in an enterprise environment. Our Apple rep told us, in no uncertain terms, that we were fine. So maybe not everybody's on the same page...?
    - christoph 12y ago
      I think the rule of thumb here is common sense. I know for certain a large number of med comms agencies in the pharmaceutical industry use enterprise certs for "internal" app distribution (think one off events, internal meetings etc. maybe needing to be run on 500 devices for 1 day only), where content may not be legally signed off/approved until 2am for an 8am start the next day. There's simply no way (other than jailbreaking) to get around this at the moment. If Apple were to start enforcing this with an iron fist, a lot of this industry (which is, believe it or not, responsible for a lot of device sales) would quickly migrate over to Android, now the devices look and function as well as their iOS counterparts. This doesn't hurt Apple's ecosystem - everyone is running Macbooks, iPads, iPhones, paying enterprise fees, etc. Most of these apps have no place clogging up the Appstore or consuming Apple reviewers time. They just don't want you buying an enterprise cert to push out your emulator bundled with illegal ROMs or illegal movie streaming app, easily downloaded by all & sundry off a public web server. This is why they put up the barriers of DUNS numbers, brief phone interviews, etc., before they will issue you an enterprise account.

marpalmin 12y ago

Is it only for ios8?

Scorponok 12y ago
Yep.

drewcrawford 12y ago

Tester? Seeking tester? I'm running a list to connect developers to people who want to test their apps. All feedback appreciated. http://sealedabstract.com/betalist http://sealedabstract.com/betalist

chrisBob 12y ago
>This list connects people who want testers, with people who need them. I get what you mean, but this doesn't quite make sense.
- drewcrawford 12y ago
  Good point. Better now?
winslow 12y ago
No android love? Also no linux option under operating system?

chrisBob 12y ago

Does anyone have experience yet with the timeline for the beta app review required for external testers? This is significant because I feel like I would just have internal users bring their device over, or install it themselves from Xcode. Its the external users that seem the most important with this. My regular submissions have ranged from 7 days to 20 hours, but I would hope they will keep the beta review to the shorter end of that.

Scorponok 12y ago
I'm very curious too! If it takes anything more than a day to get an app review, this tool is going to be pretty useless for us. :( I've submitted an update to our app to external app review, so hopefully I'll have more information soon. [Update: Now "In Review", for what that's worth.]
- Scorponok 12y ago
  And we just got approved! Took ~2 hours.
  - chrisBob 12y ago
    Wow. Thats not bad at all. It must be a pretty cursory look, or maybe just a static analysis looking for something specific. That's good news
  - marpalmin 12y ago
    One a half hour for a new app
  - davier26 12y ago
    ~4 hours and still in review...
  - mrborgen 12y ago
    Took about half a day for me. The waiting time is well worth the 900 extra beta testers you can add. At least for us.

deleted 12y ago

[deleted]

Rafert 12y ago

Sorry to see that the older TestFlight seems to be losing functionality. Sessions show up as anonymous and remote logging doesn't work for me anymore. I'm thinking of switching to http://hockeyapp.net/ http://hockeyapp.net/

mrborgen 12y ago

I read that the beta app review only applies for the 1.0 version, or if you make significant changes. Not sure about the time frame though.

iheart2code 12y ago

It's disappointing seeing how TestFlight has been integrated into iTunes Connect. What mostly upsets me is the seemingly arbitrary review before the app is released to testers. It's exactly this kind of a delay ("several hours") that keeps me from using Google Play for Android app beta & alpha testing. I end up with tight deadlines and my testers include clients & superiors. I can't afford a variable chunk of time between app compilation and release to testers. I've had really good experiences with Crashlytics and have heard good things about Hockeyapp. I'll stick with 3rd party distribution for the forseeable future.

mcmatterson 12y ago

I'm going to do a bit of shameless self-promotion for my CLI tool for ad-hoc deployments, hawk: https://github.com/mtrudel/hawk https://github.com/mtrudel/hawk Using a bit of XML glue, it uses simple building blocks of xcodebuild, S3, and email to give you one command building and distribution of your app, all driven by a plaintext Hawkfile. I (and a few colleagues) have been using it for a couple of years now, and it's been a real boon. Would love feedback.