4 ms·

> The app is a full-fat release, and is constantly being vetted by Apple. Huh? How does Apple know what your users want?

by anamax 5y ago

ChrisMarshallNY 5y ago
Why would I think that Apple will know what my users want? Apple vets TestFlight, the same way they vet release apps; except not as stridently. They look for things like obvious crash bugs (indeed, they have sometimes found bugs I missed in my testing), security issues, private API access, obvious HIG deviations, etc. They do an intensive, human, test, every time I bump the fix version, and fast, automated tests, when I do release bumps. The intensive tests usually take a couple of days, but the release tests take about 15 minutes. I generally do version bumps, once a month or so, but several release bumps, every day. TestFlight is simply another Quality tool. Apple’s testing is yet another set of eyeballs on the product, which is always welcome, and, more importantly, it allows me to put working, near-release-quality product into the hands of people that represent the target demographics, very early in the project. The product has gone through tens of thousands of testing sessions, by a number of folks, that represent the target users, but aren’t actually unrestricted end users. These are not “kludgy, crashy, hold-your-nose” testing sessions. These are “running release-quality code” testing sessions. I often find obscure, weird, bugs, minutes after creating them, and that is awesome. It’s impossible to overstate its importance to developing a high-quality, relevant, and robust end product. I have made over 600 TestFlight releases, during a development cycle of 18 months. I often make several releases, every day. That’s quite a statement. I have not encountered another team that has done anything close to this. It means that the product has been beta-quality, since one month after code start (it’s actually a technique that’s very old. I call it “constant beta”), and has been under daily end-user evaluation, the entire time, without risk of brand damage, and with the ability to do massive pivots (which have occurred multiple times). These pivots are possible, because we don’t have a large end-user base that we need to keep happy. I have just nuked the entire database, several times. Try that, with an established customer base.

> The app is a full-fat release, and is constantly being vetted by Apple. Huh? How does Apple know what your users want?

by anamax 5y ago

> The app is a full-fat release, and is constantly being vetted by Apple. Huh? How does Apple know what your users want?

ChrisMarshallNY 5y ago

Why would I think that Apple will know what my users want? Apple vets TestFlight, the same way they vet release apps; except not as stridently. They look for things like obvious crash bugs (indeed, they have sometimes found bugs I missed in my testing), security issues, private API access, obvious HIG deviations, etc. They do an intensive, human, test, every time I bump the fix version, and fast, automated tests, when I do release bumps. The intensive tests usually take a couple of days, but the release tests take about 15 minutes. I generally do version bumps, once a month or so, but several release bumps, every day. TestFlight is simply another Quality tool. Apple’s testing is yet another set of eyeballs on the product, which is always welcome, and, more importantly, it allows me to put working, near-release-quality product into the hands of people that represent the target demographics, very early in the project. The product has gone through tens of thousands of testing sessions, by a number of folks, that represent the target users, but aren’t actually unrestricted end users. These are not “kludgy, crashy, hold-your-nose” testing sessions. These are “running release-quality code” testing sessions. I often find obscure, weird, bugs, minutes after creating them, and that is awesome. It’s impossible to overstate its importance to developing a high-quality, relevant, and robust end product. I have made over 600 TestFlight releases, during a development cycle of 18 months. I often make several releases, every day. That’s quite a statement. I have not encountered another team that has done anything close to this. It means that the product has been beta-quality, since one month after code start (it’s actually a technique that’s very old. I call it “constant beta”), and has been under daily end-user evaluation, the entire time, without risk of brand damage, and with the ability to do massive pivots (which have occurred multiple times). These pivots are possible, because we don’t have a large end-user base that we need to keep happy. I have just nuked the entire database, several times. Try that, with an established customer base.