7 ms·

Apple, It’s Time To Block iOS 5 Beta Users From Reviewing Apps

163 points by davidedicillo 15y ago

pat2man 15y ago
This is just obvious. Isn't pointing out a bug thats due to an unreleased OS breaking your NDA anyway?
- stevenwei 15y ago
  Yeah, but non-developers who manage to install iOS 5 probably haven't actually signed the NDA. The funny thing is as a developer you're not even allowed to mention in your description that something in your app is broken on iOS 5, since that would be breaking the NDA.
  - psychotik 15y ago
    Apple also doesn't like it when you fix an issue, release an update and mention that you fixed a bug which affected iOS 5 compatibility. I needed to change it say "compatibility fixes for future iOS versions".
spacemanaki 15y ago
I pointed this out the last time this came up, but an additional measure would be to also crack down on sites that sell access to iOS betas, if they aren't doing that already. That would (likely) drastically reduce the number of non-developers running beta OSes since isn't it $100/year for a developer account, but you can install betas on many devices? There are sites selling access for less than $10. If it's not against the TOS of the developer accounts, it definitely should be.
- zhoutong 15y ago
  Apparently Apple has noticed this issue and they have taken down a few sites. (http://appadvice.com/appnn/2011/08/apple-takes-steps-against-developers-selling-ios-5-access-what-you-need-to-know http://appadvice.com/appnn/2011/08/apple-takes-steps-against...) In fact Apple can try to increase the technical barrier to install iOS beta. (Must upgrade in Xcode or must own at least one app in App Store to be eligible for beta access.) This way can block the vast majority of non-developers while not affecting real iOS developers.
  - silencio 15y ago
    There was speculation that that might have just been betas expiring, and not developers selling access getting their access revoked (http://www.macrumors.com/2011/08/05/apple-disabling-ios-5-beta-installs-and-developer-accounts-over-sales-of-device-slots/ http://www.macrumors.com/2011/08/05/apple-disabling-ios-5-be...). > must own at least one app in App Store to be eligible for beta access That doesn't work for many reasons. App Review, as if they didn't have enough of a hard time with a bad reputation, would just be flooded by trash from first time developers or contract developers/companies that just created an account. Apple could make it mandatory to install using Xcode, but in the past, Xcode upgrades have bricked devices where iTunes doesn't. So I have a feeling that's not really a priority and more of a burden. Plus, it's braindead simple to install and use Xcode to install betas, and anyone desperate enough to pay $10 but not $100 should easily be able to get around that. I really think all Apple needs to do is to block reviews from iOS 5 devices. Developers running betas won't be reviewing apps in the store (that's incredibly hypocritical and cruel if so) so nobody's going to be missing that. This doesn't harm beta testers and devs that want to install iOS betas.
    - dunham 15y ago
      I'd also block reviews from beta versions of iTunes, since many reviews are entered via iTunes.
  - shaggyfrog 15y ago
    Your proposals take things a bit too far without any real return. Blocking access to devs who don't have an app in the store is counterproductive, because that prevents the developers from testing against the latest iOS and finding bugs -- which is the entire point of access to the iOS beta! Not to mention some devs might be working for other clients and not yet shipped something themselves (as in my case). Requiring installs via Xcode would only be a temporary slowdown before someone figures out the secret handshake and hacks around it. Blocking reviews from devices with the latest iOS beta installed should be sufficient to eliminate most of these unfair reviews.
    - 15y ago
jhawk28 15y ago
An alternative fix would be to have the iOS version used when submitting a review. It doesn't limit "free speech", but it gives context on what the problem actually was.
- j79 15y ago
  I think this is a great idea. Apple could allow filtering to see _IF_ I really want to upgrade to iOS 5 right away or wait until iOS 5 reviewers are happy.
deleted 15y ago
[deleted]
kogir 15y ago
They should also ban reviews from jailbroken devices, where users may have installed custom fonts or other "features" for which you can't possibly test and account. And yes -- any test Apple does for this could be defeated, but I expect most jailbreak developers would side with their app writing comrades and let it stand.
- Xuzz 15y ago
  The reason that Cydia doesn't have reviews (or even ratings) is that we haven't found a way to make a working review system either, so we don't even try. I don't think that any of us would argue or work against something like that, but I also seriously doubt that Apple will do it. (I'm a contributor to the Cydia open source project, but I don't speak for saurik or Cydia, and that's probably not the official stance.)
  - saurik 15y ago
    So, I /am/ saurik, and I probably /would/ argue with and work around something like that, as I think I should be allowed to review things in the App Store. Really, the "official stance" I have is that developers who believe that having their application fail horribly when the platform is even slightly different is not their fault are deluded: Apple is going to change the platform over time, and when your app fails for Apple they tend to get pissed and just block your app from the store and may even de-prioritize your updates. Seriously: if your app fails because the font is different, there is something epically wrong with your coding methodology; and yes: Apple actually did change the font slightly for the iPhone 4 (on retina screens they switched to Helvetica Neue from Helvetica). Meanwhile, in desktop-land, everyone from large companies to small independent developers deal with this supposedly horrible problem of random fonts, different operating system versions, upgraded libraries, client-side firewalls, or any other number of things that would make developers like kogir freak out, and somehow the world doesn't end, and almost all of the software I download works without issue (with the exception of Chrome on Tablet editions of Windows, and that's squarely Google's fault). (Hell: even almost all of the software on the App Store works under these conditions. What should /really/ happen is that Apple should internally have a different "test version" of iOS that developers are not allowed to see; one that has a newer version of SQLite (which I state in specific as developers make all kinds of incorrect assumptions about the SQLite ABI) and has every core library compiled to the opposite instruction set (Thumb if normally ARM, and vice versa), that has every font replaced with Comic Sans, and uses a different pixel density, on a screen with a slightly different aspect ratio, using a CPU that runs slightly faster than the shipping devices. The reason this needs to be done is that an app that fails to run correctly in that (hidden) configuration is pretty much guaranteed to be an app that is going to stop working when any of the following occur: iOS 6, iPhone Nano, iPad+Retina, or another "game changing" device (such as the iPad when it came out). Rather than let these apps in the store just to fail later, they should be weeded out before they even get in. (And yes: Cydia itself strives not to make any assumptions about any of the things listed, and in fact worked almost entirely correctly when loaded for the first time on an iPad with the flag flipped for "don't show me in a 1x/2x box"; certainly well enough that the reviewer in this scenario would have given it a pass were they testing it in that environment. The one place it tends to get burnt is with what it does to WebKit; and the current implementation, which works fine on everything from 2.0 through 5.0, is likely stable enough to "stand the test of time".) (FWIW: I do believe that people using beta versions of iOS should not be able to review applications. These people are using a version of the firmware that is under NDA... they should not be tempted to be providing feedback that might violate that agreement to applications that fail to operate correctly, possibly due to no fault of the app. I have a good example of this related to Cydia on 5.x, but I can't tell you what it is, for this very reason ;P.)
benatkin 15y ago
This reminds me of how PyPI ratings got disabled after a long discussion with the community, started by jacobian: http://code.activestate.com/lists/python-catalog-sig/3482/ http://code.activestate.com/lists/python-catalog-sig/3482/ I strongly agree that the PyPI ratings were mostly useless and that removing the feature was a good decision. iPhone app ratings can be much more useful for popular apps, but for the long tail of apps, they suffer the same problems as PyPI's rating and review system did. Amazon and Yelp ratings, on the other hand, are quite useful. I think it's partly because users have profiles and are better able to express themselves.
- dman 15y ago
  It might also be the fact that Amazon and Yelp do a much better job of curating the ratings.
- ojilles 15y ago
  I think the app ratings for the popular apps are all useless (they're well know apps, other people rave about them, so no need to read all the reviews). I agree on the long tail being useless as well. Instead, I think it's most useful for the "middle tail": those apps that do get a fair bit of usage, but are not considered popular: the developer may not have build up a reputation yet, but the reviews help confirming the application does what it is supposed to do (or not of course).
chris24 15y ago
Apple should remove the reviews section from the App Store for people running pre-release versions of iOS and replace it with a form to send a bug report to the app developer (and possibly even a place for a list of known issues that the app has on iOS 5).
- hellweaver666 15y ago
  Apple should just add a bug reports section to the app store period.
  - vaporstun 15y ago
    Potentially, but if so, it should be something which you'd have to opt into or open only for fellow developers. The last thing I want is to get spammed with "bug reports" from lay users with trivial or non-existent issues. If they implemented this but made it take even one extra step like flipping some switch in the settings, it would filter out most lay users and garner some more useful feedback.
msftguy 15y ago
Maybe Apple doesn't block beta reviews because it incentivizes developers to fix their apps before the release? Most of those problems should be trivial to at least work around with version-dependent logic to minimizes regressions on versions < 5.x. Bad reviews don't exactly hurt Apple, and it's not like iOS developers are scarce these days, so it might just be Apple having their priorities set straight (users > devs) and not a coding blunder..or maybe it's just me overthinking it.
- X-Istence 15y ago
  Except that developers are not even given the chance to fix issues before Johnny Public is allowed to start rating apps. The whole point of the beta is to ALLOW the developer to test their apps and find the bugs, as well as to iron out the bugs in the core OS. What if it is an issue with a core API that is now returning something that it shouldn't and it wasn't caught by Apple in testing, then the developer is getting blamed for something they can't even fix. You have to be a part of the developer program ($99) to get pre-release software, and as part of that you go with the understanding that it isn't meant for everyday use. Johnny Public shouldn't be running the beta in the first place. If by the time iOS 5 is out the developer still hasn't fixed the bugs in their app, THEN it is perfectly fine to start leaving bad reviews as the developer has had plenty of time to fix their app, but during the beta period, no.
  - sgarman 15y ago
    It is additionally time consuming building work arounds for issues that might be altered by the time the OS makes it to gold master. Beta is beta, it might mean the OS does not work with your app not your app does not work in the OS.
  - ugh 15y ago
    I guess the grandparent was suggesting that, yes, it’s unfair but Apple has no incentive for banning reviews. They are ok with users heckling developers for updates since developers aren’t exactly scarce.
- cageface 15y ago
desaiguddu 15y ago
There are reasons to block the iOS5 users comment or review the applications. 1. iOS5 is beta , so chances are there that inclusion of new APIs , pre release iOS version can adversely affect the already approved application. But as the problem is not with the Application but with the Operating System , the review can be biased or completely irrelevant. 2. As a Developer I will welcome the step by Apple that, this will allow users to comment and review apps from the users who run most stable Operating system, so any bug on my application is because of me, not because of the iOS5.
ledlight 15y ago
? We have corresponding discount rate from 10%--30% for the order ? Our Led lights have 3 years quality warranty ? All of our LED Lighting products have passed the CE and ROHS test ? Another 5‰ of your order will be offered for free if the quantity is over 1000pcs ? New LED lighting samples will be offered for free in the future after being our old customers http://www.aoneled.com http://www.aoneled.com http://www.led-gu10.com http://www.led-gu10.com ? More favourable policies.please contact us or visit our cheap LED lights website ? The newest catalog with prices are prepared well for you if you send inquire to 445249140@qq.com
RyanKearney 15y ago
All the effort developers are putting into complaining about people on iOS 5 Beta leaving negative reviews could better be spent testing and fixing their app for iOS 5. Are you going to complain when iOS 5 goes live and your app still isn't ready because you stayed up until 2am writing that blog post rant instead of ironing out that last bug? Even if Apple blocks iOS 5 devices from leaving reviews, you can still leave a review through iTunes.

163 points by davidedicillo 15y ago

pat2man 15y ago

This is just obvious. Isn't pointing out a bug thats due to an unreleased OS breaking your NDA anyway?

stevenwei 15y ago
Yeah, but non-developers who manage to install iOS 5 probably haven't actually signed the NDA. The funny thing is as a developer you're not even allowed to mention in your description that something in your app is broken on iOS 5, since that would be breaking the NDA.
- psychotik 15y ago
  Apple also doesn't like it when you fix an issue, release an update and mention that you fixed a bug which affected iOS 5 compatibility. I needed to change it say "compatibility fixes for future iOS versions".

spacemanaki 15y ago

I pointed this out the last time this came up, but an additional measure would be to also crack down on sites that sell access to iOS betas, if they aren't doing that already. That would (likely) drastically reduce the number of non-developers running beta OSes since isn't it $100/year for a developer account, but you can install betas on many devices? There are sites selling access for less than $10. If it's not against the TOS of the developer accounts, it definitely should be.

zhoutong 15y ago
Apparently Apple has noticed this issue and they have taken down a few sites. (http://appadvice.com/appnn/2011/08/apple-takes-steps-against-developers-selling-ios-5-access-what-you-need-to-know http://appadvice.com/appnn/2011/08/apple-takes-steps-against...) In fact Apple can try to increase the technical barrier to install iOS beta. (Must upgrade in Xcode or must own at least one app in App Store to be eligible for beta access.) This way can block the vast majority of non-developers while not affecting real iOS developers.
- silencio 15y ago
  There was speculation that that might have just been betas expiring, and not developers selling access getting their access revoked (http://www.macrumors.com/2011/08/05/apple-disabling-ios-5-beta-installs-and-developer-accounts-over-sales-of-device-slots/ http://www.macrumors.com/2011/08/05/apple-disabling-ios-5-be...). > must own at least one app in App Store to be eligible for beta access That doesn't work for many reasons. App Review, as if they didn't have enough of a hard time with a bad reputation, would just be flooded by trash from first time developers or contract developers/companies that just created an account. Apple could make it mandatory to install using Xcode, but in the past, Xcode upgrades have bricked devices where iTunes doesn't. So I have a feeling that's not really a priority and more of a burden. Plus, it's braindead simple to install and use Xcode to install betas, and anyone desperate enough to pay $10 but not $100 should easily be able to get around that. I really think all Apple needs to do is to block reviews from iOS 5 devices. Developers running betas won't be reviewing apps in the store (that's incredibly hypocritical and cruel if so) so nobody's going to be missing that. This doesn't harm beta testers and devs that want to install iOS betas.
  - dunham 15y ago
    I'd also block reviews from beta versions of iTunes, since many reviews are entered via iTunes.
- shaggyfrog 15y ago
  Your proposals take things a bit too far without any real return. Blocking access to devs who don't have an app in the store is counterproductive, because that prevents the developers from testing against the latest iOS and finding bugs -- which is the entire point of access to the iOS beta! Not to mention some devs might be working for other clients and not yet shipped something themselves (as in my case). Requiring installs via Xcode would only be a temporary slowdown before someone figures out the secret handshake and hacks around it. Blocking reviews from devices with the latest iOS beta installed should be sufficient to eliminate most of these unfair reviews.
  - 15y ago

jhawk28 15y ago

An alternative fix would be to have the iOS version used when submitting a review. It doesn't limit "free speech", but it gives context on what the problem actually was.

j79 15y ago
I think this is a great idea. Apple could allow filtering to see _IF_ I really want to upgrade to iOS 5 right away or wait until iOS 5 reviewers are happy.

deleted 15y ago

[deleted]

kogir 15y ago

They should also ban reviews from jailbroken devices, where users may have installed custom fonts or other "features" for which you can't possibly test and account. And yes -- any test Apple does for this could be defeated, but I expect most jailbreak developers would side with their app writing comrades and let it stand.

Xuzz 15y ago
The reason that Cydia doesn't have reviews (or even ratings) is that we haven't found a way to make a working review system either, so we don't even try. I don't think that any of us would argue or work against something like that, but I also seriously doubt that Apple will do it. (I'm a contributor to the Cydia open source project, but I don't speak for saurik or Cydia, and that's probably not the official stance.)
- saurik 15y ago
  So, I /am/ saurik, and I probably /would/ argue with and work around something like that, as I think I should be allowed to review things in the App Store. Really, the "official stance" I have is that developers who believe that having their application fail horribly when the platform is even slightly different is not their fault are deluded: Apple is going to change the platform over time, and when your app fails for Apple they tend to get pissed and just block your app from the store and may even de-prioritize your updates. Seriously: if your app fails because the font is different, there is something epically wrong with your coding methodology; and yes: Apple actually did change the font slightly for the iPhone 4 (on retina screens they switched to Helvetica Neue from Helvetica). Meanwhile, in desktop-land, everyone from large companies to small independent developers deal with this supposedly horrible problem of random fonts, different operating system versions, upgraded libraries, client-side firewalls, or any other number of things that would make developers like kogir freak out, and somehow the world doesn't end, and almost all of the software I download works without issue (with the exception of Chrome on Tablet editions of Windows, and that's squarely Google's fault). (Hell: even almost all of the software on the App Store works under these conditions. What should /really/ happen is that Apple should internally have a different "test version" of iOS that developers are not allowed to see; one that has a newer version of SQLite (which I state in specific as developers make all kinds of incorrect assumptions about the SQLite ABI) and has every core library compiled to the opposite instruction set (Thumb if normally ARM, and vice versa), that has every font replaced with Comic Sans, and uses a different pixel density, on a screen with a slightly different aspect ratio, using a CPU that runs slightly faster than the shipping devices. The reason this needs to be done is that an app that fails to run correctly in that (hidden) configuration is pretty much guaranteed to be an app that is going to stop working when any of the following occur: iOS 6, iPhone Nano, iPad+Retina, or another "game changing" device (such as the iPad when it came out). Rather than let these apps in the store just to fail later, they should be weeded out before they even get in. (And yes: Cydia itself strives not to make any assumptions about any of the things listed, and in fact worked almost entirely correctly when loaded for the first time on an iPad with the flag flipped for "don't show me in a 1x/2x box"; certainly well enough that the reviewer in this scenario would have given it a pass were they testing it in that environment. The one place it tends to get burnt is with what it does to WebKit; and the current implementation, which works fine on everything from 2.0 through 5.0, is likely stable enough to "stand the test of time".) (FWIW: I do believe that people using beta versions of iOS should not be able to review applications. These people are using a version of the firmware that is under NDA... they should not be tempted to be providing feedback that might violate that agreement to applications that fail to operate correctly, possibly due to no fault of the app. I have a good example of this related to Cydia on 5.x, but I can't tell you what it is, for this very reason ;P.)

benatkin 15y ago

This reminds me of how PyPI ratings got disabled after a long discussion with the community, started by jacobian: http://code.activestate.com/lists/python-catalog-sig/3482/ http://code.activestate.com/lists/python-catalog-sig/3482/ I strongly agree that the PyPI ratings were mostly useless and that removing the feature was a good decision. iPhone app ratings can be much more useful for popular apps, but for the long tail of apps, they suffer the same problems as PyPI's rating and review system did. Amazon and Yelp ratings, on the other hand, are quite useful. I think it's partly because users have profiles and are better able to express themselves.

dman 15y ago
It might also be the fact that Amazon and Yelp do a much better job of curating the ratings.
ojilles 15y ago
I think the app ratings for the popular apps are all useless (they're well know apps, other people rave about them, so no need to read all the reviews). I agree on the long tail being useless as well. Instead, I think it's most useful for the "middle tail": those apps that do get a fair bit of usage, but are not considered popular: the developer may not have build up a reputation yet, but the reviews help confirming the application does what it is supposed to do (or not of course).

chris24 15y ago

Apple should remove the reviews section from the App Store for people running pre-release versions of iOS and replace it with a form to send a bug report to the app developer (and possibly even a place for a list of known issues that the app has on iOS 5).

hellweaver666 15y ago
Apple should just add a bug reports section to the app store period.
- vaporstun 15y ago
  Potentially, but if so, it should be something which you'd have to opt into or open only for fellow developers. The last thing I want is to get spammed with "bug reports" from lay users with trivial or non-existent issues. If they implemented this but made it take even one extra step like flipping some switch in the settings, it would filter out most lay users and garner some more useful feedback.

msftguy 15y ago

Maybe Apple doesn't block beta reviews because it incentivizes developers to fix their apps before the release? Most of those problems should be trivial to at least work around with version-dependent logic to minimizes regressions on versions < 5.x. Bad reviews don't exactly hurt Apple, and it's not like iOS developers are scarce these days, so it might just be Apple having their priorities set straight (users > devs) and not a coding blunder..or maybe it's just me overthinking it.

X-Istence 15y ago
Except that developers are not even given the chance to fix issues before Johnny Public is allowed to start rating apps. The whole point of the beta is to ALLOW the developer to test their apps and find the bugs, as well as to iron out the bugs in the core OS. What if it is an issue with a core API that is now returning something that it shouldn't and it wasn't caught by Apple in testing, then the developer is getting blamed for something they can't even fix. You have to be a part of the developer program ($99) to get pre-release software, and as part of that you go with the understanding that it isn't meant for everyday use. Johnny Public shouldn't be running the beta in the first place. If by the time iOS 5 is out the developer still hasn't fixed the bugs in their app, THEN it is perfectly fine to start leaving bad reviews as the developer has had plenty of time to fix their app, but during the beta period, no.
- sgarman 15y ago
  It is additionally time consuming building work arounds for issues that might be altered by the time the OS makes it to gold master. Beta is beta, it might mean the OS does not work with your app not your app does not work in the OS.
- ugh 15y ago
  I guess the grandparent was suggesting that, yes, it’s unfair but Apple has no incentive for banning reviews. They are ok with users heckling developers for updates since developers aren’t exactly scarce.
cageface 15y ago

desaiguddu 15y ago

There are reasons to block the iOS5 users comment or review the applications. 1. iOS5 is beta , so chances are there that inclusion of new APIs , pre release iOS version can adversely affect the already approved application. But as the problem is not with the Application but with the Operating System , the review can be biased or completely irrelevant. 2. As a Developer I will welcome the step by Apple that, this will allow users to comment and review apps from the users who run most stable Operating system, so any bug on my application is because of me, not because of the iOS5.

ledlight 15y ago

? We have corresponding discount rate from 10%--30% for the order ? Our Led lights have 3 years quality warranty ? All of our LED Lighting products have passed the CE and ROHS test ? Another 5‰ of your order will be offered for free if the quantity is over 1000pcs ? New LED lighting samples will be offered for free in the future after being our old customers http://www.aoneled.com http://www.aoneled.com http://www.led-gu10.com http://www.led-gu10.com ? More favourable policies.please contact us or visit our cheap LED lights website ? The newest catalog with prices are prepared well for you if you send inquire to 445249140@qq.com

RyanKearney 15y ago

All the effort developers are putting into complaining about people on iOS 5 Beta leaving negative reviews could better be spent testing and fixing their app for iOS 5. Are you going to complain when iOS 5 goes live and your app still isn't ready because you stayed up until 2am writing that blog post rant instead of ironing out that last bug? Even if Apple blocks iOS 5 devices from leaving reviews, you can still leave a review through iTunes.