5 ms·

> How is it that simultaneously, so many apps fail review for tiny details, showing that apps really are having manual review done on them... > ...and yet all

by lapcat 4y ago

> How is it that simultaneously, so many apps fail review for tiny details, showing that apps really are having manual review done on them... > ...and yet all this spam is getting through? It's important to understand that App Review is not consistently nitpicky but rather randomly nitpicky. Every indie dev has stories of urgent bug fix updates getting rejected for stupid, unrelated, nitpicky reasons, things that had already been in the app for months or years. Yet every indie dev also has stories of major updates flying through review in minutes, so fast you know it's impossible that anyone looked carefully. I do get the impression that they're doing random inspections. "Every week, over 500 dedicated experts around the world review over 100K apps." https://www.apple.com/app-store/ https://www.apple.com/app-store/ Apple seems to be bragging about that, but if you estimate how much time is spent on each app, the numbers look pretty bad. If 500 reviewers spend full-time 40 hours every week doing nothing but reviewing apps — no breaks, no meetings, no training, no vacations, etc. — that leaves only 12 minutes per app, maximum. And they're reviewing not just the app but the App Store metadata: release notes, description, screenshots. They definitely do metadata rejections.

kitsunesoba 4y ago
> I do get the impression that they're doing random inspections. As someone who's been responsible for various iOS App Store submissions for closing in on a decade, the impression I get is that the variation is primarily due to inconsistencies in process between reviewers. The corners probed by one reviewer might go entirely ignored by another and what one reviewer finds perfectly fine constitutes a violation in the eyes of another. The Play Store review process on the other hand comes off as feeling almost entirely automated barring a handful of exceptions, which works fine until some innocuous change in code or the automated review system trips something somewhere and your app listing is removed. Good luck getting in touch with a real person to fix it.
- saagarjha 4y ago
  I mean there’s that too. Inconsistency and lack of resources are both present.
MathMonkeyMan 4y ago
> bug fix updates getting rejected for stupid, unrelated, nitpicky reasons, things that had already been in the app for months or years This reminds me of code review in general. Some reviewers have the etiquette of "if it was there already, then it's outside the scope of this review," and some don't. Maybe a reviewer should have two responsibilities: 1. Review the proposed _change_ and verify that it meets all criteria and doesn't introduce defects. 2. Make note of any existing defects, which are then recorded. Maybe an app developer can be required to resolve everything found in (2) before the _next_ review. I've never released an app before.
- eru 4y ago
  > Some reviewers have the etiquette of "if it was there already, then it's outside the scope of this review," and some don't. > Maybe a reviewer should have two responsibilities: [...] For code review: you can make lots of different choices for responsibilities and still have a good code review process. It just needs to fit your goals and people in your organisation should be on the same page about it. So for example for code review, I don't think the reviewer should try very hard to understand the change. Explaining the change so that it is easy to understand is one of the responsibilities of the author who proposes the change. Similarly it's upon the author to demonstrate that the proposed change doesn't introduce any defects and to explain how it meets all criteria. It's not on the reviewer to go bug hunting. That is so that the next guy who view the change in a few years in version control history to understand how the software developed (or how a bug was introduced) has a fighting chance to understand what happened from what was preserved in version control alone.
- joshspankit 4y ago
  1. Since most changes are compiled binaries, you would have to rely on the submitter to self-report what changed.
  - dvzk 4y ago
    Things open source developers say because they don’t have experience with binary analysis. Not to say that Apple’s reviewers are going to be looking at IDA or Ghidra dumps, but they can see added capabilities, frameworks, symbols, or assembly and call graph changes. Not that it matters, because they don’t have the time to manually review source code changes either. At best there’s likely some automated static analysis for undocumented API symbols and common malware signatures.
dwighttk 4y ago
12 min per app should be plenty, but they should be rejecting a bunch of this garbage in about 20seconds.
fatnoah 4y ago
> It's important to understand that App Review is not consistently nitpicky but rather randomly nitpicky. This has been my experience as well. A long time ago, my company had an iOS communications app that we'd occasionally white-label for different clients (about 7, in total). The only changes to the apps were the color palette and the name/logo that appeared on the login page. For whatever reason, we were never able to release the white label for the 4th client. All other versions sailed through review, with the occasional holdup because the terms of service web page linked in the app was part of a web site which, through its navigational elements, would allow one to sign up for our service online. This is against Apple terms if you don't also allow sign up in the app. We were usually able to get this resolved because it wasn't actually a sign-up form, but a form to request more information from our sales team. For that 4th white label, that "sign-up page" became an issue we could never resolve. Our app was for enterprises, so a per-user purchase/sign-up in the app didn't make sense. We tried to move the terms of service and other required pages to their own web site with no links to anything on our company web page, but that was disallowed because it didn't link to our company web page. Apparently the only allowed solution was remove the request info page from our corporate web site. In the end, we simply resubmitted as a new app and it was approved without issue.

by lapcat 4y ago

kitsunesoba 4y ago

> I do get the impression that they're doing random inspections. As someone who's been responsible for various iOS App Store submissions for closing in on a decade, the impression I get is that the variation is primarily due to inconsistencies in process between reviewers. The corners probed by one reviewer might go entirely ignored by another and what one reviewer finds perfectly fine constitutes a violation in the eyes of another. The Play Store review process on the other hand comes off as feeling almost entirely automated barring a handful of exceptions, which works fine until some innocuous change in code or the automated review system trips something somewhere and your app listing is removed. Good luck getting in touch with a real person to fix it.

saagarjha 4y ago
I mean there’s that too. Inconsistency and lack of resources are both present.

MathMonkeyMan 4y ago

> bug fix updates getting rejected for stupid, unrelated, nitpicky reasons, things that had already been in the app for months or years This reminds me of code review in general. Some reviewers have the etiquette of "if it was there already, then it's outside the scope of this review," and some don't. Maybe a reviewer should have two responsibilities: 1. Review the proposed _change_ and verify that it meets all criteria and doesn't introduce defects. 2. Make note of any existing defects, which are then recorded. Maybe an app developer can be required to resolve everything found in (2) before the _next_ review. I've never released an app before.

eru 4y ago
> Some reviewers have the etiquette of "if it was there already, then it's outside the scope of this review," and some don't. > Maybe a reviewer should have two responsibilities: [...] For code review: you can make lots of different choices for responsibilities and still have a good code review process. It just needs to fit your goals and people in your organisation should be on the same page about it. So for example for code review, I don't think the reviewer should try very hard to understand the change. Explaining the change so that it is easy to understand is one of the responsibilities of the author who proposes the change. Similarly it's upon the author to demonstrate that the proposed change doesn't introduce any defects and to explain how it meets all criteria. It's not on the reviewer to go bug hunting. That is so that the next guy who view the change in a few years in version control history to understand how the software developed (or how a bug was introduced) has a fighting chance to understand what happened from what was preserved in version control alone.
joshspankit 4y ago
1. Since most changes are compiled binaries, you would have to rely on the submitter to self-report what changed.
- dvzk 4y ago
  Things open source developers say because they don’t have experience with binary analysis. Not to say that Apple’s reviewers are going to be looking at IDA or Ghidra dumps, but they can see added capabilities, frameworks, symbols, or assembly and call graph changes. Not that it matters, because they don’t have the time to manually review source code changes either. At best there’s likely some automated static analysis for undocumented API symbols and common malware signatures.

dwighttk 4y ago

12 min per app should be plenty, but they should be rejecting a bunch of this garbage in about 20seconds.

fatnoah 4y ago

> It's important to understand that App Review is not consistently nitpicky but rather randomly nitpicky. This has been my experience as well. A long time ago, my company had an iOS communications app that we'd occasionally white-label for different clients (about 7, in total). The only changes to the apps were the color palette and the name/logo that appeared on the login page. For whatever reason, we were never able to release the white label for the 4th client. All other versions sailed through review, with the occasional holdup because the terms of service web page linked in the app was part of a web site which, through its navigational elements, would allow one to sign up for our service online. This is against Apple terms if you don't also allow sign up in the app. We were usually able to get this resolved because it wasn't actually a sign-up form, but a form to request more information from our sales team. For that 4th white label, that "sign-up page" became an issue we could never resolve. Our app was for enterprises, so a per-user purchase/sign-up in the app didn't make sense. We tried to move the terms of service and other required pages to their own web site with no links to anything on our company web page, but that was disallowed because it didn't link to our company web page. Apparently the only allowed solution was remove the request info page from our corporate web site. In the end, we simply resubmitted as a new app and it was approved without issue.