12 ms·

Xcode 14 unintentionally increases app size

250 points by trevor-e 4y ago

johnthuss 4y ago
"With Xcode 14, any app that relied on bitcode is no longer stripping binary symbols from their production app." This is a thoroughly researched article that clearly shows the cause and provides a solution. I hope the developers of these apps will take heed.
- dkonofalski 4y ago
  Absolutely. In fairness, though, Apple should also catch this and either prompt or offer some kind of solution to this for devs that may be unaware of this change. The author seems to have found just this so a little education or notification would go a long way.
- snvzz 4y ago
  Yay, symbols. Perfect for reversing.
- gardaani 4y ago
  Would stripping binary symbols affect stack traces in crash reports?
  - saagarjha 4y ago
    Most iOS developers symbolicate their crash reports with separately-packaged symbols.
mig39 4y ago
Are they dropping support for bitcode altogether in the future?
- plorkyeran 4y ago
  Despite the release notes claiming it'll be removed in the future, trying to build with bitcode enabled is already a hard error in some cases with Xcode 14. Similarly the app store doesn't strip bitcode from things uploaded to it; it instead just tells you to rebuild with bitcode disabled.
- trevor-e 4y ago
  https://developer.apple.com/documentation/xcode-release-notes/xcode-14-release-notes https://developer.apple.com/documentation/xcode-release-note... "The capability to build with bitcode will be removed in a future Xcode release." Yea that is the plan from Apple.
- regular 4y ago
ridiculous_fish 4y ago
Some speculation about why Apple is deprecating bitcode in this SO post: https://stackoverflow.com/questions/72543728/xcode-14-deprecates-bitcode-but-why https://stackoverflow.com/questions/72543728/xcode-14-deprec...
- 0x0 4y ago
  Bitcode could never really help with big architecture changes, there's too many #ifdef / #define'd integers and enums that have different values on different platforms, which would cause platform-specific integer values to be embedded hardcoded in the bitcode
- newaccount74 4y ago
  I'm pretty sure the SO answer is (partially) wrong. As far as I understand, Bitcode is not architecture agnostic, you can't just take bitcode from a 32bit compilation and turn it into a 64bit executable since there still are a ton of architecture dependent things (like memory alignment) in the bitcode.
  - lilyball 4y ago
    That's true. However it is microarchitecture agnostic.
    - throw10920 4y ago
      Isn't machine code also microarchitecture agnostic? I thought that the definition of a microarchitecture was a particular implementation of an ISA that externally conformed to the ISA in the same way as any other microarchitecture.
      - valleyer 4y ago
        Mostly, yeah. But microarchitectures do sometimes add new instruction set extensions, like how Intel's Skylake added AVX-512. Bitcode can be re-lowered to the new extended ISA to use these instructions (assuming you don't need to target earlier uarchs). Machine code, not as easily (and I don't think anyone really tries, though I'd be interested to learn if someone does).
AnotherGoodName 4y ago
Does app size matter for install rates? I suspect not on high end mobile but this accident should give some data towards this. Will throw a spanner into the 'we can't release that feature as it will increase app size' thinking that I've witnessed in my mobile dev career. I've never seen it actually impact metrics that much.
- arcticbull 4y ago
  They answer is yes, but only in some cases: - Getting warned about app size over cellular, which in iOS happens at 200MB. - In geographies where cellular data is expensive, slow or intermittent. - At the margin when you have many millions of users. The data bears out that it does have a small but measurable impact.
- wlesieutre 4y ago
  There's a 200 MB size threshold where users on cellular connections get prompted if they're sure they want to download it, so if this pushes an app over 200 MB I would assume yes some users will bounce off of that. And in some places 200 MB can still be a big deal, the whole world isn't on 20 GB data plans.
  - ajsnigrutin 4y ago
    200MB for a 3d game with a bunch of assests... sure... but somehow there are more and more >200MB apps, that are basically a packed webapp that should be below 10MB.
- newaccount74 4y ago
  There are A LOT of people who ran out of space on their phones and can't install anything anymore. So if your app is 500MB, they'll first have to find 500MB worth of stuff to delete. If they really want your app, they'll look through the storage settings on their phone, and start deleting their biggest apps. So if your app is too big, many people won't even install it, and if they do, it'll be the first app to go if they need more space.
- mring33621 4y ago
gumby 4y ago
I thought the idea of bitcode is that Apple would produce binaries for each kind of supported device (i.e. different CPUs for supported models). Basically the benefit of a fat binary without having to generate one. Did they give up on this idea? Was it not worth it? EDIT: ridiculous_fish posted a link with some surmises on this question.
- pjmlp 4y ago
  Because to gain some stability about the raw LLVM bitcode they had to maintain their own fork, and most likely decided it wasn't worth the effort any longer. Somehow people keep missing that Apple's clang and LLVM aren't the same one gets from clang.org.
- plorkyeran 4y ago
  The one publicly known production use of bitcode was that it allowed them to recompile all existing armv7 watchOS apps as arm64_32, which made it so that the first 64-bit apple watch could run existing apps. This required specifically designing arm64_32 (which is arm64 but with 32-bit pointers) to enable this, and isn't really a viable approach for anything other than this specific scenario where they knew that they'd be migrating to a specific new architecture in the future. Before the first 64-bit watch was announced they were very vague and hand-wavy about what bitcode was for because they had to keep the upcoming products a secret, and sort of implied that it'd be used for things which didn't actually make very much sense.
  - alerighi 4y ago
    I think that here Android is a better ecosystem: you distribute your application as a bytecode, and then it's the phone itself to compile it to a real binary when you install it. This allow far greater compatibility with even very old versions of the OS or old applications in newer OS and simplifies the build process. Working in a company that builds both Android and iOS application I have to say that building iOS apps generates 10x the problems that you have with building Android applications, in terms of CI infrastructure, that means that Android you run gradlew and it works, provided you have the right JVM version installed, with XCode it's always a mess. Also for how it works the ecosystem with Android I'm not forced to upgrade, that means that if a project uses SDK X I can build it forever with that SDK and it will still work even on latest devices, with Apple you have to build with the latest toolchain if you want to execute on newer terminals. That makes Android far more suitable for b2b applications.
fcknnhvn 4y ago
This is excellent news for anyone reverse engineering these apps, getting all these unintentionally-released symbols. Thanks Apple!
- crazygringo 4y ago
  I'm actually curious if this has security implications? Is this making public anything that some app developers have thought important to obfuscate?
  - can16358p 4y ago
    If any developer is relying on security by obscurity to protect anything sensitive, they deserve to be accessed better sooner than later. However there are some practical purposes od obscurity like in games that madd reverse engineering harder (though possible of course), it might become easier in some situations.
    - ihatepython 4y ago
      Hmm. Recently I have been using my api-keys as actual function names. Maybe that's not such a good idea
      - crazygringo 4y ago
        You just brought me this close to spitting out my coffee from laughing this morning. Thanks for the great comment to start my day :)
lzooz 4y ago
Symbols not stripped? That could be fun... is there a simple way of downloading unencrypted ipas from the app store without access to a jailbroken phone?
- malhal 4y ago
  Download from App Store on M1 Mac (if it supports iPad)
- saagarjha 4y ago
  Bitcode≠unencrypted app.
- cprecioso 4y ago
  ipatool (https://github.com/majd/ipatool https://github.com/majd/ipatool) + mremap_encrypted (syscall on Apple Silicon)
YetAnotherNick 4y ago
How can companies not check app size increase before submitting it.
- WirelessGigabit 4y ago
  Because no one cares. It's all about getting it out there asap. All kinds of libraries get included for 1 simple function and this bloats the size of the app. Blindness has set in. It used to be that an exe was 5MB. Now it's 150MB because it has a browser with it.
- TheRealPomax 4y ago
  Why would they care? Whether an app is 100mb or 500mb has zero financial repercussions for them. (In a consumer-friendly world they would care, and we want them to care, but as is, they have no reason to care)
  - Loranubi 4y ago
    I am always a few years behind on smart phone models, so I don't have that much storage space. Every few months I go through my app list sort by largest size and delete the top few apps which are not necessary for me. So it definitely has financial repercussions for them. Smaller apps stay on my phone longer and have a higher chance to be used again and bring them more ad revenue..
    - TheRealPomax 4y ago
      Kinda makes you an outlier though, the people whose money they want are the people who are at most one model behind.
  - darren_ 4y ago
    There's one major reason to care - the AppStore has a threshold (200MB i think) over which it will warn users about the app's size if they are on cellular connection (IIRC it used to block downloads entirely). So there's a fair bit of incentive to stay under 200MB.
thought_alarm 4y ago
Seems to be a Swift problem; an oversight with the default Xcode settings, perhaps? I tried running `strip -rSTx` on an ad-hoc binary that contained only C/C++/ObjC code (no Swift) and it had no effect. It does have an effect on binaries with Swift code.
- saghm 4y ago
  Not sure about on MacOS, but on Linux running the `file` command on a binary indicates whether it's stripped or not: $ echo 'int main() { return 0; }' > main.c && gcc main.c && file a.out a.out: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=6e4c5817c9bf2ddcc5a2dfe6d9d9729526c81aed, for GNU/Linux 4.4.0, with debug_info, not stripped $ strip a.out && file a.out a.out: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=6e4c5817c9bf2ddcc5a2dfe6d9d9729526c81aed, for GNU/Linux 4.4.0, stripped
deleted 4y ago
[deleted]
contravariant 4y ago
> Nike iOS app install size was 182.2 MB. A week later, it was 322.1 MB This is besides the point, but is anyone else concerned just how insane those sizes are? There are operating systems smaller than that, why is that needed for a single app from a sports brand? What is this app even supposed to do? How much functionality did they need to cram into it to get a 180MB binary? Why does the app even exist? Why does it apparently need 160MB of just text to do whatever unfathomable job it has?
- stock_toaster 4y ago
  Launch images (png) for every supported screen size? Sound/media files? Libraries? I’m sure it starts to add up.
  - csande17 4y ago
    Haven't launch screens used resizable Storyboard layouts for several years now?
    - drEv0 4y ago
      They have. I believe Xcode even warns if a project is not using the resizable option.
  - outcoldman 4y ago
    I used to have an app built for some AWS services (mac, ios). First decision to use some AWS libraries added size to the binary, it grew from 1MB to 70MB, I wrote simple AWS library myself just for my needs and that resulted in the binary of 2-3MB. Most of those apps use a lot of common libraries that add a lot of size to the binaries.
    - version_five 4y ago
      I'm working on a machine learning project where we're trying to package something up, and bringing in all the packages that get installed by default with the framework we're using adds gigabytes to the size of the application, almost all of which is probably not necessary. The challenge is it's bloat on top of bloat, without an obvious way to pare it down, so it becomes a separate engineering effort to try and make something small and efficient. Under a certain size and for many applications, it's more cost effective just not to worry about it and keep the bloat
xbar 4y ago
"* American: app is way too large. breaks constantly. " It is good to know that there are ways around the bloat. It does beg: why do app developers release such bloated garbage when it would be easy to address?
- carstenhag 4y ago
  Because few companies have insights on how much each branch/change affects the download / installation size. And I can imagine that at a medium-large company, noone is really taking the lead on overreaching topics like this.
- regular 4y ago
regular 4y ago
This neglects to address WHY bitcode was removed. Wasn't this supposed to be a big deal, and required of all app submissions only a couple of years ago? I thought it was to facilitate the porting of applications to whatever architecture Apple saw fit to deploy. Update: This sounds like a pretty good explanation: https://stackoverflow.com/questions/72543728/xcode-14-deprecates-bitcode-but-why https://stackoverflow.com/questions/72543728/xcode-14-deprec...

250 points by trevor-e 4y ago

"With Xcode 14, any app that relied on bitcode is no longer stripping binary symbols from their production app." This is a thoroughly researched article that clearly shows the cause and provides a solution. I hope the developers of these apps will take heed.

dkonofalski 4y ago
Absolutely. In fairness, though, Apple should also catch this and either prompt or offer some kind of solution to this for devs that may be unaware of this change. The author seems to have found just this so a little education or notification would go a long way.
snvzz 4y ago
Yay, symbols. Perfect for reversing.
gardaani 4y ago
Would stripping binary symbols affect stack traces in crash reports?
- saagarjha 4y ago
  Most iOS developers symbolicate their crash reports with separately-packaged symbols.

mig39 4y ago

Are they dropping support for bitcode altogether in the future?

plorkyeran 4y ago
Despite the release notes claiming it'll be removed in the future, trying to build with bitcode enabled is already a hard error in some cases with Xcode 14. Similarly the app store doesn't strip bitcode from things uploaded to it; it instead just tells you to rebuild with bitcode disabled.
trevor-e 4y ago
https://developer.apple.com/documentation/xcode-release-notes/xcode-14-release-notes https://developer.apple.com/documentation/xcode-release-note... "The capability to build with bitcode will be removed in a future Xcode release." Yea that is the plan from Apple.
regular 4y ago

ridiculous_fish 4y ago

Some speculation about why Apple is deprecating bitcode in this SO post: https://stackoverflow.com/questions/72543728/xcode-14-deprecates-bitcode-but-why https://stackoverflow.com/questions/72543728/xcode-14-deprec...

0x0 4y ago
Bitcode could never really help with big architecture changes, there's too many #ifdef / #define'd integers and enums that have different values on different platforms, which would cause platform-specific integer values to be embedded hardcoded in the bitcode
newaccount74 4y ago
I'm pretty sure the SO answer is (partially) wrong. As far as I understand, Bitcode is not architecture agnostic, you can't just take bitcode from a 32bit compilation and turn it into a 64bit executable since there still are a ton of architecture dependent things (like memory alignment) in the bitcode.
- lilyball 4y ago
  That's true. However it is microarchitecture agnostic.
  - throw10920 4y ago
    Isn't machine code also microarchitecture agnostic? I thought that the definition of a microarchitecture was a particular implementation of an ISA that externally conformed to the ISA in the same way as any other microarchitecture.
    - valleyer 4y ago
      Mostly, yeah. But microarchitectures do sometimes add new instruction set extensions, like how Intel's Skylake added AVX-512. Bitcode can be re-lowered to the new extended ISA to use these instructions (assuming you don't need to target earlier uarchs). Machine code, not as easily (and I don't think anyone really tries, though I'd be interested to learn if someone does).

AnotherGoodName 4y ago

Does app size matter for install rates? I suspect not on high end mobile but this accident should give some data towards this. Will throw a spanner into the 'we can't release that feature as it will increase app size' thinking that I've witnessed in my mobile dev career. I've never seen it actually impact metrics that much.

arcticbull 4y ago
They answer is yes, but only in some cases: - Getting warned about app size over cellular, which in iOS happens at 200MB. - In geographies where cellular data is expensive, slow or intermittent. - At the margin when you have many millions of users. The data bears out that it does have a small but measurable impact.
wlesieutre 4y ago
There's a 200 MB size threshold where users on cellular connections get prompted if they're sure they want to download it, so if this pushes an app over 200 MB I would assume yes some users will bounce off of that. And in some places 200 MB can still be a big deal, the whole world isn't on 20 GB data plans.
- ajsnigrutin 4y ago
  200MB for a 3d game with a bunch of assests... sure... but somehow there are more and more >200MB apps, that are basically a packed webapp that should be below 10MB.
newaccount74 4y ago
There are A LOT of people who ran out of space on their phones and can't install anything anymore. So if your app is 500MB, they'll first have to find 500MB worth of stuff to delete. If they really want your app, they'll look through the storage settings on their phone, and start deleting their biggest apps. So if your app is too big, many people won't even install it, and if they do, it'll be the first app to go if they need more space.
mring33621 4y ago

gumby 4y ago

I thought the idea of bitcode is that Apple would produce binaries for each kind of supported device (i.e. different CPUs for supported models). Basically the benefit of a fat binary without having to generate one. Did they give up on this idea? Was it not worth it? EDIT: ridiculous_fish posted a link with some surmises on this question.

pjmlp 4y ago
Because to gain some stability about the raw LLVM bitcode they had to maintain their own fork, and most likely decided it wasn't worth the effort any longer. Somehow people keep missing that Apple's clang and LLVM aren't the same one gets from clang.org.
plorkyeran 4y ago
The one publicly known production use of bitcode was that it allowed them to recompile all existing armv7 watchOS apps as arm64_32, which made it so that the first 64-bit apple watch could run existing apps. This required specifically designing arm64_32 (which is arm64 but with 32-bit pointers) to enable this, and isn't really a viable approach for anything other than this specific scenario where they knew that they'd be migrating to a specific new architecture in the future. Before the first 64-bit watch was announced they were very vague and hand-wavy about what bitcode was for because they had to keep the upcoming products a secret, and sort of implied that it'd be used for things which didn't actually make very much sense.
- alerighi 4y ago
  I think that here Android is a better ecosystem: you distribute your application as a bytecode, and then it's the phone itself to compile it to a real binary when you install it. This allow far greater compatibility with even very old versions of the OS or old applications in newer OS and simplifies the build process. Working in a company that builds both Android and iOS application I have to say that building iOS apps generates 10x the problems that you have with building Android applications, in terms of CI infrastructure, that means that Android you run gradlew and it works, provided you have the right JVM version installed, with XCode it's always a mess. Also for how it works the ecosystem with Android I'm not forced to upgrade, that means that if a project uses SDK X I can build it forever with that SDK and it will still work even on latest devices, with Apple you have to build with the latest toolchain if you want to execute on newer terminals. That makes Android far more suitable for b2b applications.

fcknnhvn 4y ago

This is excellent news for anyone reverse engineering these apps, getting all these unintentionally-released symbols. Thanks Apple!

crazygringo 4y ago
I'm actually curious if this has security implications? Is this making public anything that some app developers have thought important to obfuscate?
- can16358p 4y ago
  If any developer is relying on security by obscurity to protect anything sensitive, they deserve to be accessed better sooner than later. However there are some practical purposes od obscurity like in games that madd reverse engineering harder (though possible of course), it might become easier in some situations.
  - ihatepython 4y ago
    Hmm. Recently I have been using my api-keys as actual function names. Maybe that's not such a good idea
    - crazygringo 4y ago
      You just brought me this close to spitting out my coffee from laughing this morning. Thanks for the great comment to start my day :)

lzooz 4y ago

Symbols not stripped? That could be fun... is there a simple way of downloading unencrypted ipas from the app store without access to a jailbroken phone?

malhal 4y ago
Download from App Store on M1 Mac (if it supports iPad)
saagarjha 4y ago
Bitcode≠unencrypted app.
cprecioso 4y ago
ipatool (https://github.com/majd/ipatool https://github.com/majd/ipatool) + mremap_encrypted (syscall on Apple Silicon)

YetAnotherNick 4y ago

How can companies not check app size increase before submitting it.

WirelessGigabit 4y ago
Because no one cares. It's all about getting it out there asap. All kinds of libraries get included for 1 simple function and this bloats the size of the app. Blindness has set in. It used to be that an exe was 5MB. Now it's 150MB because it has a browser with it.
TheRealPomax 4y ago
Why would they care? Whether an app is 100mb or 500mb has zero financial repercussions for them. (In a consumer-friendly world they would care, and we want them to care, but as is, they have no reason to care)
- Loranubi 4y ago
  I am always a few years behind on smart phone models, so I don't have that much storage space. Every few months I go through my app list sort by largest size and delete the top few apps which are not necessary for me. So it definitely has financial repercussions for them. Smaller apps stay on my phone longer and have a higher chance to be used again and bring them more ad revenue..
  - TheRealPomax 4y ago
    Kinda makes you an outlier though, the people whose money they want are the people who are at most one model behind.
- darren_ 4y ago
  There's one major reason to care - the AppStore has a threshold (200MB i think) over which it will warn users about the app's size if they are on cellular connection (IIRC it used to block downloads entirely). So there's a fair bit of incentive to stay under 200MB.

thought_alarm 4y ago

Seems to be a Swift problem; an oversight with the default Xcode settings, perhaps? I tried running `strip -rSTx` on an ad-hoc binary that contained only C/C++/ObjC code (no Swift) and it had no effect. It does have an effect on binaries with Swift code.

saghm 4y ago
Not sure about on MacOS, but on Linux running the `file` command on a binary indicates whether it's stripped or not: $ echo 'int main() { return 0; }' > main.c && gcc main.c && file a.out a.out: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=6e4c5817c9bf2ddcc5a2dfe6d9d9729526c81aed, for GNU/Linux 4.4.0, with debug_info, not stripped $ strip a.out && file a.out a.out: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=6e4c5817c9bf2ddcc5a2dfe6d9d9729526c81aed, for GNU/Linux 4.4.0, stripped

deleted 4y ago

[deleted]

contravariant 4y ago

> Nike iOS app install size was 182.2 MB. A week later, it was 322.1 MB This is besides the point, but is anyone else concerned just how insane those sizes are? There are operating systems smaller than that, why is that needed for a single app from a sports brand? What is this app even supposed to do? How much functionality did they need to cram into it to get a 180MB binary? Why does the app even exist? Why does it apparently need 160MB of just text to do whatever unfathomable job it has?

stock_toaster 4y ago
Launch images (png) for every supported screen size? Sound/media files? Libraries? I’m sure it starts to add up.
- csande17 4y ago
  Haven't launch screens used resizable Storyboard layouts for several years now?
  - drEv0 4y ago
    They have. I believe Xcode even warns if a project is not using the resizable option.
- outcoldman 4y ago
  I used to have an app built for some AWS services (mac, ios). First decision to use some AWS libraries added size to the binary, it grew from 1MB to 70MB, I wrote simple AWS library myself just for my needs and that resulted in the binary of 2-3MB. Most of those apps use a lot of common libraries that add a lot of size to the binaries.
  - version_five 4y ago
    I'm working on a machine learning project where we're trying to package something up, and bringing in all the packages that get installed by default with the framework we're using adds gigabytes to the size of the application, almost all of which is probably not necessary. The challenge is it's bloat on top of bloat, without an obvious way to pare it down, so it becomes a separate engineering effort to try and make something small and efficient. Under a certain size and for many applications, it's more cost effective just not to worry about it and keep the bloat

xbar 4y ago

"* American: app is way too large. breaks constantly. " It is good to know that there are ways around the bloat. It does beg: why do app developers release such bloated garbage when it would be easy to address?

carstenhag 4y ago
Because few companies have insights on how much each branch/change affects the download / installation size. And I can imagine that at a medium-large company, noone is really taking the lead on overreaching topics like this.
regular 4y ago

regular 4y ago

This neglects to address WHY bitcode was removed. Wasn't this supposed to be a big deal, and required of all app submissions only a couple of years ago? I thought it was to facilitate the porting of applications to whatever architecture Apple saw fit to deploy. Update: This sounds like a pretty good explanation: https://stackoverflow.com/questions/72543728/xcode-14-deprecates-bitcode-but-why https://stackoverflow.com/questions/72543728/xcode-14-deprec...