17 ms·

How Uber Deals with Large iOS App Size

257 points by rakingleaves 6y ago

ohiovr 6y ago
I'd take a sledge hammer to it. The app doesn't have to be an app at all. It could simply be a stream with an os interactive overlay that intercepts touches. Like a thin client for phones.
- lxe 6y ago
  That could trigger the App Store review block. But this is Uber, so I'm guessing there's special treatment :)
  - ohiovr 6y ago
    Interesting. I've not looked at the long rule list in a while.
- km3r 6y ago
  A stream may also make it harder for the app to work in areas with poor connection. Which, given Uber's use case, is probably a likely scenario and one that could lose a lot of customers to competitors.
  - adrr 6y ago
    Pull it on first run and cache it. If you're in place with a poor internet connection, you aren't going to be able to download the app from app store any way. I am going to assume 80% of the code packaged in the app in the app is never used by a majority of their customer base. Like all the business features where a company give their employees allowances.
  - ohiovr 6y ago
    From some simple experiences with recording my desktop to a mp4 file I've found the delta compression to be extremely efficient when there is only a little motion. Perhaps still a deal killer, true.
- doggodaddo78 6y ago
  That's called a web browser and a web app. And then there wouldn't be anything to hog-up 1/3 of a GiB on every customer's phone, and it would always be up-to-date. Just don't ever lose internet access.
  - ohiovr 6y ago
    Especially getting lost in a cell dead zone. Its not exactly a web app but it could be made that way with WebRTC.
  - amelius 6y ago
    How do you expect to hail a taxi when you don't have internet access in the first place? Does the app use SMS when the internet connection is lost?
  - depressedpanda 6y ago
    Web apps can be cached and run entirely offline. https://en.m.wikipedia.org/wiki/Progressive_web_application#Service_workers https://en.m.wikipedia.org/wiki/Progressive_web_application#...
dmitriid 6y ago
Complementary: This thread on Uber's transition to Swift that almost broke them https://twitter.com/StanTwinB/status/1336890442768547845 https://twitter.com/StanTwinB/status/1336890442768547845 Includes, among other things: forcing Apple to increase cellular download limits, 45 seconds for letters to appear in XCode, 12 seconds to call main, rewriting the linker and so on.
- satya71 6y ago
  Good read. Good to know that Uber engineering culture was as much of a dumpster fire (despite the brilliant moves) as their product.
  - pid_0 6y ago
    Brilliant is a big word for a company that still hasn't earned a profit
    - ohiovr 6y ago
      They took a lot of investor's money.. So in a sense it was a success.
  - satya71 6y ago
    Hmm, looks like this comment didn't go over well. I think they had brilliant engineers (far beyond what I can even imagine) that pulled-off incredible feats to avoid disaster. But the disaster need not have loomed. Rewrite-the-world development smacks of what I call "cowboy engineering". Why didn't they migrate the existing code-base one layer at a time?
- ohiovr 6y ago
  I'm sure swift's shifting sand castle language changes didn't help.
speedgoose 6y ago
It's maybe a naive question, but from my point of view I don't understand why it's even a problem. Why is the Uber app so big ?
- donarb 6y ago
  This article is bogus. They spend a whole bunch of time benchmarking build sizes, nothing about why they need so much code in an iOS app to begin with. Apple has dropped limits for large app downloads on cellular. They now put up a dialog to tell the user how big the download is and if they wish to defer to a WiFi download. I checked the size of the Uber app and it's about 300MB. Uber Driver is 232MB and Uber Eats is 228MB.
thegeomaster 6y ago
It has... a few million... lines of code? What? Linux has 30 million of C! I'm speechless. I cannot fathom how & why.
- edoceo 6y ago
  Cannot fathom why LOC is a metric? Me neither. Lots of stuff has millions of lines of code in various languages with wide ranging feature-sets and functionality. LOC has near zero meaning across the language/project boundary.
  - deleted 6y ago
    [deleted]
- travisgriggs 6y ago
  I've observed that lines of code are measured differently based on whether the writer is trying to convince the audience that the subject matter is big and complicated and the reader should respect the magnitude of dealing with this particular piece of software OR whether the author wants you to appreciate the brevity/simplicity/approachability of the software in question. The first decision made in this decision tree is whether you just use wc , or whether you filter out empty lines. Next goes the comments. Next goes syntactically less significant lines (just a closing brace that could* go on the previous line). Wash rinse repeat. It's a variant of the "I didn't have time to write you a short ____ so I wrote a long one instead" adage. I would guess (but only guess) that this article erred on the side overstating size.
- dntrkv 6y ago
  How? 1000 engineers x 1000 LoC/each = 1M LoC Why? If you have 100+ engineers at any given time, shipping features over a period of a few years, you'll hit 1M in no time. It sounds like a lot, but it really isn't when you consider the amount of people working on it. Now whether or not you can build the same thing with less LoC, probably. But it's not like it was built from the ground up with every piece of functionality planned out from day 1, so there will be inefficiencies. Comparing it to Linux is pointless. Platforms should be relatively stable, products are ever changing and the shelf-life of the code is sometimes measured in weeks/months.
bhupy 6y ago
An old (but fantastic) comment from the previous discussion about Uber's app size that addresses why the Uber app is so big: https://news.ycombinator.com/item?id=25376346 https://news.ycombinator.com/item?id=25376346
- jedberg 6y ago
  The summary of that comment is "we have to include a ton of stuff that will never be relevant to most users like payments APIs that only work in India." This is why some global apps have different apps for different countries. It's a trade off. Would you rather have a single fat Uber app, or have to download Uber India when you arrive there?
  - bhupy 6y ago
    > This is why some global apps have different apps for different countries Do you have some examples? Genuinely curious. To my knowledge, most of the major FAANG apps are single-binary.
    - mkipper 6y ago
      Dedicated apps for fast food chains are one example. A quick search gave me "Burger King India", "PizzaHut Egypt" and "KFC UAE". Why? I have no idea.
      - jedberg 6y ago
        Probably payment APIs, just like Uber. Or they are developed by different local app shops.
      - jdminhbg 6y ago
        I think fast food chains are generally not actually owned by the same parent company. A company in India is licensing the Burger King branding and presumably some of the recipes from Burger King USA, rather than being a subsidiary thereof.
ketralnis 6y ago
Can someone help me understand this? They blame the source of the large bundle size on: > The choice of Swift as our primary programming language, our fast-paced development environment and feature additions, layered software and its dependencies, and statically linked platform libraries result in large app binaries but can somebody familiar with iOS development explain what makes app bundles so big? Actual CPU instructions or config can't contribute this significantly. The entire Bible is about 4.5mb. If you're writing an app by yourself you almost certainly didn't write that much text in the source code. A sibling comment links to https://news.ycombinator.com/item?id=25376346 https://news.ycombinator.com/item?id=25376346 which says that they have a lot of screens but even something like "PayTM (15+ screens)" is still just textual source code and config that I don't follow how it gets beyond kilobytes. The App Store places them at 309mb, so ~68 bibles. I understand when games are large because they typically ship with images and videos included in the binary for game assets. But for a normal application where does the size come from? Is it dependencies? (And how did _they_ get so big?) That weird intro video they have on the loading screen? Are they shipping bitmaps of the cities they have markets in?
- bhupy 6y ago
  I'd say this is the most comprehensive breakdown: https://news.ycombinator.com/item?id=25376346 https://news.ycombinator.com/item?id=25376346
  - ketralnis 6y ago
    Let's say there are a hundred screens in the app and the app is 300mb. Does it really take 3mb of source code, about 3/4 of a bible, to render one screen? (I do understand that source code isn't what ships in the binary, but for the sake argument let's say they're 1:1 in size.)
    - johncolanduoni 6y ago
      The argument you’ll have by saying that for its sake will be pretty useless, because source code and machine code are nowhere near 1:1 in size.
BigBalli 6y ago
328.9 MB I'm sure it could easily be 1/10 of that while keeping all features important to the user.
- jamestenglish 6y ago
  Important to a user in 1 region/country. https://news.ycombinator.com/item?id=25376346 https://news.ycombinator.com/item?id=25376346 Uber is a global app, so the other 9/10s of the code is for all the features and functionality you'll never see outside your region since there is currently no way to split up binaries by region.
  - jbverschoor 6y ago
    328 mb more than a complete OS. No doubt they can EASILY trim 30-60%. Asset optimization, stripping SDK libraries and you're done. It's like graffiti.. the app is so big already, that the devs don't give a damn about optimizing.. why bother if they are just A/B-feature tests? 30mb for some unoptimized screens for example
    - jakub_g 6y ago
      If you click the link OP posted and read the previous thread you will see they can "easily" do ass. They already spent millions in engineering time to cut the app size as the company existence literally depended on it when Apple's bundle size limit was 150 MB. If your app is serving hundreds of cities with specific per-city customizations and all code and assets are in a single binary, life gets tough.
      - jbverschoor 6y ago
        If you open up the IPA, you'll see that basically they've recreated some anroid-xml compatible rendering engine. The localization files (50MB) -> All the strings files are double the size (unpacked), because of useless comments. There's 25MB already. In the assets catalog -> half an MB for an upscaled (!) visa card. Other images where jpgs of heif are a better choice. probably in total 10-20 mb. Strip all ICC/Gamma from the PNGS -> another 10mb. pngcrush the images -> about 40% And then of course the binary itself which is probably full of unused information.
whoisburbansky 6y ago
> While power law and fractal patterns have revealed themselves in several physical, biological, and man-made phenomena, to our knowledge we are the first to identify their presence in machine-code sequences in computer executable code. Presumably, machine code is a human expression of instructions to a computer and it is well established that all human languages show a power-law in the frequency of the words. Made me chuckle. Maybe the authors should look at getting an ACM subscription. [https://dl.acm.org/doi/pdf/10.1145/1391984.1391986 https://dl.acm.org/doi/pdf/10.1145/1391984.1391986]
- bangara 6y ago
  It is an over claim to attribute the findings in the blog here to the "Power Laws in Software" paper published in 2008 at ACM Transactions on Software Engineering and Methodology. The ACM paper is casting a wide net on many places where distributions show power law in software and is focused on software modules/libraries/classes and their dependencies. There is a mention of CPU ISA using instruction frequency in CISC architectures but no in-depth treatment of the subject. This blog focuses on the machine instructions and analyzes just not instruction frequency but a whole sequence of instructions and their frequencies plus their lengths in an exhaustive manner.
superkuh 6y ago
Uber is just a scam to launder saudi blood money through softbank. There is zero chance that human driven cars will go away. And self-driving cars cannot drive on roads with human drivers. Uber is in the later stages of the scam now and have "sold" (actually, they gave 400 million USD to the company they "sold" the division too) their self driving setup. They've admitted the only business model that would make them profitable is impossible. It's over. They're just trying to take the money and run now. That their "app" is large is irrelevant to the scam.
- viscanti 6y ago
  According to their public investor reports, they've been EBIDTA profitable on Rides for years. They're also profitable in more rationalized Eats markets (markets where there aren't other VC funded companies burning hundreds of millions of dollars on subsidies). What do they need self driving for?
  - superkuh 6y ago
    To still be profitable after they stop breaking local laws and regulation catches up with them. https://horanaviation.com/publications-uber https://horanaviation.com/publications-uber
  - ohiovr 6y ago
    Not exactly: https://finance.yahoo.com/quote/UBER/financials?p=UBER https://finance.yahoo.com/quote/UBER/financials?p=UBER It is weird that something as exploitative as Uber can't even stay in business in the long term. Eventually profit does matter and the weak stocks will be culled in the next panic. Uber will be one of them.
danpalmer 6y ago
I'd like to see Apple expose more control over their app thinning technologies. Currently they only deliver the binary for the device's CPU, and only the assets for the device's asset class. There's then some tech targeted at game devs for on-demand assets for things like game levels that you don't need all of on device at one time. I suspect the limitations of this are around the binary not being subject to this, but maybe it could be. I can see a couple of options, one is some way of extending the asset classes to code features, so that the App Store doesn't have to download iPad screens for iPhones, etc. Perhaps this could be extended with either App Store account region or locale so that, Uber in this example could not include the Venmo SDK outside of the US where no one has heard of Venmo. Or perhaps Apple could extend the on-demand assets to allow for some sort of plugin system, perhaps backed by Swift Packages, such that apps can on-demand decide they need the Venmo SDK because they're in the US, and download just that. I don't think we want a generalised package manager here, I don't envision that SDK coming from Venmo directly, but allowing an app author to upload all their separate packages if they want to. With feature heavy, international apps such as Uber I'd expect this to dramatically improve things. I'm not sure whether this benefit would translate to that much demand across the whole App Store though as I think this matters more to a very few big apps. Apple is at that optimisation point in the iOS lifecycle though so perhaps it's worth it to them.
- novok 6y ago
  Well since on demand resources were made for games (lua is specifically called out as 'ok'), I could imagine many games also making their initial size far smaller if they used binary code to make new levels or regions. Games are very large chunk of the app store and it's revenue.
  - nightowl_games 6y ago
    Games don't really use those technologies because we want to use the same tech on Android and Apple so we typically roll our own or pay a third party BaaS. The big players probably almost always roll there own.
    - 6y ago
sond813 6y ago
I’m the founder of a YC company in the current batch focused on solving this exact problem! https://www.emergetools.com https://www.emergetools.com We parse Obj-C and Swift runtime metadata to determine size contributions of individual types and functions in your app. We use this analysis to post PR comments with granular size diffs to help devs write smaller, better code. I tried it out on the Uber app and immediately noticed a disproportionate impact from their code-gen dependency injection framework, Needle. The codegen is responsible for over 30k classes in the app binary, and contributes over 10mb! In general codegen is a common problem with Swift binary sizes, and the fewer reference types generated the better, it even helps with startup time! We’ve written a blog post with case studies about how 7 of the most popular iOS apps could reduce their size: https://medium.com/swlh/how-7-ios-apps-could-save-you-500mb-of-storage-a828782c973e https://medium.com/swlh/how-7-ios-apps-could-save-you-500mb-...
- deleted 6y ago
  [deleted]
- rememberlenny 6y ago
  "The Lyft app has hundreds of duplicate files, the largest consumer of space is a single asset catalog copied 73 times in separate bundles. Another asset catalog that is virtually identical except for the timestamp at which it was created is copied 67 times. Each of these contain nothing but 482 colors (colors can be stored in asset catalogs to simplify management of dark mode). With each one taking ~250kb these quickly eat up 35mb." I read this as: Lyft installed Dark mode for 35mb. I can only imagine what my JavaScript modules are doing behind the scenes.
  - trevor-e 6y ago
    My biggest complaint with iOS development is how confusing Xcode's build system is. Extracting code out to shared frameworks is a confusing process and I can understand how so many of the top apps get it wrong. Also, it's clearly not a priority for Apple because they don't provide easy inspection tools. Best case for them is the user buys a new phone with more storage.
tomaskafka 6y ago
This seems like a case of sloppy product management saved (or rather having its consequences delayed) by person-years of ingenious engineering. Build times in tens of minutes seem terrible.
- Spivak 6y ago
  Oh I would kill for a sub half hour build time at $dayjob.
StreamBright 6y ago
It is still amazing to me how Uber cannot narrow down the use cases enough. To me it was a done product in 2014, no need to additional features. I think the software industry as a whole does not have the concept of 1.0. We are trying to ship one more thing all the time.
abductee_hg 6y ago
...as a demoscener who has released multiple 64k/4k intros I hereby formally say: LOL.
- deleted 6y ago
  [deleted]
elpakal 6y ago
I was quite surprised by the increase in build times: > Overall, 5 rounds of outlining builds in 66 minutes — a 45-minutes addition to the baseline.
viktorcode 6y ago
> The app has a couple of millions of lines of code I wonder if Uber is planning to do anything about that? The technique described in the article (whole program instructions outlining optimisation) is a band aid style solution, merely delaying the inevitable: the code produced by numerous teams independent of each other will inevitably cross first the download size limit threshold, and later maintainability threshold.
- alien_ 6y ago
  I also find it unbelievable to have so much code for a single app, this is approaching the level of magnitude of OS code bases: if you think that the entire Linux kernel is around 28M lines so roughly 15x the Uber app. The binary size is also from the same ballpark as the entire Windows 98 needed for installation. I'm glad Uber is doing something about this, but in my opinion Apple should tackle this across their entire ecosystem at the toolchain level, devices with less than 64GB of storage can quickly run out of space with just a handful of applications installed. Unfortunately it's in Apple's interest that people buy devices with more storage, so I don't expect them to invest much effort in this.
  - tenacious_tuna 6y ago
    For both technical and UX reasons it's in Apple's interest to make app updates lightweight, and those likely far outweigh any driving force they might have to encourage people to move to higher storage sizes. (1) if it takes a long time for people to update their apps, that's a crap experience that people are having on Apple devices, which goes directly against the grain of Apple's whole value proposition ("use our stuff and your life will be great!") (2) For technical reasons, it's in Apple's interest to reduce app image sizes; less strain on infrastructure, easier to scale, etc. (300MB * 1.2M (# of app store reviews) = 360 terabytes transiting their networks whenever Uber pushes an update. All that has to be load balanced, CDN'd, etc.)
  - ajconway 6y ago
    > I also find it unbelievable to have so much code for a single app This point comes up in a lot of discussions about non-trivial software. My theory is that it's of the same nature as underestimating development complexity when planning your own work as an engineer. Project after project, everyone (me included) keeps forgetting that they _will_ spend 80% of their time (and code) dealing with small issues and edge cases in their product. Who hasn't thought at some point that they can write a Twitter clone in a weekend, or hasn't been fascinated by the amount of simple bugs in someone's else product, thinking that they are obviously just bad engineers.
etaioinshrdlu 6y ago
Machine-code outlining sounds kind of like the opposite of function inlining. Right down to the name! I am amazed I've never heard of this optimization technique being used in compilers before -- it sounds like it could improve performance in many cases by making code smaller (or hurt performance for the same reason that inlining can help performance)...
rockmeamedee 6y ago
There's apparently a bananas crunch/backstory to this, where they committed to Swift before realizing they would hit its limits, and had to come up with a bunch of this optimization madness on the fly. I guess this is the cleaned up version and the more final, stable optimizations for the company blog: https://twitter.com/StanTwinB/status/1336890442768547845 https://twitter.com/StanTwinB/status/1336890442768547845
alien_ 6y ago
Wouldn't UPX give similar results in deduplicating binary code?
- lgg 6y ago
  No. UPX won't work on iOS for various reasons, but it is also generally speaking a poor choice to use on macOS. There are several issues here: * It usually does not reduce the size of the file in transit, as most files are compressed for distribution, and even if they are not most http servers will use transparent gzip compression * It does not actually reduce the size of files at rest since APFS (and HFS+ before it) support transparent decompression. The layer this is handled at is sufficiently low level most people do not even realize it is happening (stat(2) returns the uncompressed size, you need to look at extended attributes to see the real on disk size). Admittedly this does not handle binaries that are drag installed on macOS. You can find out more details here: https://github.com/RJVB/afsctool https://github.com/RJVB/afsctool * UPX slows down app launch because you know have to decompress the entire executable before you launch it, which means you need need to read the entire compressed executable from disk before you launch it * UPX greatly increases the memory overhead of running an application. Because you decompressed it in memory all the executable pages are dirty memory that need to be kept in memory or written to swap. That means you immediately loaded everything into memory instead of just the pages you needed. Normally a binaries pages are brought in from disk as necessary, and because of that they are unmodified clean memory. The built in compression support compresses smaller blocks of the binary and thus can still bring them in individually (technically this reduces the compression, but the trade off of being able to keep page demand loading working is more than worth it). * UPX makes the system perform worse under memory pressure. The fact that it generates decompresses the pages in userspace means that from the kernels perspective they are dirty. If the kernel needs to evict them due to memory pressure it needs to swap them out. Uncompressed files (or those compressed with the builtin filesystem compression) are clean memory, which means that under memory pressure the kernel can just through them out and then reload them from the file later, no need to write out the pages. In the past there were legitimate reasons for tools like UPX, and there may still be on other operating systems, but it simply does not make sense on Darwin platforms.
pgp001 6y ago
wow, all that code and it's still absolutely terrible to use. Imagine
saagarjha 6y ago
This is certainly an great read, and working on it must have very interesting. That being said, in my experience things like these are invariably technical band-aids over social problems. Whenever I see things like this, often paired with statements like “there’s so many screens and feature flags”, usually the problem is not there but actually in many other processes: for example, the design team adds assets in a way that is not enforced by the usual tooling that checks binary size, or the build process adds duplicate files into the bundle that nobody notices. Sometimes the underlying issue is hard to fix, like if it’s code size explosion due to a custom templating engine, but they really should get addressed at some point. Changes like these don’t actually solve the underlying issues, which can be a benefit for a while, but eventually they become so complex that it is hard to maintain them and they start impacting productivity in harder to measure ways by doing things like increasing build times and reducing the quality of debugging information.
- draw_down 6y ago
  > usually the problem is Usually, sure. But sometimes there is a lot to do, and if I may, Uber is not your usual app. At the point where you're being very choosy about the access modifiers on your classes, you probably thought about icon assets already. Someone elsewhere in thread linked to a partial list of concerns the app needs to cover, many of which are location-specific. You might say "well split the app by geography", but that just trades one set of problems for another, and that new set of problems could well be worse for the business overall. Paying a team of people to do this junk may be a whole lot cheaper than suffering a reduction in customer engagement when they fly to a new country and don't have the right app anymore.
  - saagarjha 6y ago
    > At the point where you're being very choosy about the access modifiers on your classes, you probably thought about icon assets already. You'd think, but many of the most popular apps accidentally ship these all the time. I think another comment mentioned that much of the code size seems to be coming from a code generation framework.
dj_mc_merlin 6y ago
I would love to work on something like this. Optimizing assembler instructions for size, speed.. just writing some. How do you get a job in this? Is embedded land like this?
pid_0 6y ago
Is part of the strategy using private iOS APIs, tracking everyone outside of ToS, lying about it, getting caught, and then being too large for Apple to actually punish them?
ketamine__ 6y ago
Wouldn't native apps be much smaller? I wish more companies did this. Uber on Android handles quite poorly.
mshockwave 6y ago
Why not do machine outlining in LTO/ThinLTO? `opt` doesn't really scale with huge module in terms of memory consumption and multi-threading, that's the reason ThinLTO was invented in the first place. I think adding machine outlining into LLVM Pass pipeline is still doable with LLVM plugin (with new PassManager)...worst case just come up with a custom LLVM/Clang
lgats 6y ago
On a smaller note, adding lossless compression to the image assets in the Uber app can save more than 14% of 2.4MB If your app has larger images, don't waste user bandwidth and optimize your assets!
Triv888 6y ago
Theare essentially compressing their app... ?
frusciante29 6y ago
not very well, i guess?
londons_explore 6y ago
So Uber has about 5000 engineers. If all of them write/click/draw 10,000 keystrokes per day, and it's all new code, and Uber has been around 10 years, thats 182 gigabytes of 'human input'. Compiling that down to 200 MB isnt too shabby!

257 points by rakingleaves 6y ago

I'd take a sledge hammer to it. The app doesn't have to be an app at all. It could simply be a stream with an os interactive overlay that intercepts touches. Like a thin client for phones.

lxe 6y ago
That could trigger the App Store review block. But this is Uber, so I'm guessing there's special treatment :)
- ohiovr 6y ago
  Interesting. I've not looked at the long rule list in a while.
km3r 6y ago
A stream may also make it harder for the app to work in areas with poor connection. Which, given Uber's use case, is probably a likely scenario and one that could lose a lot of customers to competitors.
- adrr 6y ago
  Pull it on first run and cache it. If you're in place with a poor internet connection, you aren't going to be able to download the app from app store any way. I am going to assume 80% of the code packaged in the app in the app is never used by a majority of their customer base. Like all the business features where a company give their employees allowances.
- ohiovr 6y ago
  From some simple experiences with recording my desktop to a mp4 file I've found the delta compression to be extremely efficient when there is only a little motion. Perhaps still a deal killer, true.
doggodaddo78 6y ago
That's called a web browser and a web app. And then there wouldn't be anything to hog-up 1/3 of a GiB on every customer's phone, and it would always be up-to-date. Just don't ever lose internet access.
- ohiovr 6y ago
  Especially getting lost in a cell dead zone. Its not exactly a web app but it could be made that way with WebRTC.
- amelius 6y ago
  How do you expect to hail a taxi when you don't have internet access in the first place? Does the app use SMS when the internet connection is lost?
- depressedpanda 6y ago
  Web apps can be cached and run entirely offline. https://en.m.wikipedia.org/wiki/Progressive_web_application#Service_workers https://en.m.wikipedia.org/wiki/Progressive_web_application#...

dmitriid 6y ago

Complementary: This thread on Uber's transition to Swift that almost broke them https://twitter.com/StanTwinB/status/1336890442768547845 https://twitter.com/StanTwinB/status/1336890442768547845 Includes, among other things: forcing Apple to increase cellular download limits, 45 seconds for letters to appear in XCode, 12 seconds to call main, rewriting the linker and so on.

satya71 6y ago
Good read. Good to know that Uber engineering culture was as much of a dumpster fire (despite the brilliant moves) as their product.
- pid_0 6y ago
  Brilliant is a big word for a company that still hasn't earned a profit
  - ohiovr 6y ago
    They took a lot of investor's money.. So in a sense it was a success.
- satya71 6y ago
  Hmm, looks like this comment didn't go over well. I think they had brilliant engineers (far beyond what I can even imagine) that pulled-off incredible feats to avoid disaster. But the disaster need not have loomed. Rewrite-the-world development smacks of what I call "cowboy engineering". Why didn't they migrate the existing code-base one layer at a time?
ohiovr 6y ago
I'm sure swift's shifting sand castle language changes didn't help.

speedgoose 6y ago

It's maybe a naive question, but from my point of view I don't understand why it's even a problem. Why is the Uber app so big ?

donarb 6y ago
This article is bogus. They spend a whole bunch of time benchmarking build sizes, nothing about why they need so much code in an iOS app to begin with. Apple has dropped limits for large app downloads on cellular. They now put up a dialog to tell the user how big the download is and if they wish to defer to a WiFi download. I checked the size of the Uber app and it's about 300MB. Uber Driver is 232MB and Uber Eats is 228MB.

thegeomaster 6y ago

It has... a few million... lines of code? What? Linux has 30 million of C! I'm speechless. I cannot fathom how & why.

edoceo 6y ago
Cannot fathom why LOC is a metric? Me neither. Lots of stuff has millions of lines of code in various languages with wide ranging feature-sets and functionality. LOC has near zero meaning across the language/project boundary.
- deleted 6y ago
  [deleted]
travisgriggs 6y ago
I've observed that lines of code are measured differently based on whether the writer is trying to convince the audience that the subject matter is big and complicated and the reader should respect the magnitude of dealing with this particular piece of software OR whether the author wants you to appreciate the brevity/simplicity/approachability of the software in question. The first decision made in this decision tree is whether you just use wc , or whether you filter out empty lines. Next goes the comments. Next goes syntactically less significant lines (just a closing brace that could* go on the previous line). Wash rinse repeat. It's a variant of the "I didn't have time to write you a short ____ so I wrote a long one instead" adage. I would guess (but only guess) that this article erred on the side overstating size.
dntrkv 6y ago
How? 1000 engineers x 1000 LoC/each = 1M LoC Why? If you have 100+ engineers at any given time, shipping features over a period of a few years, you'll hit 1M in no time. It sounds like a lot, but it really isn't when you consider the amount of people working on it. Now whether or not you can build the same thing with less LoC, probably. But it's not like it was built from the ground up with every piece of functionality planned out from day 1, so there will be inefficiencies. Comparing it to Linux is pointless. Platforms should be relatively stable, products are ever changing and the shelf-life of the code is sometimes measured in weeks/months.

bhupy 6y ago

An old (but fantastic) comment from the previous discussion about Uber's app size that addresses why the Uber app is so big: https://news.ycombinator.com/item?id=25376346 https://news.ycombinator.com/item?id=25376346

jedberg 6y ago
The summary of that comment is "we have to include a ton of stuff that will never be relevant to most users like payments APIs that only work in India." This is why some global apps have different apps for different countries. It's a trade off. Would you rather have a single fat Uber app, or have to download Uber India when you arrive there?
- bhupy 6y ago
  > This is why some global apps have different apps for different countries Do you have some examples? Genuinely curious. To my knowledge, most of the major FAANG apps are single-binary.
  - mkipper 6y ago
    Dedicated apps for fast food chains are one example. A quick search gave me "Burger King India", "PizzaHut Egypt" and "KFC UAE". Why? I have no idea.
    - jedberg 6y ago
      Probably payment APIs, just like Uber. Or they are developed by different local app shops.
    - jdminhbg 6y ago
      I think fast food chains are generally not actually owned by the same parent company. A company in India is licensing the Burger King branding and presumably some of the recipes from Burger King USA, rather than being a subsidiary thereof.

ketralnis 6y ago

Can someone help me understand this? They blame the source of the large bundle size on: > The choice of Swift as our primary programming language, our fast-paced development environment and feature additions, layered software and its dependencies, and statically linked platform libraries result in large app binaries but can somebody familiar with iOS development explain what makes app bundles so big? Actual CPU instructions or config can't contribute this significantly. The entire Bible is about 4.5mb. If you're writing an app by yourself you almost certainly didn't write that much text in the source code. A sibling comment links to https://news.ycombinator.com/item?id=25376346 https://news.ycombinator.com/item?id=25376346 which says that they have a lot of screens but even something like "PayTM (15+ screens)" is still just textual source code and config that I don't follow how it gets beyond kilobytes. The App Store places them at 309mb, so ~68 bibles. I understand when games are large because they typically ship with images and videos included in the binary for game assets. But for a normal application where does the size come from? Is it dependencies? (And how did _they_ get so big?) That weird intro video they have on the loading screen? Are they shipping bitmaps of the cities they have markets in?

bhupy 6y ago
I'd say this is the most comprehensive breakdown: https://news.ycombinator.com/item?id=25376346 https://news.ycombinator.com/item?id=25376346
- ketralnis 6y ago
  Let's say there are a hundred screens in the app and the app is 300mb. Does it really take 3mb of source code, about 3/4 of a bible, to render one screen? (I do understand that source code isn't what ships in the binary, but for the sake argument let's say they're 1:1 in size.)
  - johncolanduoni 6y ago
    The argument you’ll have by saying that for its sake will be pretty useless, because source code and machine code are nowhere near 1:1 in size.

BigBalli 6y ago

328.9 MB I'm sure it could easily be 1/10 of that while keeping all features important to the user.

jamestenglish 6y ago
Important to a user in 1 region/country. https://news.ycombinator.com/item?id=25376346 https://news.ycombinator.com/item?id=25376346 Uber is a global app, so the other 9/10s of the code is for all the features and functionality you'll never see outside your region since there is currently no way to split up binaries by region.
- jbverschoor 6y ago
  328 mb more than a complete OS. No doubt they can EASILY trim 30-60%. Asset optimization, stripping SDK libraries and you're done. It's like graffiti.. the app is so big already, that the devs don't give a damn about optimizing.. why bother if they are just A/B-feature tests? 30mb for some unoptimized screens for example
  - jakub_g 6y ago
    If you click the link OP posted and read the previous thread you will see they can "easily" do ass. They already spent millions in engineering time to cut the app size as the company existence literally depended on it when Apple's bundle size limit was 150 MB. If your app is serving hundreds of cities with specific per-city customizations and all code and assets are in a single binary, life gets tough.
    - jbverschoor 6y ago
      If you open up the IPA, you'll see that basically they've recreated some anroid-xml compatible rendering engine. The localization files (50MB) -> All the strings files are double the size (unpacked), because of useless comments. There's 25MB already. In the assets catalog -> half an MB for an upscaled (!) visa card. Other images where jpgs of heif are a better choice. probably in total 10-20 mb. Strip all ICC/Gamma from the PNGS -> another 10mb. pngcrush the images -> about 40% And then of course the binary itself which is probably full of unused information.

whoisburbansky 6y ago

> While power law and fractal patterns have revealed themselves in several physical, biological, and man-made phenomena, to our knowledge we are the first to identify their presence in machine-code sequences in computer executable code. Presumably, machine code is a human expression of instructions to a computer and it is well established that all human languages show a power-law in the frequency of the words. Made me chuckle. Maybe the authors should look at getting an ACM subscription. [https://dl.acm.org/doi/pdf/10.1145/1391984.1391986 https://dl.acm.org/doi/pdf/10.1145/1391984.1391986]

bangara 6y ago
It is an over claim to attribute the findings in the blog here to the "Power Laws in Software" paper published in 2008 at ACM Transactions on Software Engineering and Methodology. The ACM paper is casting a wide net on many places where distributions show power law in software and is focused on software modules/libraries/classes and their dependencies. There is a mention of CPU ISA using instruction frequency in CISC architectures but no in-depth treatment of the subject. This blog focuses on the machine instructions and analyzes just not instruction frequency but a whole sequence of instructions and their frequencies plus their lengths in an exhaustive manner.

superkuh 6y ago

Uber is just a scam to launder saudi blood money through softbank. There is zero chance that human driven cars will go away. And self-driving cars cannot drive on roads with human drivers. Uber is in the later stages of the scam now and have "sold" (actually, they gave 400 million USD to the company they "sold" the division too) their self driving setup. They've admitted the only business model that would make them profitable is impossible. It's over. They're just trying to take the money and run now. That their "app" is large is irrelevant to the scam.

viscanti 6y ago
According to their public investor reports, they've been EBIDTA profitable on Rides for years. They're also profitable in more rationalized Eats markets (markets where there aren't other VC funded companies burning hundreds of millions of dollars on subsidies). What do they need self driving for?
- superkuh 6y ago
  To still be profitable after they stop breaking local laws and regulation catches up with them. https://horanaviation.com/publications-uber https://horanaviation.com/publications-uber
- ohiovr 6y ago
  Not exactly: https://finance.yahoo.com/quote/UBER/financials?p=UBER https://finance.yahoo.com/quote/UBER/financials?p=UBER It is weird that something as exploitative as Uber can't even stay in business in the long term. Eventually profit does matter and the weak stocks will be culled in the next panic. Uber will be one of them.

danpalmer 6y ago

I'd like to see Apple expose more control over their app thinning technologies. Currently they only deliver the binary for the device's CPU, and only the assets for the device's asset class. There's then some tech targeted at game devs for on-demand assets for things like game levels that you don't need all of on device at one time. I suspect the limitations of this are around the binary not being subject to this, but maybe it could be. I can see a couple of options, one is some way of extending the asset classes to code features, so that the App Store doesn't have to download iPad screens for iPhones, etc. Perhaps this could be extended with either App Store account region or locale so that, Uber in this example could not include the Venmo SDK outside of the US where no one has heard of Venmo. Or perhaps Apple could extend the on-demand assets to allow for some sort of plugin system, perhaps backed by Swift Packages, such that apps can on-demand decide they need the Venmo SDK because they're in the US, and download just that. I don't think we want a generalised package manager here, I don't envision that SDK coming from Venmo directly, but allowing an app author to upload all their separate packages if they want to. With feature heavy, international apps such as Uber I'd expect this to dramatically improve things. I'm not sure whether this benefit would translate to that much demand across the whole App Store though as I think this matters more to a very few big apps. Apple is at that optimisation point in the iOS lifecycle though so perhaps it's worth it to them.

novok 6y ago
Well since on demand resources were made for games (lua is specifically called out as 'ok'), I could imagine many games also making their initial size far smaller if they used binary code to make new levels or regions. Games are very large chunk of the app store and it's revenue.
- nightowl_games 6y ago
  Games don't really use those technologies because we want to use the same tech on Android and Apple so we typically roll our own or pay a third party BaaS. The big players probably almost always roll there own.
  - 6y ago

sond813 6y ago

I’m the founder of a YC company in the current batch focused on solving this exact problem! https://www.emergetools.com https://www.emergetools.com We parse Obj-C and Swift runtime metadata to determine size contributions of individual types and functions in your app. We use this analysis to post PR comments with granular size diffs to help devs write smaller, better code. I tried it out on the Uber app and immediately noticed a disproportionate impact from their code-gen dependency injection framework, Needle. The codegen is responsible for over 30k classes in the app binary, and contributes over 10mb! In general codegen is a common problem with Swift binary sizes, and the fewer reference types generated the better, it even helps with startup time! We’ve written a blog post with case studies about how 7 of the most popular iOS apps could reduce their size: https://medium.com/swlh/how-7-ios-apps-could-save-you-500mb-of-storage-a828782c973e https://medium.com/swlh/how-7-ios-apps-could-save-you-500mb-...

deleted 6y ago
[deleted]
rememberlenny 6y ago
"The Lyft app has hundreds of duplicate files, the largest consumer of space is a single asset catalog copied 73 times in separate bundles. Another asset catalog that is virtually identical except for the timestamp at which it was created is copied 67 times. Each of these contain nothing but 482 colors (colors can be stored in asset catalogs to simplify management of dark mode). With each one taking ~250kb these quickly eat up 35mb." I read this as: Lyft installed Dark mode for 35mb. I can only imagine what my JavaScript modules are doing behind the scenes.
- trevor-e 6y ago
  My biggest complaint with iOS development is how confusing Xcode's build system is. Extracting code out to shared frameworks is a confusing process and I can understand how so many of the top apps get it wrong. Also, it's clearly not a priority for Apple because they don't provide easy inspection tools. Best case for them is the user buys a new phone with more storage.

tomaskafka 6y ago

This seems like a case of sloppy product management saved (or rather having its consequences delayed) by person-years of ingenious engineering. Build times in tens of minutes seem terrible.

Spivak 6y ago
Oh I would kill for a sub half hour build time at $dayjob.

StreamBright 6y ago

It is still amazing to me how Uber cannot narrow down the use cases enough. To me it was a done product in 2014, no need to additional features. I think the software industry as a whole does not have the concept of 1.0. We are trying to ship one more thing all the time.

abductee_hg 6y ago

...as a demoscener who has released multiple 64k/4k intros I hereby formally say: LOL.

deleted 6y ago
[deleted]

elpakal 6y ago

I was quite surprised by the increase in build times: > Overall, 5 rounds of outlining builds in 66 minutes — a 45-minutes addition to the baseline.

viktorcode 6y ago

> The app has a couple of millions of lines of code I wonder if Uber is planning to do anything about that? The technique described in the article (whole program instructions outlining optimisation) is a band aid style solution, merely delaying the inevitable: the code produced by numerous teams independent of each other will inevitably cross first the download size limit threshold, and later maintainability threshold.

alien_ 6y ago
I also find it unbelievable to have so much code for a single app, this is approaching the level of magnitude of OS code bases: if you think that the entire Linux kernel is around 28M lines so roughly 15x the Uber app. The binary size is also from the same ballpark as the entire Windows 98 needed for installation. I'm glad Uber is doing something about this, but in my opinion Apple should tackle this across their entire ecosystem at the toolchain level, devices with less than 64GB of storage can quickly run out of space with just a handful of applications installed. Unfortunately it's in Apple's interest that people buy devices with more storage, so I don't expect them to invest much effort in this.
- tenacious_tuna 6y ago
  For both technical and UX reasons it's in Apple's interest to make app updates lightweight, and those likely far outweigh any driving force they might have to encourage people to move to higher storage sizes. (1) if it takes a long time for people to update their apps, that's a crap experience that people are having on Apple devices, which goes directly against the grain of Apple's whole value proposition ("use our stuff and your life will be great!") (2) For technical reasons, it's in Apple's interest to reduce app image sizes; less strain on infrastructure, easier to scale, etc. (300MB * 1.2M (# of app store reviews) = 360 terabytes transiting their networks whenever Uber pushes an update. All that has to be load balanced, CDN'd, etc.)
- ajconway 6y ago
  > I also find it unbelievable to have so much code for a single app This point comes up in a lot of discussions about non-trivial software. My theory is that it's of the same nature as underestimating development complexity when planning your own work as an engineer. Project after project, everyone (me included) keeps forgetting that they _will_ spend 80% of their time (and code) dealing with small issues and edge cases in their product. Who hasn't thought at some point that they can write a Twitter clone in a weekend, or hasn't been fascinated by the amount of simple bugs in someone's else product, thinking that they are obviously just bad engineers.

etaioinshrdlu 6y ago

Machine-code outlining sounds kind of like the opposite of function inlining. Right down to the name! I am amazed I've never heard of this optimization technique being used in compilers before -- it sounds like it could improve performance in many cases by making code smaller (or hurt performance for the same reason that inlining can help performance)...

rockmeamedee 6y ago

There's apparently a bananas crunch/backstory to this, where they committed to Swift before realizing they would hit its limits, and had to come up with a bunch of this optimization madness on the fly. I guess this is the cleaned up version and the more final, stable optimizations for the company blog: https://twitter.com/StanTwinB/status/1336890442768547845 https://twitter.com/StanTwinB/status/1336890442768547845

alien_ 6y ago

Wouldn't UPX give similar results in deduplicating binary code?

lgg 6y ago
No. UPX won't work on iOS for various reasons, but it is also generally speaking a poor choice to use on macOS. There are several issues here: * It usually does not reduce the size of the file in transit, as most files are compressed for distribution, and even if they are not most http servers will use transparent gzip compression * It does not actually reduce the size of files at rest since APFS (and HFS+ before it) support transparent decompression. The layer this is handled at is sufficiently low level most people do not even realize it is happening (stat(2) returns the uncompressed size, you need to look at extended attributes to see the real on disk size). Admittedly this does not handle binaries that are drag installed on macOS. You can find out more details here: https://github.com/RJVB/afsctool https://github.com/RJVB/afsctool * UPX slows down app launch because you know have to decompress the entire executable before you launch it, which means you need need to read the entire compressed executable from disk before you launch it * UPX greatly increases the memory overhead of running an application. Because you decompressed it in memory all the executable pages are dirty memory that need to be kept in memory or written to swap. That means you immediately loaded everything into memory instead of just the pages you needed. Normally a binaries pages are brought in from disk as necessary, and because of that they are unmodified clean memory. The built in compression support compresses smaller blocks of the binary and thus can still bring them in individually (technically this reduces the compression, but the trade off of being able to keep page demand loading working is more than worth it). * UPX makes the system perform worse under memory pressure. The fact that it generates decompresses the pages in userspace means that from the kernels perspective they are dirty. If the kernel needs to evict them due to memory pressure it needs to swap them out. Uncompressed files (or those compressed with the builtin filesystem compression) are clean memory, which means that under memory pressure the kernel can just through them out and then reload them from the file later, no need to write out the pages. In the past there were legitimate reasons for tools like UPX, and there may still be on other operating systems, but it simply does not make sense on Darwin platforms.

pgp001 6y ago

wow, all that code and it's still absolutely terrible to use. Imagine

saagarjha 6y ago

This is certainly an great read, and working on it must have very interesting. That being said, in my experience things like these are invariably technical band-aids over social problems. Whenever I see things like this, often paired with statements like “there’s so many screens and feature flags”, usually the problem is not there but actually in many other processes: for example, the design team adds assets in a way that is not enforced by the usual tooling that checks binary size, or the build process adds duplicate files into the bundle that nobody notices. Sometimes the underlying issue is hard to fix, like if it’s code size explosion due to a custom templating engine, but they really should get addressed at some point. Changes like these don’t actually solve the underlying issues, which can be a benefit for a while, but eventually they become so complex that it is hard to maintain them and they start impacting productivity in harder to measure ways by doing things like increasing build times and reducing the quality of debugging information.

draw_down 6y ago
> usually the problem is Usually, sure. But sometimes there is a lot to do, and if I may, Uber is not your usual app. At the point where you're being very choosy about the access modifiers on your classes, you probably thought about icon assets already. Someone elsewhere in thread linked to a partial list of concerns the app needs to cover, many of which are location-specific. You might say "well split the app by geography", but that just trades one set of problems for another, and that new set of problems could well be worse for the business overall. Paying a team of people to do this junk may be a whole lot cheaper than suffering a reduction in customer engagement when they fly to a new country and don't have the right app anymore.
- saagarjha 6y ago
  > At the point where you're being very choosy about the access modifiers on your classes, you probably thought about icon assets already. You'd think, but many of the most popular apps accidentally ship these all the time. I think another comment mentioned that much of the code size seems to be coming from a code generation framework.

dj_mc_merlin 6y ago

I would love to work on something like this. Optimizing assembler instructions for size, speed.. just writing some. How do you get a job in this? Is embedded land like this?

pid_0 6y ago

Is part of the strategy using private iOS APIs, tracking everyone outside of ToS, lying about it, getting caught, and then being too large for Apple to actually punish them?

ketamine__ 6y ago

Wouldn't native apps be much smaller? I wish more companies did this. Uber on Android handles quite poorly.

mshockwave 6y ago

Why not do machine outlining in LTO/ThinLTO? `opt` doesn't really scale with huge module in terms of memory consumption and multi-threading, that's the reason ThinLTO was invented in the first place. I think adding machine outlining into LLVM Pass pipeline is still doable with LLVM plugin (with new PassManager)...worst case just come up with a custom LLVM/Clang

lgats 6y ago

On a smaller note, adding lossless compression to the image assets in the Uber app can save more than 14% of 2.4MB If your app has larger images, don't waste user bandwidth and optimize your assets!

Triv888 6y ago

Theare essentially compressing their app... ?

frusciante29 6y ago

not very well, i guess?

londons_explore 6y ago

So Uber has about 5000 engineers. If all of them write/click/draw 10,000 keystrokes per day, and it's all new code, and Uber has been around 10 years, thats 182 gigabytes of 'human input'. Compiling that down to 200 MB isnt too shabby!