13 ms·

Swift architecture at Uber

305 points by tsycho 9y ago

biocomputation 9y ago
I'm more interested in Waymo v Uber at the moment. kthxbai!
xyzzy4 9y ago
It seems the Uber app was extremely over-engineered. Call me crazy but I don't think you need 100 engineers to recreate the front-end of Uber.
- ReligiousFlames 9y ago
  Hire lots of people... complexity, arcaneness, process and knowledge-hoarding becomes vital to job-security.
  - tmh79 9y ago
    also when you need to develop a new feature and it needs to go through A:B testing frameworks, localization (including Arabic Right to Left text/layouts), phased feature roll outs, 99.999% reliability, integration testing, planning for use cases anywhere from LTE/WI-FI in USA to EDGE network in developing countries, and you need to build the tools that do those things too. Clearly the 100 engineers are sitting there twiddling their thumbs.
    - tyingq 9y ago
      They have a few other apps too, right? Driver app, Uber Eats, Uber Freight, etc.
    - ReligiousFlames 9y ago
      I think you might of missed the point... everyone has regulations and current problems at any scale, but the ability of BS job roles to hide in a large workforce and for managers to build pyramidal layers of organizations of little added value (but lots of expensive management layers with ever loftier titles).
      - venture_lol 9y ago
        I think you have been to a few rodeos :) Yeah, when a team goes from 2 to 100, a serious business/engineering manager got to ask a few questions. However, if money honey flow is not a problem, it could be in everyone's interest just to ride along
- htormey 9y ago
  Well your not taking into account all the A/B tests they are running, the fact that they have custom experiences for different locations (SFO, India, etc). I can easily see how they would get up to that number pretty quickly at the scale they are at.
  - venture_lol 9y ago
    At some point, consider the use of different apps altogether. Perhaps having a front loader to detect which local app should pop up based on information gleaned from the phone and GPS.
    - htormey 9y ago
      What would be the criteria for splitting something out? They do this with uber eats but I can't imagine them doing this for anything else. Why would I as a user want to download a variety of separate apps to hail cars, etc in different locations? People got really pissed off when Messenger was split out of Facebook (it's still mentioned in app store reviews to this day) I'd imagine they would not be wild about Uber doing something similar. I could see this getting really annoying really quickly. Imagine landing at a new airport and having to download a new app just to hail a car or a tuk tuk or whatever.
      - venture_lol 9y ago
        From a user perspective, you may not have to download a new app. Different locale, different customs, regulations may dictate entire different screens. How an "app" is designed and packaged internally to enable the business to respond quickly does not have to impact the use experience. There are plenty of design patterns and engineering experience to draw from. It's a somewhat boring topic. I find it more interesting to find answers to question such as: 1. A native person in India will be presented with an "Indian" app, satisfying the locale, regulations, laws, culture, etc. 2. A tourist landing in India, well, er, what version should that person use? Laws, regulations still apply (well, Uber may have a different take :) ) but how about the user experience and colloquial details? Would the tourist prefer something from home or something that more accurately, and perhaps more apt for India?
- jjtheblunt 9y ago
  He mentioned in the very beginning that there are city teams, so perhaps it's a number of people proportional (not exponential, like he says) to the growing # of cities.
- panic 9y ago
  You're coming at this backwards. They had 100 engineers already. They needed to design their software so that the engineers could all keep working on it. They explicitly say as much: > This application has served us very well for the past four years. But as we've expanded and exponentially grown our mobile engineering team, we started seeing breakages of the architecture, we started seeing problems where feature development would become pretty hard. We had to test multiple code-paths because we shared a lot of view controllers amongst different teams. We really started to hurt with the old architecture because it was written by two engineers and we had grown the team to over one hundred, at that point. The actual quality of the software isn't as important, since Uber drivers and riders don't have a choice anyway.
  - perfmode 9y ago
    Uber riders have more than one ride-sharing app on their phone. If Uber can't match me to a driver, I'm still going to get where I want to go.
    - deleted 9y ago
      [deleted]
- yawaworht12 9y ago
  Any software from a big company like Uber, Facebook, Google, etc. is far more complex than what you see. There are typically dozens of features and experiments rolled out to some users and not others. There is also a lot of work necessary to serve so many different markets, such as localization and internationalization. Furthermore, in Uber's case they have a lot more iOS applications than the ones riders use. In terms of complexity, Uber iOS apps are probably comparable to Facebook's apps.
- battwell 9y ago
  It's definitely a lot of engineers. But we cram a lot of features into the app. I expect that most people will only experience 10% of them. Consider: 1) We exist in a tonne of countries. And different counties often require different product optimizations and payment methods 2) We have our own map provider 3) We experiment with everything 4) We support countries that have terrible networking. That creates lots of challenges.
  - venture_lol 9y ago
    From an engineering perspective, that sounds uninteresting :) Terrible networking? how many engineers does it take to implement a few flavors of retries? :) Uber growth in term of getting drivers, that's what I want to know. It's not just ride subsidies, but the flow of investor honey helps, of course, but they do have some alpha growth managers.
    - uberengineer 9y ago
      Oh, it goes well beyond retries. You have issues like unpredictable latencies and lost network connectivity. A feature that works great when RTT is 400ms may require an entirely different approach when RTT is 30000ms. There are also issues with clocks due to both fraud and misconfigured cell towers. India for example has notoriously bad RTT times and Jakarta and several cities in LatAm have issues with clocks. At scale, across so many different markets, with so many different features, things get complex fast. We want the user experience for both riders and drivers to be magical. More magic requires more engineering. The goal is transportation as reliable and available as running water everywhere all the time. Here's an example of telematics engineering used to improve safety by detecting harsh braking: https://eng.uber.com/telematics/ https://eng.uber.com/telematics/ We also use the IMU measurements to detect drivers who aren't using a phone mount and are instead holding their iPhones while driving. Features like these allow us to advise drivers on how to be safer and get better ratings since not using a phone mount and harsh braking both correlate with negative user ratings. A lot more goes into the Uber app to make things magical. I used to read the front page of HN daily to read about all the cool things people around are working on. Since joining Uber, I check far less often now because there are so many cool problems being worked on internally that many of the front page stories sometimes seem quaint in comparison (not that many HNers aren't working on awesome things but the quantity and quality of cool problems my colleagues work on easily captures most of my attention these days). The best way I can describe working at Uber: it's like building that transportation component of sim-city for every city, everywhere, but it's not a simulation. Despite all the click bait negative press, its bar none the best place I've ever worked.
- unoti 9y ago
  > Call me crazy but I don't think you need 100 engineers to recreate the front-end of Uber. Your sentiment is valid, but consider this: whenever you're working with a project with a ton of money on the table, it becomes possible to add small bits of functionality which more than pay for themselves in terms of return on investment for the engineering effort. This is why you can often write a basic version of a more established application in "a weekend". We've had extensive discussion and meditation on this topic on HN before[1]. Basically apps will grow and add features as long as the engineering effort will pay for itself (I suppose I use that term loosely in this case). Also consider that the UX is different in different markets, and the reason for that is related to the first issue. I sympathize with what you're saying, though, and couldn't help but think about Alan Kay's talk that we discussed the other day, about natural complexity vs artificial complication.[2] [1] https://news.ycombinator.com/item?id=12626314 https://news.ycombinator.com/item?id=12626314 [2] https://news.ycombinator.com/item?id=14188759 https://news.ycombinator.com/item?id=14188759
- sudhirj 9y ago
  It's basically tens / hundreds of apps rolled into one, with frontends loaded based on your currently stated or GPS location. They all share a common foundation, but features vary wildly depending on city / country.
- deleted 9y ago
  [deleted]
perfmode 9y ago
I would love to see an implementation of the router component.
MooMooMooney 9y ago
"Lastly, we started combining files, and we found out that combining all of our 200 models into one file decreased the compilation time from 1min35sec, to just 17sec. So we are like, "Hold on, this is interesting, combining everything into one makes it much faster." The reason for this is that, as much as I know, that a compiler does type checking for every single file. So if you spawn 200 processes of Swift compilers, it needs to 200x check all the other files and make sure that you're using the correct types. So combining everything into one makes it much faster. " Good to know
- flamedoge 9y ago
  It's probably time spent in the linker. The advice of using WPO without opt is verrry circumstantial.
  - slavapestov 9y ago
    No, it's valid advice. Non-WMO starts a frontend job for each file, WMO runs one job. The frontend jobs do not share state, so each one parses all the files in the module. In general the parser is very fast, and the type checker tries to only type check declarations in files other than the primary file when absolutely necessary, so it's not always O(n^2). But there are pathological cases you can construct today where the type checker ends up doing too much work.
    - flamedoge 9y ago
      It just feels hacky to do it the way author suggested. You can just invoke all the files in cmdline, swift *.swift to same effect I think.
      - slavapestov 9y ago
        `swiftc * .swift` spawns one frontend job per file and then runs the linker to link together the .o's. `swiftc -whole-module-optimization * .swift` compiles all files in a single frontend job. Note that -O is independent of -whole-module-optimization, which is perhaps a bit confusing.
asimpletune 9y ago
Awesome post!
santaclaus 9y ago
Why didn't they use react native?
- eddieroger 9y ago
  Why should they have? Native is a great, if not arguably, the best option.
- rezashirazian 9y ago
  Fast forward to 9:53
  - twelve40 9y ago
    ...and there he talks about IDE crashes? didn't catch any mention of react.
    - proyb2 9y ago
      There is no mentioned of RN at all, it is because Swift works well for Uber. UberEats uses RN because of multiplatform support with their partners. Uber Everything is hiring iOS developer for logistic service? I used to work in German logistic which uses Java technology and does having lot of issues everyday. Swift would prove logistic industry in a new way and probably use Grand Central Dispatch (GCD) which is powerful. If Swift will supported Corountines after 4.0, useful for server side swift. Sorry, I find UberEats UI is distracting with animations. It does not feel native to me however I tried to, the font is awkward to read when Sans Francisco font looks much cleaner on native iOS app.
    - rezashirazian 9y ago
      There are no mentions of RN at all, but all the positive points for why they used Swift is probably an implicit argument for using that instead of RN.
- Tanegashima 9y ago
  Because it's a high performance app that thousands of people depend daily for living.
JustSomeNobody 9y ago
I just threw up a little in my mouth.
- htormey 9y ago
  Why the hate for react native? Also, they use react native for uber eats: https://eng.uber.com/ubereats-react-native/ https://eng.uber.com/ubereats-react-native/
  - JustSomeNobody 9y ago
    I don't hate react native per se. I hate that people blindly throw that out there like it in and of itself will solve real engineering problems. It's the naivety of it all that is so annoying.
    - tuxracer 9y ago
      Blindly shooting it down like it wouldn't solve engineering problems is just the other side of the same coin. You haven't really provided any constructive info why it wouldn't be a good fit here.
      - kitsunesoba 9y ago
        Blindly shooting it down probably isn't the right approach, but neither is treating it at as a sort of silver bullet. There are plenty of situations where RN isn't the best solution or just isn't a good fit. In this particular case, a company the size of Uber isn't going to reap the benefits of RN the same way a smaller one would, and RN's downsides will be magnified. Further, to fix the issues it has with RN it'll need its engineers to do native work anyway, making the extra abstraction layer something of a distraction that could've been avoided.
      - JustSomeNobody 9y ago
        How about the linked article at the top? Don't you think if React Native would have solved all of those engineering issues they would have said, "I guess we use React Native!"? The comment I replied to read, "Why didn't they use react native?" As if this would have solved EVERYTHING.
Entangled 9y ago
> Android engineers are more welcome now. Especially if they write Kotlin. I love Swift, and I love Kotlin. To any young programmer out there (I'm in my fifties), learn these two languages and you will be highly employable for the next decade, on top of the wave.
- jacquesm 9y ago
  A decade is a pretty short time for an investment of that magnitude. Learn C (yes, I know), COBOL, Java, Erlang, Python or any other non-niche language and you'll hopefully be employable for decades, or at least until general AI rolls around.
  - WhitneyLand 9y ago
    The languages actually aren't really the big investment. It's really two things: 1) Learning new concepts in computer science. For example Swift uses a lot of modern concepts around type safety, closures, parallelism, synchronization, etc. These are a big learning curve but they are not specific to Swift and you'll notice other modern languages are adopting the same concepts. 2) The UI frameworks are fundamentally different. If you're an option expert at XCode constraints, storyboards, etc, none of the applies to android and vice versa. I can tell you this, if you do bite the bullet and learn both very well you'll be surprised how much it helps you learn other systems more quickly because you've see all the concepts before.
    - jacquesm 9y ago
      It's never just the language. The libraries are where the time goes and a language without libraries is fairly useless these days.
      - Tanegashima 9y ago
        This, and package managers. For example, node.js, many people say its popular because lots or webdevs knew JS already, but that's not even close to true, most people went to node.js because they found a couple of usable libraries on npm that gave the trust necessary to start the project.
unpopular11333 9y ago
I was hoping to find something about how the Swift architecture affected their ability to cleanly implement a tipping feature, since they've been a tad behind the curve on this very highly demanded functionality (just pinging my general group).
- Humdeee 9y ago
  The functionality of that feature is completely subjective and I'm sure top level management has had many, many meetings with that very topic as the headliner since the first weeks of development. They're very likely ahead of the curve.
- valuearb 9y ago
  Pretty sure a tipping feature would be the single worst possible feature they could ever add.
beaconstudios 9y ago
hm, so they rewrote the whole platform from scratch in <totally hip language of the month> and it didn't all crash and burn? That's kind of surprising - this is usually a really stupid idea because you often end up mostly solving the problems of the v1 architecture but introducing a whole bunch of different, equally painful problems - but with the added headache of the whole codebase being newish.
- melling 9y ago
  Problems aside, Swift is the future of the Apple platform. It's a much less verbose language than Objective C. (no headers, type inference, no @, no ;) https://h4labs.wordpress.com/2016/02/09/should-i-use-objective-c-or-swift-for-writing-ios-apps/ https://h4labs.wordpress.com/2016/02/09/should-i-use-objecti...
  - beaconstudios 9y ago
    my point is that a full switch-over is usually a terrible idea. I don't know if an incremental change-over from ObjC to Swift is possible in the Apple ecosystem, but if it is that's almost always the better option in my experience.
    - valuearb 9y ago
      Swift is so far superior to Objective C that I switched permanently with version 1.2 three years ago. No more dangling pointers is huge, and worth the toolchain issues.
  - anonred 9y ago
    It may be the future, and I love the syntax, but I've run into many of the same problems mentioned in the article (long compile times, SourceKit issues, increase in binary size, etc.) with my own smaller apps. It's honestly frustrating, sometimes more so than the eyesore that is ObjC.
    - melling 9y ago
jordansmithnz 9y ago
In regards to the tool mentioned that provides information about binary size contribution... ("If you want to see this open-source. Just scream out loud") I am screaming out loud. Please open source this!
deleted 9y ago
[deleted]
dankohn1 9y ago
I feel very appreciative to have Uber working through all these bugs in the tooling so that the rest of us can take advantage once things are more reliable.
fred_is_fred 9y ago
It's really irritating that Apple picked the name of an existing software product for their language. When I saw "<large Company> swift architecture" I was pretty excited to see how they were using object storage.
- ktzar 9y ago
  and iOS had been the name of Cisco hardware OS for ages... They don't really care.
mgoblu3 9y ago
Does anyone have any other good articles/examples of maintaining large iOS applications like this? I found this super helpful and interesting to my current project so was also interested in other cases like this
- rudros 9y ago
  https://www.objc.io/issues/22-scale/ https://www.objc.io/issues/22-scale/ has some nice articles.
hota_mazi 9y ago
> The reason for this is that, as much as I know, that a compiler does type checking for every single file. So if you spawn 200 processes of Swift compilers, it needs to 200x check all the other files and make sure that you're using the correct types. I'm a bit baffled by that. Is the Swift compiler that naive? Surely you know how to assess the number of processors/cores on your system and you spawn threads in a way that doesn't lead to diminishing returns. You use a bounded thread pool and you stay within these bounds. Seeing such a speed up from simply merging text files is really puzzling to me. You have to type check the code anyway, surely the overhead of opening a new file is completely negligible compared to running this type checker on the same file? Especially since all these instances of the type checkers have to share a lot of data anyway. Swift is very cool and I'm excited to see it and Kotlin become our next generation languages for mobile, front end and back end alike, but it seems to me the Swift compiler is still very immature.
- edgyswingset 9y ago
  The Swift compiler has some serious issues with the way it does type inference. Try compiling code with a line of basic arithmetic on a few numeric literals.
  - eridius 9y ago
    That has literally nothing to do with what the parent comment is talking about. And that's not naivety either. That's because the way type inference, operating overloading, and literals work together ends up being a combinatorial explosion.
    - bendecoste 9y ago
      Kotlin has all of these features and build times don't (noticeably) suffer.
      - arcticbull 9y ago
        As does Rust.
nnain 9y ago
It's so great that they shared all these findings: I hope the XCode team takes notice and fixes some of the nagging compile time and indexing issues!
- bsaul 9y ago
  Unfortunately the xcode team knows that for a long time, since people inside appl have to use that tool themselves. The reason they don't solve the issues crippling the tool is probably like everywhere else : the codebase has become unmanageable and nobody wants to budget a rewrite.
andymatuschak 9y ago
One other big contributor to app size we've noticed at Khan Academy: extensive use of value types, particularly ones whose fields require more than a couple words of storage. The big picture observation is that for value types with storage larger than a few words, several instructions must be emitted per call and per storage word because they cannot be passed-by-value in registers. And Swift often emits more calls than you can see (e.g. thunks, protocol witnesses, weak sentinels, etc). This is not new to Swift—the same requirement exists when passing large C values around—but we use value types a lot more in Swift for various reasons, so the issue becomes more salient. In terms of remediation, I audited our app for all structs larger than a few words. I just did this manually; there were something like 120 structs to look at. For each, I converted it to a class then evaluated the impact on generated code size. Only four structs had meaningful impact on generated code size (to the tune of ~13MB), and happily, they were fully immutable, so they retained their value semantics even when converted to classes. If they had not already been fully immutable, I would have had to spend some time either adapting the classes to achieve value semantics, or adapting their clients to tolerate reference semantics. Then I audited our app for all enums larger than a few words. These can be made pass-by-reference by using Swift’s `indirect` feature, which implicitly boxes associated storage. We had one enum for which this made a substantial difference, to the tune of several MB. Then I had to make sure runtime performance hadn’t been too badly damaged by all the new dynamic allocations and indirections. In the end, I observed nothing noticeable. We don’t have formal repeatable performance tests, though—it would have been interesting to see the impact on those. C++ has this issue, too; it largely handles it by using reference arguments when consuming large stack values. In the future, it’s possible for Swift to optimize many cases (especially intramodule) where this occurs by allowing deeper stack frames to reference values types stored in parent stack frames when it can prove it’s safe. Rust has a lot of fanciness here you might find interesting!
- slavapestov 9y ago
  There is some work ongoing to improve this situation in Swift 4.0. Take/copy/destroy operations for enums are now outlined and shared between call sites, and a further improvement is in the works to pass large structs indirectly even when they are otherwise loadable (no address-only fields). Another recent improvement is copy-on-write existentials. There's no code size win here, but it improves runtime performance by avoiding copying the payload when passing existentials by value.
bsaul 9y ago
Funny he didn't mention a command line tool that gives compile time per function ( explained here http://irace.me/swift-profiling http://irace.me/swift-profiling ). That proved to be the greatest help in my case to reduce compile times drastically. He did mention running a tool to add explicit types everywhere, but it's very often a matter of just writing the most generic ones. Maybe not in uber case, but for everybody else you should try it.
- dep_b 9y ago
  Especially when having chained calls like .filter .flatMap .reduce etcetera having an explicitly declared types will take a lot of burden off the compiler. Apart from that dictionaries and arrays should be typed. Also some overloads are really hard to process, especially the '+' operators are pretty expensive when used on non-integers and not (previously) strictly typed values. Never do something like "This is a " + birdName + " on it's nest" but use the "This is a \(birdName) on it's nest" method instead. But sadly it also applies to libraries like Cartography that rely on such overloads to create more elegant declarations.
discreteevent 9y ago
"First, you have to be aware that structs can increase your binary size. If you have structs into lists they are created on the stack and they can increase your binary size." I don't get this. Is it saying that structs can increase your binary size and as a separate issue they are created on the stack. Or is it saying that because structs are created on the stack they can increase your binary size? How would that work if stack allocation is something that happens at runtime and affects your memory footprint rather than binary size? (I don't use swift so I might be missing something here)
- _pmf_ 9y ago
  In C, static structs that are not zero-initialized or uninitialized (i.e. that are statically to something else than zero) increase the binary size (they go into the data segment, i.e. even if you only initialize one field of a struct, the binary contains a full image of the full struct); this has nothing to do with the stack in C, but I don't know about Swift.
jeznf 9y ago
The Uber app has become slow and unreliable. I wish they won't kill with with features. It would be sad to see another Facebook App-like fiasco.

305 points by tsycho 9y ago

I'm more interested in Waymo v Uber at the moment. kthxbai!

It seems the Uber app was extremely over-engineered. Call me crazy but I don't think you need 100 engineers to recreate the front-end of Uber.

ReligiousFlames 9y ago
Hire lots of people... complexity, arcaneness, process and knowledge-hoarding becomes vital to job-security.
- tmh79 9y ago
  also when you need to develop a new feature and it needs to go through A:B testing frameworks, localization (including Arabic Right to Left text/layouts), phased feature roll outs, 99.999% reliability, integration testing, planning for use cases anywhere from LTE/WI-FI in USA to EDGE network in developing countries, and you need to build the tools that do those things too. Clearly the 100 engineers are sitting there twiddling their thumbs.
  - tyingq 9y ago
    They have a few other apps too, right? Driver app, Uber Eats, Uber Freight, etc.
  - ReligiousFlames 9y ago
    I think you might of missed the point... everyone has regulations and current problems at any scale, but the ability of BS job roles to hide in a large workforce and for managers to build pyramidal layers of organizations of little added value (but lots of expensive management layers with ever loftier titles).
    - venture_lol 9y ago
      I think you have been to a few rodeos :) Yeah, when a team goes from 2 to 100, a serious business/engineering manager got to ask a few questions. However, if money honey flow is not a problem, it could be in everyone's interest just to ride along
htormey 9y ago
Well your not taking into account all the A/B tests they are running, the fact that they have custom experiences for different locations (SFO, India, etc). I can easily see how they would get up to that number pretty quickly at the scale they are at.
- venture_lol 9y ago
  At some point, consider the use of different apps altogether. Perhaps having a front loader to detect which local app should pop up based on information gleaned from the phone and GPS.
  - htormey 9y ago
    What would be the criteria for splitting something out? They do this with uber eats but I can't imagine them doing this for anything else. Why would I as a user want to download a variety of separate apps to hail cars, etc in different locations? People got really pissed off when Messenger was split out of Facebook (it's still mentioned in app store reviews to this day) I'd imagine they would not be wild about Uber doing something similar. I could see this getting really annoying really quickly. Imagine landing at a new airport and having to download a new app just to hail a car or a tuk tuk or whatever.
    - venture_lol 9y ago
      From a user perspective, you may not have to download a new app. Different locale, different customs, regulations may dictate entire different screens. How an "app" is designed and packaged internally to enable the business to respond quickly does not have to impact the use experience. There are plenty of design patterns and engineering experience to draw from. It's a somewhat boring topic. I find it more interesting to find answers to question such as: 1. A native person in India will be presented with an "Indian" app, satisfying the locale, regulations, laws, culture, etc. 2. A tourist landing in India, well, er, what version should that person use? Laws, regulations still apply (well, Uber may have a different take :) ) but how about the user experience and colloquial details? Would the tourist prefer something from home or something that more accurately, and perhaps more apt for India?
jjtheblunt 9y ago
He mentioned in the very beginning that there are city teams, so perhaps it's a number of people proportional (not exponential, like he says) to the growing # of cities.
panic 9y ago
You're coming at this backwards. They had 100 engineers already. They needed to design their software so that the engineers could all keep working on it. They explicitly say as much: > This application has served us very well for the past four years. But as we've expanded and exponentially grown our mobile engineering team, we started seeing breakages of the architecture, we started seeing problems where feature development would become pretty hard. We had to test multiple code-paths because we shared a lot of view controllers amongst different teams. We really started to hurt with the old architecture because it was written by two engineers and we had grown the team to over one hundred, at that point. The actual quality of the software isn't as important, since Uber drivers and riders don't have a choice anyway.
- perfmode 9y ago
  Uber riders have more than one ride-sharing app on their phone. If Uber can't match me to a driver, I'm still going to get where I want to go.
  - deleted 9y ago
    [deleted]
yawaworht12 9y ago
Any software from a big company like Uber, Facebook, Google, etc. is far more complex than what you see. There are typically dozens of features and experiments rolled out to some users and not others. There is also a lot of work necessary to serve so many different markets, such as localization and internationalization. Furthermore, in Uber's case they have a lot more iOS applications than the ones riders use. In terms of complexity, Uber iOS apps are probably comparable to Facebook's apps.
battwell 9y ago
It's definitely a lot of engineers. But we cram a lot of features into the app. I expect that most people will only experience 10% of them. Consider: 1) We exist in a tonne of countries. And different counties often require different product optimizations and payment methods 2) We have our own map provider 3) We experiment with everything 4) We support countries that have terrible networking. That creates lots of challenges.
- venture_lol 9y ago
  From an engineering perspective, that sounds uninteresting :) Terrible networking? how many engineers does it take to implement a few flavors of retries? :) Uber growth in term of getting drivers, that's what I want to know. It's not just ride subsidies, but the flow of investor honey helps, of course, but they do have some alpha growth managers.
  - uberengineer 9y ago
    Oh, it goes well beyond retries. You have issues like unpredictable latencies and lost network connectivity. A feature that works great when RTT is 400ms may require an entirely different approach when RTT is 30000ms. There are also issues with clocks due to both fraud and misconfigured cell towers. India for example has notoriously bad RTT times and Jakarta and several cities in LatAm have issues with clocks. At scale, across so many different markets, with so many different features, things get complex fast. We want the user experience for both riders and drivers to be magical. More magic requires more engineering. The goal is transportation as reliable and available as running water everywhere all the time. Here's an example of telematics engineering used to improve safety by detecting harsh braking: https://eng.uber.com/telematics/ https://eng.uber.com/telematics/ We also use the IMU measurements to detect drivers who aren't using a phone mount and are instead holding their iPhones while driving. Features like these allow us to advise drivers on how to be safer and get better ratings since not using a phone mount and harsh braking both correlate with negative user ratings. A lot more goes into the Uber app to make things magical. I used to read the front page of HN daily to read about all the cool things people around are working on. Since joining Uber, I check far less often now because there are so many cool problems being worked on internally that many of the front page stories sometimes seem quaint in comparison (not that many HNers aren't working on awesome things but the quantity and quality of cool problems my colleagues work on easily captures most of my attention these days). The best way I can describe working at Uber: it's like building that transportation component of sim-city for every city, everywhere, but it's not a simulation. Despite all the click bait negative press, its bar none the best place I've ever worked.
unoti 9y ago
> Call me crazy but I don't think you need 100 engineers to recreate the front-end of Uber. Your sentiment is valid, but consider this: whenever you're working with a project with a ton of money on the table, it becomes possible to add small bits of functionality which more than pay for themselves in terms of return on investment for the engineering effort. This is why you can often write a basic version of a more established application in "a weekend". We've had extensive discussion and meditation on this topic on HN before[1]. Basically apps will grow and add features as long as the engineering effort will pay for itself (I suppose I use that term loosely in this case). Also consider that the UX is different in different markets, and the reason for that is related to the first issue. I sympathize with what you're saying, though, and couldn't help but think about Alan Kay's talk that we discussed the other day, about natural complexity vs artificial complication.[2] [1] https://news.ycombinator.com/item?id=12626314 https://news.ycombinator.com/item?id=12626314 [2] https://news.ycombinator.com/item?id=14188759 https://news.ycombinator.com/item?id=14188759
sudhirj 9y ago
It's basically tens / hundreds of apps rolled into one, with frontends loaded based on your currently stated or GPS location. They all share a common foundation, but features vary wildly depending on city / country.
deleted 9y ago
[deleted]

perfmode 9y ago

I would love to see an implementation of the router component.

MooMooMooney 9y ago

"Lastly, we started combining files, and we found out that combining all of our 200 models into one file decreased the compilation time from 1min35sec, to just 17sec. So we are like, "Hold on, this is interesting, combining everything into one makes it much faster." The reason for this is that, as much as I know, that a compiler does type checking for every single file. So if you spawn 200 processes of Swift compilers, it needs to 200x check all the other files and make sure that you're using the correct types. So combining everything into one makes it much faster. " Good to know

flamedoge 9y ago
It's probably time spent in the linker. The advice of using WPO without opt is verrry circumstantial.
- slavapestov 9y ago
  No, it's valid advice. Non-WMO starts a frontend job for each file, WMO runs one job. The frontend jobs do not share state, so each one parses all the files in the module. In general the parser is very fast, and the type checker tries to only type check declarations in files other than the primary file when absolutely necessary, so it's not always O(n^2). But there are pathological cases you can construct today where the type checker ends up doing too much work.
  - flamedoge 9y ago
    It just feels hacky to do it the way author suggested. You can just invoke all the files in cmdline, swift *.swift to same effect I think.
    - slavapestov 9y ago
      `swiftc * .swift` spawns one frontend job per file and then runs the linker to link together the .o's. `swiftc -whole-module-optimization * .swift` compiles all files in a single frontend job. Note that -O is independent of -whole-module-optimization, which is perhaps a bit confusing.

asimpletune 9y ago

Awesome post!

santaclaus 9y ago

Why didn't they use react native?

eddieroger 9y ago
Why should they have? Native is a great, if not arguably, the best option.
rezashirazian 9y ago
Fast forward to 9:53
- twelve40 9y ago
  ...and there he talks about IDE crashes? didn't catch any mention of react.
  - proyb2 9y ago
    There is no mentioned of RN at all, it is because Swift works well for Uber. UberEats uses RN because of multiplatform support with their partners. Uber Everything is hiring iOS developer for logistic service? I used to work in German logistic which uses Java technology and does having lot of issues everyday. Swift would prove logistic industry in a new way and probably use Grand Central Dispatch (GCD) which is powerful. If Swift will supported Corountines after 4.0, useful for server side swift. Sorry, I find UberEats UI is distracting with animations. It does not feel native to me however I tried to, the font is awkward to read when Sans Francisco font looks much cleaner on native iOS app.
  - rezashirazian 9y ago
    There are no mentions of RN at all, but all the positive points for why they used Swift is probably an implicit argument for using that instead of RN.
Tanegashima 9y ago
Because it's a high performance app that thousands of people depend daily for living.

JustSomeNobody 9y ago

I just threw up a little in my mouth.

htormey 9y ago
Why the hate for react native? Also, they use react native for uber eats: https://eng.uber.com/ubereats-react-native/ https://eng.uber.com/ubereats-react-native/
- JustSomeNobody 9y ago
  I don't hate react native per se. I hate that people blindly throw that out there like it in and of itself will solve real engineering problems. It's the naivety of it all that is so annoying.
  - tuxracer 9y ago
    Blindly shooting it down like it wouldn't solve engineering problems is just the other side of the same coin. You haven't really provided any constructive info why it wouldn't be a good fit here.
    - kitsunesoba 9y ago
      Blindly shooting it down probably isn't the right approach, but neither is treating it at as a sort of silver bullet. There are plenty of situations where RN isn't the best solution or just isn't a good fit. In this particular case, a company the size of Uber isn't going to reap the benefits of RN the same way a smaller one would, and RN's downsides will be magnified. Further, to fix the issues it has with RN it'll need its engineers to do native work anyway, making the extra abstraction layer something of a distraction that could've been avoided.
    - JustSomeNobody 9y ago
      How about the linked article at the top? Don't you think if React Native would have solved all of those engineering issues they would have said, "I guess we use React Native!"? The comment I replied to read, "Why didn't they use react native?" As if this would have solved EVERYTHING.

Entangled 9y ago

> Android engineers are more welcome now. Especially if they write Kotlin. I love Swift, and I love Kotlin. To any young programmer out there (I'm in my fifties), learn these two languages and you will be highly employable for the next decade, on top of the wave.

jacquesm 9y ago
A decade is a pretty short time for an investment of that magnitude. Learn C (yes, I know), COBOL, Java, Erlang, Python or any other non-niche language and you'll hopefully be employable for decades, or at least until general AI rolls around.
- WhitneyLand 9y ago
  The languages actually aren't really the big investment. It's really two things: 1) Learning new concepts in computer science. For example Swift uses a lot of modern concepts around type safety, closures, parallelism, synchronization, etc. These are a big learning curve but they are not specific to Swift and you'll notice other modern languages are adopting the same concepts. 2) The UI frameworks are fundamentally different. If you're an option expert at XCode constraints, storyboards, etc, none of the applies to android and vice versa. I can tell you this, if you do bite the bullet and learn both very well you'll be surprised how much it helps you learn other systems more quickly because you've see all the concepts before.
  - jacquesm 9y ago
    It's never just the language. The libraries are where the time goes and a language without libraries is fairly useless these days.
    - Tanegashima 9y ago
      This, and package managers. For example, node.js, many people say its popular because lots or webdevs knew JS already, but that's not even close to true, most people went to node.js because they found a couple of usable libraries on npm that gave the trust necessary to start the project.

unpopular11333 9y ago

I was hoping to find something about how the Swift architecture affected their ability to cleanly implement a tipping feature, since they've been a tad behind the curve on this very highly demanded functionality (just pinging my general group).

Humdeee 9y ago
The functionality of that feature is completely subjective and I'm sure top level management has had many, many meetings with that very topic as the headliner since the first weeks of development. They're very likely ahead of the curve.
valuearb 9y ago
Pretty sure a tipping feature would be the single worst possible feature they could ever add.

beaconstudios 9y ago

hm, so they rewrote the whole platform from scratch in <totally hip language of the month> and it didn't all crash and burn? That's kind of surprising - this is usually a really stupid idea because you often end up mostly solving the problems of the v1 architecture but introducing a whole bunch of different, equally painful problems - but with the added headache of the whole codebase being newish.

melling 9y ago
Problems aside, Swift is the future of the Apple platform. It's a much less verbose language than Objective C. (no headers, type inference, no @, no ;) https://h4labs.wordpress.com/2016/02/09/should-i-use-objective-c-or-swift-for-writing-ios-apps/ https://h4labs.wordpress.com/2016/02/09/should-i-use-objecti...
- beaconstudios 9y ago
  my point is that a full switch-over is usually a terrible idea. I don't know if an incremental change-over from ObjC to Swift is possible in the Apple ecosystem, but if it is that's almost always the better option in my experience.
  - valuearb 9y ago
    Swift is so far superior to Objective C that I switched permanently with version 1.2 three years ago. No more dangling pointers is huge, and worth the toolchain issues.
- anonred 9y ago
  It may be the future, and I love the syntax, but I've run into many of the same problems mentioned in the article (long compile times, SourceKit issues, increase in binary size, etc.) with my own smaller apps. It's honestly frustrating, sometimes more so than the eyesore that is ObjC.
  - melling 9y ago

jordansmithnz 9y ago

In regards to the tool mentioned that provides information about binary size contribution... ("If you want to see this open-source. Just scream out loud") I am screaming out loud. Please open source this!

deleted 9y ago

[deleted]

dankohn1 9y ago

I feel very appreciative to have Uber working through all these bugs in the tooling so that the rest of us can take advantage once things are more reliable.

fred_is_fred 9y ago

It's really irritating that Apple picked the name of an existing software product for their language. When I saw "<large Company> swift architecture" I was pretty excited to see how they were using object storage.

ktzar 9y ago
and iOS had been the name of Cisco hardware OS for ages... They don't really care.

mgoblu3 9y ago

Does anyone have any other good articles/examples of maintaining large iOS applications like this? I found this super helpful and interesting to my current project so was also interested in other cases like this

rudros 9y ago
https://www.objc.io/issues/22-scale/ https://www.objc.io/issues/22-scale/ has some nice articles.

hota_mazi 9y ago

> The reason for this is that, as much as I know, that a compiler does type checking for every single file. So if you spawn 200 processes of Swift compilers, it needs to 200x check all the other files and make sure that you're using the correct types. I'm a bit baffled by that. Is the Swift compiler that naive? Surely you know how to assess the number of processors/cores on your system and you spawn threads in a way that doesn't lead to diminishing returns. You use a bounded thread pool and you stay within these bounds. Seeing such a speed up from simply merging text files is really puzzling to me. You have to type check the code anyway, surely the overhead of opening a new file is completely negligible compared to running this type checker on the same file? Especially since all these instances of the type checkers have to share a lot of data anyway. Swift is very cool and I'm excited to see it and Kotlin become our next generation languages for mobile, front end and back end alike, but it seems to me the Swift compiler is still very immature.

edgyswingset 9y ago
The Swift compiler has some serious issues with the way it does type inference. Try compiling code with a line of basic arithmetic on a few numeric literals.
- eridius 9y ago
  That has literally nothing to do with what the parent comment is talking about. And that's not naivety either. That's because the way type inference, operating overloading, and literals work together ends up being a combinatorial explosion.
  - bendecoste 9y ago
    Kotlin has all of these features and build times don't (noticeably) suffer.
    - arcticbull 9y ago
      As does Rust.

nnain 9y ago

It's so great that they shared all these findings: I hope the XCode team takes notice and fixes some of the nagging compile time and indexing issues!

bsaul 9y ago
Unfortunately the xcode team knows that for a long time, since people inside appl have to use that tool themselves. The reason they don't solve the issues crippling the tool is probably like everywhere else : the codebase has become unmanageable and nobody wants to budget a rewrite.

andymatuschak 9y ago

One other big contributor to app size we've noticed at Khan Academy: extensive use of value types, particularly ones whose fields require more than a couple words of storage. The big picture observation is that for value types with storage larger than a few words, several instructions must be emitted per call and per storage word because they cannot be passed-by-value in registers. And Swift often emits more calls than you can see (e.g. thunks, protocol witnesses, weak sentinels, etc). This is not new to Swift—the same requirement exists when passing large C values around—but we use value types a lot more in Swift for various reasons, so the issue becomes more salient. In terms of remediation, I audited our app for all structs larger than a few words. I just did this manually; there were something like 120 structs to look at. For each, I converted it to a class then evaluated the impact on generated code size. Only four structs had meaningful impact on generated code size (to the tune of ~13MB), and happily, they were fully immutable, so they retained their value semantics even when converted to classes. If they had not already been fully immutable, I would have had to spend some time either adapting the classes to achieve value semantics, or adapting their clients to tolerate reference semantics. Then I audited our app for all enums larger than a few words. These can be made pass-by-reference by using Swift’s `indirect` feature, which implicitly boxes associated storage. We had one enum for which this made a substantial difference, to the tune of several MB. Then I had to make sure runtime performance hadn’t been too badly damaged by all the new dynamic allocations and indirections. In the end, I observed nothing noticeable. We don’t have formal repeatable performance tests, though—it would have been interesting to see the impact on those. C++ has this issue, too; it largely handles it by using reference arguments when consuming large stack values. In the future, it’s possible for Swift to optimize many cases (especially intramodule) where this occurs by allowing deeper stack frames to reference values types stored in parent stack frames when it can prove it’s safe. Rust has a lot of fanciness here you might find interesting!

slavapestov 9y ago
There is some work ongoing to improve this situation in Swift 4.0. Take/copy/destroy operations for enums are now outlined and shared between call sites, and a further improvement is in the works to pass large structs indirectly even when they are otherwise loadable (no address-only fields). Another recent improvement is copy-on-write existentials. There's no code size win here, but it improves runtime performance by avoiding copying the payload when passing existentials by value.

bsaul 9y ago

Funny he didn't mention a command line tool that gives compile time per function ( explained here http://irace.me/swift-profiling http://irace.me/swift-profiling ). That proved to be the greatest help in my case to reduce compile times drastically. He did mention running a tool to add explicit types everywhere, but it's very often a matter of just writing the most generic ones. Maybe not in uber case, but for everybody else you should try it.

dep_b 9y ago
Especially when having chained calls like .filter .flatMap .reduce etcetera having an explicitly declared types will take a lot of burden off the compiler. Apart from that dictionaries and arrays should be typed. Also some overloads are really hard to process, especially the '+' operators are pretty expensive when used on non-integers and not (previously) strictly typed values. Never do something like "This is a " + birdName + " on it's nest" but use the "This is a \(birdName) on it's nest" method instead. But sadly it also applies to libraries like Cartography that rely on such overloads to create more elegant declarations.

discreteevent 9y ago

"First, you have to be aware that structs can increase your binary size. If you have structs into lists they are created on the stack and they can increase your binary size." I don't get this. Is it saying that structs can increase your binary size and as a separate issue they are created on the stack. Or is it saying that because structs are created on the stack they can increase your binary size? How would that work if stack allocation is something that happens at runtime and affects your memory footprint rather than binary size? (I don't use swift so I might be missing something here)

_pmf_ 9y ago
In C, static structs that are not zero-initialized or uninitialized (i.e. that are statically to something else than zero) increase the binary size (they go into the data segment, i.e. even if you only initialize one field of a struct, the binary contains a full image of the full struct); this has nothing to do with the stack in C, but I don't know about Swift.

jeznf 9y ago

The Uber app has become slow and unreliable. I wish they won't kill with with features. It would be sad to see another Facebook App-like fiasco.