7 ms·

Uber actually has a much more difficult engineering problems than both WeWork or Airbnb in the sense that they are operating a real time service that requires n

by tempsy 7y ago

Uber actually has a much more difficult engineering problems than both WeWork or Airbnb in the sense that they are operating a real time service that requires near perfect uptime 24/7 and work across multiple timezones in multiple languages, payment methods, and currencies with varying connection speeds. That isn’t easy.

goatinaboat 7y ago
they are operating a real time service that requires near perfect uptime 24/7 and work across multiple timezones in multiple languages, payment methods, and currencies with varying connection speeds. That isn’t easy It isn’t easy, but there are dozens of companies that have been doing it successfully for 40-50 years. Credit cards, airlines, telcos and more. Those challenges are a matter of meticulous application of well established best practices, and none of those are “tech companies”. I mean, Interflora probably has as good or better uptime than Uber!
- tempsy 7y ago
  Do you really believe that Uber is run on a technology stack that is 40-50 years old? It’s not just real time but streaming data. Handling millions of concurrent trips, and processing all those trips in real time is not something that any of the industries you’ve mentioned do.
  - goatinaboat 7y ago
    Do you really believe that Uber is run on a technology stack that is 40-50 years old? Of course not, but I do believe - because it’s demonstrably true - that those challenges are not unique to Uber, and that they have been solved many, many times over. For example sure, Uber handles a lot of trips. But every one of those trips is backed by a credit card processor, without the latter even breaking a sweat. I guarantee that Mastercard or whoever barely even notices the additional workload they get from Uber.
    - adt2bt 7y ago
      You’re correct, the transactional part is true, but MasterCard is following a relatively basic process: 1. Authenticate the transaction (is this ‘uber’ Company ok and is it reasonable this user is using it for fraud detection) 2. Check that the users balance + the cost of the trip < the users credit limit 3. Post the change to a transaction log 4. Some hours/days later, finalize the pending transactions. That’s a whole lot simpler (outside of fraud detection) than what Uber has to do to get you from A to B. *edited for formatting
  - legulere 7y ago
    x86 is 41 years old, which Uber uses for sure. Newer often doesn’t mean better or better enough. Telephone companies have been streaming voice data for millions of people with a reliability that I haven’t seen from any modern internet service.
  - Barrin92 7y ago
    This whole situation is really funny because it looks a little bit like VC funded companies slowly discovering why soviet planning doesn't work. We already had figured out how to move people around in cars in a way that doesn't require software at all, it's called a market and taxis have operated in it for a really long time. Uber is going to figure out shortly why there wasn't a giant taxi monopoly on the planet before they existed.
  - geofft 7y ago
    The question is not whether it is, it's whether it could be. Here's an example: the stock exchanges also need to handle huge amounts of concurrent and real-time data (with much greater constraints on latency and correctness and linearizability than Uber's customers demand). They're not running on literally unchanged 50-year-old tech any more, but they're also not doing the sort of technology innovation that Uber's doing, and they're doing just fine.
- mping 7y ago
  All of those probably have huge tech. Imho engineering wise, Uber really needs tech. If you read uber tech posts, you can see the challenges they face. Even if you want to apply so called best practices you need an above average tech team at the very least. The proof is in the failure rate of our industry.
  - goatinaboat 7y ago
    you want to apply so called best practices you need an above average tech team Actually, the opposite is true. To apply best practices you need people with attention to detail who can follow a runbook precisely. You need relatively few to keep the runbooks up to date. Traditionally the former were referred to as “operators”.
    - NeverFade 7y ago
      Best practices are guidelines, not exact and fully-specified recipes that an ant could follow. If only implementing complex tech solutions was so easy.
    - marcinzm 7y ago
      Rulebooks only work perfectly when everything is static. Even then they only work when you also know all failure cases perfectly which you almost surely don't. Uber is in a competitive market and can't freeze features for years at a time. Furthermore, it in a growing market which means scale alone will cause new failure points to come up.
  - geofft 7y ago
    > If you read uber tech posts, you can see the challenges they face. Even if you want to apply so called best practices you need an above average tech team at the very least. The proof is in the failure rate of our industry. This is one possible interpretation. The other possible interpretation (which I favor) is that good engineers like working on hard problems, and will gravitate towards places where they can work on hard problems—whether or not the business actually fundamentally requires hard problems. There could be simpler architectures. (Remember that pre-Uber, nobody really said that the problem with the taxi industry was that you had to call a dispatcher on the phone and that system couldn't scale. Dispatchers were bad at things, sure, but we didn't think that phones were fundamentally too low-bandwidth to handle the request rate.) Some recent posts on Uber's tech blog: - A zero-downtime migration from a payments architecture with eight microservices that needed a code update every time they launched something like Uber Eats, which was "an unsustainable use of development time," to one that was generic across new businesses Uber might launch in the future. - A tool to automatically generate an iOS app with the same build structure as their current app or a slightly modified one, to see if certain approaches would speed up their build times. - A new web interface, with feature parity to their mobile interface, based on the JavaScript framework previously developed by Uber Engineering and also using fancy new web platform features like service workers. - A new framework for automatically submitting data analysis jobs to their scalable computing environment with the ability to update configs to match changes to which databases exist, which computing clusters exist, how to access them, etc. - A conversational AI platform. - An AI ant that walks around in better ways. None of these, I think, fall into the category of Uber really needing tech. They're all cool, they all have long-term business value, but you could just declare that you're tech-skeptical and don't want to make platforms for platforms for platforms and still deliver Uber's core business - matching users with cars and moving money around - just fine.
- taurath 7y ago
  Poorly, and at great cost. They’re also not developing any new features, and if they do they have a product cycle measured in years if not decades. To say it’s easy to build consistently running real time services in a rapidly changing regulatory and business environment is a bit dismissive I’d say.
  - nindalf 7y ago
    Agree. It’s easy to have 5 9s of uptime when you’re not shipping any new features. In fact that’s the response that some companies have when their uptime SLAs are in danger of being violated - fewer features are shipped.
    - thaumasiotes 7y ago
      > It’s easy to have 5 9s of uptime when you’re not shipping any new features. 5 9s of uptime is five minutes and fifteen seconds of downtime per year. That's difficult to hit reliably no matter what, because pretty much anything that causes any amount of downtime can easily cause more than five minutes of it.
- walshemj 7y ago
  Those companies have large tech workforces to service those requirements
fnord77 7y ago
uber pool is an interesting optimization problem. an eng there was telling me about the graph theory she used to tackle it.
thaumasiotes 7y ago
> they are operating a real time service that requires near perfect uptime 24/7 and work across multiple timezones in multiple languages, payment methods, and currencies Uber is present around the world, but I don't see much reason Uber India needs to interact with Uber North America in real time. Operationally, they're completely independent -- no one is going to connect a ride in Canada to a ride in India -- so why do they need to be strongly coupled in terms of the software service? What would go wrong if Uber India only handled one timezone and didn't really support French?
- danpalmer 7y ago
  An Uber competitor, Hailo, did this. Rather than adapting their services to support multiple languages, time zones, etc, they instead just ran a fully separate instance of the service for each region. Their costs, downtime, and tech debt spiralled out of control, mostly due to this architecture decision, and then spent a year or more rebuilding to try to get it under control. Uber probably have the better approach, at least I don’t hear bad things about Uber’s tech where I have heard multiple horror stories about Hailo’s.
  - arusahni 7y ago
    Do you have a source for this? Not questioning the assertion - just interested in the issues and the design process for the solution.
    - danpalmer 7y ago
      I'm afraid I don't have a single source – this is from discussions about tech teams, culture, engineering, etc, at meet-ups in London over the past ~7 years. They had a relatively large engineering team so there are plenty of ex-Hailo devs around here. There were some good things to come out of it, from what I understand. I believe Hailo were avid Cassandra users and many of the team went to Monzo who have deployed Cassandra very effectively.
- marcinzm 7y ago
  >What would go wrong if Uber India only handled one timezone and didn't really support French? People travel. That aside, you'd have 100+ different deployments that need to be independently updated, debugged, managed, etc. That takes people and effort and is still going to cause issues. For example, each country speaking French might have a different deployment so updating language files needs to take into account every single version running live.

by tempsy 7y ago

goatinaboat 7y ago

they are operating a real time service that requires near perfect uptime 24/7 and work across multiple timezones in multiple languages, payment methods, and currencies with varying connection speeds. That isn’t easy It isn’t easy, but there are dozens of companies that have been doing it successfully for 40-50 years. Credit cards, airlines, telcos and more. Those challenges are a matter of meticulous application of well established best practices, and none of those are “tech companies”. I mean, Interflora probably has as good or better uptime than Uber!

tempsy 7y ago
Do you really believe that Uber is run on a technology stack that is 40-50 years old? It’s not just real time but streaming data. Handling millions of concurrent trips, and processing all those trips in real time is not something that any of the industries you’ve mentioned do.
- goatinaboat 7y ago
  Do you really believe that Uber is run on a technology stack that is 40-50 years old? Of course not, but I do believe - because it’s demonstrably true - that those challenges are not unique to Uber, and that they have been solved many, many times over. For example sure, Uber handles a lot of trips. But every one of those trips is backed by a credit card processor, without the latter even breaking a sweat. I guarantee that Mastercard or whoever barely even notices the additional workload they get from Uber.
  - adt2bt 7y ago
    You’re correct, the transactional part is true, but MasterCard is following a relatively basic process: 1. Authenticate the transaction (is this ‘uber’ Company ok and is it reasonable this user is using it for fraud detection) 2. Check that the users balance + the cost of the trip < the users credit limit 3. Post the change to a transaction log 4. Some hours/days later, finalize the pending transactions. That’s a whole lot simpler (outside of fraud detection) than what Uber has to do to get you from A to B. *edited for formatting
- legulere 7y ago
  x86 is 41 years old, which Uber uses for sure. Newer often doesn’t mean better or better enough. Telephone companies have been streaming voice data for millions of people with a reliability that I haven’t seen from any modern internet service.
- Barrin92 7y ago
  This whole situation is really funny because it looks a little bit like VC funded companies slowly discovering why soviet planning doesn't work. We already had figured out how to move people around in cars in a way that doesn't require software at all, it's called a market and taxis have operated in it for a really long time. Uber is going to figure out shortly why there wasn't a giant taxi monopoly on the planet before they existed.
- geofft 7y ago
  The question is not whether it is, it's whether it could be. Here's an example: the stock exchanges also need to handle huge amounts of concurrent and real-time data (with much greater constraints on latency and correctness and linearizability than Uber's customers demand). They're not running on literally unchanged 50-year-old tech any more, but they're also not doing the sort of technology innovation that Uber's doing, and they're doing just fine.
mping 7y ago
All of those probably have huge tech. Imho engineering wise, Uber really needs tech. If you read uber tech posts, you can see the challenges they face. Even if you want to apply so called best practices you need an above average tech team at the very least. The proof is in the failure rate of our industry.
- goatinaboat 7y ago
  you want to apply so called best practices you need an above average tech team Actually, the opposite is true. To apply best practices you need people with attention to detail who can follow a runbook precisely. You need relatively few to keep the runbooks up to date. Traditionally the former were referred to as “operators”.
  - NeverFade 7y ago
    Best practices are guidelines, not exact and fully-specified recipes that an ant could follow. If only implementing complex tech solutions was so easy.
  - marcinzm 7y ago
    Rulebooks only work perfectly when everything is static. Even then they only work when you also know all failure cases perfectly which you almost surely don't. Uber is in a competitive market and can't freeze features for years at a time. Furthermore, it in a growing market which means scale alone will cause new failure points to come up.
- geofft 7y ago
  > If you read uber tech posts, you can see the challenges they face. Even if you want to apply so called best practices you need an above average tech team at the very least. The proof is in the failure rate of our industry. This is one possible interpretation. The other possible interpretation (which I favor) is that good engineers like working on hard problems, and will gravitate towards places where they can work on hard problems—whether or not the business actually fundamentally requires hard problems. There could be simpler architectures. (Remember that pre-Uber, nobody really said that the problem with the taxi industry was that you had to call a dispatcher on the phone and that system couldn't scale. Dispatchers were bad at things, sure, but we didn't think that phones were fundamentally too low-bandwidth to handle the request rate.) Some recent posts on Uber's tech blog: - A zero-downtime migration from a payments architecture with eight microservices that needed a code update every time they launched something like Uber Eats, which was "an unsustainable use of development time," to one that was generic across new businesses Uber might launch in the future. - A tool to automatically generate an iOS app with the same build structure as their current app or a slightly modified one, to see if certain approaches would speed up their build times. - A new web interface, with feature parity to their mobile interface, based on the JavaScript framework previously developed by Uber Engineering and also using fancy new web platform features like service workers. - A new framework for automatically submitting data analysis jobs to their scalable computing environment with the ability to update configs to match changes to which databases exist, which computing clusters exist, how to access them, etc. - A conversational AI platform. - An AI ant that walks around in better ways. None of these, I think, fall into the category of Uber really needing tech. They're all cool, they all have long-term business value, but you could just declare that you're tech-skeptical and don't want to make platforms for platforms for platforms and still deliver Uber's core business - matching users with cars and moving money around - just fine.
taurath 7y ago
Poorly, and at great cost. They’re also not developing any new features, and if they do they have a product cycle measured in years if not decades. To say it’s easy to build consistently running real time services in a rapidly changing regulatory and business environment is a bit dismissive I’d say.
- nindalf 7y ago
  Agree. It’s easy to have 5 9s of uptime when you’re not shipping any new features. In fact that’s the response that some companies have when their uptime SLAs are in danger of being violated - fewer features are shipped.
  - thaumasiotes 7y ago
    > It’s easy to have 5 9s of uptime when you’re not shipping any new features. 5 9s of uptime is five minutes and fifteen seconds of downtime per year. That's difficult to hit reliably no matter what, because pretty much anything that causes any amount of downtime can easily cause more than five minutes of it.
walshemj 7y ago
Those companies have large tech workforces to service those requirements

fnord77 7y ago

uber pool is an interesting optimization problem. an eng there was telling me about the graph theory she used to tackle it.

thaumasiotes 7y ago

> they are operating a real time service that requires near perfect uptime 24/7 and work across multiple timezones in multiple languages, payment methods, and currencies Uber is present around the world, but I don't see much reason Uber India needs to interact with Uber North America in real time. Operationally, they're completely independent -- no one is going to connect a ride in Canada to a ride in India -- so why do they need to be strongly coupled in terms of the software service? What would go wrong if Uber India only handled one timezone and didn't really support French?

danpalmer 7y ago
An Uber competitor, Hailo, did this. Rather than adapting their services to support multiple languages, time zones, etc, they instead just ran a fully separate instance of the service for each region. Their costs, downtime, and tech debt spiralled out of control, mostly due to this architecture decision, and then spent a year or more rebuilding to try to get it under control. Uber probably have the better approach, at least I don’t hear bad things about Uber’s tech where I have heard multiple horror stories about Hailo’s.
- arusahni 7y ago
  Do you have a source for this? Not questioning the assertion - just interested in the issues and the design process for the solution.
  - danpalmer 7y ago
    I'm afraid I don't have a single source – this is from discussions about tech teams, culture, engineering, etc, at meet-ups in London over the past ~7 years. They had a relatively large engineering team so there are plenty of ex-Hailo devs around here. There were some good things to come out of it, from what I understand. I believe Hailo were avid Cassandra users and many of the team went to Monzo who have deployed Cassandra very effectively.
marcinzm 7y ago
>What would go wrong if Uber India only handled one timezone and didn't really support French? People travel. That aside, you'd have 100+ different deployments that need to be independently updated, debugged, managed, etc. That takes people and effort and is still going to cause issues. For example, each country speaking French might have a different deployment so updating language files needs to take into account every single version running live.