7 ms·

I'd love to know how many people are responsible for devops/operations/app at various stages of any company's journey. Wikipedia says Uber employs 6,500 people

• dkarapetyan 10y ago
  That's all bloat. Pure and simple. At the end of the day Uber just does routing and basic allocation. It's a simple operations problem that has been solved since the 70s and no one back then needed ELK, Docker, Cassandra, etc. I've seen this bloat everywhere. It is usually a result of internal politics and posturing by management types. The kinds of people Steve Jobs would have called B and C players. Now the actual people operations is another matter entirely but the tech stack definitely doesn't need to be that complicated.
  • Animats 10y ago
    I have to admit, I could see this running for a city the size of SF on a desktop machine under the table at the taxi depot. Uber has 11,000 drivers in SF, but probably only a few thousand are on at any one time. A ride takes a few minutes, so if you figure 3,000 active drivers and 4 rides per hour, that's only about 3 ride transactions per second. You have a transaction at ordering, one at ride start, and one at ride end. Plus you have tracking of where all the active drivers are, pinging maybe once a minute. That adds up to only at 10-20TPS. You can offload the routine web and app stuff to some front end machines. And you want to do some analysis every minute or two to see where there are "surges". The only non-trivial part of this is assigning drivers to rides.
    • dkarapetyan 10y ago
      Indeed. Most SaaS operations are over-provisioned both in terms of hardware and software.
      • goldenkey 10y ago
        And over-provisioned in terms of employees, I'd add.
    • mkagenius 10y ago
      > tracking of where all the active drivers are, pinging maybe once a minute. Once every 5 seconds will be more accurate.
      • ambicapter 10y ago
        And will 30x the number of requests...
      • dzhiurgis 10y ago
        Driver app pings central server every 15 seconds. If any ride requests are available, it displays it to driver.
    • ricardobeat 10y ago
      It's easy to imagine the simplest stack that can serve the core features of any service, and that is well served by a single box. What's missing from the picture is the infrastructure to replicate this 500 times by separate teams, monitoring all of it, backup, auditing, aggregating customer and business metrics, back-office systems, and more. Plus the fact that these things always grow organically and embed a host of imperfect decisions - the imaginary system will always be better designed.
      • tlrobinson 10y ago
        This. Armchair software architecture is so easy when you can gloss over the details that make a product great. Also routing, ETAs, geocoding/search, etc. WebGL visualizations and such are probably overkill, but if it makes the company more fun to work for then it probably breaks even, at worst.
  • asdfaoeu 10y ago
    > At the end of the day Uber just does routing and basic allocation. The thing is though that that algorithm is easily copied as evidenced by lyft and etc. So really Uber's business model needs to be all about differentiation, marketing, analytics and prediction otherwise they won't survive. I think analytics and other fuzzy avenues are where those technologies shine.
    • gaius 10y ago
      That's their business model? Really? I think we both know that their business is nothing to do with the technology they use.
  • kccqzy 10y ago
    You certainly have an interesting argument but it doesn't sound convincing. Can you explain more about how 1970s technology can solve the problems Uber is facing in 2016?
    • 1128128112 10y ago
      https://blogs.harvard.edu/philg/2009/05/18/ruby-on-rails-and-the-importance-of-being-stupid/ https://blogs.harvard.edu/philg/2009/05/18/ruby-on-rails-and...
    • dkarapetyan 10y ago
      Here you go https://en.wikipedia.org/wiki/Operations_research https://en.wikipedia.org/wiki/Operations_research. Go to town. The history section and the problems addressed section is more than sufficient. You can expand further if necessary and decide for yourself whether what Uber does is in any way novel and whether it requires all the bloat.
  • rimantas 10y ago
    Exactly. I saw the presentation of one of thei tech guys and was very surprised by it. First, by a number of IT people they have, second, by the work they do. It looked inventibg problems and solving problems for the sake if problems and solutions, i. e. no business value in it.
  • garyclarke27 10y ago
    True, I think their CTO must be confused and easily mislead by techies who just want to get the latest buzz words onto their cvs. Silly comments such as - we gain true insight from pretty graphics rather than tedious sql queries - says it all for me. I suppose they have to burn the insane amount of capital they raised $50 Billion somehow?
    • danpalmer 10y ago
      I recommend reading "The Visual Display of Quantitive Information" by Tufte. I would have partially agreed with you before, but I really do think that correct visualisation of data can make it vastly more useful, and as a few other commenters have noted, Uber has a big challenge to differentiate themselves from Lyft and others, and effective use of data could well be one of their differentiators.
    • icebraining 10y ago
      That's quite condescending; their CTO is no rookie.
  • UK-AL 10y ago
    Docker and Cassandra don't solve basic routing. They solve the fact you have have millions of people hitting you site, and many servers to manage. You didn't have that problem in the 1970s
    • gaius 10y ago
      Credit card processors and airlines did millions of transactions a day in the 1970s, and they managed it with CICS.
      • uberonhn 10y ago
        It sounds like you have no idea how credit card transactions were or are processed. They are almost exclusively file and batched once a day, even today. Back in the 1970s it was even worse because there was no real time authorization.
        • dang 10y ago
          This is helpful information if accurate, but please edit incivility like "It sounds like you have no idea" out of your comments here. The site guidelines ask you to omit this sort of thing, so please post civil and substantive comments only: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newswelcome.html
          • gaius 10y ago
            He or she is correct that the cash settlements were and are batched, but authorizations, reservations etc were online - to the mainframe that's just another transaction. The reason settlements were batched was to save on wire fees.
  • gaius 10y ago
    Like bureaucracy, the complexity of your tech stack grows to accommodate the number of people available to work on it. Go crazy on the hiring, and what do you expect all those people to do all day?
    • danpalmer 10y ago
      That's a very uncharitable views, and it could be argued the other way around, that the hiring occurs to support the need for more people in engineering. I'm not certain which way around it goes with Uber, but I've seen both.
  • candidates 10y ago
    I disagree. Even on hacker news, people rarely express such absurd things with so little confidence. You fail to take into account many of the following: * Extremely high volume. Uber has indicated elsewhere that they receive upwards of a few hundred thousand requests per second on just one service. Please show me the logistics stack that did this in the 70s. * Yes, building the first version of something is extremely cheap and easy. But being able to improve it becomes harder and harder. Especially given high volume, modern companies need sophisticated analytical tools that provides reliable data to both technical and non technical staff. Please show me the analytics stack that was able to ingest, store, and analyze terabytes of business data in realtime from the 70s. * Reliability. Modern web applications need to fail gracefully and be debugged quickly. Please show me the logistics and routing stack that was capable of extremely high uptime while being deployed constantly and serving hundreds of thousands of requests per second from the 70s. * Extensibility. Businesses need to extend to new markets. Moves like this often invalidate past assumptions. In order to support business flexibility, modern engineers deliberately invests considerable time into building decoupled components that can be reused as platforms instead of stuffed into a monolithic codebase. Please show me the operations and routing stack that could easily be reconfigured to enable such products as Amazon Web Services, Uber's external API, the google maps API, or Uber EATS—from the 70s. To make this more concrete, I worked on a routing stack at another company which probably works similarly to Uber's ETA systems. When considering these things, it's important to keep in mind the dependency tree of each new problem set and the work required to make those dependencies work reliably at big scale. To give you an idea of what this area alone entails: 1. Machine learning. - wiring together of and improving algorithms: linear regression to begin with, then random forest, then neural networks. - ensuring data required for learning is reliably available and correctly computed. - tools to launch, deploy, test these models. 2. Working with map data in memory many times larger than what fits onto the smallest consumer laptop. - how do you handle updates of data? - what if you want to use different data sets in different places, because they're more accurate? - how do you debug errors in the data without visual tools (hint: it's really hard and time consuming)? - how do you optimize loading this data into memory without requiring hours to deploy your application? - where do you even store this data? 3. Requests per second in the hundreds of thousands and latency requirements (in order to ensure the app responds quickly) hovering around 10ms. - how do you profile complex distributed applications? - what optimizations are available to make graph search faster (hint: A* isn't fast enough)? - how hard is it to implement these optimizations? 4. Data science and data science tools - Visualizations! - again, reliable data pipelines That's about what one team works on over the course of a year. Note the dependencies we have here: 1. We assume access to cloud infrastructure that doesn't require us to do all of our own devops. 2. We assume mature and automatically scaling data infrastructure: that kafka and storm have been set up and tuned to a degree that we don't have to worry about it. In reality, kafka alone requires a team of at least a dozen at linkedin to keep up with the maintenance, operations, and optimization burden of keeping up with scale. 3. We assume mature and scalable service oriented architecture tooling—if a call to another service is slow, I should be able to see on a dashboard what service is slow, how frequently, it's slow, why it's slow (if it depends on another service) etc. and countless other things I could spend days enumerating for you but I guess it'd be wasted on you because you're pretty convinced you already solved these problems in the 70s, so why am i wasting my breath
  • pfarnsworth 10y ago
    I love how any armchair quarterback on HN can sit back and dismiss the work of thousands of engineers as bloat, with no actual qualifications of their own. Build a multi-billion dollar company that has satisfied customers around the world, and then let's hear what you have to say. Let me guess, you also came up with the idea for Google Adsense and the iPhone in high school, right?
    • dkarapetyan 10y ago
      The business idea is a separate concern. A solid business model can withstand all sorts of abuse and incompetence at all levels. Many eBay CEOs besides their best efforts to destroy the company have been unable to do so. Similarly for PayPal and a few other companies that have excellent product/market fit. Each of those companies survives despite the best efforts of 1000s of engineers and managers to over-engineer and justify their salaries. So the logic of "1000s of people have worked on this so it must be valuable" is incorrect reasoning. The more pertinent question is how do these companies survive despite all the over-engineering that is happening? Once you ask that question you are almost surely led to the conclusion that the technology is not as relevant as people would like to think and inefficiencies at the technology level have very little effect on actual business outcomes when the product itself provides value people are willing to pay for.
• jlarocco 10y ago
  I don't know the numbers, but at least some of Uber's tech employees are working on things that aren't directly connected to the app and rides, like mapping and self-driving cars. One of their recruiters contacted me a while back, and it sounds like they're working on some really neat stuff, but I don't agree with all their business practices, so I didn't pursue it :-/ In any case, he pointed to their website: https://www.uberatc.com/ https://www.uberatc.com/
• wheresmypwd 10y ago
  Assume it's a team of 3-8 eng/ops/Devops managing each complex tech on the list, then 5-10 more teams of 5-20 engineers managing areas of the product, and interfacing with the infra teams to get things done.