5 ms·

It amazes me they have 1,700 services. It would be hilarious satire to actually see the description of each. And the debugging scenarios he listed make it sound

It amazes me they have 1,700 services. It would be hilarious satire to actually see the description of each. And the debugging scenarios he listed make it sound like they have very poor engineers working for them. Who on earth lets an application get to prod that iterates through a list of items and makes a request for each thing? When did we loose our heads and think such an architecture is sane? The UNIX philosophy is do one thing and do it well, but that doesn't mean be foolish about the size of one said thing. Doing one thing means solving a problem, and limited the scope of said problem so as to have a cap on cognitive overhead, not having a notch in your "I have a service belt". We don't see the LS command divided into 50 separate commands and git repo's.....

venantius 10y ago
While I'd concede that 1,700 services is a lot, and probably a function of the fact that they're hiring too many engineers too quickly (there's no possible way to indoctrinate people into sane and standardized engineering practices at this sort of hiring growth), I don't actually think what he's describing is that unusual for a company at Uber's scale or experiencing Uber's rate of growth. I work at Airbnb and while we have many fewer engineers and services than Uber does, most of the issues he talked about in the talk resonated with me to one degree or another.
- iamspoilt 10y ago
  I couldn't agree more.
willejs 10y ago
Saying something like '...they have very poor engineers working for them.' is pretty unfounded, and naive. It's easy to say that having 1700 services is overkill from our point of view, but we don't know the complete architecture, problems being solved and environment that they operate in. One thing to possibly consider is that once you have set up tooling for a platform; logging, request tracing, alerting, orchestration, common libraries, standards and deployment. Deploying and operating new services becomes relatively straight forward. Saying that, 1700 is a lot, which makes me intrigued to see inside uber.
- bartread 10y ago
  Yeah, that's completely ridiculous. I'm currently working with a very (very) large UK-based retailer, with various outlets globally, and I can tell you from first hand experience that their selection process is ruthless. There are no, or at least very few, lamers in the tech team. And yet, when you look at the systems, and the way things are built, some of it just seems crazy. Only it's not, and for many reasons, of which here are only a couple: - The business has grown and evolved over time - with significant restructuring occurring either gradually, or more quickly and in a more intentionally managed way; systems have grown and evolved with it, often organically (which is something I see at every company I've worked with). - Legacy systems that make it really hard to change things, and migrate to newer and better ways of doing things because they're still in production and depended upon by a host of other systems and services. - The degree of interconnectedness between systems and services is high across the board; this isn't a reflection of bad design, so much as a reflection of the complexity of the business. In any large organisation these kinds of things are apparent. I would imagine that in a large organisation that has become large very quickly, like Uber, if anything the problems would be magnified. To say that they have poor engineers is therefore very unfair: these are people who would have needed to ship RIGHT NOW on a regular basis in order to facilitate the growth of the business. That's going to lead to technical compromises along the way, and there's little to be done to avoid it. There's also a difference between being aware of a problem and having time, or it being a priority, to fix it (e.g., iterating and retrieving).
- iamleppert 10y ago
  This wasn't a derivation from the repo count, it was in response to the debug part of the talk where he was talking about fan out. Some app was iterating through a list of things and for each thing was making a network request, instead of figuring out what it needed up front. It's either complete laziness or incompetence, probably the later. I assume with any company that has that many services their culture is partly to blame, too. It's interesting because this kind of technical debt will eventually open the door to competitors once Uber is under more financial pressure...i.e. they have burned through all the investor money.
  - 10y ago
deleted 10y ago
[deleted]
stephansta 10y ago
>When did we loose our heads and think such an architecture is sane? It's not sane. That's the entire purpose of the talk.

venantius 10y ago

While I'd concede that 1,700 services is a lot, and probably a function of the fact that they're hiring too many engineers too quickly (there's no possible way to indoctrinate people into sane and standardized engineering practices at this sort of hiring growth), I don't actually think what he's describing is that unusual for a company at Uber's scale or experiencing Uber's rate of growth. I work at Airbnb and while we have many fewer engineers and services than Uber does, most of the issues he talked about in the talk resonated with me to one degree or another.

iamspoilt 10y ago
I couldn't agree more.

willejs 10y ago

Saying something like '...they have very poor engineers working for them.' is pretty unfounded, and naive. It's easy to say that having 1700 services is overkill from our point of view, but we don't know the complete architecture, problems being solved and environment that they operate in. One thing to possibly consider is that once you have set up tooling for a platform; logging, request tracing, alerting, orchestration, common libraries, standards and deployment. Deploying and operating new services becomes relatively straight forward. Saying that, 1700 is a lot, which makes me intrigued to see inside uber.

bartread 10y ago
Yeah, that's completely ridiculous. I'm currently working with a very (very) large UK-based retailer, with various outlets globally, and I can tell you from first hand experience that their selection process is ruthless. There are no, or at least very few, lamers in the tech team. And yet, when you look at the systems, and the way things are built, some of it just seems crazy. Only it's not, and for many reasons, of which here are only a couple: - The business has grown and evolved over time - with significant restructuring occurring either gradually, or more quickly and in a more intentionally managed way; systems have grown and evolved with it, often organically (which is something I see at every company I've worked with). - Legacy systems that make it really hard to change things, and migrate to newer and better ways of doing things because they're still in production and depended upon by a host of other systems and services. - The degree of interconnectedness between systems and services is high across the board; this isn't a reflection of bad design, so much as a reflection of the complexity of the business. In any large organisation these kinds of things are apparent. I would imagine that in a large organisation that has become large very quickly, like Uber, if anything the problems would be magnified. To say that they have poor engineers is therefore very unfair: these are people who would have needed to ship RIGHT NOW on a regular basis in order to facilitate the growth of the business. That's going to lead to technical compromises along the way, and there's little to be done to avoid it. There's also a difference between being aware of a problem and having time, or it being a priority, to fix it (e.g., iterating and retrieving).
iamleppert 10y ago
This wasn't a derivation from the repo count, it was in response to the debug part of the talk where he was talking about fan out. Some app was iterating through a list of things and for each thing was making a network request, instead of figuring out what it needed up front. It's either complete laziness or incompetence, probably the later. I assume with any company that has that many services their culture is partly to blame, too. It's interesting because this kind of technical debt will eventually open the door to competitors once Uber is under more financial pressure...i.e. they have burned through all the investor money.
- 10y ago

deleted 10y ago

[deleted]

stephansta 10y ago

>When did we loose our heads and think such an architecture is sane? It's not sane. That's the entire purpose of the talk.