4 ms·

Somewhat off topic but maybe someone can enlighten me: > Uber has grown to around 2,200 critical microservices Even thinking about the very largest systems I’

Somewhat off topic but maybe someone can enlighten me: > Uber has grown to around 2,200 critical microservices Even thinking about the very largest systems I’ve worked on, I can’t think of what could possibly be split into 2000 separate individual services. What are these thousands of microservices and how micro are they?

nevir 6y ago
I'd be curious how that compares to Amazon's count of services (my guess is 10k+)
- beastcoast 6y ago
  I’d say we’re closer to an order of magnitude larger When you consider all the different businesses and domains. My org alone has about 1000 services. But we have massive scale and the employees to support that number of services. You also won’t find one architecture; it runs the gamut from lambda to REST/RPC to pubsub, event bus and workflow architectures. Typically new orgs start out with a few larger services that get broken out over time, as the complexity and number of employees continues to grow.
drol3 6y ago
I am working for a company that has moved from 3 to 15 backend developers and 25 to 50 services over the last year. The amount of services is really quite meaninglessness compared to the amount of developers. As with anything the efficiency good down as the amount of people goes up. There is no way that 2000 people can agree on anything. So most likely there is lots and lots of overlap. You could probably refactor some of this duplication out, but by the time you would be done new duplication would have emerged. Keeping many people in sync is just difficult :) Anyway the point is that 2200 services is meaningless without telling how many employees are working on said services :)
eikenberry 6y ago
Remember that those aren't 2200 separate things going on, but 2200 instances of the services. I'm sure that includes very large clusters running many copies of the same service.
- wikibob 6y ago
  That is an incorrect interpretation. I assure you there are /significantly/ more than 2200 instances of the services running. They do indeed mean 2200 separate deployable services.
Ozzie_osman 6y ago
Not at Uber, but I've heard a service there can be as small as a database table or two with some CRUD. Or a lambda-style function or two that provides some simple logic when queried or aggregates from a couple other services. Not saying that's a good way to build systems but it's definitely one way.
- dkersten 6y ago
  Who has 2000 database tables though? Or 2000 lambda functions for that matter.
  - mekael 6y ago
    Multi-line insurance systems generally start at 2000 tables. Heck, simple auto insurance systems with multiple products in a single large state can hit 5k really easy.
  - jessaustin 6y ago
    Any actual business that's been around a few years and enforces 3NF is going to have close to 500 rows in the DB. Stretch that business to multiple continents and service types and one would expect 2000.
    - jessaustin 6y ago
      Haha of course I should have said "tables" not "rows".
daxfohl 6y ago
For every important service there is the current one, the old one that is still in use, the previous previous version that one dumb team still hasn't migrated from, the next gen prototype, the service that deploys the service, the service that monitors it, the data analysis tool, data migration services, the intern project, several upstream services that continuously test the integration points, blah blah blah. Then you look at the verticals and there's maps, payments, analytics, hosting, marketing, security, identity, partnerships, third party integrations, and a million other things you don't think about. It's pretty easy to get into those numbers if you want zero downtime and code your own verticals, and have the resources to do it.
jeswin 6y ago
I don't find this surprising at all. Uber operates across 5 dozen countries (or more) each with different regulations and offerings. For instance, in India you get autorickshaws (called "Autos" sometimes) in addition to cabs as a seperate option. Autos have different rules, billing, driver compliance and safety standards. There are probably a bunch of services around Autos. Similarly, safety regulatons differ in each country, and in some cases teams are forced to act quickly. Having them in separate microservices again makes sense. Same goes for offers, eats etc. 2000 services is not inconceivable for a company operating across the world.
etxm 6y ago
pico services.
jmchuster 6y ago
Hmm, would it be easier to imagine PayPal or Stripe requiring a lot of services? Given its volume of payments, Uber has basically had to re-implement PayPal internally, custom-tailored to its own flow. Companies at this size/scale basically contain multiple whole companies within them. Think of like all the SAAS services that your company relies on, and then imagine that your company now owns all of them and all the services that come with them.
- clintonb 6y ago
  Uber uses Stripe and PayPal to process payments. They don’t need to reimplement either, and doing so would be a bit of a waste. It’s worth pointing out that not all of the 2200 services may be user-facing. Some may be internal, such as admin tooling or CI services. That said, 2200 seems like a lot!
  - jmchuster 6y ago
    Sorry, I think you are a bit mistaken about how payment platforms work. Stripe and PayPal provide an interface to a country's banks. In exchange for abstracting away the underlying bank infrastructure, you pay them a fee for every transaction. Stripe and PayPal support a limited number of countries and a subset of the bank's functionality. So if you want to interact with any new countries or access any non-supported financial instruments, you'll have to directly integrate with the banks and implement those yourselves. To give you a sense of scale, Uber's payments volume is like 10% of Stripe's.
    - wpnx 6y ago
      Curious; where did you get the 10% number from
      - jmchuster 6y ago
        Here's an estimate of Stripe's yearly transaction volume at $200 billion https://www.forbes.com/sites/christianowens/2019/09/21/stripe-isnt-overvalued-at-35bn-you-just-dont-understand-twenty-first-century-commerce/ https://www.forbes.com/sites/christianowens/2019/09/21/strip... Here's Uber's announced results of $18 billion in gross bookings https://investor.uber.com/news-events/news/press-release-details/2020/Uber-Announces-Results-for-Fourth-Quarter-and-Full-Year-2019/ https://investor.uber.com/news-events/news/press-release-det... Round that to significant figures and you get 10%.

nevir 6y ago

I'd be curious how that compares to Amazon's count of services (my guess is 10k+)

beastcoast 6y ago
I’d say we’re closer to an order of magnitude larger When you consider all the different businesses and domains. My org alone has about 1000 services. But we have massive scale and the employees to support that number of services. You also won’t find one architecture; it runs the gamut from lambda to REST/RPC to pubsub, event bus and workflow architectures. Typically new orgs start out with a few larger services that get broken out over time, as the complexity and number of employees continues to grow.

drol3 6y ago

I am working for a company that has moved from 3 to 15 backend developers and 25 to 50 services over the last year. The amount of services is really quite meaninglessness compared to the amount of developers. As with anything the efficiency good down as the amount of people goes up. There is no way that 2000 people can agree on anything. So most likely there is lots and lots of overlap. You could probably refactor some of this duplication out, but by the time you would be done new duplication would have emerged. Keeping many people in sync is just difficult :) Anyway the point is that 2200 services is meaningless without telling how many employees are working on said services :)

eikenberry 6y ago

Remember that those aren't 2200 separate things going on, but 2200 instances of the services. I'm sure that includes very large clusters running many copies of the same service.

wikibob 6y ago
That is an incorrect interpretation. I assure you there are /significantly/ more than 2200 instances of the services running. They do indeed mean 2200 separate deployable services.

Ozzie_osman 6y ago

Not at Uber, but I've heard a service there can be as small as a database table or two with some CRUD. Or a lambda-style function or two that provides some simple logic when queried or aggregates from a couple other services. Not saying that's a good way to build systems but it's definitely one way.

dkersten 6y ago
Who has 2000 database tables though? Or 2000 lambda functions for that matter.
- mekael 6y ago
  Multi-line insurance systems generally start at 2000 tables. Heck, simple auto insurance systems with multiple products in a single large state can hit 5k really easy.
- jessaustin 6y ago
  Any actual business that's been around a few years and enforces 3NF is going to have close to 500 rows in the DB. Stretch that business to multiple continents and service types and one would expect 2000.
  - jessaustin 6y ago
    Haha of course I should have said "tables" not "rows".

daxfohl 6y ago

For every important service there is the current one, the old one that is still in use, the previous previous version that one dumb team still hasn't migrated from, the next gen prototype, the service that deploys the service, the service that monitors it, the data analysis tool, data migration services, the intern project, several upstream services that continuously test the integration points, blah blah blah. Then you look at the verticals and there's maps, payments, analytics, hosting, marketing, security, identity, partnerships, third party integrations, and a million other things you don't think about. It's pretty easy to get into those numbers if you want zero downtime and code your own verticals, and have the resources to do it.

jeswin 6y ago

I don't find this surprising at all. Uber operates across 5 dozen countries (or more) each with different regulations and offerings. For instance, in India you get autorickshaws (called "Autos" sometimes) in addition to cabs as a seperate option. Autos have different rules, billing, driver compliance and safety standards. There are probably a bunch of services around Autos. Similarly, safety regulatons differ in each country, and in some cases teams are forced to act quickly. Having them in separate microservices again makes sense. Same goes for offers, eats etc. 2000 services is not inconceivable for a company operating across the world.

etxm 6y ago

pico services.

jmchuster 6y ago

Hmm, would it be easier to imagine PayPal or Stripe requiring a lot of services? Given its volume of payments, Uber has basically had to re-implement PayPal internally, custom-tailored to its own flow. Companies at this size/scale basically contain multiple whole companies within them. Think of like all the SAAS services that your company relies on, and then imagine that your company now owns all of them and all the services that come with them.

clintonb 6y ago
Uber uses Stripe and PayPal to process payments. They don’t need to reimplement either, and doing so would be a bit of a waste. It’s worth pointing out that not all of the 2200 services may be user-facing. Some may be internal, such as admin tooling or CI services. That said, 2200 seems like a lot!
- jmchuster 6y ago
  Sorry, I think you are a bit mistaken about how payment platforms work. Stripe and PayPal provide an interface to a country's banks. In exchange for abstracting away the underlying bank infrastructure, you pay them a fee for every transaction. Stripe and PayPal support a limited number of countries and a subset of the bank's functionality. So if you want to interact with any new countries or access any non-supported financial instruments, you'll have to directly integrate with the banks and implement those yourselves. To give you a sense of scale, Uber's payments volume is like 10% of Stripe's.
  - wpnx 6y ago
    Curious; where did you get the 10% number from
    - jmchuster 6y ago
      Here's an estimate of Stripe's yearly transaction volume at $200 billion https://www.forbes.com/sites/christianowens/2019/09/21/stripe-isnt-overvalued-at-35bn-you-just-dont-understand-twenty-first-century-commerce/ https://www.forbes.com/sites/christianowens/2019/09/21/strip... Here's Uber's announced results of $18 billion in gross bookings https://investor.uber.com/news-events/news/press-release-details/2020/Uber-Announces-Results-for-Fourth-Quarter-and-Full-Year-2019/ https://investor.uber.com/news-events/news/press-release-det... Round that to significant figures and you get 10%.