5 ms·

There's a famous 'rebuttal' post to this here https://medium.com/@buckhx/unwinding-uber-s-most-efficient-service-406413c5871d https://medium.com/@buckhx/unwindi

There's a famous 'rebuttal' post to this here https://medium.com/@buckhx/unwinding-uber-s-most-efficient-service-406413c5871d https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

abgfm 7y ago
Thank you for this :))
- buckhx 7y ago
  np ;)
merb 7y ago
well basically ubers blog posts are really low quality and from the outside their engineering descisions are kinda vague. like the switch from postgres to mysql. I'm still clueless how you can have so much money and one of the biggest engineering team, but still can't correctly engineer your stuff. I mean, everybody makes wrong decisions or errors in production code.
- Slartie 7y ago
  From what I gathered off of several HN comments of Uber insiders, the size of their engineering team is actually more likely to be one of the causes for bad engineering decisions. They seem to have too many engineers running around for too little actual work to be done, which results in those engineers coming up with stuff to do to keep them busy (and of course to ensure they appear to be busy and worth their money to their superiors). One of the best ways to create yourself some work is to not choose an already-proven path to a solution, but invent a new one just for the sake of inventing a new one. Of course that's not how this kind of doing is justified - the justification is usually "the proven path does not scale to our needs" or "by using a special approach adapted to our needs we can be more efficient" or "the proven path is too complex, we can get by with something simpler and easier to maintain". Which might actually all be proper justifications, it's just that you should have some hard proof for these statements, like benchmark results of a comparison of different approaches. That part often gets skipped, which is actually ironic, because doing extensive evaluation and benchmarking and implementing different approaches first before choosing one for production actually serves quite well to create even more work to do.
  - asdfman123 7y ago
    Also, it's resume-driven development. Five years later they want to be able to be the guy who can say, "Yeah, I was the original designer behind Kafka," but substituting in a brand-new Uber technology.
  - triceratops 7y ago
    It might be sub-optimal for Uber but it's better for the industry as a whole to have many different implementations of the same type of solution.
- jeltz 7y ago
  And none of their blog posts about their own geo stuff ever come close to answer the why. What is wrong with e.g. PostGIS or Elasticsearch? Why do they need to reinvent geo databases several times?
  - loriverkutya 7y ago
    Too much time and not enough things to do.
    - drchickensalad 7y ago
      How does this happen?! They're so expensive
      - Kye 7y ago
        Give me twenty billion dollars to make a ride hailing app and I'll show you. More seriously: they raised tens of billions of dollars to make a ride hailing app and research self-driving cars. Throwing more money at a problem doesn't solve it faster, but it does pay for hiring people, and headcount is seen as a proxy for doing stuff. When a measure becomes a target, it ceases to be a good measure. https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law Some big start-ups[0] solve this conundrum by investing in adjacent companies to find the solutions they're paid to find. Outsourcing! This still has limits because there's so much money flowing around right now. Tossing a few million at a company with hundreds still won't solve the problem faster. [0] Start-up definition for this post: a company that has taken funding but hasn't yet found a sustainable business model. That's how Uber is still a start-up with more money than most companies make in decades.
  - uber42163 7y ago
    Teams outside Infra aren't allowed persistent disk. Service authors get to pick from a small catalog of managed operational storage systems. Life would be a lot easier if a scalable Postgres were one of them, but it isn't. For the moment, the offerings with high scalability and good SLAs are all KV stores. We do have ES but it's operated for log search, not production critical paths.
- deleted 7y ago
  [deleted]
GrumpyNl 7y ago
Thats a great write up. They should hire him on the spot
- buckhx 7y ago
  That'd be nice. For real though I did meet with them after, but it was basically the same as getting a referral so the same interview process which I chose to forgo.
jeltz 7y ago
Another rebuttal which is less through due to the benchmark being very synthetic but still points out that Uber's numbers are nothing impressive. https://www.cybertec-postgresql.com/en/beating-uber-with-a-postgresql-prototype/ https://www.cybertec-postgresql.com/en/beating-uber-with-a-p...
- Thaxll 7y ago
  This is a crap article tbh They don't take any requierments and build a 5min poc with no constrains. For instance their demo doesn't update anything, Uber updates location every seconds, I'd like to see how PG behaves when you rebuild index thousand time per seconds.
  - jeltz 7y ago
    No, that was not a requirement. This is not the database for the locations of cars, but the database with human defined geofences which probably only update a couple of times per day. The database is used to check e.g. if a customer is currently at an airport or not. That is a very normal and simple GIS workload.
twic 7y ago
So Uber's algorithm takes 58756 ns/op, and the fastest non-brutal algorithm took 471 ns/op. That's about 100 times faster. They used 40 machines to serve New Year's Eve traffic. Perhaps with a better algorithm, they could have got away with one. (Possibly not, because there's still a lot of HTTP and JSON munging work to be done, and the network card becomes a bottleneck at some point)

There's a famous 'rebuttal' post to this here https://medium.com/@buckhx/unwinding-uber-s-most-efficient-service-406413c5871d https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

abgfm 7y ago

Thank you for this :))

buckhx 7y ago
np ;)

merb 7y ago

well basically ubers blog posts are really low quality and from the outside their engineering descisions are kinda vague. like the switch from postgres to mysql. I'm still clueless how you can have so much money and one of the biggest engineering team, but still can't correctly engineer your stuff. I mean, everybody makes wrong decisions or errors in production code.

Slartie 7y ago
From what I gathered off of several HN comments of Uber insiders, the size of their engineering team is actually more likely to be one of the causes for bad engineering decisions. They seem to have too many engineers running around for too little actual work to be done, which results in those engineers coming up with stuff to do to keep them busy (and of course to ensure they appear to be busy and worth their money to their superiors). One of the best ways to create yourself some work is to not choose an already-proven path to a solution, but invent a new one just for the sake of inventing a new one. Of course that's not how this kind of doing is justified - the justification is usually "the proven path does not scale to our needs" or "by using a special approach adapted to our needs we can be more efficient" or "the proven path is too complex, we can get by with something simpler and easier to maintain". Which might actually all be proper justifications, it's just that you should have some hard proof for these statements, like benchmark results of a comparison of different approaches. That part often gets skipped, which is actually ironic, because doing extensive evaluation and benchmarking and implementing different approaches first before choosing one for production actually serves quite well to create even more work to do.
- asdfman123 7y ago
  Also, it's resume-driven development. Five years later they want to be able to be the guy who can say, "Yeah, I was the original designer behind Kafka," but substituting in a brand-new Uber technology.
- triceratops 7y ago
  It might be sub-optimal for Uber but it's better for the industry as a whole to have many different implementations of the same type of solution.
jeltz 7y ago
And none of their blog posts about their own geo stuff ever come close to answer the why. What is wrong with e.g. PostGIS or Elasticsearch? Why do they need to reinvent geo databases several times?
- loriverkutya 7y ago
  Too much time and not enough things to do.
  - drchickensalad 7y ago
    How does this happen?! They're so expensive
    - Kye 7y ago
      Give me twenty billion dollars to make a ride hailing app and I'll show you. More seriously: they raised tens of billions of dollars to make a ride hailing app and research self-driving cars. Throwing more money at a problem doesn't solve it faster, but it does pay for hiring people, and headcount is seen as a proxy for doing stuff. When a measure becomes a target, it ceases to be a good measure. https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law Some big start-ups[0] solve this conundrum by investing in adjacent companies to find the solutions they're paid to find. Outsourcing! This still has limits because there's so much money flowing around right now. Tossing a few million at a company with hundreds still won't solve the problem faster. [0] Start-up definition for this post: a company that has taken funding but hasn't yet found a sustainable business model. That's how Uber is still a start-up with more money than most companies make in decades.
- uber42163 7y ago
  Teams outside Infra aren't allowed persistent disk. Service authors get to pick from a small catalog of managed operational storage systems. Life would be a lot easier if a scalable Postgres were one of them, but it isn't. For the moment, the offerings with high scalability and good SLAs are all KV stores. We do have ES but it's operated for log search, not production critical paths.
deleted 7y ago
[deleted]

GrumpyNl 7y ago

Thats a great write up. They should hire him on the spot

buckhx 7y ago
That'd be nice. For real though I did meet with them after, but it was basically the same as getting a referral so the same interview process which I chose to forgo.

jeltz 7y ago

Another rebuttal which is less through due to the benchmark being very synthetic but still points out that Uber's numbers are nothing impressive. https://www.cybertec-postgresql.com/en/beating-uber-with-a-postgresql-prototype/ https://www.cybertec-postgresql.com/en/beating-uber-with-a-p...

Thaxll 7y ago
This is a crap article tbh They don't take any requierments and build a 5min poc with no constrains. For instance their demo doesn't update anything, Uber updates location every seconds, I'd like to see how PG behaves when you rebuild index thousand time per seconds.
- jeltz 7y ago
  No, that was not a requirement. This is not the database for the locations of cars, but the database with human defined geofences which probably only update a couple of times per day. The database is used to check e.g. if a customer is currently at an airport or not. That is a very normal and simple GIS workload.

twic 7y ago

So Uber's algorithm takes 58756 ns/op, and the fastest non-brutal algorithm took 471 ns/op. That's about 100 times faster. They used 40 machines to serve New Year's Eve traffic. Perhaps with a better algorithm, they could have got away with one. (Possibly not, because there's still a lot of HTTP and JSON munging work to be done, and the network card becomes a bottleneck at some point)