3 ms·
Uber Serves 40M Qps from Online Storage Using an Integrated Cache
- MuffinFlavored 3y ago> The stateless query engine layer is responsible for query planning, routing, sharding, schema management, node health monitoring, request parsing, validation, and AuthN/AuthZ. Why don't the nodes expose health checks and other software like Promteheus/Alertmanager check node health? Similar question for AuthN/AuthZ Obviously Uber knows more than I do... just curious if this design decision (probably just splitting hairs) makes the most sense
- supriyo-biswas 3y agoThis is an internal managed database service, so exposing Prometheus metrics still doesn’t solve the issue of reliability for the consumers of said service. Of course, the “query engine layer” probably has multiple components, and could have Prometheus running somewhere in the stack.
- deleted 3y ago[deleted]
- withinboredom 3y agoReminds me of FASTER but for querying instead of WALs and k/v stores. I have a feeling we will be seeing a lot more of these types of optimizations in the near future.
- HackerThemAll 3y agoThis smells bad architecture to me. Been there, seen that many times. That load could be decreased by factor of 10 by some simple clever ideas and changes. Then it could further be lowered. I just can't imagine a map app and a supporting business to be _that_ demanding in terms of storage use. I do see congestion points, but it's not storage. Unless they're adding up cache lookups to the overall number, which is then just a clickbait.
- IamLoading 3y agoI dont agree with your criticism. It is a unified cache for multiple microservices, deployed in multiple regions, has at least 20 million DAU. Not hard to imagine that peak can hit 40M total reads.