4 ms·

Does anyone else have something similar? What are some use cases that you found are useful?

• vivek-jain 9mo ago
  Sharded in-memory caching turns out to be rather useful at scale :) Some of the key examples highlighted on our blog are Unity Catalog, which is essentially the metadata layer for Databricks, our Query Orchestration Engine, and our distributed remote cache. See the blog post for more!
• louis-paul 9mo ago
  Sounds related to Google Slicer: https://research.google/pubs/slicer-auto-sharding-for-datacenter-applications/ https://research.google/pubs/slicer-auto-sharding-for-datace...
  • atuladya 9mo ago
    It is similar to Slicer in terms of the abstraction (I built Slicer at Google) but the architecture, implementation and algorithms have a lot of differences
    • bigwheels 9mo ago
      Did you also work on this databricks dicery?
      • hiyer 9mo ago
        Yes he did. I attended a talk from him on the same, so that's how I know.
• WookieRushing 9mo ago
  These show up once you have a certain scale where it is either cost inefficient or the hot spots are very dynamic. They also try to avoid latency by being eventually consistent sidecars instead of proxies. I’ve seen them used for traffic routing, storage system metadata systems, distributed cache etc