4 ms·

Maybe. I have more experience on the UI side, so my intuition may be off here, but I still have trouble believing that these kinds of basic slicing-and-dicing-r

Maybe. I have more experience on the UI side, so my intuition may be off here, but I still have trouble believing that these kinds of basic slicing-and-dicing-records problems aren't solved already. Especially when we're talking separate workspaces per-org, which could each get their own DB or shard; you can't hold all those messages in client-side memory at once, but surely at the DB level it's still much less data to process than what eg. Twitter deals with. Databases are fast, and this seems like a very vanilla database-shaped problem

muglug 4y ago
Twitter is very read-heavy. Think about the average timeline — Tweets don't change (they recently introduced "editing", but it's not an in-place edit). That means many things can be heavily cached, and most people on Twitter aren't writing Tweets, so the write volume is relatively low compared to the read volume. Now think about the average chat app like Slack or Teams. Those have a very different read-write ratio. Some things can be cached for long periods of time, but many other entities become invalidated quickly, and are subject to intricate permissions policies. As for sharding, Slack uses Vitess: https://slack.engineering/scaling-datastores-at-slack-with-vitess/ https://slack.engineering/scaling-datastores-at-slack-with-v...
- brundolf 4y ago
  I guess, although: > most people on Twitter aren't writing Tweets I don't know how true this is. Most people in my corner of twitter tweet several times a day, some of them have rapid-fire conversations in the replies. And that cache-busts (or at least appends to) a whole lot of people's feeds, just like it would people's chat history (while in both cases, almost all writes are appends) But idk, I'll stop speculating here because I'm not basing it on much
  - fmajid 4y ago
    Twitter is the first implementation of write-only memory.

muglug 4y ago

Twitter is very read-heavy. Think about the average timeline — Tweets don't change (they recently introduced "editing", but it's not an in-place edit). That means many things can be heavily cached, and most people on Twitter aren't writing Tweets, so the write volume is relatively low compared to the read volume. Now think about the average chat app like Slack or Teams. Those have a very different read-write ratio. Some things can be cached for long periods of time, but many other entities become invalidated quickly, and are subject to intricate permissions policies. As for sharding, Slack uses Vitess: https://slack.engineering/scaling-datastores-at-slack-with-vitess/ https://slack.engineering/scaling-datastores-at-slack-with-v...

brundolf 4y ago
I guess, although: > most people on Twitter aren't writing Tweets I don't know how true this is. Most people in my corner of twitter tweet several times a day, some of them have rapid-fire conversations in the replies. And that cache-busts (or at least appends to) a whole lot of people's feeds, just like it would people's chat history (while in both cases, almost all writes are appends) But idk, I'll stop speculating here because I'm not basing it on much
- fmajid 4y ago
  Twitter is the first implementation of write-only memory.