3 ms·
So I am a little unsure about where all the data is. Am I right in thinking that a user has a cached merge of all their follower's events. So when they first re
by tlarkworthy 12y ago
So I am a little unsure about where all the data is. Am I right in thinking that a user has a cached merge of all their follower's events. So when they first read they get an inconsistent view which can be populated in the background on demand. The massive data gains being: most people don't read all their data so you waste a lot of time on fan-out writing data that is never used.
The read view just has to be caught up from when it was last accessed. You don't do this all the time, only when a user requests their timeline. So still an individual CDRT set still has to be persisted for each individual user a bit like an out-of-date inbox. You don't create the whole inbox from scratch each request or do you?
maybe you do because you only need recent events??
----------------
EDIT.
I think I worked it out ... the inboxes are created dynamically. This is what was meant by stateless. All the time series data is able to fit into one server's memory, so the IO overhead of assemblage is low
- sagichmal 12y agoYou got it on your edit. The inbox is generated dynamically from multiple outboxes. The time series data, however, is not able to fit in one server's memory; it's sharded over multiple non-communicating Redis instances, collectively known as a pool. And multiple pools (complete replica sets) constitute the system.
- tlarkworthy 12y agoAhhhhhhh. yes yes. Nice. Weirdly enough I am implementing a CRDT list today (not a set). The TreeDoc example in the main paper has a problem I think, if two writes with same key go in, the disambiguator keeps an ordering. But if that occurs, you now can't insert between those two edits because the disambiguators are not dense. The set stuff looks good though.