6 ms·

Reading the article I didn't see it, but does FB have any innovation/papers/infra around "entity A changed, so cache keys x/y/z need invalidated"? Basically sim

Reading the article I didn't see it, but does FB have any innovation/papers/infra around "entity A changed, so cache keys x/y/z need invalidated"? Basically similar to Noria mentioned at the beginning of the post.

thephyber 3y ago
There is a diagram of a process that reads the MySQL commit log and uses that to send Memcache delete commands. Not exactly groundbreaking. But I would also guess that FB would have to explain lots of their middleware concepts in order to explain how data in the database maps to memcache keys defined in middleware. Personally, I think the hard part is ensuring the entire company/system accesses data through the correct abstractions (eg. the newest data model code) and does not bypass the data model code (eg. by hitting the database directly).
- bagels 3y ago
  There are a few layers above this as well as safeguards to help make sure the right abstractions are used.
- YZF 3y ago
  So the cache can be out of sync with the database before that log processing gets done with deleting?
  - AlphaCharlie 3y ago
    The binlogs are applied first, then the deletes are emitted. There’s also support for read after write consistency within a single cluster by adding a canary to the memcache key warning to read from the master replica after the write succeeds, that’s a really cool feature for a multi region eventually consistent system
amiga-workbench 3y ago
This can be fairly gracefully handled by using a caching library with tag support. On a webpage that collates content from disparate parts of a CMS, you just tag the page content cache with the ID's of all the entities that make it up. Then you define a save/delete handler that purges caches tagged with the ID of whatever you just altered.
yuliyp 3y ago
Not sure about what's been published, but from what I recall, for data stored in memcache it was manual: people would need to know the right keys to invalidate when doing DB mutations (naturally there were functions encoding this, so multiple queries updating, say, a user could all just call a single function which returned the cache keys for that user).
- AlphaCharlie 3y ago
  That’s correct :)

thephyber 3y ago

There is a diagram of a process that reads the MySQL commit log and uses that to send Memcache delete commands. Not exactly groundbreaking. But I would also guess that FB would have to explain lots of their middleware concepts in order to explain how data in the database maps to memcache keys defined in middleware. Personally, I think the hard part is ensuring the entire company/system accesses data through the correct abstractions (eg. the newest data model code) and does not bypass the data model code (eg. by hitting the database directly).

bagels 3y ago
There are a few layers above this as well as safeguards to help make sure the right abstractions are used.
YZF 3y ago
So the cache can be out of sync with the database before that log processing gets done with deleting?
- AlphaCharlie 3y ago
  The binlogs are applied first, then the deletes are emitted. There’s also support for read after write consistency within a single cluster by adding a canary to the memcache key warning to read from the master replica after the write succeeds, that’s a really cool feature for a multi region eventually consistent system

amiga-workbench 3y ago

This can be fairly gracefully handled by using a caching library with tag support. On a webpage that collates content from disparate parts of a CMS, you just tag the page content cache with the ID's of all the entities that make it up. Then you define a save/delete handler that purges caches tagged with the ID of whatever you just altered.

yuliyp 3y ago

Not sure about what's been published, but from what I recall, for data stored in memcache it was manual: people would need to know the right keys to invalidate when doing DB mutations (naturally there were functions encoding this, so multiple queries updating, say, a user could all just call a single function which returned the cache keys for that user).

AlphaCharlie 3y ago
That’s correct :)