3 ms·

> with a simpler cache model (say you just cache a single item), do you agree that solving the "when" problem is very managable? The "when" problem is dependen

> with a simpler cache model (say you just cache a single item), do you agree that solving the "when" problem is very managable? The "when" problem is dependent on your application architecture, not on your cache backend nor the number of keys in it. If, overall, you have an extremely simplistic application architecture, then cache invalidation may be quite easy, but you'll either: 1. forgo advanced user interaction or dynamic updates, in which case cache invalidation may not even be required at all (excepting publishing) 2. have scalability problems, and need to increase the complexity of your application to meet those challenges The difficulty of cache invalidation scales with the complexity of your application (and not necessarily linear scaling) > My argument is that #1 can be very managable with simpler data models. #2 can't be avoided; and #2 is very hard. Yes, #1 can be manageable for low-traffic simple static applications. It is a general problem who's difficulty relates to the complexity of the application. Yes, #2, can't be avoided. But, while it is interesting, and - in some cases, given a specific caching stack - it may be relatively hard, it's not a general problem. Difficulties with it are implementation-specific, not broadly applicable. Significantly, it's not the general and fundamental hard problem being referred when people talk about the universal difficulty of "cache invalidation".

uvdn7 4y ago
> The "when" problem is dependent on your application architecture, not on your cache backend nor the number of keys in it. I would argue the "when" problem is dependent on data models. And it's possible that we are referring to the same thing with different names. > If, overall, you have an extremely simplistic application architecture I mean FB is not a simple app. A complicated app can be built on a relatively simple data model as well (that's essentially how TAO works). But we do have memcache as well; and in some cases it can be complicated/hard. I can see that, and I won't argue against it. > Yes, #2, can't be avoided. But, while it is interesting, and - in some cases, given a specific caching stack - it may be relatively hard, it's not a general problem. I respectfully disagree. I explained in the blog post about why #2 is a generally hard problem. The analogy I like to use is Paxos. The protocol fits on a single slide. It's easier to feel like you have Paxos work; but it's very hard to have Paxos actually work.
- lucideer 4y ago
  Paxos is an implementation (of protocols). I think you fundamentally misunderstand what the word "general" means here.
  - uvdn7 4y ago
    > Paxos is an implementation (of protocols). I have nothing more to add, if that's your standpoint. But I am glad that we "debugged" to close to the "root" of our assumptions/context.

uvdn7 4y ago

> The "when" problem is dependent on your application architecture, not on your cache backend nor the number of keys in it. I would argue the "when" problem is dependent on data models. And it's possible that we are referring to the same thing with different names. > If, overall, you have an extremely simplistic application architecture I mean FB is not a simple app. A complicated app can be built on a relatively simple data model as well (that's essentially how TAO works). But we do have memcache as well; and in some cases it can be complicated/hard. I can see that, and I won't argue against it. > Yes, #2, can't be avoided. But, while it is interesting, and - in some cases, given a specific caching stack - it may be relatively hard, it's not a general problem. I respectfully disagree. I explained in the blog post about why #2 is a generally hard problem. The analogy I like to use is Paxos. The protocol fits on a single slide. It's easier to feel like you have Paxos work; but it's very hard to have Paxos actually work.

lucideer 4y ago
Paxos is an implementation (of protocols). I think you fundamentally misunderstand what the word "general" means here.
- uvdn7 4y ago
  > Paxos is an implementation (of protocols). I have nothing more to add, if that's your standpoint. But I am glad that we "debugged" to close to the "root" of our assumptions/context.