3 ms·

Cache invalidation isn’t hard, it’s just where hard things show up for easy problems. Cache invalidation in a nutshell: For some given memory constraint, give

Cache invalidation isn’t hard, it’s just where hard things show up for easy problems. Cache invalidation in a nutshell: For some given memory constraint, give me the same result with lookup performance for known inputs. That’s not hard because of memory constraints (excepting very specialized use cases), that’s hard because the things that benefit from caching usually have implicit inputs. You’re trying to cache “database query” where “database” has other stuff mutating state. You’re not caching the call, you’re caching the side effects it depends on. This use case is trivially solved if you plan for it. Use the algorithms from a history based system. Caching the state of a given object in git is easy: its cache is its hash. Either that part of the tree changed or it didn’t. The key is to remove implicit dependencies from your cache path. Everything that you cache states its dependencies, anything that changes in them reports back. You can get more specific than that for finer grained control but we’re already well past invalidation being the hard problem. It’s just dependency granularity.

kstenerud 6y ago
Which is great until your backdraft cache invalidation signals start clogging things up because they're happening too often. Caching isn't always something you can pre-plan for; you're not going to always have a clean, fully encapsulated space like a git database. Quite often the need for caching only comes about as a result of profiling. I've seen plenty of crazy cache systems that turned out to be completely unnecessary and incurred much technical debt in my days because we guessed wrong about the performance profile and expected usage graph at the start. Sometimes only a subsection of the queries need to be cached, which is important because you may be facing constraints where a full cache would saturate your storage space. Like in the game of go, it looks simple but the devil's in the details. Caching in the real world is hard, and because of that cache invalidation is also hard.
- eyelidlessness 6y ago
  > Which is great until your backdraft cache invalidation signals start clogging things up because they're happening too often. There are plenty of simple solutions. Batching, throttling, scheduling, partitioning. Any or several could help. > you're not going to always have a clean, fully encapsulated space like a git database This was kinda my point: the problem isn’t cache invalidation, it’s external concurrent state. This, and the rest of your response, comes down to the hard problems being design, analysis, and ability to adapt. Those problems show up in cache complexity because it reveals where those other considerations are suboptimal.

kstenerud 6y ago

Which is great until your backdraft cache invalidation signals start clogging things up because they're happening too often. Caching isn't always something you can pre-plan for; you're not going to always have a clean, fully encapsulated space like a git database. Quite often the need for caching only comes about as a result of profiling. I've seen plenty of crazy cache systems that turned out to be completely unnecessary and incurred much technical debt in my days because we guessed wrong about the performance profile and expected usage graph at the start. Sometimes only a subsection of the queries need to be cached, which is important because you may be facing constraints where a full cache would saturate your storage space. Like in the game of go, it looks simple but the devil's in the details. Caching in the real world is hard, and because of that cache invalidation is also hard.

eyelidlessness 6y ago
> Which is great until your backdraft cache invalidation signals start clogging things up because they're happening too often. There are plenty of simple solutions. Batching, throttling, scheduling, partitioning. Any or several could help. > you're not going to always have a clean, fully encapsulated space like a git database This was kinda my point: the problem isn’t cache invalidation, it’s external concurrent state. This, and the rest of your response, comes down to the hard problems being design, analysis, and ability to adapt. Those problems show up in cache complexity because it reveals where those other considerations are suboptimal.