3 ms·

The key problem with caching, not alluded to in this article, isn't that it hides performance problems or is hard to invalidate or causes slow startups. It's th

by barrkel 10y ago

The key problem with caching, not alluded to in this article, isn't that it hides performance problems or is hard to invalidate or causes slow startups. It's that it entangles state in a global way. Responses to a request are usually dependent on a wide variety of different internal states: properties of the user, cookies, state in one or more databases often across multiple tables, or state in multiple dependent services. A cached value is a snapshot of a calculation over this internal state: but it needs to be invalidated when any of the states that it is composed from changes. What used to be a simple local calculation now becomes a global concern; anything that modifies any of the underlying states needs to be aware of the global state in the cache, and evict or invalidate cached items that were dependent on the underlying state. No longer is local reasoning enough. You need to have a global understanding, possibly across multiple subsystems. Caching worthy of the name - caching things that are complex to calculate in time or space - inflicts serious architectural harm on a software system. It impairs maintenance and understanding of the system, and will cause bugs when people who don't have sufficient global understanding make local changes. It's not all bad. Caching static assets, or a small well-understood layer (like a disk cache), or designing calculations in such a way that dependencies have immutable mappings from identifiers to values - there are ways to make caching work. But it needs to be done very carefully in a large system or you risk long-term damage.

kazinator 10y ago
> isn't that it hides performance problems or is hard to invalidate ... > ... it needs to be invalidated when any of the states that it is composed from change ... need to have a global understanding, possibly across multiple subsystems :)
- barrkel 10y ago
  "hard to invalidate" doesn't really capture the nature of the architectural harm.
webmaven 10y ago
> there are ways to make caching work. But it needs to be done very carefully in a large system or you risk long-term damage. There are ways of using caching that are relatively simple to reason about that ought to be evangelized a bit. For example, many caching proxies expose quite a bit more functionality than just caching, such as replacing placeholders in server-side rendered pages with transclusions (by the proxy) from another URL (which can be part of the same web app). This lets you compose server responses from fragments that have different caching rules, neatly side-stepping many of the issues mentioned in the OP as well keeping global state disentangled. For example, you can cache most of a rendered web-page with a generous TTL so that the cache only queries the web app behind it once an hour or even less often, and have the proxy insert the portion of the page that must be current, which is also cached, but with a TTL of <1 second and relying on if-modified-since to speed up the usual case where nothing has changed. The resulting setup is easy to reason about, doesn't have any particular gotchas (unless you are doing a lot of A/B testing), and will absolutely solve the problem of your site slowing to a crawl because 50k impatient users are obsessively hitting Ctrl-R over and over on your homepage at the same time every Monday morning to check to see if that one thing has changed yet. Notably, this type of setup works without having to beef up with more server instances to handle the load, or having to accept a slightly stale homepage being served for several minutes, or even accepting a longer response time for the very first user to hit the server after the content does change and the cache is invalidated and repopulated. All you've done with the cache is eliminate wasted cycles at the cost of adding less complexity to the code than you need for feature-switching. This approach of fast composition (transclusion or even simpler concatenation) of separately cached values avoids the problem of global state entanglement because the final transclusion or concatenation is simple and fast enough to not (or hardly) bother with caching the final result at all.

by barrkel 10y ago

kazinator 10y ago

> isn't that it hides performance problems or is hard to invalidate ... > ... it needs to be invalidated when any of the states that it is composed from change ... need to have a global understanding, possibly across multiple subsystems :)

barrkel 10y ago
"hard to invalidate" doesn't really capture the nature of the architectural harm.

webmaven 10y ago

> there are ways to make caching work. But it needs to be done very carefully in a large system or you risk long-term damage. There are ways of using caching that are relatively simple to reason about that ought to be evangelized a bit. For example, many caching proxies expose quite a bit more functionality than just caching, such as replacing placeholders in server-side rendered pages with transclusions (by the proxy) from another URL (which can be part of the same web app). This lets you compose server responses from fragments that have different caching rules, neatly side-stepping many of the issues mentioned in the OP as well keeping global state disentangled. For example, you can cache most of a rendered web-page with a generous TTL so that the cache only queries the web app behind it once an hour or even less often, and have the proxy insert the portion of the page that must be current, which is also cached, but with a TTL of <1 second and relying on if-modified-since to speed up the usual case where nothing has changed. The resulting setup is easy to reason about, doesn't have any particular gotchas (unless you are doing a lot of A/B testing), and will absolutely solve the problem of your site slowing to a crawl because 50k impatient users are obsessively hitting Ctrl-R over and over on your homepage at the same time every Monday morning to check to see if that one thing has changed yet. Notably, this type of setup works without having to beef up with more server instances to handle the load, or having to accept a slightly stale homepage being served for several minutes, or even accepting a longer response time for the very first user to hit the server after the content does change and the cache is invalidated and repopulated. All you've done with the cache is eliminate wasted cycles at the cost of adding less complexity to the code than you need for feature-switching. This approach of fast composition (transclusion or even simpler concatenation) of separately cached values avoids the problem of global state entanglement because the final transclusion or concatenation is simple and fast enough to not (or hardly) bother with caching the final result at all.