4 ms·

This runs afoul of the article’s warning to “ Never require a cache hit” It's important never to require a cache hit - even as a soft requirement. Evi

This runs afoul of the article’s warning to “ Never require a cache hit” It's important never to require a cache hit - even as a soft requirement. Evictions can happen at inconvenient times - such as when some other part of the system is under load - and there mustn't be any negative consequences to a cache miss If the cache does empty unexpectedly and that is going to cause your infrastructure to die in a cascade, probably some kind of fallback will be needed.

stetrain 6y ago
If you have a result that takes 10 seconds to calculate, and you need to read it faster than that, so you make a background processes to calculate it and keep it updated somewhere, you could argue that it's no longer caching. It's an asynchronous read model or a projection and it's now just part of your application.
- hinkley 6y ago
  I find that when I'm in a dead-end conversation with people about caching that often they can't accept that pre-calculation and caching are two different concepts, and you conflate them at our peril. For instance if I look up a value and pass it to three pure functions to ask about it, versus having the 3 functions look up the value and try to share a cache entry. This is doubly wrong because the final question may see a cache eviction and answer a different question due to changes in the data. Now I may have a situation where True:False:True was not a corner case we cover because it's logically impossible, but it's happening in the code because the last one changed from False to True due to concurrent read and write. The latter is also harder to test, because it requires mocks instead of manufactured inputs. And at this point the code is using the cache as global, mutable state, instead of semi-local state. Three strikes, you're out.
Thaxll 6y ago
If the computation of the data takes 10sec and your hammered with request, you should def require a cache hit and return a 500 if there is no cache and do not go the DB or somebackup mechanism.
- devonkim 6y ago
  Caches in web architectures seem to be similar to NUMA style addressing and makes me think that cache invalidations in web should have failovers to an L2 or L3 if the L1 is gone differentiating between an expected invalidation or an unexpected invalidation (Redis down, let’s say). Sometimes in an emergency (security incident, let’s say) all cache items should be invalidated though resulting in said 500 or temporarily unavailable response while the cache is repopulated again and that’s something else that Djikstra would shake his fist angrily at from beyond the grave just the same.
- dharmab 6y ago
  I'd argue in that case, the cache is more of a local read-only DB replica :) It's a subtle word change, but can communicate the slightly difference architecture and break the reader away from an assumption.
  - jerf 6y ago
    Yeah, I've had a similar thought to that before... as you scale up, the whole "cache" thing is just not a service you can have, because a cache must only speed up the thing being cached, it should ideally have no other visible impacts. As a site scales up and becomes dependent on a cache, it is really not a "cache" anymore, and both terminology and mindset should change as a result. If your site comes crashing down because the cache went down, it's not a cache anymore and you should use a different mindset in approaching it. The answer varies depending on a whole lot of relevant numbers that can vary widely, but an example of an answer I have in one of my systems is that I store all the cache queries more persistently than the cache answers. This particular system gets hammered with requests that are mostly the same as last time at certain times but spends a lot of time idle. The numbers work out that if I'm trying to compute the cache during one of my crunches, I need a huge amount of resources for just a couple of minutes, but if I restart the service between these crunches, there's plenty of time to recompute the entire cache before it is necessary with no additional resources. But that's just one particular example of not really being a "cache" anymore suitable in my very particular case.

stetrain 6y ago

If you have a result that takes 10 seconds to calculate, and you need to read it faster than that, so you make a background processes to calculate it and keep it updated somewhere, you could argue that it's no longer caching. It's an asynchronous read model or a projection and it's now just part of your application.

hinkley 6y ago
I find that when I'm in a dead-end conversation with people about caching that often they can't accept that pre-calculation and caching are two different concepts, and you conflate them at our peril. For instance if I look up a value and pass it to three pure functions to ask about it, versus having the 3 functions look up the value and try to share a cache entry. This is doubly wrong because the final question may see a cache eviction and answer a different question due to changes in the data. Now I may have a situation where True:False:True was not a corner case we cover because it's logically impossible, but it's happening in the code because the last one changed from False to True due to concurrent read and write. The latter is also harder to test, because it requires mocks instead of manufactured inputs. And at this point the code is using the cache as global, mutable state, instead of semi-local state. Three strikes, you're out.

Thaxll 6y ago

If the computation of the data takes 10sec and your hammered with request, you should def require a cache hit and return a 500 if there is no cache and do not go the DB or somebackup mechanism.

devonkim 6y ago
Caches in web architectures seem to be similar to NUMA style addressing and makes me think that cache invalidations in web should have failovers to an L2 or L3 if the L1 is gone differentiating between an expected invalidation or an unexpected invalidation (Redis down, let’s say). Sometimes in an emergency (security incident, let’s say) all cache items should be invalidated though resulting in said 500 or temporarily unavailable response while the cache is repopulated again and that’s something else that Djikstra would shake his fist angrily at from beyond the grave just the same.
dharmab 6y ago
I'd argue in that case, the cache is more of a local read-only DB replica :) It's a subtle word change, but can communicate the slightly difference architecture and break the reader away from an assumption.
- jerf 6y ago
  Yeah, I've had a similar thought to that before... as you scale up, the whole "cache" thing is just not a service you can have, because a cache must only speed up the thing being cached, it should ideally have no other visible impacts. As a site scales up and becomes dependent on a cache, it is really not a "cache" anymore, and both terminology and mindset should change as a result. If your site comes crashing down because the cache went down, it's not a cache anymore and you should use a different mindset in approaching it. The answer varies depending on a whole lot of relevant numbers that can vary widely, but an example of an answer I have in one of my systems is that I store all the cache queries more persistently than the cache answers. This particular system gets hammered with requests that are mostly the same as last time at certain times but spends a lot of time idle. The numbers work out that if I'm trying to compute the cache during one of my crunches, I need a huge amount of resources for just a couple of minutes, but if I restart the service between these crunches, there's plenty of time to recompute the entire cache before it is necessary with no additional resources. But that's just one particular example of not really being a "cache" anymore suitable in my very particular case.