3 ms·

If the computation of the data takes 10sec and your hammered with request, you should def require a cache hit and return a 500 if there is no cache and do not g

by Thaxll 6y ago

If the computation of the data takes 10sec and your hammered with request, you should def require a cache hit and return a 500 if there is no cache and do not go the DB or somebackup mechanism.

devonkim 6y ago
Caches in web architectures seem to be similar to NUMA style addressing and makes me think that cache invalidations in web should have failovers to an L2 or L3 if the L1 is gone differentiating between an expected invalidation or an unexpected invalidation (Redis down, let’s say). Sometimes in an emergency (security incident, let’s say) all cache items should be invalidated though resulting in said 500 or temporarily unavailable response while the cache is repopulated again and that’s something else that Djikstra would shake his fist angrily at from beyond the grave just the same.
dharmab 6y ago
I'd argue in that case, the cache is more of a local read-only DB replica :) It's a subtle word change, but can communicate the slightly difference architecture and break the reader away from an assumption.
- jerf 6y ago
  Yeah, I've had a similar thought to that before... as you scale up, the whole "cache" thing is just not a service you can have, because a cache must only speed up the thing being cached, it should ideally have no other visible impacts. As a site scales up and becomes dependent on a cache, it is really not a "cache" anymore, and both terminology and mindset should change as a result. If your site comes crashing down because the cache went down, it's not a cache anymore and you should use a different mindset in approaching it. The answer varies depending on a whole lot of relevant numbers that can vary widely, but an example of an answer I have in one of my systems is that I store all the cache queries more persistently than the cache answers. This particular system gets hammered with requests that are mostly the same as last time at certain times but spends a lot of time idle. The numbers work out that if I'm trying to compute the cache during one of my crunches, I need a huge amount of resources for just a couple of minutes, but if I restart the service between these crunches, there's plenty of time to recompute the entire cache before it is necessary with no additional resources. But that's just one particular example of not really being a "cache" anymore suitable in my very particular case.

by Thaxll 6y ago

If the computation of the data takes 10sec and your hammered with request, you should def require a cache hit and return a 500 if there is no cache and do not go the DB or somebackup mechanism.

devonkim 6y ago

Caches in web architectures seem to be similar to NUMA style addressing and makes me think that cache invalidations in web should have failovers to an L2 or L3 if the L1 is gone differentiating between an expected invalidation or an unexpected invalidation (Redis down, let’s say). Sometimes in an emergency (security incident, let’s say) all cache items should be invalidated though resulting in said 500 or temporarily unavailable response while the cache is repopulated again and that’s something else that Djikstra would shake his fist angrily at from beyond the grave just the same.

dharmab 6y ago

I'd argue in that case, the cache is more of a local read-only DB replica :) It's a subtle word change, but can communicate the slightly difference architecture and break the reader away from an assumption.

jerf 6y ago
Yeah, I've had a similar thought to that before... as you scale up, the whole "cache" thing is just not a service you can have, because a cache must only speed up the thing being cached, it should ideally have no other visible impacts. As a site scales up and becomes dependent on a cache, it is really not a "cache" anymore, and both terminology and mindset should change as a result. If your site comes crashing down because the cache went down, it's not a cache anymore and you should use a different mindset in approaching it. The answer varies depending on a whole lot of relevant numbers that can vary widely, but an example of an answer I have in one of my systems is that I store all the cache queries more persistently than the cache answers. This particular system gets hammered with requests that are mostly the same as last time at certain times but spends a lot of time idle. The numbers work out that if I'm trying to compute the cache during one of my crunches, I need a huge amount of resources for just a couple of minutes, but if I restart the service between these crunches, there's plenty of time to recompute the entire cache before it is necessary with no additional resources. But that's just one particular example of not really being a "cache" anymore suitable in my very particular case.