14 ms·

"I think now caching is probably best understood as a tool for making software simpler" - that's cute. Caching might be beneficial for many cases, but if it doe

by ckdot2 1y ago

"I think now caching is probably best understood as a tool for making software simpler" - that's cute. Caching might be beneficial for many cases, but if it doesn't do one thing then this is simplifying software. There's that famous quote "There are only two hard things in Computer Science: cache invalidation and naming things.", and, sure, it's a bit ironical, but there's some truth in there.

bell-cot 1y ago
(You forgot off-by-1 errors.) All software has to name things, and count. Caching (including invalidation) is best understood as a liability. If you can foist it off on your CPU and OS and DB, good for you. Programming whatever you're actually trying to get done is already hard enough.
- yxhuvud 1y ago
  Off by 1-errors is not part of the original quote, but is just a later addon to make it funny. They also tend not to be very hard.
  - TeMPOraL 1y ago
    Except when they're part of some base assumptions in the domain or dozen of layers of abstractions below you. They are hard to prevent from happening.
  - tombert 1y ago
    They're not hard but I will say that when I was writing an app that was using both JavaScript and Julia, I kept getting off-by-one errors because Julia starts at 1 instead of 0. Really the only time in my entire professional career that off-by-one errors have actually given me headaches.
    - bobthepanda 1y ago
      I think that is from a time when popular programming languages varied in their behavior and also when people were writing for loops with incrementation all the time. A lot of languages have just settled on zero indexing, and many now have some variation of for/each or for/of that would eliminate a lot of potential ways to encounter this error.
      - tombert 1y ago
        Yeah, I mostly will do map/reduce/filter when possible, and obviously in those cases indexes don’t matter. It could start at index 12345 for all I care with that stuff. Occasionally, though, I need to use the same index across multiple items, there’s not a trivial means in which to zip, and at that point I have to use an old school for loop. That’s when the 1-index vs 0-index bites me.
- Cthulhu_ 1y ago
  If you omit the off-by-1 error from the two hard things joke, you're still off by 1, right? Kind of?
whateveracct 1y ago
caching often does simplify software though when done well and - as the OP suggests - it works best when the cache is a well-defined abstraction with properties and rules about how it works just because "caching" is mentioned in a meme doesn't mean it can't be true that it can simplify software
- BowBun 1y ago
  > caching often does simplify software though when done well I have to push back here, I think this is objectively untrue. By definition a system or piece of code on where you add a condition where something else happens (cache) that behaves differently than the uncached path increases complexity. I'm not saying it's wrong to cache things or that they aren't useful, but I think they absolutely are an abstraction and an optimization at the cost of complexity. Good code bases hide complexity from the devs all the time, so it's not a question of whether you can code it away, but rather how difficult is it to troubleshoot the internals of the system.
- PaulHoule 1y ago
  Trying some other way to explicitly manage multiple storage tiers could get pretty complicated.
- jameshart 1y ago
  If you hide caching away as an implementation detail behind an abstraction, it comes back and bites you as a leaky abstraction later. Look at how CPU cache line behaviors radically change the performance of superficially similar algorithms. Look at how query performance for a database server drops off a cliff the moment the working cache no longer fits in memory. Hiding complexity can be a simplification, until you exceed the bounds of the simplification and the complexity you hid demands your attention anyway.
  - atq2119 1y ago
    CPUs are still a great example for how caching simplifies things. There's a long history in computer architecture of cores and accelerators that don't have a cache but instead rely on explicitly programmed local scratchpads. They are universally more difficult to program than general purpose CPUs because of that.
EGreg 1y ago
I never understood about cache invalidation or naming things Both are not that difficult, honestly. Aren’t there a lot harder things out there
- szundi 1y ago
  [dead]
- gryfft 1y ago
  It's a little bit tongue in cheek; no one is seriously suggesting it's harder than P=NP or the problem of consciousness. But there's something a bit "death and taxes" to the inevitability that any large enough project is going to have some corner cases involving these old chestnuts. Heck you can probably prove that any system for naming things is either inconsistent or incomplete.
  - TeMPOraL 1y ago
    > no one is seriously suggesting it's harder than P=NP or the problem of consciousness. Well, I for one feel that "naming things" ultimately boils down to the latter, which may or may not be harder than the former.
- Valodim 1y ago
  In my experience, the larger the software you write, the truer these become. At some point all obvious names will have collisions, and getting caching right is crucial to do but difficult to achieve because it transcends the entire stack. You could group these two things into "getting the data model right" as the single hard thing, perhaps that rings more true to you :)
- quuxplusone 1y ago
  For "only two hard problems," read "two candidates for among the hardest problems (but we feel strongly that these are indeed good candidates)," or something along those lines, more or less. It's also possible that these used to be the only two hard problems at the time the aphorism was first recorded, but the underlying state of the world has changed since then and the aphorism, as recorded, is no longer current.
bloppe 1y ago
"Two programs could have similar behaviour but structured very differently, the difference being that one utilizes caching as an abstraction and one explicitly has the concept of different tiers of storage." The author is comparing "off-the-shelf" caching with custom caching. They're coming from the assumption that you must be caching somehow and arguing that the word "caching" should be understood to mean only particular approaches to the general idea of caching. And obviously the whole point of the general idea is to optimize things. It's a rhetorical mess
heikkilevanto 1y ago
Caching is simple, yes. The hard part is in the last word, invalidation. Even that is manageable for a single process. But as soon as you have multiple (threads / processes / nodes / data centers) updating the data, it does get quite complex, pretty fast. Likewise, naming things is simple as long as you alone, or a in a small team. But as soon as there are multiple organizations with all their own traditions, it gets tricky. Just witness the eternal flame wars about camelCase, PascalCase, snake_case, kebab-case, and UPPER_CASE. It is almost as hopeless culture clash as Emacs vs Vi vs PowerPoint... (I leave the off-by-one errors as an exercise for the reader)
- TeMPOraL 1y ago
  I'd say this is not the "naming things" that's hard. Beyond picking a common identifier format in the team, there are at least two dimensions that are much harder: - The language dimension - choice of words, that are good enough for the purpose, and not confusing. For example, "Manager" is as ambiguous as it gets, it can mean many thing, except we've been using it long enough that there's a more specific shape of meaning[0] for that word in code/program architecture contexts - so you still would use it instead of, say "Coordinator", which would raise all kinds of questions that "Manager" no longer does. - The epistemological dimension - whether the word you chose correctly names the concept you meant, and whether the concept you meant is actually the right one to describe the thing you're trying to describe. Ultimately, this is the hard thing at the root of philosophy. In practice, it manifests like e.g. choice between digging into some obscure branches of mathematics to correctly name the thing "endofunctor" or something, or calling it "Square" and saying "fuck it, we'll clarify the exceptions in the comments". -- [0] - I mean "more specific" in the sense it's distinct from the other meanings and somewhat narrow - but still it's fuzzy as heck and you can't describe it fully in words; it's basically tacit knowledge.
  - Xss3 1y ago
    I try to name things descriptively in simple terms and often end up with NamesAboutThisLong, once they get too long i know the thing is doing too much and some refactoring is needed for readability. I also avoid letting the reader make assumptions. HasPlayerJumpedRecently is bad. What does recently mean? HasPlayerJumpedInLastTenMs is better, even if it's a bit long...Which highlights that it should probably be refactored into a more flexible value; MsSincePlayerLastJumped. If you arent assuming a time var wth Ms is milliseconds you aren't doing games dev so that one slides with me.
Traubenfuchs 1y ago
I never understood this meme. We use caching a lot, anything that gets cached can only be written by one service each. The writing services emit cache invalidation messages via SNS that cache users must listen to via SQS, to clear/update their cache. Alternatively we cache stuff with just a TTL, when immediate cache invalidation is not important. Where‘s the struggle?
- porridgeraisin 1y ago
  Here's one: everybody invalidating and refreshing their cache at the same time can cause a thundering herd problem.
- hmottestad 1y ago
  Does SQS guarantee delivery to all clients? If it does then that’s doing a lot of heavy lifting for you. If it doesn’t guarantee delivery, then I believe you will at some point have a client that reads a cached value thinking it’s still valid because the invalidation message got lost in the network.
  - maccard 1y ago
    Eventually. The problem is that eventually delivering that message will result in clients assuming that it will always be the same, when it’s not.
- williamdclt 1y ago
  You don’t support read-your-own-write and your cache data might be stale for arbitrarily long. These relaxed consistency constraints make caching a lot easier. If that’s acceptable to your use cases then you’re in a great place! If not… well, at scale you often need to find a way for it to be acceptable anyway
- pton_xd 1y ago
  > Where‘s the struggle? If there are no real consequences for reading stale data, and your writes are infrequent enough, then indeed you're lucky and have a relatively simple problem.
deleted 1y ago
[deleted]
hatthew 1y ago
If you have a system with "slow storage", caching is a way to optimize that to "storage that is sometimes fast". If you have a system with "slow storage" and "fast storage", caching is a way to abstract that away to just "storage". The author is arguing that the latter is the default way we should think about the concept of caching, which is a valid opinion to have.
deleted 1y ago
[deleted]
AdieuToLogic 1y ago
> There's that famous quote "There are only two hard things in Computer Science: cache invalidation and naming things.", and, sure, it's a bit ironical, but there's some truth in there. The joke form of this quote goes along the lines of: There are only two hard things in Computer Science: cache invalidation, naming things, and off-by-one errors. :-D
- dcminter 1y ago
  I rather like the snark of: there's two hard problems in computer science: we only have one joke and it's not funny. Apparently⁰ by Philip Scott Bowden¹ ⁰ https://martinfowler.com/bliki/TwoHardThings.html https://martinfowler.com/bliki/TwoHardThings.html ¹ https://x.com/pbowden/status/468855097879830528 https://x.com/pbowden/status/468855097879830528
  - aorth 1y ago
    Just remembered another one: there are 10 types of people in the world: those who understand binary and those who don't. :)
- AndrewOMartin 1y ago
  Which leads to > I don't see what's so hard about DNS, it's just cache invalidation and naming things.
  - manyaoman 1y ago
    ... and avoiding off-by-one errors.
  - dcminter 1y ago
    Oh that's gooood. Got a cite or is it yours?
- SAI_Peregrinus 1y ago
  My favorite variation only really works in text: There are three hard problems in Computer Science: 1) Cache invalidation 2) Naming th3) Concurings rency 4) Off-by-one errors
deleted 1y ago
[deleted]

by ckdot2 1y ago

bell-cot 1y ago

(You forgot off-by-1 errors.) All software has to name things, and count. Caching (including invalidation) is best understood as a liability. If you can foist it off on your CPU and OS and DB, good for you. Programming whatever you're actually trying to get done is already hard enough.

yxhuvud 1y ago
Off by 1-errors is not part of the original quote, but is just a later addon to make it funny. They also tend not to be very hard.
- TeMPOraL 1y ago
  Except when they're part of some base assumptions in the domain or dozen of layers of abstractions below you. They are hard to prevent from happening.
- tombert 1y ago
  They're not hard but I will say that when I was writing an app that was using both JavaScript and Julia, I kept getting off-by-one errors because Julia starts at 1 instead of 0. Really the only time in my entire professional career that off-by-one errors have actually given me headaches.
  - bobthepanda 1y ago
    I think that is from a time when popular programming languages varied in their behavior and also when people were writing for loops with incrementation all the time. A lot of languages have just settled on zero indexing, and many now have some variation of for/each or for/of that would eliminate a lot of potential ways to encounter this error.
    - tombert 1y ago
      Yeah, I mostly will do map/reduce/filter when possible, and obviously in those cases indexes don’t matter. It could start at index 12345 for all I care with that stuff. Occasionally, though, I need to use the same index across multiple items, there’s not a trivial means in which to zip, and at that point I have to use an old school for loop. That’s when the 1-index vs 0-index bites me.
Cthulhu_ 1y ago
If you omit the off-by-1 error from the two hard things joke, you're still off by 1, right? Kind of?

whateveracct 1y ago

caching often does simplify software though when done well and - as the OP suggests - it works best when the cache is a well-defined abstraction with properties and rules about how it works just because "caching" is mentioned in a meme doesn't mean it can't be true that it can simplify software

BowBun 1y ago
> caching often does simplify software though when done well I have to push back here, I think this is objectively untrue. By definition a system or piece of code on where you add a condition where something else happens (cache) that behaves differently than the uncached path increases complexity. I'm not saying it's wrong to cache things or that they aren't useful, but I think they absolutely are an abstraction and an optimization at the cost of complexity. Good code bases hide complexity from the devs all the time, so it's not a question of whether you can code it away, but rather how difficult is it to troubleshoot the internals of the system.
PaulHoule 1y ago
Trying some other way to explicitly manage multiple storage tiers could get pretty complicated.
jameshart 1y ago
If you hide caching away as an implementation detail behind an abstraction, it comes back and bites you as a leaky abstraction later. Look at how CPU cache line behaviors radically change the performance of superficially similar algorithms. Look at how query performance for a database server drops off a cliff the moment the working cache no longer fits in memory. Hiding complexity can be a simplification, until you exceed the bounds of the simplification and the complexity you hid demands your attention anyway.
- atq2119 1y ago
  CPUs are still a great example for how caching simplifies things. There's a long history in computer architecture of cores and accelerators that don't have a cache but instead rely on explicitly programmed local scratchpads. They are universally more difficult to program than general purpose CPUs because of that.

EGreg 1y ago

I never understood about cache invalidation or naming things Both are not that difficult, honestly. Aren’t there a lot harder things out there

szundi 1y ago
[dead]
gryfft 1y ago
It's a little bit tongue in cheek; no one is seriously suggesting it's harder than P=NP or the problem of consciousness. But there's something a bit "death and taxes" to the inevitability that any large enough project is going to have some corner cases involving these old chestnuts. Heck you can probably prove that any system for naming things is either inconsistent or incomplete.
- TeMPOraL 1y ago
  > no one is seriously suggesting it's harder than P=NP or the problem of consciousness. Well, I for one feel that "naming things" ultimately boils down to the latter, which may or may not be harder than the former.
Valodim 1y ago
In my experience, the larger the software you write, the truer these become. At some point all obvious names will have collisions, and getting caching right is crucial to do but difficult to achieve because it transcends the entire stack. You could group these two things into "getting the data model right" as the single hard thing, perhaps that rings more true to you :)
quuxplusone 1y ago
For "only two hard problems," read "two candidates for among the hardest problems (but we feel strongly that these are indeed good candidates)," or something along those lines, more or less. It's also possible that these used to be the only two hard problems at the time the aphorism was first recorded, but the underlying state of the world has changed since then and the aphorism, as recorded, is no longer current.

bloppe 1y ago

"Two programs could have similar behaviour but structured very differently, the difference being that one utilizes caching as an abstraction and one explicitly has the concept of different tiers of storage." The author is comparing "off-the-shelf" caching with custom caching. They're coming from the assumption that you must be caching somehow and arguing that the word "caching" should be understood to mean only particular approaches to the general idea of caching. And obviously the whole point of the general idea is to optimize things. It's a rhetorical mess

heikkilevanto 1y ago

Caching is simple, yes. The hard part is in the last word, invalidation. Even that is manageable for a single process. But as soon as you have multiple (threads / processes / nodes / data centers) updating the data, it does get quite complex, pretty fast. Likewise, naming things is simple as long as you alone, or a in a small team. But as soon as there are multiple organizations with all their own traditions, it gets tricky. Just witness the eternal flame wars about camelCase, PascalCase, snake_case, kebab-case, and UPPER_CASE. It is almost as hopeless culture clash as Emacs vs Vi vs PowerPoint... (I leave the off-by-one errors as an exercise for the reader)

TeMPOraL 1y ago
I'd say this is not the "naming things" that's hard. Beyond picking a common identifier format in the team, there are at least two dimensions that are much harder: - The language dimension - choice of words, that are good enough for the purpose, and not confusing. For example, "Manager" is as ambiguous as it gets, it can mean many thing, except we've been using it long enough that there's a more specific shape of meaning[0] for that word in code/program architecture contexts - so you still would use it instead of, say "Coordinator", which would raise all kinds of questions that "Manager" no longer does. - The epistemological dimension - whether the word you chose correctly names the concept you meant, and whether the concept you meant is actually the right one to describe the thing you're trying to describe. Ultimately, this is the hard thing at the root of philosophy. In practice, it manifests like e.g. choice between digging into some obscure branches of mathematics to correctly name the thing "endofunctor" or something, or calling it "Square" and saying "fuck it, we'll clarify the exceptions in the comments". -- [0] - I mean "more specific" in the sense it's distinct from the other meanings and somewhat narrow - but still it's fuzzy as heck and you can't describe it fully in words; it's basically tacit knowledge.
- Xss3 1y ago
  I try to name things descriptively in simple terms and often end up with NamesAboutThisLong, once they get too long i know the thing is doing too much and some refactoring is needed for readability. I also avoid letting the reader make assumptions. HasPlayerJumpedRecently is bad. What does recently mean? HasPlayerJumpedInLastTenMs is better, even if it's a bit long...Which highlights that it should probably be refactored into a more flexible value; MsSincePlayerLastJumped. If you arent assuming a time var wth Ms is milliseconds you aren't doing games dev so that one slides with me.

Traubenfuchs 1y ago

I never understood this meme. We use caching a lot, anything that gets cached can only be written by one service each. The writing services emit cache invalidation messages via SNS that cache users must listen to via SQS, to clear/update their cache. Alternatively we cache stuff with just a TTL, when immediate cache invalidation is not important. Where‘s the struggle?

porridgeraisin 1y ago
Here's one: everybody invalidating and refreshing their cache at the same time can cause a thundering herd problem.
hmottestad 1y ago
Does SQS guarantee delivery to all clients? If it does then that’s doing a lot of heavy lifting for you. If it doesn’t guarantee delivery, then I believe you will at some point have a client that reads a cached value thinking it’s still valid because the invalidation message got lost in the network.
- maccard 1y ago
  Eventually. The problem is that eventually delivering that message will result in clients assuming that it will always be the same, when it’s not.
williamdclt 1y ago
You don’t support read-your-own-write and your cache data might be stale for arbitrarily long. These relaxed consistency constraints make caching a lot easier. If that’s acceptable to your use cases then you’re in a great place! If not… well, at scale you often need to find a way for it to be acceptable anyway
pton_xd 1y ago
> Where‘s the struggle? If there are no real consequences for reading stale data, and your writes are infrequent enough, then indeed you're lucky and have a relatively simple problem.

deleted 1y ago

[deleted]

hatthew 1y ago

If you have a system with "slow storage", caching is a way to optimize that to "storage that is sometimes fast". If you have a system with "slow storage" and "fast storage", caching is a way to abstract that away to just "storage". The author is arguing that the latter is the default way we should think about the concept of caching, which is a valid opinion to have.

deleted 1y ago

[deleted]

AdieuToLogic 1y ago

> There's that famous quote "There are only two hard things in Computer Science: cache invalidation and naming things.", and, sure, it's a bit ironical, but there's some truth in there. The joke form of this quote goes along the lines of: There are only two hard things in Computer Science: cache invalidation, naming things, and off-by-one errors. :-D

dcminter 1y ago
I rather like the snark of: there's two hard problems in computer science: we only have one joke and it's not funny. Apparently⁰ by Philip Scott Bowden¹ ⁰ https://martinfowler.com/bliki/TwoHardThings.html https://martinfowler.com/bliki/TwoHardThings.html ¹ https://x.com/pbowden/status/468855097879830528 https://x.com/pbowden/status/468855097879830528
- aorth 1y ago
  Just remembered another one: there are 10 types of people in the world: those who understand binary and those who don't. :)
AndrewOMartin 1y ago
Which leads to > I don't see what's so hard about DNS, it's just cache invalidation and naming things.
- manyaoman 1y ago
  ... and avoiding off-by-one errors.
- dcminter 1y ago
  Oh that's gooood. Got a cite or is it yours?
SAI_Peregrinus 1y ago
My favorite variation only really works in text: There are three hard problems in Computer Science: 1) Cache invalidation 2) Naming th3) Concurings rency 4) Off-by-one errors

deleted 1y ago

[deleted]