10 ms·

Counterfactuals are not causality

89 points by grzm 5y ago

osipov 5y ago
counterfactuals based definition of causality is one of many definitions of causality
pixl97 5y ago
I get asked for root causes all the time. Many times I dont have visibility into the entire system so the best I can give the client a good set of counterfactuals on the bad policy they used to get to that place.
Jiro 5y ago
By this reasoning, "I'm tired because I didn't sleep" is fundamentally different from "I'm tired because I exercised a lot". Also, "the car didn't run because there wasn't enough gas" is fundamentally different from "the car didn't run because the engine broke down".
- kc0bfv 5y ago
  Sure. "I'm tired because - I stayed up all night playing games", or, "I'm tired - because my child screamed all night long," are more along the lines of what the post suggests are useful, and I can see how they're fundamentally more useful to getting at the root cause than the counterfactual, "I'm tired because I didn't sleep." But this example is very simple and makes the argument less important I'd say.
  - yorwba 5y ago
    All three statements using "because" are counterfactual. If you said "I'm tired and I stayed up all night playing games." you're only making a factual claim. But using "X because Y" implies that Y matters for X, i.e. in the counterfactual situation where Y is not true, X wouldn't be true either. The author of TFA seems to suggest that explanations where Y is "not Z" are inherently less useful (I don't think so) and also calls them "counterfactual" even when "not Z" is a fact, which is confusing.
    - kc0bfv 5y ago
      'All three statements using "because" are counterfactual.' No - the definition the article refers to is this: adjective - relating to or expressing what has not happened or is not the case. I used "because" to illustrate the "looking for root cause" use case, like what the article was referring to. Regardless, that doesn't make it counterfactual per the definition the author is using. To me the part that makes the counterfactual less useful when determining root causes is the human element the author mentions - looking for root cause tends to stop once they're brought up (they're the end of the causal chain in discussion), and blame starts to get assigned. I've seen that happen, and it hasn't helped resolve the issue, so avoiding that seems useful. I don't see a problem with using a counterfactual statement if it really is the root cause and I'm blaming myself for something.
- mannykannot 5y ago
  > By this reasoning, "I'm tired because I didn't sleep" is fundamentally different from "I'm tired because I exercised a lot". Well, they are different because they propose different causes. Either of these statements (and the so-called counterfactuals in the article) might be true, depending on the actual cause. The distinction that matters here is that statements of causes are not solutions, and solutions are elicited by different questions than are causes ("what can we do about it?", rather than "why did it happen?") It does not really have anything to do with counterfactuals vs. causes, and the author's actual point seems to be about how to present solutions.
- tshaddox 5y ago
  You're tired because you stayed awake. The volume was full because the admin failed to configure file purging.
- taeric 5y ago
  I thought the counterfactual for that is, "if I didn't didn't exercise, I wouldn't be tried." Which... May not be true.
- deleted 5y ago
  [deleted]
skmurphy 5y ago
The shorthand is "if only" => "next time." Key points from a good article. Definition: "A counterfactual is a statement about how the world might be different now if something had happened differently in the past. It’s a kind of “alternate history” idea." "Here’s the rub: a counterfactual cannot be a cause. By definition the counterfactual did not happen, therefore it cannot have caused anything. Only events that actually occur can be causes of other events. Causality should be stated in a form “Because X then Y”. The statement “If not X then not Y” is not an explanation, it is a kind of wishful thinking about how the past might have unfolded differently." "When performing Five Whys it is important to avoid this counterfactual leap. Stick to the events that actually occurred. [..]" "Try to reformulate the counterfactual as a statement about future prevention:" 1. If we configure file purging, then this won’t happen again 2. If we monitor for “volume full” conditions, then this won’t happen again 3. If we clean up files from old builds, then this won’t happen again. These are useful statements. When formulated this way, they’re clearly talking about the future and not hypothesizing an alternate history. "
- deleted 5y ago
  [deleted]
ihumanable 5y ago
This post would have been stronger if the author had gone back to their original 5 whys and continued it without counterfactuals. - Build didn't happen, why? - Pod didn't start, why? - Disk was full, why? At this point if counterfactuals are off the table, the answers become tautologies. Although if everyone is in agreement that counterfactuals are not helpful, this could become a powerful heuristic in determining that you've hit root cause and can shift from searching for root cause to designing mitigations for the root cause. I guess that's the point of the post, but it seemed weird to start a scenario saying "it goes off the rails here, because of XYZ" and not come back and give the better version of that scenario.
- cwp 5y ago
  Yeah, I was waiting for that too. - A process wrote 125 GB of logs, why? - It was compiled with VERBOSE=true, why? - The build config was changed on 7/12/2021 by Steve, why? - Issue #5988, need to collect data one why that process crashes so much.
benlivengood 5y ago
I mostly agree with this author but I think counterfactual isn't the most useful word in the article. For example "The admin did not configure file purging." isn't a counterfactual unless admins were in fact purging files. A better phrase might be subverted expectations, along with the admonition to make expectations more explicit with the author's if-then examples. Inaccurate expectations are just as important to analyze and fix as hardware and software bugs. Counterfactuals are actually very powerful and useful because they do let us analyze the past instead of just the future. I have found that chasing counterfactuals usually bottoms out in known failure modes in practice. For example tracing back that lack of disk space in build nodes would probably arrive at either "insufficient design" or "insufficient review" or even "insufficient understanding", all of which are pretty well-known and common engineering failure modes and we already know the solutions in the future; spend more time on design, review with more people, ensure education is happening. It's actually quite useful to do this because as a trend over many postmortems "insufficient design" has very different organizational solutions than "insufficient understanding" and it's a meta-level counterfactual of "if we were better at (design|understanding) what problems would we avoid?"
deleted 5y ago
[deleted]
Jtsummers 5y ago
Those aren't counterfactuals, they're negative statements. "Why are the lights off?" "Because I didn't turn them on." That's not a counterfactual, it's a fact. Just like the admins not configuring file purging is a (presumed) fact in the scenario under discussion in the article. Negative statements are not counterfactuals. Now, they may not be helpful for a 5 Whys analysis, but that's a separate thing. Counterfactual: If the admins had turned on file purging, then the volume would not have been full. Why? Because they didn't turn it on so this is a counterfactual in that it is dealing with facts not in existence in the reality under examination and reaching a conclusion of how things would have been different with those different facts. But nearly all the causes in a 5 Whys can be seen as counterfactuals if phrased correctly: "Builds did not complete." Why? "Kubernetes could not start the pod, and the operation timed out after 1 hour." As a counterfactual: If Kurbernetes could have started the pod, then the operation would not have timed out after 1 hour and the builds would have completed.
- np_tedious 5y ago
  I think those counterfactuals can be the most useful explanation, while not being the direct cause. "why did the disk fill up" would be most directly explained by the most recent write. That's not useful though. It doesn't really matter what the last write was.
  - bb88 5y ago
    "The disk filled up because the website was trying to save a configuration backup to disk." -> Stop backing up configurations.
- cwp 5y ago
  "I didn't turn them on" is a fact. It's the "because" that's not a fact. The whole point of the article is that you can't get causality from things that didn't happen. There are seven billion other people in the world who also didn't turn on the lights. What makes your inaction special? In order to get the "because" we have to imagine another state of the world where you DID turn on the lights, and further speculate that this would have been effective to illuminate the room. That's counterfactual, because the fact is that it's dark in here.
nerdponx 5y ago
Ironically, "counterfactual" happens to be a technical term in the subfield of causal analysis within data analysis and statistics. See, e.g. https://david-salazar.github.io/2020/08/10/causality-counterfactuals-clash-of-worlds/ https://david-salazar.github.io/2020/08/10/causality-counter...
- carbocation 5y ago
  Based on the (apparently accidental) use of terms of art in the field, I clicked on this article genuinely expecting to see a refutation of Pearl or something.
  - tpoacher 5y ago
    same. oh well.
- deleted 5y ago
  [deleted]
drdeca 5y ago
This seems to rely on a distinction between positive and negative statements. I'm not sure whether that that distinction works everywhere? (e.g. is "x is less than or equal to y" positive or negative? what about "x is less than y"? The negation of the first of these (under the assumption that x and y are comparable) is "y is greater than x". So, it seems that the two have to have opposite uh, .... I'm going to say "polarities"? But we can also say that "x is less than or equal to y" is the same as "either x is less than y, or x is equal to y", and, err if the conjunction or disjunction of two positive statements is positive, and similarly of two negative statements is a negative statement, ...? I feel like maybe something goes wrong here, but I'm not sure.) ___ If there is a domino, and I push it, and it falls over, I think most will agree that "It fell over because I pushed it.". If I don't push it, this article seems to say that "It didn't fall over because I didn't push it." is not a causal explanation. And, maybe that's right, but I don't think the "because 'if I had pushed it, it would have fallen down' is a counterfactual" is convincing reason for that. Because in the case where "I don't not push it" and it "doesn't stay up", shouldn't "It didn't stay up because I didn't not push it" be a valid causal explanation, being equivalent to "It fell down because I pushed it", yet, in this case, "if I hadn't pushed it, it wouldn't have fallen down" would be a true counterfactual? -- oh, this article is about advice for language use in interpersonal stuff (especially in a work environment), and maybe practical problem solving in general, not studying metaphyics-or-whatever of causation, ok.
wly_cdgr 5y ago
Bro lost me at "Only a finite number of events actually happened..." Actually, he lost me earlier, cos "The admin didn't configure file purging" is something that did happen: the admin failed to take an action. The event happened even if the action didn't; the article confuses the two It doesn't really matter tho since this article is really just a piece of advice on how to handle incident reports in a tactful and constructive manner, not a serious argument about causality
bluecheese33 5y ago
I found this insightful. However, it does seem a lot of the discussion is around the concept of causation by absence. I don't think it directly impacts the point but there is a good paper by Helen BeeBee [1] on the topic for those interested. She discusses some views on how to reason about common sense statements like "Smith's failure to water her office plants caused their death". The salient views covered are: 1. causation by absence is impossible (making the plants statement false) 2. causation by absence is possible and attributing an absence to a cause is sound when the absence is "abnormal". This could be because Smith normally waters the plants, it was Smith's job to water the plants, or Smith made a promise to water the plants. 3. causation by absence is not possible, but it is valid to offer "causal explanations". This is the suggested approach in the paper. A causal explanation provides information about the causal history of an event, but doesn't need to specify a specific node in the causal graph. C causes E is a causal statement. E because C is a "causal explanation". E because C doesn't necessarily imply C causes E. So adopting this approach, The plants died because Smith failed to water them is a true statement that provides some additional explanatory information, Smith failing to water the plants caused their death is not. I think these distinctions may have some minor implications on how you perceive blame. [1] https://philpapers.org/archive/BEECAN.pdf https://philpapers.org/archive/BEECAN.pdf
deltaonefour 5y ago
The author doesn't know what causation is. Heck not even correlation establishes causation so OF course a counterfactual can't establish causation. Not only are counterfactuals infinite in number but ALL hypothesis are infinite in number. You can literally say ANYTHING about why something didn't build. I can say someone didn't purge some files or I can say someone purged files or I can say aliens destroyed the server with a laser. There are an infinite number of statements that can be made here and whether it's a "counterfactual" is irrelevant. What the author doesn't know is that NOTHING can establish causation except for a very specific type of experiment. Basically there is nothing that can establish causality UNLESS you have control over the causal switch. You flick the switch randomly and if the randomness of the switch flicking correlates with the consequence you have then have established causation. BUT you have only established it to a certain correlative degree. It is fundamentally impossible in this reality to establish causation to be 100%. True causation can never be fully verified and at best the experiment I described is the best fuzzy verification. If you truly study how science works you will find that this flaw extends to all of science and therefore reality. In reality as we know it, nothing can be proven. Proof is fundamentally impossible. If you understand this concept then you will truly understand what science is. Anyway, Going to the authors Kubernetes example, his "counterfactuals" are basically null hypothesis and can be helpful in establishing causation but by themselves are pointless. It's simple... If you hypothesize that something didn't build because the admins did not "configure file purging" then to establish causation is SIMPLE. Configure the file purging and see if your system builds. Then remove the configuration and see if the system stops building. Do this enough times and you have established causation or zero causation to a certain degree. That's it. This is science and probability and statistics. So crazy to see someone mention causality without even knowing what it really is.
- zwaps 5y ago
  You hint at it, but causation is a fuzzy concept even with a RCT. Causation is "established" in the usual sense by having a causal model of a data generating process. Such model may or may not identify causal parameters given the data. That is, it is not the "randomized control trial" itself that establishes causation, rather it is a more general concept of statistical identification. Case in point, there are instances where a RCT fails. For example, even if you can assign treatments, you might find that treatment and control groups influence each other or whatever else can go wrong in a RCT. By contrast, there are instances where we do not control the causal switch, but we can observe "natural experiments" nonetheless. Usually, causation is fuzzy, since there are at least some assumptions which need to be made for us to believe the analysis to be causal. However, really, the same is true for an RCT. For example, if you observe an engineer sometimes purging the cache, and sometimes not purging the cache, you do not actually control the assignment of treatment. However, it is easy to think of a mental model where observations of this engineer, and the state of the cluster, gives you high confidence to proclaim: "Purging the cache makes the cluster start!" So, this is just to expand on what you wrote: Causation is not a precise concept, because it depends on our model of the DGP and the data we actually have. Identification of causal parameters depend on these factors, and thereby on assumptions - this is true for RCTs where I control the treatment, just as much for observational studies where I do not. Even if I assign treatments, I have to make the assumption that my assignment is truly random, and that there is no non-random factors influencing my observations (etc). In quite a broad sense, causation as a scientific concept depends on assumptions I need to make. If I get to control the treatment variable (or variable of interest), it is typically easier to create "exogenous" variation that identifies causal parameters... in English: Allows me to establish causation. However, the whole concept does not depend on controlling or not controlling treatment. RCTs can fail, or there are instances in which I can establish and measure causation even without controlling the treatment.
tpoacher 5y ago
This article gets a lot of things backwards. 1. The point of counterfactuals is not to "prove" causality; it's to help you detect possible causal routes, in the presence of given events under examination. E.g. if you know events X, Y and Z co-occurred, and you want to know if X or Y could have possibly caused Z to happen, the counterfactual formulation can answer that question. If not X implies not Z, whereas Y does not alter Z, then you have eliminated Y as a causal agent, and have identified X as a possible causal agent (in fact the only possible causal agent, if Z could not be influenced by another external, unidentified agent, or by itself). Furthermore, in the presence of a number of counterfactual statements, you can allocate responsibility and blame, linked to expectations of events. So aliens is a silly example, unless for whatever reason the author has a very high prior expectation for their existence, which exceeds that of file purging errors.
zwaps 5y ago
I was trying to find out what rubs me the wrong way in this article, and I think it is this: Counterfactuals require a causal model. More so, they belong to at least one causal model. This can be a mental model, a formal one, a precise one or a diffuse one. Nevertheless, counterfactuals "come" with an implied model. People may disagree about which model they talk about, and which model is correct. Such a model may or may not be "precise" enough to identify useful parameters, which is the real rub here. For example, the three "Counterfactuals" the author presents presume a model where purging the cache (etc.) directly causes the problem to go away. Each of those statements might come from the same or from a different causal model. The problem, which I feel make this approach not useful, is that the author has no model in mind. In other words, they switch between different models. One counterfactual is, for instance, "If we don't live on earth, then we don't care about the cluster". Let's ignore that the outcome variable is different. The model with "We are not even alive" as a variable is not a reasonable one. First, it is empirically wrong. Second, it does not "identify" parameters (relationships) of interest. These two points are what we strive for when using a model (which we do when we construct counterfactual, even if it is a verbal model!) The author posits that one can add unlimited counterfactuals. What the author means is that one can add unlimited variables and relationships (however diffuse) to the model. This is true. However, we still need to judge whether the model is good, whether it is helpful. In drawing such analogies to statistics, one should not forget the other concepts. From a statistical standpoint, the author begins with a model that does not identify causal parameters associated with the failure to start the cluster. They then add "counterfactuals", that is, they expand their mental model to new causes. They find that any of these variables may cause a counterfactual where the cluster starts. In my view, the author is only half right. Counterfactuals, as presented here, imply a causal model (perhaps not a precise one). Counterfactuals can be useful, if the underlying model is useful. Whether this is the case, depends largely on factors the author does not discuss in the article. Edit: It may also be helpful to distinguish a counterfactual from "observations with exogenous variation". For example, if we purged the cache some time ago purely by accident, and the cluster did start, we might take this as a hint that purging the cache allows the cluster to run. This is not a counterfactual. It can be used to identify causal chains. If we were sure that nothing else was different (in a certain statistical sense), then we could "identify" a causal path from cache purging to staring the cluster. However, to determine that, we again need a model. In fact, we need a model and assumptions about the observation (what is called data generating process) jointly - because really, they are the same thing. For example, if the cache was purged because an engineer discovered an error somewhere, and then did some other action Y - perhaps it was not the purging of the cache that allowed the cluster to run. It may have been action Y. Here, purging the cache was not "exogenous" enough for us to infer the correct reason. Identification of a model/DPG is a deep concept, but a crucial one to finding cause and effect.
erdii 5y ago
Since no-one else posted it: The Five Dirty Words of CI - J. Paul Reed - https://www.youtube.com/watch?v=ZXXaCCbpNYw https://www.youtube.com/watch?v=ZXXaCCbpNYw
morelandjs 5y ago
Is it fair to say that the counterfactual is hypothesizing the B in an A/B test? I.e. I administer a pill with an active drug to a treatment cohort. What would have happened had I administered no drug (sugar pill) to the same treatment cohort.
a_humean 5y ago
Funny enough, reductions of the causal relation to a counterfactual one is a common theme in philosophy (and adjacently in statistical sciences - see Pearl). Most philosophers would contend that a true counterfactual can be sufficient condition for causation, but not a necessary one. From that seed there are quite a few different accounts (David Lewis' being the most famous) that attempt to reduce cauation to counterfactual relations + something else: https://plato.stanford.edu/entries/causation-counterfactual/ https://plato.stanford.edu/entries/causation-counterfactual/ Its still a somewhat live debate in contemporary philosophy. The medicine saved the patient's life is true if the patient would have died had the medicine not been administered. How do we determine the truth of such a counterfactual? In practice we do large scale randomised trails to attempt to simulate a counterfactual to test the causal powers of the the medince - but its probably beyond our powers to know for certain that the counterfactual is true in the singular case. Philosophers are interested in what makes a counterfactual truth or false, and whether understanding a counterfactual presupposes causation or whether a causation can be understood in terms of counterfactuals. Nobody doubts that counterfactuals and causation are releated, and few doubt that counterfactuals can be reduced to their logical or epistemic counsins.
- jfengel 5y ago
  The counterfactual has never seemed epistemologically satisfying to me, because of the way the influence spreads. If I had in fact flicked the light switch, what is taking up the empty space where my body would have been? What happens at the power plant which is now generating a bit more current than it was before? Implicitly there's this idea that every counterfactual can be contained and localized, but that runs counter to my understanding. Change one thing and you change the entire universe. That's true even classically -- even before you bring in quantum physics and its problems with counterfactuals. Perhaps counterfactuals are a good model of the psychology of causation. And in fact, I suspect that psychology is as deep as causation runs; it doesn't in fact have a direct physical or epistemic analogue. All of which is fine and important, but I feel like it diminishes the value of the effort put into it. It's not studying VLTIMATE TRVTH, just a quirk of how brains think about stuff. And that would mean you need to use a very different toolset to work on it.
tom-thistime 5y ago
Among other problems, the author seems to be confusing counterfactuals with negative statements.
WesolyKubeczek 5y ago
The lawmakers should read this article, take it to their hearts and rewrite criminal codes all over the world to not punish people for the lack of action.

89 points by grzm 5y ago

counterfactuals based definition of causality is one of many definitions of causality

I get asked for root causes all the time. Many times I dont have visibility into the entire system so the best I can give the client a good set of counterfactuals on the bad policy they used to get to that place.

Jiro 5y ago

By this reasoning, "I'm tired because I didn't sleep" is fundamentally different from "I'm tired because I exercised a lot". Also, "the car didn't run because there wasn't enough gas" is fundamentally different from "the car didn't run because the engine broke down".

kc0bfv 5y ago
Sure. "I'm tired because - I stayed up all night playing games", or, "I'm tired - because my child screamed all night long," are more along the lines of what the post suggests are useful, and I can see how they're fundamentally more useful to getting at the root cause than the counterfactual, "I'm tired because I didn't sleep." But this example is very simple and makes the argument less important I'd say.
- yorwba 5y ago
  All three statements using "because" are counterfactual. If you said "I'm tired and I stayed up all night playing games." you're only making a factual claim. But using "X because Y" implies that Y matters for X, i.e. in the counterfactual situation where Y is not true, X wouldn't be true either. The author of TFA seems to suggest that explanations where Y is "not Z" are inherently less useful (I don't think so) and also calls them "counterfactual" even when "not Z" is a fact, which is confusing.
  - kc0bfv 5y ago
    'All three statements using "because" are counterfactual.' No - the definition the article refers to is this: adjective - relating to or expressing what has not happened or is not the case. I used "because" to illustrate the "looking for root cause" use case, like what the article was referring to. Regardless, that doesn't make it counterfactual per the definition the author is using. To me the part that makes the counterfactual less useful when determining root causes is the human element the author mentions - looking for root cause tends to stop once they're brought up (they're the end of the causal chain in discussion), and blame starts to get assigned. I've seen that happen, and it hasn't helped resolve the issue, so avoiding that seems useful. I don't see a problem with using a counterfactual statement if it really is the root cause and I'm blaming myself for something.
mannykannot 5y ago
> By this reasoning, "I'm tired because I didn't sleep" is fundamentally different from "I'm tired because I exercised a lot". Well, they are different because they propose different causes. Either of these statements (and the so-called counterfactuals in the article) might be true, depending on the actual cause. The distinction that matters here is that statements of causes are not solutions, and solutions are elicited by different questions than are causes ("what can we do about it?", rather than "why did it happen?") It does not really have anything to do with counterfactuals vs. causes, and the author's actual point seems to be about how to present solutions.
tshaddox 5y ago
You're tired because you stayed awake. The volume was full because the admin failed to configure file purging.
taeric 5y ago
I thought the counterfactual for that is, "if I didn't didn't exercise, I wouldn't be tried." Which... May not be true.
deleted 5y ago
[deleted]

skmurphy 5y ago

The shorthand is "if only" => "next time." Key points from a good article. Definition: "A counterfactual is a statement about how the world might be different now if something had happened differently in the past. It’s a kind of “alternate history” idea." "Here’s the rub: a counterfactual cannot be a cause. By definition the counterfactual did not happen, therefore it cannot have caused anything. Only events that actually occur can be causes of other events. Causality should be stated in a form “Because X then Y”. The statement “If not X then not Y” is not an explanation, it is a kind of wishful thinking about how the past might have unfolded differently." "When performing Five Whys it is important to avoid this counterfactual leap. Stick to the events that actually occurred. [..]" "Try to reformulate the counterfactual as a statement about future prevention:" 1. If we configure file purging, then this won’t happen again 2. If we monitor for “volume full” conditions, then this won’t happen again 3. If we clean up files from old builds, then this won’t happen again. These are useful statements. When formulated this way, they’re clearly talking about the future and not hypothesizing an alternate history. "

deleted 5y ago
[deleted]

ihumanable 5y ago

This post would have been stronger if the author had gone back to their original 5 whys and continued it without counterfactuals. - Build didn't happen, why? - Pod didn't start, why? - Disk was full, why? At this point if counterfactuals are off the table, the answers become tautologies. Although if everyone is in agreement that counterfactuals are not helpful, this could become a powerful heuristic in determining that you've hit root cause and can shift from searching for root cause to designing mitigations for the root cause. I guess that's the point of the post, but it seemed weird to start a scenario saying "it goes off the rails here, because of XYZ" and not come back and give the better version of that scenario.

cwp 5y ago
Yeah, I was waiting for that too. - A process wrote 125 GB of logs, why? - It was compiled with VERBOSE=true, why? - The build config was changed on 7/12/2021 by Steve, why? - Issue #5988, need to collect data one why that process crashes so much.

benlivengood 5y ago

I mostly agree with this author but I think counterfactual isn't the most useful word in the article. For example "The admin did not configure file purging." isn't a counterfactual unless admins were in fact purging files. A better phrase might be subverted expectations, along with the admonition to make expectations more explicit with the author's if-then examples. Inaccurate expectations are just as important to analyze and fix as hardware and software bugs. Counterfactuals are actually very powerful and useful because they do let us analyze the past instead of just the future. I have found that chasing counterfactuals usually bottoms out in known failure modes in practice. For example tracing back that lack of disk space in build nodes would probably arrive at either "insufficient design" or "insufficient review" or even "insufficient understanding", all of which are pretty well-known and common engineering failure modes and we already know the solutions in the future; spend more time on design, review with more people, ensure education is happening. It's actually quite useful to do this because as a trend over many postmortems "insufficient design" has very different organizational solutions than "insufficient understanding" and it's a meta-level counterfactual of "if we were better at (design|understanding) what problems would we avoid?"

deleted 5y ago

[deleted]

Jtsummers 5y ago

Those aren't counterfactuals, they're negative statements. "Why are the lights off?" "Because I didn't turn them on." That's not a counterfactual, it's a fact. Just like the admins not configuring file purging is a (presumed) fact in the scenario under discussion in the article. Negative statements are not counterfactuals. Now, they may not be helpful for a 5 Whys analysis, but that's a separate thing. Counterfactual: If the admins had turned on file purging, then the volume would not have been full. Why? Because they didn't turn it on so this is a counterfactual in that it is dealing with facts not in existence in the reality under examination and reaching a conclusion of how things would have been different with those different facts. But nearly all the causes in a 5 Whys can be seen as counterfactuals if phrased correctly: "Builds did not complete." Why? "Kubernetes could not start the pod, and the operation timed out after 1 hour." As a counterfactual: If Kurbernetes could have started the pod, then the operation would not have timed out after 1 hour and the builds would have completed.

np_tedious 5y ago
I think those counterfactuals can be the most useful explanation, while not being the direct cause. "why did the disk fill up" would be most directly explained by the most recent write. That's not useful though. It doesn't really matter what the last write was.
- bb88 5y ago
  "The disk filled up because the website was trying to save a configuration backup to disk." -> Stop backing up configurations.
cwp 5y ago
"I didn't turn them on" is a fact. It's the "because" that's not a fact. The whole point of the article is that you can't get causality from things that didn't happen. There are seven billion other people in the world who also didn't turn on the lights. What makes your inaction special? In order to get the "because" we have to imagine another state of the world where you DID turn on the lights, and further speculate that this would have been effective to illuminate the room. That's counterfactual, because the fact is that it's dark in here.

nerdponx 5y ago

Ironically, "counterfactual" happens to be a technical term in the subfield of causal analysis within data analysis and statistics. See, e.g. https://david-salazar.github.io/2020/08/10/causality-counterfactuals-clash-of-worlds/ https://david-salazar.github.io/2020/08/10/causality-counter...

carbocation 5y ago
Based on the (apparently accidental) use of terms of art in the field, I clicked on this article genuinely expecting to see a refutation of Pearl or something.
- tpoacher 5y ago
  same. oh well.
deleted 5y ago
[deleted]

drdeca 5y ago

This seems to rely on a distinction between positive and negative statements. I'm not sure whether that that distinction works everywhere? (e.g. is "x is less than or equal to y" positive or negative? what about "x is less than y"? The negation of the first of these (under the assumption that x and y are comparable) is "y is greater than x". So, it seems that the two have to have opposite uh, .... I'm going to say "polarities"? But we can also say that "x is less than or equal to y" is the same as "either x is less than y, or x is equal to y", and, err if the conjunction or disjunction of two positive statements is positive, and similarly of two negative statements is a negative statement, ...? I feel like maybe something goes wrong here, but I'm not sure.) ___ If there is a domino, and I push it, and it falls over, I think most will agree that "It fell over because I pushed it.". If I don't push it, this article seems to say that "It didn't fall over because I didn't push it." is not a causal explanation. And, maybe that's right, but I don't think the "because 'if I had pushed it, it would have fallen down' is a counterfactual" is convincing reason for that. Because in the case where "I don't not push it" and it "doesn't stay up", shouldn't "It didn't stay up because I didn't not push it" be a valid causal explanation, being equivalent to "It fell down because I pushed it", yet, in this case, "if I hadn't pushed it, it wouldn't have fallen down" would be a true counterfactual? -- oh, this article is about advice for language use in interpersonal stuff (especially in a work environment), and maybe practical problem solving in general, not studying metaphyics-or-whatever of causation, ok.

wly_cdgr 5y ago

Bro lost me at "Only a finite number of events actually happened..." Actually, he lost me earlier, cos "The admin didn't configure file purging" is something that did happen: the admin failed to take an action. The event happened even if the action didn't; the article confuses the two It doesn't really matter tho since this article is really just a piece of advice on how to handle incident reports in a tactful and constructive manner, not a serious argument about causality

bluecheese33 5y ago

I found this insightful. However, it does seem a lot of the discussion is around the concept of causation by absence. I don't think it directly impacts the point but there is a good paper by Helen BeeBee [1] on the topic for those interested. She discusses some views on how to reason about common sense statements like "Smith's failure to water her office plants caused their death". The salient views covered are: 1. causation by absence is impossible (making the plants statement false) 2. causation by absence is possible and attributing an absence to a cause is sound when the absence is "abnormal". This could be because Smith normally waters the plants, it was Smith's job to water the plants, or Smith made a promise to water the plants. 3. causation by absence is not possible, but it is valid to offer "causal explanations". This is the suggested approach in the paper. A causal explanation provides information about the causal history of an event, but doesn't need to specify a specific node in the causal graph. C causes E is a causal statement. E because C is a "causal explanation". E because C doesn't necessarily imply C causes E. So adopting this approach, The plants died because Smith failed to water them is a true statement that provides some additional explanatory information, Smith failing to water the plants caused their death is not. I think these distinctions may have some minor implications on how you perceive blame. [1] https://philpapers.org/archive/BEECAN.pdf https://philpapers.org/archive/BEECAN.pdf

deltaonefour 5y ago

The author doesn't know what causation is. Heck not even correlation establishes causation so OF course a counterfactual can't establish causation. Not only are counterfactuals infinite in number but ALL hypothesis are infinite in number. You can literally say ANYTHING about why something didn't build. I can say someone didn't purge some files or I can say someone purged files or I can say aliens destroyed the server with a laser. There are an infinite number of statements that can be made here and whether it's a "counterfactual" is irrelevant. What the author doesn't know is that NOTHING can establish causation except for a very specific type of experiment. Basically there is nothing that can establish causality UNLESS you have control over the causal switch. You flick the switch randomly and if the randomness of the switch flicking correlates with the consequence you have then have established causation. BUT you have only established it to a certain correlative degree. It is fundamentally impossible in this reality to establish causation to be 100%. True causation can never be fully verified and at best the experiment I described is the best fuzzy verification. If you truly study how science works you will find that this flaw extends to all of science and therefore reality. In reality as we know it, nothing can be proven. Proof is fundamentally impossible. If you understand this concept then you will truly understand what science is. Anyway, Going to the authors Kubernetes example, his "counterfactuals" are basically null hypothesis and can be helpful in establishing causation but by themselves are pointless. It's simple... If you hypothesize that something didn't build because the admins did not "configure file purging" then to establish causation is SIMPLE. Configure the file purging and see if your system builds. Then remove the configuration and see if the system stops building. Do this enough times and you have established causation or zero causation to a certain degree. That's it. This is science and probability and statistics. So crazy to see someone mention causality without even knowing what it really is.

zwaps 5y ago
You hint at it, but causation is a fuzzy concept even with a RCT. Causation is "established" in the usual sense by having a causal model of a data generating process. Such model may or may not identify causal parameters given the data. That is, it is not the "randomized control trial" itself that establishes causation, rather it is a more general concept of statistical identification. Case in point, there are instances where a RCT fails. For example, even if you can assign treatments, you might find that treatment and control groups influence each other or whatever else can go wrong in a RCT. By contrast, there are instances where we do not control the causal switch, but we can observe "natural experiments" nonetheless. Usually, causation is fuzzy, since there are at least some assumptions which need to be made for us to believe the analysis to be causal. However, really, the same is true for an RCT. For example, if you observe an engineer sometimes purging the cache, and sometimes not purging the cache, you do not actually control the assignment of treatment. However, it is easy to think of a mental model where observations of this engineer, and the state of the cluster, gives you high confidence to proclaim: "Purging the cache makes the cluster start!" So, this is just to expand on what you wrote: Causation is not a precise concept, because it depends on our model of the DGP and the data we actually have. Identification of causal parameters depend on these factors, and thereby on assumptions - this is true for RCTs where I control the treatment, just as much for observational studies where I do not. Even if I assign treatments, I have to make the assumption that my assignment is truly random, and that there is no non-random factors influencing my observations (etc). In quite a broad sense, causation as a scientific concept depends on assumptions I need to make. If I get to control the treatment variable (or variable of interest), it is typically easier to create "exogenous" variation that identifies causal parameters... in English: Allows me to establish causation. However, the whole concept does not depend on controlling or not controlling treatment. RCTs can fail, or there are instances in which I can establish and measure causation even without controlling the treatment.

tpoacher 5y ago

This article gets a lot of things backwards. 1. The point of counterfactuals is not to "prove" causality; it's to help you detect possible causal routes, in the presence of given events under examination. E.g. if you know events X, Y and Z co-occurred, and you want to know if X or Y could have possibly caused Z to happen, the counterfactual formulation can answer that question. If not X implies not Z, whereas Y does not alter Z, then you have eliminated Y as a causal agent, and have identified X as a possible causal agent (in fact the only possible causal agent, if Z could not be influenced by another external, unidentified agent, or by itself). Furthermore, in the presence of a number of counterfactual statements, you can allocate responsibility and blame, linked to expectations of events. So aliens is a silly example, unless for whatever reason the author has a very high prior expectation for their existence, which exceeds that of file purging errors.

zwaps 5y ago

I was trying to find out what rubs me the wrong way in this article, and I think it is this: Counterfactuals require a causal model. More so, they belong to at least one causal model. This can be a mental model, a formal one, a precise one or a diffuse one. Nevertheless, counterfactuals "come" with an implied model. People may disagree about which model they talk about, and which model is correct. Such a model may or may not be "precise" enough to identify useful parameters, which is the real rub here. For example, the three "Counterfactuals" the author presents presume a model where purging the cache (etc.) directly causes the problem to go away. Each of those statements might come from the same or from a different causal model. The problem, which I feel make this approach not useful, is that the author has no model in mind. In other words, they switch between different models. One counterfactual is, for instance, "If we don't live on earth, then we don't care about the cluster". Let's ignore that the outcome variable is different. The model with "We are not even alive" as a variable is not a reasonable one. First, it is empirically wrong. Second, it does not "identify" parameters (relationships) of interest. These two points are what we strive for when using a model (which we do when we construct counterfactual, even if it is a verbal model!) The author posits that one can add unlimited counterfactuals. What the author means is that one can add unlimited variables and relationships (however diffuse) to the model. This is true. However, we still need to judge whether the model is good, whether it is helpful. In drawing such analogies to statistics, one should not forget the other concepts. From a statistical standpoint, the author begins with a model that does not identify causal parameters associated with the failure to start the cluster. They then add "counterfactuals", that is, they expand their mental model to new causes. They find that any of these variables may cause a counterfactual where the cluster starts. In my view, the author is only half right. Counterfactuals, as presented here, imply a causal model (perhaps not a precise one). Counterfactuals can be useful, if the underlying model is useful. Whether this is the case, depends largely on factors the author does not discuss in the article. Edit: It may also be helpful to distinguish a counterfactual from "observations with exogenous variation". For example, if we purged the cache some time ago purely by accident, and the cluster did start, we might take this as a hint that purging the cache allows the cluster to run. This is not a counterfactual. It can be used to identify causal chains. If we were sure that nothing else was different (in a certain statistical sense), then we could "identify" a causal path from cache purging to staring the cluster. However, to determine that, we again need a model. In fact, we need a model and assumptions about the observation (what is called data generating process) jointly - because really, they are the same thing. For example, if the cache was purged because an engineer discovered an error somewhere, and then did some other action Y - perhaps it was not the purging of the cache that allowed the cluster to run. It may have been action Y. Here, purging the cache was not "exogenous" enough for us to infer the correct reason. Identification of a model/DPG is a deep concept, but a crucial one to finding cause and effect.

erdii 5y ago

Since no-one else posted it: The Five Dirty Words of CI - J. Paul Reed - https://www.youtube.com/watch?v=ZXXaCCbpNYw https://www.youtube.com/watch?v=ZXXaCCbpNYw

morelandjs 5y ago

Is it fair to say that the counterfactual is hypothesizing the B in an A/B test? I.e. I administer a pill with an active drug to a treatment cohort. What would have happened had I administered no drug (sugar pill) to the same treatment cohort.

a_humean 5y ago

Funny enough, reductions of the causal relation to a counterfactual one is a common theme in philosophy (and adjacently in statistical sciences - see Pearl). Most philosophers would contend that a true counterfactual can be sufficient condition for causation, but not a necessary one. From that seed there are quite a few different accounts (David Lewis' being the most famous) that attempt to reduce cauation to counterfactual relations + something else: https://plato.stanford.edu/entries/causation-counterfactual/ https://plato.stanford.edu/entries/causation-counterfactual/ Its still a somewhat live debate in contemporary philosophy. The medicine saved the patient's life is true if the patient would have died had the medicine not been administered. How do we determine the truth of such a counterfactual? In practice we do large scale randomised trails to attempt to simulate a counterfactual to test the causal powers of the the medince - but its probably beyond our powers to know for certain that the counterfactual is true in the singular case. Philosophers are interested in what makes a counterfactual truth or false, and whether understanding a counterfactual presupposes causation or whether a causation can be understood in terms of counterfactuals. Nobody doubts that counterfactuals and causation are releated, and few doubt that counterfactuals can be reduced to their logical or epistemic counsins.

jfengel 5y ago
The counterfactual has never seemed epistemologically satisfying to me, because of the way the influence spreads. If I had in fact flicked the light switch, what is taking up the empty space where my body would have been? What happens at the power plant which is now generating a bit more current than it was before? Implicitly there's this idea that every counterfactual can be contained and localized, but that runs counter to my understanding. Change one thing and you change the entire universe. That's true even classically -- even before you bring in quantum physics and its problems with counterfactuals. Perhaps counterfactuals are a good model of the psychology of causation. And in fact, I suspect that psychology is as deep as causation runs; it doesn't in fact have a direct physical or epistemic analogue. All of which is fine and important, but I feel like it diminishes the value of the effort put into it. It's not studying VLTIMATE TRVTH, just a quirk of how brains think about stuff. And that would mean you need to use a very different toolset to work on it.

tom-thistime 5y ago

Among other problems, the author seems to be confusing counterfactuals with negative statements.

WesolyKubeczek 5y ago

The lawmakers should read this article, take it to their hearts and rewrite criminal codes all over the world to not punish people for the lack of action.