6 ms·
>The language of algebra is symmetric: If x tells us about y, then y tells us about x. I’m talking about deterministic relationships. There’s no way to write in
by daenz 8y ago
>The language of algebra is symmetric: If x tells us about y, then y tells us about x. I’m talking about deterministic relationships. There’s no way to write in mathematics a simple fact—for example, that the upcoming storm causes the barometer to go down, and not the other way around.
Is this true? It kind of blows my mind if it is.
- kabbalf 8y agoSeems like it's a fallacy of limiting oneself to stationary reasoning. Differential equations and difference equations that capture temporal and causual behaviors are a thing. A large part of control theory is model identification, which tries to fit a temporal model to data. In a difference equation x(t+1) = f(x(t),u(t)), u causes changes to x and not the other way around
- PeterisP 8y agoIf we don't know that the true equation is x(t+1) = f(x(t),u(t)) but have to infer it from data, then observing that such an equation seems to hold true for some particular u and x does not mean that u causes changes to x. It may simply be a predictor of x, as in the 'barometer readings may precede and imply a storm but not cause one' example given by others above.
- tgb 8y agoDifferential equations say nothing about causality they just describe what happens. Please remember that Pearl literally wrote the book on this topic before assuming he missed an elementary counter example.
- salty_biscuits 8y agoI feel like reversiblity is a big complication here. Very interesting and subtle area.
- tgb 8y agoI don't see the relevance. You can have (non-differential )algebraic equations that involve a "time" parameter. They can be non-reversible in the sense that I think you mean - that knowing a point "back" in time tells you values in the future but knowing a value in the future does not (uniquely) determine the solution before that point. [For example x^2 = t^2 for t <= 0 and x^2 = 0 for t >= 0. There are two continuous solutions x(t) which are equal at say t=1.] For differential or non-differential equations, they're still just describing how something is and not the causality behind it. It's always possible that the equation is merely a result of hidden variables and there is no casual relationship between any two points of the solution in any meaningful sense.
- salty_biscuits 8y agoI meant in how people talk about "causality", it is always poorly defined. It seems like there is a implicit assumption about irreversible systems at play.
- throwaway37585 8y agoDid you mean that, under reversibility + determinism, the future “causes” the past just as much as the past “causes” the future?
- salty_biscuits 8y agoFor some closed physical systems this is pretty much true, but I think the way people think about the world is at odds with this. That is why we find things like this so surprising https://youtu.be/p08_KlTKP50 https://youtu.be/p08_KlTKP50 It gets even murkier when you think of statements like "Hitler coming into power caused world war 2". There are so many things going on in that system that it couldn't possibly be true (e.g. if you change Hitler out for another person maybe world war 2 still happens), but works as a plausible line of causal reasoning for a lot of people.
- virmundi 8y agoCan't this be modeled with baysian math? The chance that the storm causes the pressure to drop is .99. there could be some slight uncertainty included.
- srean 8y agoNo because conditioning is fundamentally a 'selection' operation. It selects/filters the full data on, say, a predicate (say gender==male) and analyses that subset. This is fundamentally different from an intervention where you turn someone into 'male' who wasn't male to begin with. Those who were turned male might behave differently from those who were male without any intervention -- the sub-population you selected for defining conditional probabilities. These two scenarios have the potential to exhibit different behavior -- probabilistic models without a notion of intervention or counterfactuals will only capture the former. But just like this 'selection' operator you c an define an analogous operator for -- selecting on those that you interved on -- then you are in the realms of causality.
- georgrwasington 8y agohe's ignoring expressions. it's not true.
- tlarkworthy 8y agoIt's a very interestiny topic. Passive observation can never determine causality ... But active experimentation with control variables can! This is super important, you need an active agent running experiments to narrow the causal structure e.g. a robot. More data collection doesn't help without a feedback loop
- xvedejas 8y agoThis is not true! There are techniques to determine causality passively. Here is one such example, which takes advantage of the behavior of noise to choose between "temperature causes altitude" and "altitude causes temperature": [PDF] http://papers.nips.cc/paper/3548-nonlinear-causal-discovery-with-additive-noise-models.pdf http://papers.nips.cc/paper/3548-nonlinear-causal-discovery-...
- tlarkworthy 8y agoI think that paper is using a heuristic that, for some problems, is a good guess which direction the causal direciton is. Use your common sense though. Do you think we would have built the modern world (e.g. electricity) by passive observation of the world, or does really getting to the truth require controlled experiments?
- xvedejas 8y agoAll experiments require repeat measurements for confidence. None of this means that passive methods are impossible, which is what you were saying earlier.
- tlarkworthy 8y agoThat paper had no proof attached, because it's a heuristic and can easily be fooled by a bad setting. So maybe some causal relationship that are obvious can be learnt passively. But what I am saying is entirely consistent with what Pearl is saying in the article. To get to the next level of ai you need an experimenting agent. Passive methods in general are impossible for untangling causal relationship.
- omalleyt 8y agoI think there was some inevitable loss of nuance as that got translated into layman's terms. For all practical purposes, you can write causality like p(x|y) = 1 and time(y) < time(x) I.e. causality is just when one event always happens after another event. Any additional requirements for causality are basically philosophy. But typical ML systems don't construct networks of causal relations, is basically what he's getting at from my reading of it
- kgwgk 8y agop(sun rises|rooster crows)=1 Is the rooster causing the dawn?
- sjg007 8y agoExcept if the rooster is dead or you put the rooster in a dark room.
- kgwgk 8y agop(sun rises|rooster crows)=1 remains valid when “rooster crows” is false.
- CuriouslyC 8y agoI think you'd have a fair argument if you considered a complement, i.e. P(sun rises|no rooster crowing) = 0.
- imh 8y ago> For all practical purposes, you can write causality like > p(x|y) = 1 and time(y) < time(x) This isn't true at all. For a counterexample, x and y may both have a common cause. Pearl's work is on this and extends the language to talk about p(y | do(x)), meaning that you talk about what happens to y when you take some hypothetical intervention to change x. Causation framed in terms of intervention talks about "what if it had been this instead of that?" and is probably the most common model of causation. For more info look up the rubin causal model, the potential outcomes framework, and pearls's "do" notation.
- pjmorris 8y agoHe's written an entire book [0] on how to add notions of causality to the existing algebraic framework for expressing correlation. He alludes to the core idea in the next part of the interview: "Mathematics has not developed the asymmetric language required to capture our understanding that if x causes y that does not mean that y causes x. It sounds like a terrible thing to say against science, I know. If I were to say it to my mother, she’d slap me." His home page [1] links to several presentations (e.g. [2]) where he lays out the key ideas. [0] "Causality: Models, Reasoning and Inference", Pearl [1] http://bayes.cs.ucla.edu/jp_home.html http://bayes.cs.ucla.edu/jp_home.html [2] http://bayes.cs.ucla.edu/IJCAI99/ijcai-99.pdf http://bayes.cs.ucla.edu/IJCAI99/ijcai-99.pdf
- threatofrain 8y agoWhat about the various concurrency algebras that basically have some composable notion of "happened before"? Aren't these developments at least 30 years old?
- qmalzp 8y agoAs a former mathematican, I was at first a little offended and dismissive of his claim. But, perhaps what one can say is that mathematicians don't seem to distinguish "causation" with "implication". After all, if the barometer goes down, that does imply a storm is coming (perhaps with some increased probability), but it still doesn't cause the storm to come (even with increased probability). In a simplified closed system, where all you have are barometers and storms, maybe there is no difference between implication and causation; all you know is these variables are correlated. Perhaps once you take every atom in the universe into account, the two start to look the same.
- goatlover 8y agoThat can't be right, because you can take the barometers out of the closed system, and it will still storm. Correlation isn't causation, and for good reasons.
- ordu 8y ago> In a simplified closed system, where all you have are barometers and storms, maybe there is no difference between implication and causation; all you know is these variables are correlated. Perhaps once you take every atom in the universe into account, the two start to look the same. Implication it is a very mathematical thing. It is like you know, that y=f(x), and then you write x=g(x), where g is inverse of f. It works both ways, there are no cause, no effect, just link between two variables. If you use math to reason about causal links in reality, you need to use some implicit knowledge which is not represented in formula. It doesn't means that math is bad. Geometry likes euclidian space while we know from Einstein that our space is not euclidian one -- it doesn't mean that geometry is bad. Euclidian geometry just solves some specific problems and doesn't solve others. Causal link reflects ability to change dependant variable by changing independant one. It is not something like "fundamental property of the Universe", it is our subjective way to structure information about the reality. It seems to me, that physicists believe the other way, that causation is the inherent property of reality. Maybe they are right in their field, but it doesn't work in everyday life. Causal link is an abstraction that helps us to know what we can do to change outcomes. In this sense there are no causal link between barometer readings and a storm: if you change barometer readings to reflect a fine weather the storm will come anyway. Maybe there is causal link between atmosphetic pressure and a storm? I do not know it, because I see no way to change atmospheric pressure and I'm not educated well enough to understand scientific weather models. Though it is relatively safe for me to believe that low atmospheric pressure causes storm: causational link or correlational one -- it will not change my behaviour, because I cannot change atmospheric pressure. If I'll find a way, than it would be cruicial to figure out the kind of the link, because I'll be able to break something if I'm wrong. But I said that it is relatively safe, because if I suppose that link is causational, I would use that link differently while reasoning about the weather, it will change my other beliefs and probably it will change my behaviour somehow. So, the main idea is: causation is just our way to structure reality. We are free to choose which links are causal, it is all up to us. If we think it will help us to reason about reality, then we should speak about casuality. And the most important difference between causation and correlation is the ability to change dependant variable by changing independant. If we can change dependant variable that way, than we should mark link as causational. If we cannot, that we should think about that link as about correlational. Implication just do not draw this difference.
- Izkata 8y agoIt kinda sounds like, despite his accolades, his experience is limited to a strict set of mathematics. For example, logical implication is what he suggests doesn't exist, but it's over in the "logic" category, not pure mathematics like algebra. Association rule mining [0] is a whole category about finding such asymmetric implications. Also: > But I’m asking about the future—what next? Can you have a robot scientist that would plan an experiment and find new answers to pending scientific questions? That’s the next step. Nearly 10 years ago [1], a robot called Adam both made and tested hypotheses about yeast. Certainly not a general AI, or even an award-winning massive breakthrough, but it's a good step in a direction that he doesn't think exists yet. --- [0] https://en.wikipedia.org/wiki/Association_rule_learning https://en.wikipedia.org/wiki/Association_rule_learning [1] http://www.sciencemag.org/news/2009/04/robotic-scientists-make-first-discoveries http://www.sciencemag.org/news/2009/04/robotic-scientists-ma...
- abalaji 8y agoI might be wrong but x -> y (read as x implies y) is something taught in discrete mathematics classes. I'm not too sure what he means by mathematics does not have notation for causality. https://en.wikipedia.org/wiki/Material_conditional https://en.wikipedia.org/wiki/Material_conditional
- bsaul 8y agoA barometer changing implies the storm is coming. It doesn’t however causes the storm to come.