13 ms·

How Judea Pearl Became One of AI's Sharpest Critics

308 points by stevenwoo 8y ago

eksemplar 8y ago
In history we call this determinism. The more you know about a historic choice and the complex mechanisms around it, the more it makes perfect sense while at the same time leaving you absolutely clueless about the why. Christianity being chosen by the Roman Empire is the typical example. To most people the choice makes perfect sense, because we look back at what it brought with it. But when you put yourself in the heads of the decision makers and look at all the options they had, well, it makes no sense at all. A lot of machine learning tells us trends, but it tells us nothing about the why, and I completely agree with the article about how useless that data is. I mean, it’s probably great at harmless things, but when my elaborate online profile still can’t figure out why I happen to read a cultural, artsy but somewhat conservative news paper, despite the fact that my data shows the algorithm that I really really shouldn’t be doing that, well, then we simply can’t use ML for any form of decision making or even as an advisor. At least not in the public sector.
- sdenton4 8y ago
  Yeah, I think it's worth also asking whether humans /actually/ are any good at answering the 'why' with anything but bullshit. I would argue that we're pretty good at understanding causality in very limited circumstances (the window broke because that kid threw the ball), and extremely overconfident in our ability to understand causation in a much broader range of circumstances (the stock price went up because...). This overconfidence drives a lot of the decisions we make, for better or for worse. It's an area where if we push hard on AI, we'll likely have to come to terms with how bad we are in this area, and ask ourselves whether we feel comfortable deploying 'thinking machines' with similar levels of incompetence and/or arrogance.
BenoitP 8y ago
If someone can ELI5 to me what is Pearl's do-calculus, that'd be quite great. I have tried to build an understanding of it since he got the Turing prize, but have failed so far.
- cbames89 8y ago
  It's a way to take interventions, for example what would happen if we can cigarettes, and model it with observational data. The do-calculus is just a couple of rules which say when we can do this. It essentially boils down to a couple of substitution rules.
- imh 8y ago
  Rain causes people to carry umbrellas. Rain causes puddles. Probabilistically, there's a strong connection between me carrying an umbrella and you seeing puddles (assuming we're both in SF or whatever). To be a bayesian, you could model this as a conditional probability: p(you see a puddle | I carry an umbrella). It will look like a strong connection, but it isn't a causal one. If it were, then I could stop carrying an umbrella and clear away the puddles. That intervention of me changing my umbrella carrying behavior is what causation is all about. If we change this, does that change? So then you talk about the probability of you seeing a puddle given some intervention that forces me to carry my umbrella regardless of anything else. We see that if you force me, independent of rain, to carry an umbrella or not to, then the connection between the umbrella and puddles is gone. p(puddles | do(umbrella)) != p(puddles | umbrella). do(X) means to take an intervention and force X regardless of other things. As contrast, you can talk about the connection between rain and and puddles. If there were some hypothetical weather machine where we could force rain or sunshine, then you'd see that intervening and forcing rain (a.k.a. do(rain)) still keeps the relationship with puddles. p(puddles | do(rain)) still shows a connection. That is a causal connection. It's all about counterfactual "what if I changed X?" questions. Using that idea, you can get all sorts of cool theory.
  - BenoitP 8y ago
    Thank you!
- jmmcd 8y ago
  Some readers might enjoy looking at code -- I came across this repo: https://github.com/akelleh/causality https://github.com/akelleh/causality
mark_l_watson 8y ago
I bought Judea Pearl’s new book The Book of Why last night after reading this article. So far I love the book. I manage a machine learning team at work and I appreciate Pearl’s discussion of how deep learning and statistics won’t lead to strong AI.
- eli_gottlieb 8y ago
  When I saw another one of these publicity articles, I basically ran to buy the book. It's really nice to have a book that will help me get the intuition and history of causal modeling rather than just giving theorems about graph structure under intervention.
  - mark_l_watson 8y ago
    I agree, it is nice to have a very clear high level approach to causal reasoning. I find his other books to be ‘slow going’ so I hope that after reading the Why book, I will have an easier time absorbing his earlier work.
    - Jach 8y ago
      I just started reading The Book of Why too, so far so good. I pre-ordered it once I found out it was coming. I've been telling people my view of it is it's like the primer to the primer (Causal Inference in Statistics: A Primer) to a subject-introduction paper (An Introduction to Causal Inference) to the OG math book (Causality). I'm hoping to eventually get back to the nice Causality hardcover I've had on my shelf for too long.
ml_thoughts 8y ago
Worth noting that there has been a fair bit of good research in causal machine learning in the last year or so, for example "Implicit Causal Models for Genome-wide Association Studies" (https://arxiv.org/pdf/1710.10742.pdf https://arxiv.org/pdf/1710.10742.pdf). The key point of this paper is that neural networks really are very good at "curve fitting" and that this curve fitting in the context of variational inference has advantages for causal reasoning, too. Neural networks can be used in a variety of structures, and these structures tend to benefit from the inclusion of powerful trainable non-linear function approximators. In this sense, deep learning will continue to be a powerful tool despite some limitations in its current use. I think Pearl, who's obviously remained very influential for many practitioners of machine learning, knows the value of "curve fitting". However I think it's a bit hard for a brief interview to sit down and have a real conversation about the state of the art of an academic field and the "Deep Learning is Broken" angle is a bit more attractive.
- deleted 8y ago
  [deleted]
- stochastic_monk 8y ago
  It's worth considering that anywhere in graphical models where coefficients of any sort learned can be augmented by neural networks (such as in the last decade of natural language processing, where the SOTA of almost all problems has been successfully neuralized). I wonder if Deep Belief Machines and their flavor of generative models, which seem closer in nature to Pearl's PGMs, have a chance to bridge the gap involved. Edit, as an aside: Given the enormously high dimensionality of personal genomes and the incredibly small sample size, for over a decade I've failed to put any trust in GWAS studies and found my suspicion supported on a number of occasions, considering difficulty in reproducibility likely brought about by the above problem. Is there any reason to think that improved statistical methods can possibly surmount the fundamental problem of limited sample size and high dimensionality?
  - et2o 8y ago
    Numerous important biomedical findings have resulted from GWAS. Most GWAS today are inherently reproducible since their hits usually come from multi-stage designs with independent samples. Sample sizes are no longer "incredibly small" either; large GWAS often have in the order of 100s of 1000s of patients. Some have over a million. I suppose the most important idea is that GWAS aren't really supposed to show causality. "Association" is in the name. GWAS are usually hypothesis generating (e.g., identification of associated variants) and then identified variants can be probed experimentally with all of the tools of molecular biology. In summary, GWAS have their problems, but I think your statement is a bit too strong.
guscost 8y ago
Aren’t we all?
Barrin92 8y ago
I really like that people asking fundamental questions about the way AI can be pushed forward beyond just ML seems to gain a little bit of traction again, at least in the public eye. Even Kissinger opined on the topic just a few days ago https://www.theatlantic.com/magazine/archive/2018/06/henry-kissinger-ai-could-mean-the-end-of-human-history/559124/ https://www.theatlantic.com/magazine/archive/2018/06/henry-k... I agree with Pearl that there's something deeply misguided about thinking that intelligence truly can be solved as a data optimization problem. I'd be happy if we'd see more research about AI at a symbolic level again. The biggest hindrance seems to be funding. ML is really successful in a lot of commercial domains, general AI is a big moonshot. With most researchers moving from long term university positions to the business sector I'm concerned about this sort of research, not just in computer science.
- alexanderdmitri 8y ago
  One thing that I never see brought into the equation but I believe is fundamental to human "general" intelligence is emotional response. It's hugely instrumental in terms of us deciding what's important to investigate, which input to analyze and how to mold our output. The importance of emotion in guiding and driving our learning is most evident in children, during the first four or five years of life of development. I think a huge leap forward in the field could be made by modeling an emotional system similar to ours. It also seems like it would be an incredibly difficult and fuzzy problem to solve. Addendum: After thinking about it for a few minutes more, I bet an implicit understanding of cause and effect could be derived by a system designed to try to optimize it's emotional state
  - Pimpus 8y ago
    I do think the impact of emotions on our thinking is understated, but I also think that focusing on this aspect is going at it backwards. A value system is necessary for emotions, otherwise happy and sad mean nothing. I think this is what you mean when you talk about "modeling emotions". At the same time, some intelligence seems like a prerequisite as blindly following a value system will lead to massive hedonistic AI orgies. Er, I mean, an override mechanism is required, and intelligence is the best possible solution, so we're back at the fundamental problem. I think there is a lot more intelligence behind even our most primitive urges than we realize. Hope that made sense, typing on mobile.
QueensGambit 8y ago
You dont need to go that far. If you want to replace an algorithm in a regulated industry like insurance (say, premium calculation based on risk score), you need to show the audit trail of how you arrived at the result. You cant have a probabilistic model that builds bias without any explanation. Only a few ML algorithms like decision tree can show the causal relationship today. It is very hard to that in neural network with multiple layers.
- deleted 8y ago
  [deleted]
- ItsMe000001 8y ago
  Isn't there a lot going on already in insurance based on algorithms - and no explanation? I'm looking at scoring/rating... People getting a lower credit rating for living in the wrong neighborhood and other things, well that would at least be an identifiable reason, but I think that you cannot find out how exactly they arrived at your personal score? How transparent are banks and insurances for consumers, today? Also on the HN frontpage right now is a link to a Guardian article "how to disappear from the internet", and the top comment in the forum there about his difficulties to deal with the results of identity theft, credit card debt, also shows a complete lack of transparency.
  - QueensGambit 8y ago
    It may not be transparent to the consumers. But, these algorithms are designed by actuaries and everyone in the company understands how it works. You need to have an audit trail of how the premium is calculated. But in deep learning algorithms, features and co-efficients are not determined by humans. In most cases, it cant even be understood by humans. Without this understanding, I highly doubt if they will be accepted in regulatory industries.
  - stevesimmons 8y ago
    The lack of transparency is for a good reason: if a model's parameters become known, they can be gamed and lose their predictive power. Not only do banks etc keep their models secret from customers, they keep them secret from other departments. The credit risk strategy team, for instance, won't want to risk customer service staff 'helping' customers alter their application details to get their scores over a cut-off. (I used to run credit risk strategy, fraud, collections, operations etc for two credit card companies)
daenz 8y ago
>The language of algebra is symmetric: If x tells us about y, then y tells us about x. I’m talking about deterministic relationships. There’s no way to write in mathematics a simple fact—for example, that the upcoming storm causes the barometer to go down, and not the other way around. Is this true? It kind of blows my mind if it is.
- kabbalf 8y ago
  Seems like it's a fallacy of limiting oneself to stationary reasoning. Differential equations and difference equations that capture temporal and causual behaviors are a thing. A large part of control theory is model identification, which tries to fit a temporal model to data. In a difference equation x(t+1) = f(x(t),u(t)), u causes changes to x and not the other way around
  - PeterisP 8y ago
    If we don't know that the true equation is x(t+1) = f(x(t),u(t)) but have to infer it from data, then observing that such an equation seems to hold true for some particular u and x does not mean that u causes changes to x. It may simply be a predictor of x, as in the 'barometer readings may precede and imply a storm but not cause one' example given by others above.
  - tgb 8y ago
    Differential equations say nothing about causality they just describe what happens. Please remember that Pearl literally wrote the book on this topic before assuming he missed an elementary counter example.
    - salty_biscuits 8y ago
      I feel like reversiblity is a big complication here. Very interesting and subtle area.
      - tgb 8y ago
        I don't see the relevance. You can have (non-differential )algebraic equations that involve a "time" parameter. They can be non-reversible in the sense that I think you mean - that knowing a point "back" in time tells you values in the future but knowing a value in the future does not (uniquely) determine the solution before that point. [For example x^2 = t^2 for t <= 0 and x^2 = 0 for t >= 0. There are two continuous solutions x(t) which are equal at say t=1.] For differential or non-differential equations, they're still just describing how something is and not the causality behind it. It's always possible that the equation is merely a result of hidden variables and there is no casual relationship between any two points of the solution in any meaningful sense.
sgt101 8y ago
The core issue is trust, explanation is one part of trust, but there are deeper issues. After all, if I explain that I have made this clinical decision because it results in lower mortality and you point out that the mortality statistic is to shit, and then I point out that we can't do the experiment required to work out the mortality statistic properly because that would mean potentially killing children... we have an issue. We trust doctors and pilots, they offer partial explanations that we can somewhat understand, but they are backed by experience and qualification. Their perspective is informed by science - some good, some bad. Most of us don't think about that. We have a perspective based on our cultural and social background, the machine must understand this and provide alternative explanations to suit us. I have written a long article on this all, but I can't finish the game theory off!
- udfalkso 8y ago
  Sounds like an interesting article
  - sgt101 8y ago
    Yeah, if only I could sort out the sums... But then.. I guess that's the point!
deleted 8y ago
[deleted]
323454 8y ago
I can't help but see this as another example of the pattern in which a big name in a field gets up and says that the current direction of their field (deep learning) is great and all but not really making progress on the big question (intelligence), and that to solve that question we need to solve another big question (what is causality) before we can make true progress. Other examples to me are Chomsky on consciousness and its implications for language, Einstein on causality w.r.t. quantum theory. This isnt to say the big name is wrong, just to point out a potential pattern.
- p1esk 8y ago
  Just want to point out that in general it's a lot harder to recognize the correct direction, than to make a progress in a direction.
fizx 8y ago
Chris Manning gave a really interesting technical yet pretty approachable talk on machine reasoning at ICLR this year. If you're interested in this topic, i recommend watching! https://www.youtube.com/watch?v=jpNLp9SnTF8 https://www.youtube.com/watch?v=jpNLp9SnTF8
hackguru 8y ago
There are a lot of efforts in developing models that understand causal relationships within mainstream machine learning community. Mostly to train models that don't require a lot of training examples. Deep learning usually requires a lot of data and trained models are not easily transferable to other tasks. Yet humans tend to transfer their knowledge from other tasks pretty easily to seemingly unrelated tasks. This seems to be due to our mental models surrounding causal relationships. One example of such efforts is schema networks. It is a model-based approach to RL that exhibits some of the strong generalization abilities that can be key to human-like general intelligence. https://www.vicarious.com/2017/08/07/general-game-playing-with-schema-networks/ https://www.vicarious.com/2017/08/07/general-game-playing-wi...
- mongodude 8y ago
  Vicarious is shit company and the only thing worthy of note they have done is raise large amounts of money!
matchagaucho 8y ago
Once ML models are routinely provided time-based features; particularly deltas between events and the 3rd derivative rate-of-change; they are going to identify some amazing causal relationships that humans are incapable of today.
- daveguy 8y ago
  Why 3rd derivative? Do you think that because we don't typically work with 3rd derivatives to explain physical phenomenon that there is a lot we are missing? Or are there particular scenarios where you see 3rd derivative as being critical to causality? 1st derivative = direct causality, 2nd derivative = ability to affect change, 3rd derivative = ?? tertiary causes. Wait. Doesn't a derivative mathematically define a causal relationship? EDIT: nevermind re: derivative = causal ... that's just a correlation relationship. dx/dt. Still I'm curious as to what is special about the 3d derivative (besides jerk).
  - scottlocklin 8y ago
    3rd derivative is a simple filter on acceleration. Obvious feature in something like fault prediction on heavy machinery. What you really want, though are the ability to decouple and infer relationships between short and long term features (something like the cepstrum transform from speech analysis).
joaorico 8y ago
Pearl's words from the Introduction of "BAYESIANISM AND CAUSALITY, OR, WHY I AM ONLY A HALF-BAYESIAN": "I turned Bayesian in 1971, as soon as I began reading Savage’s monograph The Foundations of Statistical Inference [Savage, 1962]. The arguments were unassailable: (i) It is plain silly to ignore what we know, (ii) It is natural and useful to cast what we know in the language of probabilities, and (iii) If our subjective probabilities are erroneous, their impact will get washed out in due time, as the number of observations increases. Thirty years later, I am still a devout Bayesian in the sense of (i), but I now doubt the wisdom of (ii) and I know that, in general, (iii) is false."
- evanmoran 8y ago
  For (ii) what do you use instead of probabilities? And for (iii) what changed for you to think this doesn't improve over time?
  - tdrd 8y ago
    OP was quoting.
  - sjg007 8y ago
    Subjective probabilities are based on the model. Increasing observations won't help if you have the wrong model to begin with. So we need causal methods to ask if the model is correct. We also need methods to propose new models or rebuild if it is wrong as well.
  - p1esk 8y ago
    For (ii) I'm guessing Pearl's do-calculus, and I agree with sjg007 regarding (iii).
salty_biscuits 8y ago
I don't necessarily agree with the assertion there has been no progress on algorithms that can propose experiments. Isn't this exactly what Bayesian Optimization with regret minimization is all about? Also seems strange to say that AI is just curve fitting in a pejorative sense. Isn't that all of science? Curve fitting is hard!
- xamuel 8y ago
  >Isn't [curve fitting] all of science? No! Astronomical observations can be shoehorned into geocentrism by adding more and more epicycles. That's curve fitting. At some point you have to realize the Earth revolves around the sun. Currently ML is on a dangerous path because any disagreement with empirical evidence can just be waved away with more data, more computation power, etc. In that sense, it's practically unfalsifiable. http://wiki.c2.com/?AddingEpicycles http://wiki.c2.com/?AddingEpicycles
  - salty_biscuits 8y ago
    Only if you don't have a complexity penalty in your fit. This is a model selection problem and you should have a prior on the structure of the model that leads to something like the BIC. Curve fitting is hard. The elimination of epicycles was due to Tyco Brahe collecting more data and a lower complexity model being proposed to explain the data.
joe_the_user 8y ago
Honest question: If two models are each trained with a single training set, tested with a single test set, and used only to predict a single stream of data and predict it approximately equally as well, is there any formal way to say one model is engaging in "casual reasoning" and another isn't?
- visarga 8y ago
  causal, casual, same thing
shahbaby 8y ago
For me, AI is now in the same category as religion; I don't talk about it because nothing good ever comes out of these type of "discussions." We should be more open minded and humble about how to approach this problem but almost everyone seems to have a strong opinion about it creating a very low signal to noise ratio.
- tempodox 8y ago
  I don't think you have to go as far as comparing it to religion. Diagnosing the field as being (currently) overhyped should be enough to explain your observations. However, Kissinger's article [1] raises worthwhile and important questions that really should be discussed broadly. [1] https://www.theatlantic.com/magazine/archive/2018/06/henry-kissinger-ai-could-mean-the-end-of-human-history/559124/ https://www.theatlantic.com/magazine/archive/2018/06/henry-k...
dontwantai 8y ago
The problem is that deep learning is not powered by an intelligent system able to suggest a way to control and filter input data. Today humans are the ones that filter and decide which loss function to use, and in order to achieve GAI deep learning must be provided with a meta-deep learning framework able to filter and adjust the loss function. Perhaps a feed-back network trying to evolve from a small model to a more general one using some kind of generative grammar for filtering and controling the input. A transfer knowledge graph powered by deep learning that select a generative grammar for designing the most valuable filtering and objective function to produce a deep learning system that learn by itself.
deleted 8y ago
[deleted]
deleted 8y ago
[deleted]

308 points by stevenwoo 8y ago

In history we call this determinism. The more you know about a historic choice and the complex mechanisms around it, the more it makes perfect sense while at the same time leaving you absolutely clueless about the why. Christianity being chosen by the Roman Empire is the typical example. To most people the choice makes perfect sense, because we look back at what it brought with it. But when you put yourself in the heads of the decision makers and look at all the options they had, well, it makes no sense at all. A lot of machine learning tells us trends, but it tells us nothing about the why, and I completely agree with the article about how useless that data is. I mean, it’s probably great at harmless things, but when my elaborate online profile still can’t figure out why I happen to read a cultural, artsy but somewhat conservative news paper, despite the fact that my data shows the algorithm that I really really shouldn’t be doing that, well, then we simply can’t use ML for any form of decision making or even as an advisor. At least not in the public sector.

sdenton4 8y ago
Yeah, I think it's worth also asking whether humans /actually/ are any good at answering the 'why' with anything but bullshit. I would argue that we're pretty good at understanding causality in very limited circumstances (the window broke because that kid threw the ball), and extremely overconfident in our ability to understand causation in a much broader range of circumstances (the stock price went up because...). This overconfidence drives a lot of the decisions we make, for better or for worse. It's an area where if we push hard on AI, we'll likely have to come to terms with how bad we are in this area, and ask ourselves whether we feel comfortable deploying 'thinking machines' with similar levels of incompetence and/or arrogance.

BenoitP 8y ago

If someone can ELI5 to me what is Pearl's do-calculus, that'd be quite great. I have tried to build an understanding of it since he got the Turing prize, but have failed so far.

cbames89 8y ago
It's a way to take interventions, for example what would happen if we can cigarettes, and model it with observational data. The do-calculus is just a couple of rules which say when we can do this. It essentially boils down to a couple of substitution rules.
imh 8y ago
Rain causes people to carry umbrellas. Rain causes puddles. Probabilistically, there's a strong connection between me carrying an umbrella and you seeing puddles (assuming we're both in SF or whatever). To be a bayesian, you could model this as a conditional probability: p(you see a puddle | I carry an umbrella). It will look like a strong connection, but it isn't a causal one. If it were, then I could stop carrying an umbrella and clear away the puddles. That intervention of me changing my umbrella carrying behavior is what causation is all about. If we change this, does that change? So then you talk about the probability of you seeing a puddle given some intervention that forces me to carry my umbrella regardless of anything else. We see that if you force me, independent of rain, to carry an umbrella or not to, then the connection between the umbrella and puddles is gone. p(puddles | do(umbrella)) != p(puddles | umbrella). do(X) means to take an intervention and force X regardless of other things. As contrast, you can talk about the connection between rain and and puddles. If there were some hypothetical weather machine where we could force rain or sunshine, then you'd see that intervening and forcing rain (a.k.a. do(rain)) still keeps the relationship with puddles. p(puddles | do(rain)) still shows a connection. That is a causal connection. It's all about counterfactual "what if I changed X?" questions. Using that idea, you can get all sorts of cool theory.
- BenoitP 8y ago
  Thank you!
jmmcd 8y ago
Some readers might enjoy looking at code -- I came across this repo: https://github.com/akelleh/causality https://github.com/akelleh/causality

mark_l_watson 8y ago

I bought Judea Pearl’s new book The Book of Why last night after reading this article. So far I love the book. I manage a machine learning team at work and I appreciate Pearl’s discussion of how deep learning and statistics won’t lead to strong AI.

eli_gottlieb 8y ago
When I saw another one of these publicity articles, I basically ran to buy the book. It's really nice to have a book that will help me get the intuition and history of causal modeling rather than just giving theorems about graph structure under intervention.
- mark_l_watson 8y ago
  I agree, it is nice to have a very clear high level approach to causal reasoning. I find his other books to be ‘slow going’ so I hope that after reading the Why book, I will have an easier time absorbing his earlier work.
  - Jach 8y ago
    I just started reading The Book of Why too, so far so good. I pre-ordered it once I found out it was coming. I've been telling people my view of it is it's like the primer to the primer (Causal Inference in Statistics: A Primer) to a subject-introduction paper (An Introduction to Causal Inference) to the OG math book (Causality). I'm hoping to eventually get back to the nice Causality hardcover I've had on my shelf for too long.

ml_thoughts 8y ago

Worth noting that there has been a fair bit of good research in causal machine learning in the last year or so, for example "Implicit Causal Models for Genome-wide Association Studies" (https://arxiv.org/pdf/1710.10742.pdf https://arxiv.org/pdf/1710.10742.pdf). The key point of this paper is that neural networks really are very good at "curve fitting" and that this curve fitting in the context of variational inference has advantages for causal reasoning, too. Neural networks can be used in a variety of structures, and these structures tend to benefit from the inclusion of powerful trainable non-linear function approximators. In this sense, deep learning will continue to be a powerful tool despite some limitations in its current use. I think Pearl, who's obviously remained very influential for many practitioners of machine learning, knows the value of "curve fitting". However I think it's a bit hard for a brief interview to sit down and have a real conversation about the state of the art of an academic field and the "Deep Learning is Broken" angle is a bit more attractive.

deleted 8y ago
[deleted]
stochastic_monk 8y ago
It's worth considering that anywhere in graphical models where coefficients of any sort learned can be augmented by neural networks (such as in the last decade of natural language processing, where the SOTA of almost all problems has been successfully neuralized). I wonder if Deep Belief Machines and their flavor of generative models, which seem closer in nature to Pearl's PGMs, have a chance to bridge the gap involved. Edit, as an aside: Given the enormously high dimensionality of personal genomes and the incredibly small sample size, for over a decade I've failed to put any trust in GWAS studies and found my suspicion supported on a number of occasions, considering difficulty in reproducibility likely brought about by the above problem. Is there any reason to think that improved statistical methods can possibly surmount the fundamental problem of limited sample size and high dimensionality?
- et2o 8y ago
  Numerous important biomedical findings have resulted from GWAS. Most GWAS today are inherently reproducible since their hits usually come from multi-stage designs with independent samples. Sample sizes are no longer "incredibly small" either; large GWAS often have in the order of 100s of 1000s of patients. Some have over a million. I suppose the most important idea is that GWAS aren't really supposed to show causality. "Association" is in the name. GWAS are usually hypothesis generating (e.g., identification of associated variants) and then identified variants can be probed experimentally with all of the tools of molecular biology. In summary, GWAS have their problems, but I think your statement is a bit too strong.

guscost 8y ago

Aren’t we all?

Barrin92 8y ago

I really like that people asking fundamental questions about the way AI can be pushed forward beyond just ML seems to gain a little bit of traction again, at least in the public eye. Even Kissinger opined on the topic just a few days ago https://www.theatlantic.com/magazine/archive/2018/06/henry-kissinger-ai-could-mean-the-end-of-human-history/559124/ https://www.theatlantic.com/magazine/archive/2018/06/henry-k... I agree with Pearl that there's something deeply misguided about thinking that intelligence truly can be solved as a data optimization problem. I'd be happy if we'd see more research about AI at a symbolic level again. The biggest hindrance seems to be funding. ML is really successful in a lot of commercial domains, general AI is a big moonshot. With most researchers moving from long term university positions to the business sector I'm concerned about this sort of research, not just in computer science.

alexanderdmitri 8y ago
One thing that I never see brought into the equation but I believe is fundamental to human "general" intelligence is emotional response. It's hugely instrumental in terms of us deciding what's important to investigate, which input to analyze and how to mold our output. The importance of emotion in guiding and driving our learning is most evident in children, during the first four or five years of life of development. I think a huge leap forward in the field could be made by modeling an emotional system similar to ours. It also seems like it would be an incredibly difficult and fuzzy problem to solve. Addendum: After thinking about it for a few minutes more, I bet an implicit understanding of cause and effect could be derived by a system designed to try to optimize it's emotional state
- Pimpus 8y ago
  I do think the impact of emotions on our thinking is understated, but I also think that focusing on this aspect is going at it backwards. A value system is necessary for emotions, otherwise happy and sad mean nothing. I think this is what you mean when you talk about "modeling emotions". At the same time, some intelligence seems like a prerequisite as blindly following a value system will lead to massive hedonistic AI orgies. Er, I mean, an override mechanism is required, and intelligence is the best possible solution, so we're back at the fundamental problem. I think there is a lot more intelligence behind even our most primitive urges than we realize. Hope that made sense, typing on mobile.

QueensGambit 8y ago

You dont need to go that far. If you want to replace an algorithm in a regulated industry like insurance (say, premium calculation based on risk score), you need to show the audit trail of how you arrived at the result. You cant have a probabilistic model that builds bias without any explanation. Only a few ML algorithms like decision tree can show the causal relationship today. It is very hard to that in neural network with multiple layers.

deleted 8y ago
[deleted]
ItsMe000001 8y ago
Isn't there a lot going on already in insurance based on algorithms - and no explanation? I'm looking at scoring/rating... People getting a lower credit rating for living in the wrong neighborhood and other things, well that would at least be an identifiable reason, but I think that you cannot find out how exactly they arrived at your personal score? How transparent are banks and insurances for consumers, today? Also on the HN frontpage right now is a link to a Guardian article "how to disappear from the internet", and the top comment in the forum there about his difficulties to deal with the results of identity theft, credit card debt, also shows a complete lack of transparency.
- QueensGambit 8y ago
  It may not be transparent to the consumers. But, these algorithms are designed by actuaries and everyone in the company understands how it works. You need to have an audit trail of how the premium is calculated. But in deep learning algorithms, features and co-efficients are not determined by humans. In most cases, it cant even be understood by humans. Without this understanding, I highly doubt if they will be accepted in regulatory industries.
- stevesimmons 8y ago
  The lack of transparency is for a good reason: if a model's parameters become known, they can be gamed and lose their predictive power. Not only do banks etc keep their models secret from customers, they keep them secret from other departments. The credit risk strategy team, for instance, won't want to risk customer service staff 'helping' customers alter their application details to get their scores over a cut-off. (I used to run credit risk strategy, fraud, collections, operations etc for two credit card companies)

daenz 8y ago

>The language of algebra is symmetric: If x tells us about y, then y tells us about x. I’m talking about deterministic relationships. There’s no way to write in mathematics a simple fact—for example, that the upcoming storm causes the barometer to go down, and not the other way around. Is this true? It kind of blows my mind if it is.

kabbalf 8y ago
Seems like it's a fallacy of limiting oneself to stationary reasoning. Differential equations and difference equations that capture temporal and causual behaviors are a thing. A large part of control theory is model identification, which tries to fit a temporal model to data. In a difference equation x(t+1) = f(x(t),u(t)), u causes changes to x and not the other way around
- PeterisP 8y ago
  If we don't know that the true equation is x(t+1) = f(x(t),u(t)) but have to infer it from data, then observing that such an equation seems to hold true for some particular u and x does not mean that u causes changes to x. It may simply be a predictor of x, as in the 'barometer readings may precede and imply a storm but not cause one' example given by others above.
- tgb 8y ago
  Differential equations say nothing about causality they just describe what happens. Please remember that Pearl literally wrote the book on this topic before assuming he missed an elementary counter example.
  - salty_biscuits 8y ago
    I feel like reversiblity is a big complication here. Very interesting and subtle area.
    - tgb 8y ago
      I don't see the relevance. You can have (non-differential )algebraic equations that involve a "time" parameter. They can be non-reversible in the sense that I think you mean - that knowing a point "back" in time tells you values in the future but knowing a value in the future does not (uniquely) determine the solution before that point. [For example x^2 = t^2 for t <= 0 and x^2 = 0 for t >= 0. There are two continuous solutions x(t) which are equal at say t=1.] For differential or non-differential equations, they're still just describing how something is and not the causality behind it. It's always possible that the equation is merely a result of hidden variables and there is no casual relationship between any two points of the solution in any meaningful sense.

sgt101 8y ago

The core issue is trust, explanation is one part of trust, but there are deeper issues. After all, if I explain that I have made this clinical decision because it results in lower mortality and you point out that the mortality statistic is to shit, and then I point out that we can't do the experiment required to work out the mortality statistic properly because that would mean potentially killing children... we have an issue. We trust doctors and pilots, they offer partial explanations that we can somewhat understand, but they are backed by experience and qualification. Their perspective is informed by science - some good, some bad. Most of us don't think about that. We have a perspective based on our cultural and social background, the machine must understand this and provide alternative explanations to suit us. I have written a long article on this all, but I can't finish the game theory off!

udfalkso 8y ago
Sounds like an interesting article
- sgt101 8y ago
  Yeah, if only I could sort out the sums... But then.. I guess that's the point!

deleted 8y ago

[deleted]

323454 8y ago

I can't help but see this as another example of the pattern in which a big name in a field gets up and says that the current direction of their field (deep learning) is great and all but not really making progress on the big question (intelligence), and that to solve that question we need to solve another big question (what is causality) before we can make true progress. Other examples to me are Chomsky on consciousness and its implications for language, Einstein on causality w.r.t. quantum theory. This isnt to say the big name is wrong, just to point out a potential pattern.

p1esk 8y ago
Just want to point out that in general it's a lot harder to recognize the correct direction, than to make a progress in a direction.

fizx 8y ago

Chris Manning gave a really interesting technical yet pretty approachable talk on machine reasoning at ICLR this year. If you're interested in this topic, i recommend watching! https://www.youtube.com/watch?v=jpNLp9SnTF8 https://www.youtube.com/watch?v=jpNLp9SnTF8

hackguru 8y ago

There are a lot of efforts in developing models that understand causal relationships within mainstream machine learning community. Mostly to train models that don't require a lot of training examples. Deep learning usually requires a lot of data and trained models are not easily transferable to other tasks. Yet humans tend to transfer their knowledge from other tasks pretty easily to seemingly unrelated tasks. This seems to be due to our mental models surrounding causal relationships. One example of such efforts is schema networks. It is a model-based approach to RL that exhibits some of the strong generalization abilities that can be key to human-like general intelligence. https://www.vicarious.com/2017/08/07/general-game-playing-with-schema-networks/ https://www.vicarious.com/2017/08/07/general-game-playing-wi...

mongodude 8y ago
Vicarious is shit company and the only thing worthy of note they have done is raise large amounts of money!

matchagaucho 8y ago

Once ML models are routinely provided time-based features; particularly deltas between events and the 3rd derivative rate-of-change; they are going to identify some amazing causal relationships that humans are incapable of today.

daveguy 8y ago
Why 3rd derivative? Do you think that because we don't typically work with 3rd derivatives to explain physical phenomenon that there is a lot we are missing? Or are there particular scenarios where you see 3rd derivative as being critical to causality? 1st derivative = direct causality, 2nd derivative = ability to affect change, 3rd derivative = ?? tertiary causes. Wait. Doesn't a derivative mathematically define a causal relationship? EDIT: nevermind re: derivative = causal ... that's just a correlation relationship. dx/dt. Still I'm curious as to what is special about the 3d derivative (besides jerk).
- scottlocklin 8y ago
  3rd derivative is a simple filter on acceleration. Obvious feature in something like fault prediction on heavy machinery. What you really want, though are the ability to decouple and infer relationships between short and long term features (something like the cepstrum transform from speech analysis).

joaorico 8y ago

Pearl's words from the Introduction of "BAYESIANISM AND CAUSALITY, OR, WHY I AM ONLY A HALF-BAYESIAN": "I turned Bayesian in 1971, as soon as I began reading Savage’s monograph The Foundations of Statistical Inference [Savage, 1962]. The arguments were unassailable: (i) It is plain silly to ignore what we know, (ii) It is natural and useful to cast what we know in the language of probabilities, and (iii) If our subjective probabilities are erroneous, their impact will get washed out in due time, as the number of observations increases. Thirty years later, I am still a devout Bayesian in the sense of (i), but I now doubt the wisdom of (ii) and I know that, in general, (iii) is false."

evanmoran 8y ago
For (ii) what do you use instead of probabilities? And for (iii) what changed for you to think this doesn't improve over time?
- tdrd 8y ago
  OP was quoting.
- sjg007 8y ago
  Subjective probabilities are based on the model. Increasing observations won't help if you have the wrong model to begin with. So we need causal methods to ask if the model is correct. We also need methods to propose new models or rebuild if it is wrong as well.
- p1esk 8y ago
  For (ii) I'm guessing Pearl's do-calculus, and I agree with sjg007 regarding (iii).

salty_biscuits 8y ago

I don't necessarily agree with the assertion there has been no progress on algorithms that can propose experiments. Isn't this exactly what Bayesian Optimization with regret minimization is all about? Also seems strange to say that AI is just curve fitting in a pejorative sense. Isn't that all of science? Curve fitting is hard!

xamuel 8y ago
>Isn't [curve fitting] all of science? No! Astronomical observations can be shoehorned into geocentrism by adding more and more epicycles. That's curve fitting. At some point you have to realize the Earth revolves around the sun. Currently ML is on a dangerous path because any disagreement with empirical evidence can just be waved away with more data, more computation power, etc. In that sense, it's practically unfalsifiable. http://wiki.c2.com/?AddingEpicycles http://wiki.c2.com/?AddingEpicycles
- salty_biscuits 8y ago
  Only if you don't have a complexity penalty in your fit. This is a model selection problem and you should have a prior on the structure of the model that leads to something like the BIC. Curve fitting is hard. The elimination of epicycles was due to Tyco Brahe collecting more data and a lower complexity model being proposed to explain the data.

joe_the_user 8y ago

Honest question: If two models are each trained with a single training set, tested with a single test set, and used only to predict a single stream of data and predict it approximately equally as well, is there any formal way to say one model is engaging in "casual reasoning" and another isn't?

visarga 8y ago
causal, casual, same thing

shahbaby 8y ago

For me, AI is now in the same category as religion; I don't talk about it because nothing good ever comes out of these type of "discussions." We should be more open minded and humble about how to approach this problem but almost everyone seems to have a strong opinion about it creating a very low signal to noise ratio.

tempodox 8y ago
I don't think you have to go as far as comparing it to religion. Diagnosing the field as being (currently) overhyped should be enough to explain your observations. However, Kissinger's article [1] raises worthwhile and important questions that really should be discussed broadly. [1] https://www.theatlantic.com/magazine/archive/2018/06/henry-kissinger-ai-could-mean-the-end-of-human-history/559124/ https://www.theatlantic.com/magazine/archive/2018/06/henry-k...

dontwantai 8y ago

The problem is that deep learning is not powered by an intelligent system able to suggest a way to control and filter input data. Today humans are the ones that filter and decide which loss function to use, and in order to achieve GAI deep learning must be provided with a meta-deep learning framework able to filter and adjust the loss function. Perhaps a feed-back network trying to evolve from a small model to a more general one using some kind of generative grammar for filtering and controling the input. A transfer knowledge graph powered by deep learning that select a generative grammar for designing the most valuable filtering and objective function to produce a deep learning system that learn by itself.

deleted 8y ago

[deleted]

deleted 8y ago

[deleted]