4 ms·

What the article describes is called an "expert system" and is what AI in the enterprise used to look like. Basically, you try to capture the instinct of a gre

What the article describes is called an "expert system" and is what AI in the enterprise used to look like. Basically, you try to capture the instinct of a great salesmen by formalizing it into computer logic. Often that's done with rules like in the article. It works good, but has its limits. The finer reasoning of human judgement are often not expressable, people don't know why they made that decision. Making it hard to capture. And human also have their limits. Too many variables, too much noise, too much data and they won't make the best prediction/decision. That's when ML shines. Instead of trying to encode an expert's intuition, instead you let the machine develop its own intuition, itself becoming an expert through training. The downside is it now similarly becomes challenging to formalize the machine's intuition. Why it made a given choice is no longer easily apparent. I do think expert systems still have value. Especially when you lack the dataset to train a machine expert.

dvfjsdhgfv 9y ago
Yes, these are two main approaches to AI, with expert systems being the "old" AI and ML being the "modern" AI. In the old approach, you knew exactly what you wanted to achieve, and very often you had a relatively clear idea how to do it; the challenge was to construct the solution manually. In the new approach you are not so much concerned with how to do it as you train on some dataset anyway. The results are less predictable though, you might even not understand why certain things happen. The number of mistakes/false positives can be relatively high. Still, with these methods, I think we're very far from real artificial intelligence, where you can really learn something new on the basis of something old. I believe until we have a profound understanding of what it means "to understand", such an advancement in AI won't be possible.
- mjburgess 9y ago
  I think we do have a good-enough understanding of "understanding" to characterize what is required. However this is to be found in the textbooks of neuroscience, not in the machine manufacture manuals of a silicon lithography plant. There isn't as much mystery as there seems. When a neuroscientist asks "how does a animal perceive the world?" the answer is reasonably methodologically obvious. When a computer scientist asks, "how does a machine perceive the world?" the mystery arises only because it doesn't.
  - dvfjsdhgfv 9y ago
    There is a huge difference between perception and understanding. http://news.itu.int/reality-check-not-nearly-close-strong-ai-many-believe/ http://news.itu.int/reality-check-not-nearly-close-strong-ai... > Marcus also observes “a bias in the field which is to assume that everything is learnt.” Marcus makes the argument that human beings do not learn everything by ‘trial-and-error’ – that part of our knowledge is innate, learnt over evolution – leading him to suggest that “we need more innateness if we are going to build intelligent agents … It’s not learning trial-by-trial in the way that our contemporary machines are.”
    - mjburgess 9y ago
      Well, not as much as you think. Perception is structured by concepts which are learnt. Perception is the prototypical mechanism of understanding. Insofar as we talking about "abstractions" we are just using language to model the conceptual structure of perception; and then higher order concepts to model the structure of these "perceptual" concepts. We tend to think of understanding as being equivalent to the linguistic act of accounting for a higher-order concept, when this is really only something humans can do -- and then only with tremendous difficulty needing quite significant methodological assistance from others, etc. This is the mistake Socrates makes when he asks for "definitions", as-if the dog doesn't understand the geography of where he lives only because he has no linguistic model of the concepts he's using. Or as-if the general has no knowledge of justice because he has no precise linguistic model of it. When Potter Steward said in the pornography case, " I know it when I see it " he was literally correct. The concept "pornography" is part of seeing pornography, it is knowledge which structures your perception. The lack of perfect linguistic model for that is neither here nor there, and a bit of a blind alley for AI (and socrates). Animals, including humans, understand a great deal more than any one person can model linguistically. It is this which structures our perception and allows even trivial engagements with our environment.
      - dvfjsdhgfv 9y ago
        > When Potter Steward said in the pornography case, " I know it when I see it " he was literally correct. Here you refer to categorization, not understanding. From the article I refer to: > “Deep learning is good at certain aspects of perception, particularly categorization, but perception is more than categorization and cognition or intelligence is more than just perception. There are many things that go into intelligence … And what we have made real progress on is perception, but the rest of it, we still haven’t made that much progress collectively in the field.” Understanding, real understanding, the "aha moment", is the very basis of new discoveries and real evolution. What we see now are systems quite good at categorizing, pattern recognition in a given set once prepared correctly and so on. There is no relation whatsoever with real intelligence, as in being able to make a new discovery based on analytical and synthetic skills. There are quite a few voices in the community saying the opposite and maintaining that mimicking intelligent action is practically the same as the intelligent action itself, but there are quite obvious limits to what can be achieved in this way.
sverhagen 9y ago
Newbie question: would you not design ML systems to be able to explain themselves? Alike: hey, query planner, what's your query plan?
- halflings 9y ago
  There is something close to what you're describing: 1) You train a complex ML model (say, deep neural net) to get the job done, solve your problem with high accuracy. 2) When you want to explain one of your model's results, you train a simpler more explainable model (say, linear regression) in the neighborhood of that point, such that it is locally similar to the more complex model. The task is too complex to be explained by a linear regression model over all possible inputs, but it often is simple enough around a specific point.
  - dmichulke 9y ago
    E.g., lime: https://github.com/marcotcr/lime https://github.com/marcotcr/lime
- bo1024 9y ago
  This is often called "interpretable machine learning" and it's an active area of research. By default, ML systems just make predictions; they don't explain anything. And it's not obvious how to get explanations from them.
- Retra 9y ago
  Is there any guarantee that every concept which a machine can learn can be represented in present-day human language in a manner compact enough to use? One worry is that a machine might learn to do something we can't represent efficiently. I mean, people write whole libraries of books on ethics, so how can you expect a machine to punch out one or two lines on its own? There's no upper limit on how long a proof to a given problem must be in any fixed language. And if you have to invent new language, people won't be able to read it anymore.
- PeterisP 9y ago
  You could, there are many approaches to design interpretable/explainable systems (Here https://distill.pub/2018/building-blocks/ https://distill.pub/2018/building-blocks/ is a nice and recent review of some of these problems with illustrations), but there are serious tradeoffs so people usually choose not to do so. The first issue is that, in general, explaining a decision is a harder task than making a decision, just as reviewing and understanding someone's decision takes more effort than it took to make it. This means that adding explainability and interpretability to a system that achieves some result will take more effort, time and money than building that system itself - you'll triple your budget and get no better actual results, just a better feeling about them. This also limits the effort invested in tools and methods for this; there's large demand and funding for making stuff work or making stuff a bit better, so many people are getting paid to work on that, and not nearly so much on interpretability. The second issue is that best performing systems take "everything" into account and thus become too complex for a human to understand. A system can be explainable, interpretable and accessible, but if making a correct decision inherently relies on complicated interrelations between many factors, then it will be difficult (or at least time-consuming) for humans to understand anyway. If you limit the complexity of the system to something that can be easily explained, then you're severely limiting its power, and you get poor results. For your query planner, the analogy is the scenario where we could build a query planner that is capable of optimizing beyond the commonly used basic blocks in query plans, instead outputting optimized machine code to perform particular query better than the standard blocks, achieving improved performance by merging and interleaving operations. The resulting output would be much harder to interpret, since no "block-level" query plan can be equivalent; and requiring the operations to be neatly separable in a limited number of blocks each with separate human-understandable meaning would mean restricting the system and inevitably getting a worse result. The same applies to humans, by the way - being able to explain all the factors why you believe your decision is likely to be correct takes more knowledge, insight and effort than simply making that decision. It's not like humans can explain why exactly they see (or don't see) an animal in picture like http://www.naute.com/images/cow.gif http://www.naute.com/images/cow.gif , consciously understanding that in a structured way is very, very hard. If you would require a human to explain the criteria they'll use before they get to make a decision (as opposed to post-decision rationalization that often has no relation to why the decision was made, and has more relation to making up a socially acceptable plausible justification why you did what you did), then you get poor, simplified criteria that can't reflect the complexity of the problem; you could get their "rules of thumb" or their major factors, but explaining why exactly it is so is generally a mentoring task that takes months (if not more) of effort both from the teacher and the student. AlphaZero system plays Go and Chess better than any human. It could be modified to describe in detail why it chose this move over the other, but would you (or any other human) have the mental capacity to truly understand it, no matter how well it's explained? The distill.pub article illustrates lots of ways how an image classifier can explain why it made one choice or another - but is it actually useful for tasks other than for R&D to debug or improve the model? It seems like a nice-to-have feature that would be liked if it's not to expensive, but I'd bet that for most models the company wouldn't actually use the explainability feature (again, outside of its builders debugging or improving it) because the whole point of ML is that people get removed out of the loop and don't look at the decisions.
BenoitP 9y ago
> The downside is it now similarly becomes challenging to formalize the machine's intuition Tooting my own horn here: I'm taking up this challenge professionally. There are new methods being discovered to extract interpretations for humans out of a model[1][2][3]; and I'll gladly work and provide advice and implementations on this, as an independent contractor. Having an interpretable/explicable ML pipeline helps in: * Knowing what drives your customers' buying patterns * Knowing where to look for signals closer to the root, helping go from correlation to causation * Orienting your marketing messages. You don't engage users the same way if they're a repeat buyer or a "hot now" lead * Concentrating on "would buy our product but won't because of X", which is where the market share battle lies; as opposed to "definitely would buy" and "definitely wouldn't buy", which are cases where you have little influence on. * Listing buying patterns exhaustively, enabling you to discover low noise niches * Providing auditable explanations of the decisions the model has taken, if it has been required legally Here is my email: pro@benoit.paris [1] https://distill.pub/2018/building-blocks/ https://distill.pub/2018/building-blocks/ [2] https://www.darpa.mil/program/explainable-artificial-intelligence https://www.darpa.mil/program/explainable-artificial-intelli... [3] http://interpretable.ml/ http://interpretable.ml/
al2o3cr 9y ago
"itself becoming an expert through training" One issue this runs into: the machine becomes an expert at something that matches what it was exposed to during training. The research on adversarial examples shows that the thing being learned != what the expert learns, frequently to an extreme degree.
- radarsat1 9y ago
  And anyone working on ML/AI who doesn't understand overfitting is selling snake oil.

dvfjsdhgfv 9y ago

Yes, these are two main approaches to AI, with expert systems being the "old" AI and ML being the "modern" AI. In the old approach, you knew exactly what you wanted to achieve, and very often you had a relatively clear idea how to do it; the challenge was to construct the solution manually. In the new approach you are not so much concerned with how to do it as you train on some dataset anyway. The results are less predictable though, you might even not understand why certain things happen. The number of mistakes/false positives can be relatively high. Still, with these methods, I think we're very far from real artificial intelligence, where you can really learn something new on the basis of something old. I believe until we have a profound understanding of what it means "to understand", such an advancement in AI won't be possible.

mjburgess 9y ago
I think we do have a good-enough understanding of "understanding" to characterize what is required. However this is to be found in the textbooks of neuroscience, not in the machine manufacture manuals of a silicon lithography plant. There isn't as much mystery as there seems. When a neuroscientist asks "how does a animal perceive the world?" the answer is reasonably methodologically obvious. When a computer scientist asks, "how does a machine perceive the world?" the mystery arises only because it doesn't.
- dvfjsdhgfv 9y ago
  There is a huge difference between perception and understanding. http://news.itu.int/reality-check-not-nearly-close-strong-ai-many-believe/ http://news.itu.int/reality-check-not-nearly-close-strong-ai... > Marcus also observes “a bias in the field which is to assume that everything is learnt.” Marcus makes the argument that human beings do not learn everything by ‘trial-and-error’ – that part of our knowledge is innate, learnt over evolution – leading him to suggest that “we need more innateness if we are going to build intelligent agents … It’s not learning trial-by-trial in the way that our contemporary machines are.”
  - mjburgess 9y ago
    Well, not as much as you think. Perception is structured by concepts which are learnt. Perception is the prototypical mechanism of understanding. Insofar as we talking about "abstractions" we are just using language to model the conceptual structure of perception; and then higher order concepts to model the structure of these "perceptual" concepts. We tend to think of understanding as being equivalent to the linguistic act of accounting for a higher-order concept, when this is really only something humans can do -- and then only with tremendous difficulty needing quite significant methodological assistance from others, etc. This is the mistake Socrates makes when he asks for "definitions", as-if the dog doesn't understand the geography of where he lives only because he has no linguistic model of the concepts he's using. Or as-if the general has no knowledge of justice because he has no precise linguistic model of it. When Potter Steward said in the pornography case, " I know it when I see it " he was literally correct. The concept "pornography" is part of seeing pornography, it is knowledge which structures your perception. The lack of perfect linguistic model for that is neither here nor there, and a bit of a blind alley for AI (and socrates). Animals, including humans, understand a great deal more than any one person can model linguistically. It is this which structures our perception and allows even trivial engagements with our environment.
    - dvfjsdhgfv 9y ago
      > When Potter Steward said in the pornography case, " I know it when I see it " he was literally correct. Here you refer to categorization, not understanding. From the article I refer to: > “Deep learning is good at certain aspects of perception, particularly categorization, but perception is more than categorization and cognition or intelligence is more than just perception. There are many things that go into intelligence … And what we have made real progress on is perception, but the rest of it, we still haven’t made that much progress collectively in the field.” Understanding, real understanding, the "aha moment", is the very basis of new discoveries and real evolution. What we see now are systems quite good at categorizing, pattern recognition in a given set once prepared correctly and so on. There is no relation whatsoever with real intelligence, as in being able to make a new discovery based on analytical and synthetic skills. There are quite a few voices in the community saying the opposite and maintaining that mimicking intelligent action is practically the same as the intelligent action itself, but there are quite obvious limits to what can be achieved in this way.

sverhagen 9y ago

Newbie question: would you not design ML systems to be able to explain themselves? Alike: hey, query planner, what's your query plan?

halflings 9y ago
There is something close to what you're describing: 1) You train a complex ML model (say, deep neural net) to get the job done, solve your problem with high accuracy. 2) When you want to explain one of your model's results, you train a simpler more explainable model (say, linear regression) in the neighborhood of that point, such that it is locally similar to the more complex model. The task is too complex to be explained by a linear regression model over all possible inputs, but it often is simple enough around a specific point.
- dmichulke 9y ago
  E.g., lime: https://github.com/marcotcr/lime https://github.com/marcotcr/lime
bo1024 9y ago
This is often called "interpretable machine learning" and it's an active area of research. By default, ML systems just make predictions; they don't explain anything. And it's not obvious how to get explanations from them.
Retra 9y ago
Is there any guarantee that every concept which a machine can learn can be represented in present-day human language in a manner compact enough to use? One worry is that a machine might learn to do something we can't represent efficiently. I mean, people write whole libraries of books on ethics, so how can you expect a machine to punch out one or two lines on its own? There's no upper limit on how long a proof to a given problem must be in any fixed language. And if you have to invent new language, people won't be able to read it anymore.
PeterisP 9y ago
You could, there are many approaches to design interpretable/explainable systems (Here https://distill.pub/2018/building-blocks/ https://distill.pub/2018/building-blocks/ is a nice and recent review of some of these problems with illustrations), but there are serious tradeoffs so people usually choose not to do so. The first issue is that, in general, explaining a decision is a harder task than making a decision, just as reviewing and understanding someone's decision takes more effort than it took to make it. This means that adding explainability and interpretability to a system that achieves some result will take more effort, time and money than building that system itself - you'll triple your budget and get no better actual results, just a better feeling about them. This also limits the effort invested in tools and methods for this; there's large demand and funding for making stuff work or making stuff a bit better, so many people are getting paid to work on that, and not nearly so much on interpretability. The second issue is that best performing systems take "everything" into account and thus become too complex for a human to understand. A system can be explainable, interpretable and accessible, but if making a correct decision inherently relies on complicated interrelations between many factors, then it will be difficult (or at least time-consuming) for humans to understand anyway. If you limit the complexity of the system to something that can be easily explained, then you're severely limiting its power, and you get poor results. For your query planner, the analogy is the scenario where we could build a query planner that is capable of optimizing beyond the commonly used basic blocks in query plans, instead outputting optimized machine code to perform particular query better than the standard blocks, achieving improved performance by merging and interleaving operations. The resulting output would be much harder to interpret, since no "block-level" query plan can be equivalent; and requiring the operations to be neatly separable in a limited number of blocks each with separate human-understandable meaning would mean restricting the system and inevitably getting a worse result. The same applies to humans, by the way - being able to explain all the factors why you believe your decision is likely to be correct takes more knowledge, insight and effort than simply making that decision. It's not like humans can explain why exactly they see (or don't see) an animal in picture like http://www.naute.com/images/cow.gif http://www.naute.com/images/cow.gif , consciously understanding that in a structured way is very, very hard. If you would require a human to explain the criteria they'll use before they get to make a decision (as opposed to post-decision rationalization that often has no relation to why the decision was made, and has more relation to making up a socially acceptable plausible justification why you did what you did), then you get poor, simplified criteria that can't reflect the complexity of the problem; you could get their "rules of thumb" or their major factors, but explaining why exactly it is so is generally a mentoring task that takes months (if not more) of effort both from the teacher and the student. AlphaZero system plays Go and Chess better than any human. It could be modified to describe in detail why it chose this move over the other, but would you (or any other human) have the mental capacity to truly understand it, no matter how well it's explained? The distill.pub article illustrates lots of ways how an image classifier can explain why it made one choice or another - but is it actually useful for tasks other than for R&D to debug or improve the model? It seems like a nice-to-have feature that would be liked if it's not to expensive, but I'd bet that for most models the company wouldn't actually use the explainability feature (again, outside of its builders debugging or improving it) because the whole point of ML is that people get removed out of the loop and don't look at the decisions.

BenoitP 9y ago

> The downside is it now similarly becomes challenging to formalize the machine's intuition Tooting my own horn here: I'm taking up this challenge professionally. There are new methods being discovered to extract interpretations for humans out of a model[1][2][3]; and I'll gladly work and provide advice and implementations on this, as an independent contractor. Having an interpretable/explicable ML pipeline helps in: * Knowing what drives your customers' buying patterns * Knowing where to look for signals closer to the root, helping go from correlation to causation * Orienting your marketing messages. You don't engage users the same way if they're a repeat buyer or a "hot now" lead * Concentrating on "would buy our product but won't because of X", which is where the market share battle lies; as opposed to "definitely would buy" and "definitely wouldn't buy", which are cases where you have little influence on. * Listing buying patterns exhaustively, enabling you to discover low noise niches * Providing auditable explanations of the decisions the model has taken, if it has been required legally Here is my email: pro@benoit.paris [1] https://distill.pub/2018/building-blocks/ https://distill.pub/2018/building-blocks/ [2] https://www.darpa.mil/program/explainable-artificial-intelligence https://www.darpa.mil/program/explainable-artificial-intelli... [3] http://interpretable.ml/ http://interpretable.ml/

al2o3cr 9y ago

"itself becoming an expert through training" One issue this runs into: the machine becomes an expert at something that matches what it was exposed to during training. The research on adversarial examples shows that the thing being learned != what the expert learns, frequently to an extreme degree.

radarsat1 9y ago
And anyone working on ML/AI who doesn't understand overfitting is selling snake oil.