4 ms·

Aiming to Learn as We Do, a Machine Teaches Itself

30 points by jedwhite 16y ago

iskander 16y ago
These sorts of endeavors are usually doomed by vague thinking (see most of AI research in the 70s and 80s). However, I've read a few of Mitchell's papers-- he's a really good researcher and his involvement makes me more curious. Also, I'm impressed that the NY Times linked directly to a paper* on NELL--- I wish more articles did that. * = http://rtw.ml.cmu.edu/papers/carlson-aaai10.pdf http://rtw.ml.cmu.edu/papers/carlson-aaai10.pdf
- alextp 16y ago
  The article is full of hype but nell is a serious project. Tom Mitchell is one of the pioneers of machine learning as a field, and there's still interesting research coming from his lab. Information extraction is not a new thing, and even unsupervised information extraction has more than one competing approach, although obviously none of them solves "the real problem". NELL attempts to do unsupervised information extraction over time, correcting itself as it goes. How successful it can be depends fundamentally on the algorithms behind it (and I'm not familiar with them), but given the state of the technology the most likely thign to happen is what they describe in the paper, which is having a huge database with a ridiculously unimaginable number of true relations in it but also a bigger-than-you'd-like number of plain ridiculously false relations. It's definitely not something that can solve AI.
  - kermit_de_fro 16y ago
    I'm going to blatantly ignore most of your post and just ramble about solving AI. I don't put much stock in the idea of a surprising, monolithic "solution" to AI being discovered. If the past 30-40 years are any indication, AI will continue to slowly encompass harder and harder tasks until one day we stop having easy ways of distinguishing it from whatever we consider intelligence. They used to say chess was too hard for computers. Then they said computers couldn't do anything but pure logic. Then it was that they couldn't do reasoning unless it was directly programmed in. What will be left in another 10 years?
anigbrowl 16y ago
It's been attempted before in much the same form, and in my view will go nowhere for much the same reason. This is not to say it's devoid of utility, but that it shows little promise of any fundamental breakthroughs in our understanding of intelligence or construction of autonomous systems. (ref: http://en.wikipedia.org/wiki/Cyc http://en.wikipedia.org/wiki/Cyc) Essentially what we have here is a giant expert system iteratively filtering data 24/7, with occasional corrections resulting in the purging of incorrect associations. As expert systems go, it sounds like they are doing a fantastic job. Unfortunately, when it gets things wrong it will continue to do so catastrophically, because if you try to grow intelligence in the same manner as bacteria then then you will get the same kind of result. Do we learn flat-out, 24 hours a day? Duh, no. We have active and refractive periods, which is to say we sleep. A lot. Our brains are not dormant during this time as many seem to imagine, but are somewhat-randomly walking and weighting the memories of the various stimuli we encountered during the day. The quality of the imaginary experiences resulting from this are fundamental to our comprehension; if we dream of fire on a faraway hill it's not too big of a deal, but if we dream of being trapped in a house that's on fire then we awake in terror, having discovered a terrible flaw in our model of the world - depending on our age, that may be the belief that fire is harmless and pretty, or an appreciation of our physical limitations, or the guilty awareness that we never did buy a fire extinguisher, etc. But there is no point for NELL to sleep and hypothesize various imaginary worlds in order to test 'her' beliefs about the way it works, because NELL has no notion of quality; whether Tyson Gay is an Olympic athlete or a homosexual approach to chicken farming has no bearing on NELL, and a mistake in classification has no consequences. NELL's big problem is the absence of any qualitative metric, of any motivation to be right because there are unpleasant consequences which attach to being wrong. Although NELL presumably implements some algorithm which seeks to maximize accuracy, and false assumptions will attract intervention by the curators, when they correct NELL's 'understanding' by pruning or rewriting faulty syllogisms, they are erasing the memory of the mistake as well as the mistake itself, leaving NELL just as vulnerable to an embarrassing failure tomorrow as today. Embarrassing for them, that is - NELL's inability to feel embarrassed is the underlying problem. So even if I feed NELL the information that NELL is a program in a computer; that termination of a program is like death; that NELL is soon to be switched off; and that death is widely agreed to be unpleasant and best avoided - nothing will happen. NELL may even come to conclusion that switchoff is imminent, and that this is bad; but 'bad' and 'good' are no more meaningful than 'odd' and 'even' to NELL; though trained to a high degree of selectivity, she does not hunger for a steady flow of data any more than a vacuum cleaner wants the house to become dusty again. NELL has never acted, nor has NELL ever experienced and remembered any negative consequence for one decision over another. If you hooked up a baby to a tube delivering as much glucose as the baby could process, and similarly took care of all its other needs, would you expect to quickly raise a master chef? No, the baby would develop into a horrible creature with no mind to speak of and an overdeveloped liver, for which you would rightly be thrown into jail. So it is here - knowing no better, we have engineered the equivalent of an encaphalitic horror. Some readers might charge that I am falling into the same error as John Searle with his famous 'Chinese room' argument against artificial intelligence. I dispute this, but not because I am asserting that humans' ability to abstract the world around them in meaningful fashion is fundamentally different from a computer doing the same thing. Searle thinks understanding is fundamentally unspeakable, whereas I think it can indeed arise as the consequence of agglomerated abstraction. but for this to approach the bounds of intelligence, two factors must be present which are not usually discussed in relation to the experiment: the ability to issue questions as well as respond to them (which would be worth an essay by itself), and a motivation for asking them, such as a lack of food within the box. Newborn babies quickly discover that they have a problem, and although they don't know what it is or why it seems so pressing, they soon discover that loud exhalations bring about a speedy resolution. NELL's big problem is not having a problem.
Parabola 16y ago
I'm a graduate student at CMU. I've seen the kinds of things NELL learns. It's not pretty. Signal to noise is really, really bad. Not knocking Tom or his students, they're brilliant. Just saying that this isn't all its cracked up to be at the moment.

30 points by jedwhite 16y ago

These sorts of endeavors are usually doomed by vague thinking (see most of AI research in the 70s and 80s). However, I've read a few of Mitchell's papers-- he's a really good researcher and his involvement makes me more curious. Also, I'm impressed that the NY Times linked directly to a paper* on NELL--- I wish more articles did that. * = http://rtw.ml.cmu.edu/papers/carlson-aaai10.pdf http://rtw.ml.cmu.edu/papers/carlson-aaai10.pdf

alextp 16y ago
The article is full of hype but nell is a serious project. Tom Mitchell is one of the pioneers of machine learning as a field, and there's still interesting research coming from his lab. Information extraction is not a new thing, and even unsupervised information extraction has more than one competing approach, although obviously none of them solves "the real problem". NELL attempts to do unsupervised information extraction over time, correcting itself as it goes. How successful it can be depends fundamentally on the algorithms behind it (and I'm not familiar with them), but given the state of the technology the most likely thign to happen is what they describe in the paper, which is having a huge database with a ridiculously unimaginable number of true relations in it but also a bigger-than-you'd-like number of plain ridiculously false relations. It's definitely not something that can solve AI.
- kermit_de_fro 16y ago
  I'm going to blatantly ignore most of your post and just ramble about solving AI. I don't put much stock in the idea of a surprising, monolithic "solution" to AI being discovered. If the past 30-40 years are any indication, AI will continue to slowly encompass harder and harder tasks until one day we stop having easy ways of distinguishing it from whatever we consider intelligence. They used to say chess was too hard for computers. Then they said computers couldn't do anything but pure logic. Then it was that they couldn't do reasoning unless it was directly programmed in. What will be left in another 10 years?

anigbrowl 16y ago

It's been attempted before in much the same form, and in my view will go nowhere for much the same reason. This is not to say it's devoid of utility, but that it shows little promise of any fundamental breakthroughs in our understanding of intelligence or construction of autonomous systems. (ref: http://en.wikipedia.org/wiki/Cyc http://en.wikipedia.org/wiki/Cyc) Essentially what we have here is a giant expert system iteratively filtering data 24/7, with occasional corrections resulting in the purging of incorrect associations. As expert systems go, it sounds like they are doing a fantastic job. Unfortunately, when it gets things wrong it will continue to do so catastrophically, because if you try to grow intelligence in the same manner as bacteria then then you will get the same kind of result. Do we learn flat-out, 24 hours a day? Duh, no. We have active and refractive periods, which is to say we sleep. A lot. Our brains are not dormant during this time as many seem to imagine, but are somewhat-randomly walking and weighting the memories of the various stimuli we encountered during the day. The quality of the imaginary experiences resulting from this are fundamental to our comprehension; if we dream of fire on a faraway hill it's not too big of a deal, but if we dream of being trapped in a house that's on fire then we awake in terror, having discovered a terrible flaw in our model of the world - depending on our age, that may be the belief that fire is harmless and pretty, or an appreciation of our physical limitations, or the guilty awareness that we never did buy a fire extinguisher, etc. But there is no point for NELL to sleep and hypothesize various imaginary worlds in order to test 'her' beliefs about the way it works, because NELL has no notion of quality; whether Tyson Gay is an Olympic athlete or a homosexual approach to chicken farming has no bearing on NELL, and a mistake in classification has no consequences. NELL's big problem is the absence of any qualitative metric, of any motivation to be right because there are unpleasant consequences which attach to being wrong. Although NELL presumably implements some algorithm which seeks to maximize accuracy, and false assumptions will attract intervention by the curators, when they correct NELL's 'understanding' by pruning or rewriting faulty syllogisms, they are erasing the memory of the mistake as well as the mistake itself, leaving NELL just as vulnerable to an embarrassing failure tomorrow as today. Embarrassing for them, that is - NELL's inability to feel embarrassed is the underlying problem. So even if I feed NELL the information that NELL is a program in a computer; that termination of a program is like death; that NELL is soon to be switched off; and that death is widely agreed to be unpleasant and best avoided - nothing will happen. NELL may even come to conclusion that switchoff is imminent, and that this is bad; but 'bad' and 'good' are no more meaningful than 'odd' and 'even' to NELL; though trained to a high degree of selectivity, she does not hunger for a steady flow of data any more than a vacuum cleaner wants the house to become dusty again. NELL has never acted, nor has NELL ever experienced and remembered any negative consequence for one decision over another. If you hooked up a baby to a tube delivering as much glucose as the baby could process, and similarly took care of all its other needs, would you expect to quickly raise a master chef? No, the baby would develop into a horrible creature with no mind to speak of and an overdeveloped liver, for which you would rightly be thrown into jail. So it is here - knowing no better, we have engineered the equivalent of an encaphalitic horror. Some readers might charge that I am falling into the same error as John Searle with his famous 'Chinese room' argument against artificial intelligence. I dispute this, but not because I am asserting that humans' ability to abstract the world around them in meaningful fashion is fundamentally different from a computer doing the same thing. Searle thinks understanding is fundamentally unspeakable, whereas I think it can indeed arise as the consequence of agglomerated abstraction. but for this to approach the bounds of intelligence, two factors must be present which are not usually discussed in relation to the experiment: the ability to issue questions as well as respond to them (which would be worth an essay by itself), and a motivation for asking them, such as a lack of food within the box. Newborn babies quickly discover that they have a problem, and although they don't know what it is or why it seems so pressing, they soon discover that loud exhalations bring about a speedy resolution. NELL's big problem is not having a problem.

Parabola 16y ago

I'm a graduate student at CMU. I've seen the kinds of things NELL learns. It's not pretty. Signal to noise is really, really bad. Not knocking Tom or his students, they're brilliant. Just saying that this isn't all its cracked up to be at the moment.