3 ms·

> Well we do actually have a purely mathematical approach to AI worked out. Supposedly with enough computer power and enough data, a one stroke solution to eve

by graycat 13y ago

> Well we do actually have a purely mathematical approach to AI worked out. Supposedly with enough computer power and enough data, a one stroke solution to everything is stochastic optimal control, but that solution takes, say, just brute force to, say, planetary motion instead of Newton's second law of motion and law of gravity. Else, need to insert such laws into the software, but we would insert only laws humans knew from the past, or have the AI software discover such laws, not so promising. This stochastic optimal control approach is not practical or even very insightful. But it is mathematical. > machine learning is making progress in leaps and bounds. I looked at Prof Ng's machine learning course, and all I saw was some old intermediate statistics, in particular, maximum likelihood estimation (MLE), done badly. I doubt that we have any solid foundation to build on for any significantly new and powerful techniques for machine learning. I see nothing in machine learning that promises to be anything like human intelligence. Sure, we can write a really good chess program, but no way do we believe that its internals are anything like human intelligence. > As is our understanding of the brain. Right, there are lots of neurons. And if someone gets a really big injury just above their left ear, then we have a good guess at what the more obvious results will be. But that's not much understanding of how the brain actually works. It's a little like we have a car, have no idea what's under the hood, and are asked to build a car. Maybe we are good with metal working, but we don't even know what a connecting rod is. > It wasn't that long after engines were invented that we got airplanes. The rest needed was relatively simple, the wind tunnel, some spruce wood, glue, linen, paint, wire, and good carpentry. For the equivalent parts of AI, I doubt that we have even a weak little hollow hint of a tiny clue. In some of the old work in AI, it was said that a core challenge was the 'representation problem'. If all that was meant was just what programming language data structures to use, then that was not significant progress. Or, sure, we have a shot at understanding the 'sensors' and 'transducers' that are connected to the brain: Sensors: Pain, sound, sight, taste, etc. Transducers: Muscles, speech, eye focus, etc. We know some about how the middle and inner ear handles sound and the gross parts of the eye. And if we show a guy a picture of a pretty girl, then we can see what parts of his brain become more active. And we know that there are neurons firing. But so far it seems that that's about it. So, that's like my computer: For sensors and transducers it has a keyboard, mouse, speakers, printer, Ethernet connection, etc. And if we look deep inside then we see a lot of circuits and transistors. But my kitty cat has no idea at all about the internals of the software that runs in my computer, and by analogy I see no understanding of the analogous details inside a human brain. Or, we have computers, and we can write software for them using If-Then-Else, Do-While, Call-Return, etc., but for writing software comparable with a human brain we don't know the first character to type into an empty file for the software. In simple terms, we don't have a software design. Or, it's like we are still in the sixth grade, have learned, say, Python, and are asked to write software to solve the ordinary differential equations of space flight to the outer planets -- we don't know where to start. Or, closer in, we're asked to write software to solve the Navier-Stokes equations -- once we get much past toy problems, our grid software goes unstable and gives wacko results. Net, we just don't yet know how to program anything like real, human intelligence.

Houshalter 13y ago
I was referring to AIXI as the perfect mathematical AI. The main recent advancement in machine learning is deep learning. It's advanced the state of the art in machine vision and speech recognition quite a bit. Machine learning is on a spectrum from "statistics" with simple models and low dimensional data, to "AI" with complicated models and high dimensional data. >if someone gets a really big injury just above their left ear, then we have a good guess at what the more obvious results will be. But that's not much understanding of how the brain actually works. Neuroscience is a bit beyond that. I believe there are also some large projects like Blue Brain working on the problem. I swear I saw a video somewhere of a simulation of a neocortex that could do IQ test type questions and respond just like a human. But the point is we do have more than nothing.
- graycat 13y ago
  > I was referring to AIXI as the perfect mathematical AI. I will have to Google AIXI. A big point about being mathematical is that that is about the only solid way we can evaluate candidate work before running software and, say, something like a Turing test. Some math is most of why we know, well before any software is written, that (1) heap sort will run in n ln(n), (2) AVL trees find leaves in ln(n), and (3) our calculations for navigating a space craft to the outer planets will work. More generally, the math is 'deductive' in a severe and powerful sense and, thus, about the only tool we have to know well in advance of, say, writing a lot of software. But math does not have 'truth' and, instead, needs hypotheses. So, for hypotheses, for some design for some software for AI, we need some. Enough hypotheses are going to be a bit tough to find. And math gives only some mathematical conclusions, and we will need to know that these are sufficient for AI; for that we will want, likely need, a sufficiently clear definition of AI, that is, something better than just an empirical test such as a Turing test or doing well on and IQ test. Tough challenge. Instead of such usage of math, about all we have in AI for a 'methodology' is, (1) here I have some intuitive ideas I like, (2) with a lot of money I can write the code and, maybe, get it to run, and (3) trust me, that program can read a plane geometry book with all the proofs cut out and, then, fill in all the proofs or some such. So, steps (1) and (2) are, in the opinion of anyone else, say, DARPA, 'long shots', and (3) will be heavily in the eye of the beholder. The challenges of (1), (2), and (3) already make AI an unpromising direction. > The main recent advancement in machine learning is deep learning. It's advanced the state of the art in machine vision and speech recognition quite a bit. AI has been talking about 'deep knowledge' for a long time. That was, say, in a program that could diagnose car problems, 'knowledge' that the engine connected to the transmission connected to the drive shaft connected to the differential connected to the rear wheels or some such and, then, be able to use this 'knowledge' in 'reasoning' to diagnose problems. E.g., a vibration could be caused by worn U-joints. When I was in AI, when I worked in the field, there were plenty of people who saw the importance of such 'deep knowledge' but had next to nothing on really how to make it real. For 'deep learning', the last I heard, that was tweaking the parameters 'deep' in some big 'neural network', basically a case of nonlinear curve fitting. Somehow I just don't accept that such a 'neural network' is nearly all that makes a human brain work; that is, I'd expect to see some promising 'organization' at a higher level than just the little elements for the nonlinear curve fitting. E.g., for speech recognition, I believe an important part of how humans do it is to take what they heard, which is often quite noisy and by itself just not nearly enough, and compare it with what they know about the subject under discussion and, then, based on that 'background knowledge', correct the noisy parts of what they heard. E.g., if the subject is a cake recipe for a party for six people, then it's not "a cup of salt" but maybe a cup or two or three of flour. If the subject is the history of US presidents and war, then "I'll be j..." may be LBJ and "sson" maybe "Nixon". Here the speech recognition is heavily from a base of 'subject understanding'. An issue will be, how the heck does the human brain sometimes make such 'corrections' so darned fast. For image recognition, the situation has to be in part similar but more so: I doubt that we have even a shot at image recognition without a 'prior library' of 'object possibilities': That is, if we are looking at an image, say, from satellite, of some woods and looking for a Russian tank hidden there, then we need to know what a Russian tank looks like so that we can guess what a hidden Russian tank would look like on the image so that we can, then, look for that on the image. Here we have to understand lighting, shadows, what a Russian tank looks like from various directions, etc. So, we are using some real 'human knowledge' of the real thing, the tank, we are looking for. E.g., my kitty cat has a food tray. He knows well the difference between that tray and everything else that might be around it -- jug of detergent, toaster, bottle of soda pop, decorative vase, a kitchen timer. Then I can move his food tray, and he doesn't get confused at all. Net, what he is doing with image recognition is not just simplistic and, instead, has within it a 'concept' of his food tray, a concept that he created. He's not stupid you know! So, I begin to conclude that for speech and image recognition, e.g., handwriting recognition, we need a large 'base' of 'prior human knowledge' about the 'subject area', e.g., with 'concepts', etc., before we start. That is, we need close to 'full, real AI' just to, say, do well reading any handwriting. From such considerations, I believe we have a very long way to go. Broadly one of my first cut guesses about how to proceed would be to roll back to something simpler in two respects. First, start with brains smaller, hopefully simpler, than those of humans. Maybe start with a worm and work up to a frog, bird, ..., in a few centuries, a kitty cat! Second, start with the baby animal and see how it learns once it starts to as an egg, once it's born, what it gets from its mother, etc. So, eventually work up to software that could start learning with just "Ma ma' and proceed from there. But can't just start with humans and "Ma ma" because a human just born likely already has somehow built in a lot that is crucial we just don't have a clue about. So, start with worms, frogs, birds, etc. Another idea for how to proceed is to try for just simple 'cognition' with just text and image input and just text output. E.g., start with something that can diagram English sentences and move from there to some 'understanding', e.g., have made progress enough with 'meaning' that, e.g., know when two sentences with quite different words and grammar really mean essentially the same thing and when they don't mean the same thing report why not and be able to revise one of the sentences so that the two do mean the same thing. So, here we are essentially assuming that AI has to stand on some capabilities with language -- do kitty cats have an 'internal language'? Hmm ...! If kitty cats don't have such an 'internal language', then I am totally stuck! Then with some text and image input, the thing should be able to cook up a good proof of the Pythagorean theorem. I can believe that some software can diagram English sentences or come close to it, but that is just a tiny start on what I am suggesting. The real challenge, as I am guessing, is to have the software keep track of and manipulate 'meaning', whatever the heck that is. And I would anticipate a 'bootstrap' approach: Postulate and program something for doing such things with meaning, 'teach' it, and then look at the 'connections' it has built internally, say, between words and meaning, and also observe that the thing appears to work well. So, it's a 'bootstrap' because it works without our having any very good prior idea just why; that is, we could not prove in advance that it could work. So, for kitty cat knowledge, have it understand its environment in terms of 'concepts' (part of 'meaning') hard, soft, strong, weak, hot, cold, and, then, know when it can use its claws to hold on to a soft, strong, not too hot or too cold surface, push out of the way a hard, weak obstacle, etc. Maybe some such research direction could be made to work. But I'm not holding my breath waiting.

by graycat 13y ago

Houshalter 13y ago

I was referring to AIXI as the perfect mathematical AI. The main recent advancement in machine learning is deep learning. It's advanced the state of the art in machine vision and speech recognition quite a bit. Machine learning is on a spectrum from "statistics" with simple models and low dimensional data, to "AI" with complicated models and high dimensional data. >if someone gets a really big injury just above their left ear, then we have a good guess at what the more obvious results will be. But that's not much understanding of how the brain actually works. Neuroscience is a bit beyond that. I believe there are also some large projects like Blue Brain working on the problem. I swear I saw a video somewhere of a simulation of a neocortex that could do IQ test type questions and respond just like a human. But the point is we do have more than nothing.

graycat 13y ago
> I was referring to AIXI as the perfect mathematical AI. I will have to Google AIXI. A big point about being mathematical is that that is about the only solid way we can evaluate candidate work before running software and, say, something like a Turing test. Some math is most of why we know, well before any software is written, that (1) heap sort will run in n ln(n), (2) AVL trees find leaves in ln(n), and (3) our calculations for navigating a space craft to the outer planets will work. More generally, the math is 'deductive' in a severe and powerful sense and, thus, about the only tool we have to know well in advance of, say, writing a lot of software. But math does not have 'truth' and, instead, needs hypotheses. So, for hypotheses, for some design for some software for AI, we need some. Enough hypotheses are going to be a bit tough to find. And math gives only some mathematical conclusions, and we will need to know that these are sufficient for AI; for that we will want, likely need, a sufficiently clear definition of AI, that is, something better than just an empirical test such as a Turing test or doing well on and IQ test. Tough challenge. Instead of such usage of math, about all we have in AI for a 'methodology' is, (1) here I have some intuitive ideas I like, (2) with a lot of money I can write the code and, maybe, get it to run, and (3) trust me, that program can read a plane geometry book with all the proofs cut out and, then, fill in all the proofs or some such. So, steps (1) and (2) are, in the opinion of anyone else, say, DARPA, 'long shots', and (3) will be heavily in the eye of the beholder. The challenges of (1), (2), and (3) already make AI an unpromising direction. > The main recent advancement in machine learning is deep learning. It's advanced the state of the art in machine vision and speech recognition quite a bit. AI has been talking about 'deep knowledge' for a long time. That was, say, in a program that could diagnose car problems, 'knowledge' that the engine connected to the transmission connected to the drive shaft connected to the differential connected to the rear wheels or some such and, then, be able to use this 'knowledge' in 'reasoning' to diagnose problems. E.g., a vibration could be caused by worn U-joints. When I was in AI, when I worked in the field, there were plenty of people who saw the importance of such 'deep knowledge' but had next to nothing on really how to make it real. For 'deep learning', the last I heard, that was tweaking the parameters 'deep' in some big 'neural network', basically a case of nonlinear curve fitting. Somehow I just don't accept that such a 'neural network' is nearly all that makes a human brain work; that is, I'd expect to see some promising 'organization' at a higher level than just the little elements for the nonlinear curve fitting. E.g., for speech recognition, I believe an important part of how humans do it is to take what they heard, which is often quite noisy and by itself just not nearly enough, and compare it with what they know about the subject under discussion and, then, based on that 'background knowledge', correct the noisy parts of what they heard. E.g., if the subject is a cake recipe for a party for six people, then it's not "a cup of salt" but maybe a cup or two or three of flour. If the subject is the history of US presidents and war, then "I'll be j..." may be LBJ and "sson" maybe "Nixon". Here the speech recognition is heavily from a base of 'subject understanding'. An issue will be, how the heck does the human brain sometimes make such 'corrections' so darned fast. For image recognition, the situation has to be in part similar but more so: I doubt that we have even a shot at image recognition without a 'prior library' of 'object possibilities': That is, if we are looking at an image, say, from satellite, of some woods and looking for a Russian tank hidden there, then we need to know what a Russian tank looks like so that we can guess what a hidden Russian tank would look like on the image so that we can, then, look for that on the image. Here we have to understand lighting, shadows, what a Russian tank looks like from various directions, etc. So, we are using some real 'human knowledge' of the real thing, the tank, we are looking for. E.g., my kitty cat has a food tray. He knows well the difference between that tray and everything else that might be around it -- jug of detergent, toaster, bottle of soda pop, decorative vase, a kitchen timer. Then I can move his food tray, and he doesn't get confused at all. Net, what he is doing with image recognition is not just simplistic and, instead, has within it a 'concept' of his food tray, a concept that he created. He's not stupid you know! So, I begin to conclude that for speech and image recognition, e.g., handwriting recognition, we need a large 'base' of 'prior human knowledge' about the 'subject area', e.g., with 'concepts', etc., before we start. That is, we need close to 'full, real AI' just to, say, do well reading any handwriting. From such considerations, I believe we have a very long way to go. Broadly one of my first cut guesses about how to proceed would be to roll back to something simpler in two respects. First, start with brains smaller, hopefully simpler, than those of humans. Maybe start with a worm and work up to a frog, bird, ..., in a few centuries, a kitty cat! Second, start with the baby animal and see how it learns once it starts to as an egg, once it's born, what it gets from its mother, etc. So, eventually work up to software that could start learning with just "Ma ma' and proceed from there. But can't just start with humans and "Ma ma" because a human just born likely already has somehow built in a lot that is crucial we just don't have a clue about. So, start with worms, frogs, birds, etc. Another idea for how to proceed is to try for just simple 'cognition' with just text and image input and just text output. E.g., start with something that can diagram English sentences and move from there to some 'understanding', e.g., have made progress enough with 'meaning' that, e.g., know when two sentences with quite different words and grammar really mean essentially the same thing and when they don't mean the same thing report why not and be able to revise one of the sentences so that the two do mean the same thing. So, here we are essentially assuming that AI has to stand on some capabilities with language -- do kitty cats have an 'internal language'? Hmm ...! If kitty cats don't have such an 'internal language', then I am totally stuck! Then with some text and image input, the thing should be able to cook up a good proof of the Pythagorean theorem. I can believe that some software can diagram English sentences or come close to it, but that is just a tiny start on what I am suggesting. The real challenge, as I am guessing, is to have the software keep track of and manipulate 'meaning', whatever the heck that is. And I would anticipate a 'bootstrap' approach: Postulate and program something for doing such things with meaning, 'teach' it, and then look at the 'connections' it has built internally, say, between words and meaning, and also observe that the thing appears to work well. So, it's a 'bootstrap' because it works without our having any very good prior idea just why; that is, we could not prove in advance that it could work. So, for kitty cat knowledge, have it understand its environment in terms of 'concepts' (part of 'meaning') hard, soft, strong, weak, hot, cold, and, then, know when it can use its claws to hold on to a soft, strong, not too hot or too cold surface, push out of the way a hard, weak obstacle, etc. Maybe some such research direction could be made to work. But I'm not holding my breath waiting.