5 ms·

I rather dislike ANNs. I recognize that ANNs perform well and find research to improve their performance interesting, but honestly they're completely opaque bla

by tel 15y ago

I rather dislike ANNs. I recognize that ANNs perform well and find research to improve their performance interesting, but honestly they're completely opaque black boxes with structure that only plausibly mimics the absolute simplest of biological neural networks. I want a "brain" which was built from principles that allow its workings to be intelligible and efficient based on a technique which reflects the structure of the problem of learning, generalization, and hypothesis search and illuminates it. So I'll buy (and often use) your ANNs, but I can't help but feel a little bit worried about the state of affairs. It's probably a whole lot like that automatically enumerated proof of the 4-color theorem. It's definitely important, but there is something unsatisfying about it. We aren't learning as much as we felt entitled to because the destination was only a small part of the journey.

Game_Ender 15y ago
That is the entire machine learning field. Very well designed black boxes which you train to classify or estimate your desired quantities based on you input data. It might not be true "AI" but it gets a lot of practical work done.
- tel 15y ago
  I don't care a thing about "true AI". I just want ML to learn more from statistics.
tansey 15y ago
> they're completely opaque black boxes with structure that only plausibly mimics the absolute simplest of biological neural networks I guess this depends on how advanced your toolkit is. If you're simply using feedforward nets with fixed weights, fixed topology, and sigmoid activation, then yes that's probably right. However, there are a lot of advanced techniques in ANNs that most people never use. Lots of more biologically-inspired approaches have been explored, such as leaky integrator neurons, Hebbian learning, and indirect encodings. > I want a "brain" which was built from principles See, but the problem is that we don't even have the knowledge of how real brains work to start forming such principals. In fact, we're starting to see computational models used to form principals of biological brains (for example, [1]). > that allow its workings to be intelligible And what does that mean? You want to be able to discern some rule base from the system that you can understand? Even biological brains do not have that property. > based on a technique which reflects the structure of the problem of learning, generalization, and hypothesis search and illuminates it. This is very vague. Happy to dive into it if you are willing to expound a bit. > It's probably a whole lot like that automatically enumerated proof of the 4-color theorem. It's definitely important, but there is something unsatisfying about it. We aren't learning as much as we felt entitled to because the destination was only a small part of the journey. Again, very vague. Do you think having children is unsatisfying because you can't understand their actions most of the time? Maybe I'm misunderstanding the issue here. Seriously though, I want to know what will impress people here. It's not going to be a fully general, "human-level" AI, obviously, as that requires huge amounts of semantic knowledge about the world that was encoded through billions of years of evolution, but that to me is not really a necessary condition for AI. [1] http://nn.cs.utexas.edu/?hoffman:biopsych11 http://nn.cs.utexas.edu/?hoffman:biopsych11
- RobertKohr 15y ago
  Thank you for your solid reply. I think ANN is the real thing, and most of the other "AI" solutions that we have give the impression of intelligence, but are simply algorithms that solve problems we have in very well defined ways. ANN is a general intelligence that gets shaped by experience and adapts figure out its own solutions. This is the path that will lead to our concept of AI presented in sci-fi and what the general public actually thinks of. It isn't some coder banging on the keyboard trying to replicate what a human would do under a specific situation, it is something that truly learns. We have a long way to go before this becomes a reality, but it will happen.
  - khafra 15y ago
    Tansey wrote a good, solid reply, but I dislike worshipping at the altar of mysteriousness. When a human or better GAI finally gets built, it won't be entirely hand-coded (although the seed for it might be). But the better we understand how our AI works, the better we can make sure it's doing what we want it to; and not, say, crashing the stock market with millions of erroneous transactions carried out too fast for humans to correct.
  - tel 15y ago
    That's very romantic, but I think instead we're going to learn that intelligence is not such a well-defined idea as to be engineered into anything imagined by science fiction. Further, there are plenty of techniques which aren't ANNs but still are "shaped by experience" and "figure out their own solutions". I side pretty strongly with the idea of building tools to bring the powerful modes of computerized learning close to the powerful modes of human learning. I think we're inordinately far away from replicating one with the other.
  - haliax 15y ago
    > I think ANN is the real thing, and most of the other "AI" solutions that we have give the impression of intelligence, but are simply algorithms that solve problems we have in very well defined ways...ANN is a general intelligence that gets shaped by experience and adapts figure out its own solutions...It isn't some coder banging on the keyboard trying to replicate what a human would do under a specific situation, it is something that truly learns. It's not ANNs on the one hand and "some coder banging on the keyboard trying to replicate what a human would do" on the other. Supervised learning is the most common use case for an ANN, and means learning from labelled training data during a training phase and then not changing; it can be done with a number of techniques, including Support Vector Machines, Decision Trees, Bayesian methods, and so on. Reinforcement Learning on the other hand, is continuous learning from trying things and making mistakes -- experience, in other words. It too can be done a number of ways, like with Evolutionary Algorithms, Markov Decision Processes, Inductive Logic Programming, etc. Hell, PG's spam classifier learns from experience, and it's nothing more than a Naive Bayes classifier ( http://www.paulgraham.com/spam.html http://www.paulgraham.com/spam.html ). Without going on for much longer, my point is that there really is no reason to exalt ANNs the way you have. Disclaimer: I'm a huge believer in Artificial General Intelligence via Competent Program Search ( see http://metacog.org/research.html http://metacog.org/research.html + http://www.google.com/url?sa=t&source=web&cd=2&ved=0CCIQtwIwAQ&url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D5hsvCib83ME&rct=j&q=goertzel%20google%20tech%20talk&ei=BMGYTtnaG8bm0QHpt7XZBA&usg=AFQjCNHrK7fPXL05JlSK3JCaXttNt_0aog&sig2=ZBa62SPhTzRFSCtMLKtneA&cad=rja http://www.google.com/url?sa=t&source=web&cd=2&v... + www.idsia.ch/~juergen/ )

by tel 15y ago

Game_Ender 15y ago

That is the entire machine learning field. Very well designed black boxes which you train to classify or estimate your desired quantities based on you input data. It might not be true "AI" but it gets a lot of practical work done.

tel 15y ago
I don't care a thing about "true AI". I just want ML to learn more from statistics.

tansey 15y ago

> they're completely opaque black boxes with structure that only plausibly mimics the absolute simplest of biological neural networks I guess this depends on how advanced your toolkit is. If you're simply using feedforward nets with fixed weights, fixed topology, and sigmoid activation, then yes that's probably right. However, there are a lot of advanced techniques in ANNs that most people never use. Lots of more biologically-inspired approaches have been explored, such as leaky integrator neurons, Hebbian learning, and indirect encodings. > I want a "brain" which was built from principles See, but the problem is that we don't even have the knowledge of how real brains work to start forming such principals. In fact, we're starting to see computational models used to form principals of biological brains (for example, [1]). > that allow its workings to be intelligible And what does that mean? You want to be able to discern some rule base from the system that you can understand? Even biological brains do not have that property. > based on a technique which reflects the structure of the problem of learning, generalization, and hypothesis search and illuminates it. This is very vague. Happy to dive into it if you are willing to expound a bit. > It's probably a whole lot like that automatically enumerated proof of the 4-color theorem. It's definitely important, but there is something unsatisfying about it. We aren't learning as much as we felt entitled to because the destination was only a small part of the journey. Again, very vague. Do you think having children is unsatisfying because you can't understand their actions most of the time? Maybe I'm misunderstanding the issue here. Seriously though, I want to know what will impress people here. It's not going to be a fully general, "human-level" AI, obviously, as that requires huge amounts of semantic knowledge about the world that was encoded through billions of years of evolution, but that to me is not really a necessary condition for AI. [1] http://nn.cs.utexas.edu/?hoffman:biopsych11 http://nn.cs.utexas.edu/?hoffman:biopsych11

RobertKohr 15y ago
Thank you for your solid reply. I think ANN is the real thing, and most of the other "AI" solutions that we have give the impression of intelligence, but are simply algorithms that solve problems we have in very well defined ways. ANN is a general intelligence that gets shaped by experience and adapts figure out its own solutions. This is the path that will lead to our concept of AI presented in sci-fi and what the general public actually thinks of. It isn't some coder banging on the keyboard trying to replicate what a human would do under a specific situation, it is something that truly learns. We have a long way to go before this becomes a reality, but it will happen.
- khafra 15y ago
  Tansey wrote a good, solid reply, but I dislike worshipping at the altar of mysteriousness. When a human or better GAI finally gets built, it won't be entirely hand-coded (although the seed for it might be). But the better we understand how our AI works, the better we can make sure it's doing what we want it to; and not, say, crashing the stock market with millions of erroneous transactions carried out too fast for humans to correct.
- tel 15y ago
  That's very romantic, but I think instead we're going to learn that intelligence is not such a well-defined idea as to be engineered into anything imagined by science fiction. Further, there are plenty of techniques which aren't ANNs but still are "shaped by experience" and "figure out their own solutions". I side pretty strongly with the idea of building tools to bring the powerful modes of computerized learning close to the powerful modes of human learning. I think we're inordinately far away from replicating one with the other.
- haliax 15y ago
  > I think ANN is the real thing, and most of the other "AI" solutions that we have give the impression of intelligence, but are simply algorithms that solve problems we have in very well defined ways...ANN is a general intelligence that gets shaped by experience and adapts figure out its own solutions...It isn't some coder banging on the keyboard trying to replicate what a human would do under a specific situation, it is something that truly learns. It's not ANNs on the one hand and "some coder banging on the keyboard trying to replicate what a human would do" on the other. Supervised learning is the most common use case for an ANN, and means learning from labelled training data during a training phase and then not changing; it can be done with a number of techniques, including Support Vector Machines, Decision Trees, Bayesian methods, and so on. Reinforcement Learning on the other hand, is continuous learning from trying things and making mistakes -- experience, in other words. It too can be done a number of ways, like with Evolutionary Algorithms, Markov Decision Processes, Inductive Logic Programming, etc. Hell, PG's spam classifier learns from experience, and it's nothing more than a Naive Bayes classifier ( http://www.paulgraham.com/spam.html http://www.paulgraham.com/spam.html ). Without going on for much longer, my point is that there really is no reason to exalt ANNs the way you have. Disclaimer: I'm a huge believer in Artificial General Intelligence via Competent Program Search ( see http://metacog.org/research.html http://metacog.org/research.html + http://www.google.com/url?sa=t&source=web&cd=2&ved=0CCIQtwIwAQ&url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D5hsvCib83ME&rct=j&q=goertzel%20google%20tech%20talk&ei=BMGYTtnaG8bm0QHpt7XZBA&usg=AFQjCNHrK7fPXL05JlSK3JCaXttNt_0aog&sig2=ZBa62SPhTzRFSCtMLKtneA&cad=rja http://www.google.com/url?sa=t&source=web&cd=2&v... + www.idsia.ch/~juergen/ )