4 ms·

Keep in mind evolution managed to make strong AI, us, through pretty much blind, random mutations, and inefficient selection. The thing about deep learning is

Keep in mind evolution managed to make strong AI, us, through pretty much blind, random mutations, and inefficient selection. The thing about deep learning is that it's not just nonlinear curve fitting. It learns increasingly high level features and representations of the input. Recurrent neural networks have the power of a Turing machine. And stuff like dropout are really efficient at generalization. My favorite example is word2vec. Creating a representation for every English word. Subtracting "man" from "king" and adding "woman" gives the representation for "queen". Speech recognition is moving that way. It outputs a probability distribution of possible words, and a good language model can use that to figure out what is most likely. But even a raw deep learning net should eventually learn those relationships. Same with image recognition. I think you'd be surprised at what is currently possible. The future looks bright.

graycat 13y ago
> It learns increasingly high level features and representations of the input. In the words of Darth Vader, impressive. In my words, astounding. Perhaps beyond belief. I'm thrilled if what you say is true, but I'm tempted to offer you a great, once in a life time deal on a bridge over the East River. > The future looks bright. From 'The Music Man', "I'm reticent. Yes, I'm reticent." Might want to make sure no one added some funny stuff to the Kool Aid! On AI, my 'deep learning' had a good 'training set', the world of 'expert systems'. My first cut view was that it was 99 44/100% hype and half of the rest polluted water. What was left was some somewhat clever software, say, the Forgy RETE algorithm. My views after my first cut view was that my first cut view was quite generous, that expert systems filled a much need gap in the literature and would be illuminating if ignited. So, from my 'training set' my Bayesian 'prior probability' is that nearly anything about AI is at least 99 44/100% hype. That a bunch of neural network nodes can somehow in effect develop internally just via adjustments in the 'weights' or whatever 'parameters' it has just from analysis of a 'training set' images of a Russian tank (no doubt complete with skill at 'spacial relations' where it is claimed that boys are better than girls) instead of somehow just 'storing' the data on the tank separately looks like rewiring the Intel processor when download a new PDF file instead of just putting the PDF file in storage. But, maybe putting the 'recognition means' somehow 'with' the storage means is how it is actually done. The old Darwinian guess I made was that early on it was darned important to understand three dimensions and paths through three dimensions. So, going after a little animal, and it goes behind a rock. So, there's a lot of advantage to understanding the rock as a concept and that can go the other way around the rock and get the animal. But it seems that the concept of a rock stays even outside the context of chasing prey. So, somehow intelligence works with concepts such a rocks and also uses that concept for chasing prey, turning the rock over and looking under it, knowing that a rock is hard and dense, etc. Net, my view is that AI is darned hard, so hard that MLE, neural nets, decision theory, etc. are hardly up to the level of even baby talk. Just my not very well informed, intuitive, largely out of date opinion. But, I have a good track record: I was correct early on that expert systems are a junk approach to AI. > The future looks bright. Yes, eventually.
- Houshalter 13y ago
  There is a degree of hype. They are really good at pattern recognition, maybe even superhuman on some problems and with enough training and data. But certainly they can't "think" in a normal sense or are a magical solution to the AI problem. And like everything in AI, once you understand how it actually works, it may not seem as impressive as it did at first. >instead of somehow just 'storing' the data on the tank separately looks like rewiring the Intel processor when download a new PDF file instead of just putting the PDF file in storage. Good analogy, but how would you even do that? One picture of a tank isn't enough to generalize. Is a tank any image colored green? Is it any object painted camouflage? Is it any vehicle that has a tube protruding from it? In order to learn, you need a lot of examples, and you need to test a lot of different hypotheses about what a tank is. That's a really difficult problem.
  - graycat 13y ago
    > That's a really difficult problem. Yup.

graycat 13y ago

> It learns increasingly high level features and representations of the input. In the words of Darth Vader, impressive. In my words, astounding. Perhaps beyond belief. I'm thrilled if what you say is true, but I'm tempted to offer you a great, once in a life time deal on a bridge over the East River. > The future looks bright. From 'The Music Man', "I'm reticent. Yes, I'm reticent." Might want to make sure no one added some funny stuff to the Kool Aid! On AI, my 'deep learning' had a good 'training set', the world of 'expert systems'. My first cut view was that it was 99 44/100% hype and half of the rest polluted water. What was left was some somewhat clever software, say, the Forgy RETE algorithm. My views after my first cut view was that my first cut view was quite generous, that expert systems filled a much need gap in the literature and would be illuminating if ignited. So, from my 'training set' my Bayesian 'prior probability' is that nearly anything about AI is at least 99 44/100% hype. That a bunch of neural network nodes can somehow in effect develop internally just via adjustments in the 'weights' or whatever 'parameters' it has just from analysis of a 'training set' images of a Russian tank (no doubt complete with skill at 'spacial relations' where it is claimed that boys are better than girls) instead of somehow just 'storing' the data on the tank separately looks like rewiring the Intel processor when download a new PDF file instead of just putting the PDF file in storage. But, maybe putting the 'recognition means' somehow 'with' the storage means is how it is actually done. The old Darwinian guess I made was that early on it was darned important to understand three dimensions and paths through three dimensions. So, going after a little animal, and it goes behind a rock. So, there's a lot of advantage to understanding the rock as a concept and that can go the other way around the rock and get the animal. But it seems that the concept of a rock stays even outside the context of chasing prey. So, somehow intelligence works with concepts such a rocks and also uses that concept for chasing prey, turning the rock over and looking under it, knowing that a rock is hard and dense, etc. Net, my view is that AI is darned hard, so hard that MLE, neural nets, decision theory, etc. are hardly up to the level of even baby talk. Just my not very well informed, intuitive, largely out of date opinion. But, I have a good track record: I was correct early on that expert systems are a junk approach to AI. > The future looks bright. Yes, eventually.

Houshalter 13y ago
There is a degree of hype. They are really good at pattern recognition, maybe even superhuman on some problems and with enough training and data. But certainly they can't "think" in a normal sense or are a magical solution to the AI problem. And like everything in AI, once you understand how it actually works, it may not seem as impressive as it did at first. >instead of somehow just 'storing' the data on the tank separately looks like rewiring the Intel processor when download a new PDF file instead of just putting the PDF file in storage. Good analogy, but how would you even do that? One picture of a tank isn't enough to generalize. Is a tank any image colored green? Is it any object painted camouflage? Is it any vehicle that has a tube protruding from it? In order to learn, you need a lot of examples, and you need to test a lot of different hypotheses about what a tank is. That's a really difficult problem.
- graycat 13y ago
  > That's a really difficult problem. Yup.