4 ms·

For me, "There are certainly some reasons to be optimistic. Andrew Ng, who worked or works on Google’s AI, has said that he believes learning comes from a singl

by rdl 13y ago

For me, "There are certainly some reasons to be optimistic. Andrew Ng, who worked or works on Google’s AI, has said that he believes learning comes from a single algorithm - the part of your brain that processes input from your ears is also capable of learning to process input from your eyes. If we can just figure out this one general-purpose algorithm, programs may be able to learn general-purpose things." is what was most interesting to me. The idea that there's a fairly simple algorithm to learning, applied in the brain, which produces at least the basic learning capability, and possibly consciousness.

elwell 13y ago
Fair enough, but can we imagine a better algorithm?
- rdl 13y ago
  I've always assumed any real machine intelligence would be scalable to the point where even if it started at a millionth the functionality of a human, it would end up far surpassing humanity.
  - derefr 13y ago
    Why do you believe this? We don't see animals self-improving to become humans (in the "consciously deciding what revisions to make to their mental architecture at each step" sense), so why would you expect electronic animal-level minds to be able to do the same to become electronic human-level minds, or beyond? Personally, I doubt even electronic human-equivalent minds would be capable of self-improvement. After all, we are not smart enough to build AIs better than us (yet), so why should electronic minds only as smart as we are be capable of that, either? A possible answer to that, I suppose, would be that the electronic mind would have far more input/training data fed to it per second than biological minds receive, and far more time to "work on" that sense data to derive patterns between each decision-step.
    - rdl 13y ago
      Because (naively) I'd assume it's some kind of software running on computers, and one can generally improve performance of that by buying more hardware or using better technology (or shorter lived components, or better cooling, or whatever). If humans created an AI to solve some kind of open-ended problem, where "more AI" made the solution better, there would be every incentive to spend the money on more or better hardware for it. It's not clear that a shark would gain much by being 100x smarter, particularly if the metabolic cost were high; for a lot of human problems, spending 100x more on hardware/power/etc. for a 10% better solution would be quite desirable.
    - henrygrew 13y ago
      What about the advantage gained by time, i think a major drawback to human improvement is the effect of death.Given enough time and accumulated experience, self improvement is possible.A machine cannot succumb to death.
  - aragot 13y ago
    I personnaly don't think the brain could be as intelligent if it were only a part of it, like if had 3 senses instead of 5 for example: The emulation of a sense by the others might be necessary to bootstrap the system. So we might just be on the wrong path when we try to teach a small machine to learn. And maybe, once you have a fully functional brain, that it constantly derives out of bounds. Like sudden overheating, which is litterally the step #1 of a depression in a human. So we might not be able to scale an AI beyond the size of one brain. Apart from clusters obviously, but then you need to sustain civilizations of brains, and civs do collapse every dozen generations. We might not be materially able to find enough energy to power all of those trials and errors.
    - aliakbarkhan 13y ago
      The amount of rampant bulshitting here by obviously unqualified people is amazing. If you don't even know that humans do not, in fact, have five senses, I'm not sure I can take your post seriously even as unqualified speculation.
      - aragot 13y ago
        The whole thread including the OP is full of speculation by unqualified people, because AI is the ultimate fantasy for all of us here. So common sense is a perfectly satisfying qualification to post in this particular thread. Concerning the 5 senses, I refer to common wisdom because it speaks to everyone. Those who know better are probably smart enough to translate "5 senses" into an accurate scientific wording.
      - membrain 13y ago
        can we please put this right on top of the whole page and close this thread? too many ai-fanboys in here, who got their knowledge about ai from hollywood movies and video games..
- chongli 13y ago
  I think we need a better machine first. The current model of one or a handful of CPU cores with very tiny caches talking to memory over a narrow, high latency bus doesn't seem very efficient for building and querying massive (multi-petabyte), highly associative (thousands of associations) data structures.
  - aragot 13y ago
    There are 100,000,000 neurons in the brain. It's not that many. A Core 2 Duo has 169m [1]. A neuron is connected to about 10.000 others. And how many computers do we have in the world? It even sounds like we scientifics are sluggish at inventing AI ;) [1] http://en.wikipedia.org/wiki/Transistor_count http://en.wikipedia.org/wiki/Transistor_count
    - tim333 13y ago
      Of course a Core 2 Duo has 169m transistors rather than neurons. Interestingly you can do a rough calculation of how much processing power would be needed to do the equivalent of a human brain and it's quite a lot. Hans Moravec calculates it a about 100 million million instructions a second ( http://www.transhumanist.com/volume1/moravec.htm http://www.transhumanist.com/volume1/moravec.htm ) based on the processing power required to fuctionally equal the human retina which is reasonably understood. A core 2 duo does something more like 2.7 thousand million instructions per second and so you'd need about 35,000 of them for brain equivalence. So you're talking in 2014 terms of a a top end supercomputer rather than a macbook.
    - chongli 13y ago
      >There are 100,000,000 neurons in the brain. You're off by 3 orders of magnitude. Try 200 Billion[0]. [0] http://en.wikipedia.org/wiki/Human_brain#Structure http://en.wikipedia.org/wiki/Human_brain#Structure
nl 13y ago
We already have a candidate algorithm[1], which is successful in a broad range of applications[2][3][4]. [1] http://en.wikipedia.org/wiki/Deep_learning#Convolutional_neural_networks http://en.wikipedia.org/wiki/Deep_learning#Convolutional_neu... [2] http://deeplearning.net/reading-list/ http://deeplearning.net/reading-list/ [3] http://en.wikipedia.org/wiki/Deep_learning#Results http://en.wikipedia.org/wiki/Deep_learning#Results [4] http://www.wired.com/wiredscience/2012/06/google-x-neural-network/ http://www.wired.com/wiredscience/2012/06/google-x-neural-ne...
- electrograv 13y ago
  "Deep Learning" is not itself a candidate for anything, because it's not any single algorithm, but a category of approaches. Deep Learning generally refers to machine learning algorithms that deal with stacking multiple layers of simpler functions to enable more complicated functions, and optimizing all the parameters to best fit your training set and generalize to new samples (the hard part). Though it usually refers to neural networks, I dont think there's any reason it doesn't also apply to other layered approaches as long as there's a relatively unified learning algorithm applied across the whole system. There are clearly many different deep learning algorithms, even if you just count the permutations of tricks you can choose from to improve layered NN generalization. Though to be fair I think very good progress is being made towards developing "better" algorithms in the sense that new ones (e.g. RBM pretraining + dropout) usual perform better than than older algorithms, no matter what data you use it on (now network architecture is another matter entirely).
  - nl 13y ago
    I actually linked to convolutional neural networks on the Deep Learning page as the algorithm. But I do agree with your point.
  - eli_gottlieb 13y ago
    One of the most interestingly general things about Deep Learning is that unsupervised learning approaches can be used at the bottom of the "stack" to learn more useful high-level features from the input data. This ends up making your higher-level learners more helpful for "Real Stuff".
stinos 13y ago
It's an intersting thought indeed, but I wonder if any neuroscientists would back it up.
membrain 13y ago
this hypothesis is around since at least the 70s (Vernon Mountcastle' paper on cortical similarities), recently (2004) hawkins also published a book in which he promoted the same idea, on which he is currently working with his AGI company numenta. to be honest, it sounds really great and it could even be the case that there is a very general underlying principle to cortical information processing and pattern recognition. But one should be careful not to mix solid scientific hypotheses with mainstream media hysteria and people who try to grab attention with their simplifications, claiming today that entropy maximization is the underlying principle and changing to sparse coding tomorrow. We are not that far and what we need is solid research instead of over-the-head assumptions and claims "to have solved the riddle" (in that respect, it might not be that far off alchemy :P)
mannykannot 13y ago
"Andrew Ng, who worked or works on Google’s AI, has said that he believes learning comes from a single algorithm." As algorithms can be combined, the existence of any set of algorithms satisfying this goal would automatically imply the existence of a single algorithm incorporating all of them. If a sufficiently-detailed physical simulation of a human's brain satisfied this goal, then that would be one such algorithm.

by rdl 13y ago

elwell 13y ago

Fair enough, but can we imagine a better algorithm?

rdl 13y ago
I've always assumed any real machine intelligence would be scalable to the point where even if it started at a millionth the functionality of a human, it would end up far surpassing humanity.
- derefr 13y ago
  Why do you believe this? We don't see animals self-improving to become humans (in the "consciously deciding what revisions to make to their mental architecture at each step" sense), so why would you expect electronic animal-level minds to be able to do the same to become electronic human-level minds, or beyond? Personally, I doubt even electronic human-equivalent minds would be capable of self-improvement. After all, we are not smart enough to build AIs better than us (yet), so why should electronic minds only as smart as we are be capable of that, either? A possible answer to that, I suppose, would be that the electronic mind would have far more input/training data fed to it per second than biological minds receive, and far more time to "work on" that sense data to derive patterns between each decision-step.
  - rdl 13y ago
    Because (naively) I'd assume it's some kind of software running on computers, and one can generally improve performance of that by buying more hardware or using better technology (or shorter lived components, or better cooling, or whatever). If humans created an AI to solve some kind of open-ended problem, where "more AI" made the solution better, there would be every incentive to spend the money on more or better hardware for it. It's not clear that a shark would gain much by being 100x smarter, particularly if the metabolic cost were high; for a lot of human problems, spending 100x more on hardware/power/etc. for a 10% better solution would be quite desirable.
  - henrygrew 13y ago
    What about the advantage gained by time, i think a major drawback to human improvement is the effect of death.Given enough time and accumulated experience, self improvement is possible.A machine cannot succumb to death.
- aragot 13y ago
  I personnaly don't think the brain could be as intelligent if it were only a part of it, like if had 3 senses instead of 5 for example: The emulation of a sense by the others might be necessary to bootstrap the system. So we might just be on the wrong path when we try to teach a small machine to learn. And maybe, once you have a fully functional brain, that it constantly derives out of bounds. Like sudden overheating, which is litterally the step #1 of a depression in a human. So we might not be able to scale an AI beyond the size of one brain. Apart from clusters obviously, but then you need to sustain civilizations of brains, and civs do collapse every dozen generations. We might not be materially able to find enough energy to power all of those trials and errors.
  - aliakbarkhan 13y ago
    The amount of rampant bulshitting here by obviously unqualified people is amazing. If you don't even know that humans do not, in fact, have five senses, I'm not sure I can take your post seriously even as unqualified speculation.
    - aragot 13y ago
      The whole thread including the OP is full of speculation by unqualified people, because AI is the ultimate fantasy for all of us here. So common sense is a perfectly satisfying qualification to post in this particular thread. Concerning the 5 senses, I refer to common wisdom because it speaks to everyone. Those who know better are probably smart enough to translate "5 senses" into an accurate scientific wording.
    - membrain 13y ago
      can we please put this right on top of the whole page and close this thread? too many ai-fanboys in here, who got their knowledge about ai from hollywood movies and video games..
chongli 13y ago
I think we need a better machine first. The current model of one or a handful of CPU cores with very tiny caches talking to memory over a narrow, high latency bus doesn't seem very efficient for building and querying massive (multi-petabyte), highly associative (thousands of associations) data structures.
- aragot 13y ago
  There are 100,000,000 neurons in the brain. It's not that many. A Core 2 Duo has 169m [1]. A neuron is connected to about 10.000 others. And how many computers do we have in the world? It even sounds like we scientifics are sluggish at inventing AI ;) [1] http://en.wikipedia.org/wiki/Transistor_count http://en.wikipedia.org/wiki/Transistor_count
  - tim333 13y ago
    Of course a Core 2 Duo has 169m transistors rather than neurons. Interestingly you can do a rough calculation of how much processing power would be needed to do the equivalent of a human brain and it's quite a lot. Hans Moravec calculates it a about 100 million million instructions a second ( http://www.transhumanist.com/volume1/moravec.htm http://www.transhumanist.com/volume1/moravec.htm ) based on the processing power required to fuctionally equal the human retina which is reasonably understood. A core 2 duo does something more like 2.7 thousand million instructions per second and so you'd need about 35,000 of them for brain equivalence. So you're talking in 2014 terms of a a top end supercomputer rather than a macbook.
  - chongli 13y ago
    >There are 100,000,000 neurons in the brain. You're off by 3 orders of magnitude. Try 200 Billion[0]. [0] http://en.wikipedia.org/wiki/Human_brain#Structure http://en.wikipedia.org/wiki/Human_brain#Structure

nl 13y ago

We already have a candidate algorithm[1], which is successful in a broad range of applications[2][3][4]. [1] http://en.wikipedia.org/wiki/Deep_learning#Convolutional_neural_networks http://en.wikipedia.org/wiki/Deep_learning#Convolutional_neu... [2] http://deeplearning.net/reading-list/ http://deeplearning.net/reading-list/ [3] http://en.wikipedia.org/wiki/Deep_learning#Results http://en.wikipedia.org/wiki/Deep_learning#Results [4] http://www.wired.com/wiredscience/2012/06/google-x-neural-network/ http://www.wired.com/wiredscience/2012/06/google-x-neural-ne...

electrograv 13y ago
"Deep Learning" is not itself a candidate for anything, because it's not any single algorithm, but a category of approaches. Deep Learning generally refers to machine learning algorithms that deal with stacking multiple layers of simpler functions to enable more complicated functions, and optimizing all the parameters to best fit your training set and generalize to new samples (the hard part). Though it usually refers to neural networks, I dont think there's any reason it doesn't also apply to other layered approaches as long as there's a relatively unified learning algorithm applied across the whole system. There are clearly many different deep learning algorithms, even if you just count the permutations of tricks you can choose from to improve layered NN generalization. Though to be fair I think very good progress is being made towards developing "better" algorithms in the sense that new ones (e.g. RBM pretraining + dropout) usual perform better than than older algorithms, no matter what data you use it on (now network architecture is another matter entirely).
- nl 13y ago
  I actually linked to convolutional neural networks on the Deep Learning page as the algorithm. But I do agree with your point.
- eli_gottlieb 13y ago
  One of the most interestingly general things about Deep Learning is that unsupervised learning approaches can be used at the bottom of the "stack" to learn more useful high-level features from the input data. This ends up making your higher-level learners more helpful for "Real Stuff".

stinos 13y ago

It's an intersting thought indeed, but I wonder if any neuroscientists would back it up.

membrain 13y ago

this hypothesis is around since at least the 70s (Vernon Mountcastle' paper on cortical similarities), recently (2004) hawkins also published a book in which he promoted the same idea, on which he is currently working with his AGI company numenta. to be honest, it sounds really great and it could even be the case that there is a very general underlying principle to cortical information processing and pattern recognition. But one should be careful not to mix solid scientific hypotheses with mainstream media hysteria and people who try to grab attention with their simplifications, claiming today that entropy maximization is the underlying principle and changing to sparse coding tomorrow. We are not that far and what we need is solid research instead of over-the-head assumptions and claims "to have solved the riddle" (in that respect, it might not be that far off alchemy :P)

mannykannot 13y ago

"Andrew Ng, who worked or works on Google’s AI, has said that he believes learning comes from a single algorithm." As algorithms can be combined, the existence of any set of algorithms satisfying this goal would automatically imply the existence of a single algorithm incorporating all of them. If a sufficiently-detailed physical simulation of a human's brain satisfied this goal, then that would be one such algorithm.