5 ms·

The author completely misses the point of these arguments. This is not about science increasing the rate of scientific progress. It's about computation improvi

The author completely misses the point of these arguments. This is not about science increasing the rate of scientific progress. It's about computation improving the rate of computational progress. Particularly, the "Singularity" is about that happening in a world where strong AI exists. And I'm not even going to harp on the fact that the number of computations per second that we perform on this planet has been undoubtedly exponential at least for a while, because that's not the real point. The real point is that like the author says, human creativity is still the driving force in computational advances, and human creativity can't be sped up by periodically doubling clock rates or core counts. If it could? Those doublings would happen even quicker, and the exponential growth we've seen so far would be nothing. This does, of course, require that you believe that strong AI is possible at all, and that the entire process of creativity could eventually be put into software. If you don't believe in that in the first place, then there's no point having the argument at all - you don't argue the details of a nuclear chain reaction with someone that doesn't believe in neutrons.

fchollet 14y ago
How is throwing more (truly intelligent) computers at the problem any different than throwing more human brains at it? It's not just computing power that had been exploding, it is also the number of people trying to crack the nut of AI (for instance).
- gd1 14y ago
  Because the human brains can only communicate and acquire knowledge at a limited rate. This causes the "exponential decrease of discovery impact of each succeeding researcher" that the original post refers to, but he misses the point completely. We can't 'stand on the shoulders of giants' until we've climbed up there, slowly and painfully. An AI can just download the lot. And share any new discoveries with the hive mind.
  - jpadkins 14y ago
    it's sci fi, but we haven't ruled out that human brains can't acquire knowledge at rates orders of magnitudes faster than current learning methods. Big advances in neuroscience and linguistics could make this possible. Right now, the limits on our bandwidth is language and our sensory inputs. It's not completely ridiculous to think we could encode certain kinds of knowledge in more compact forms, and transmit them via an artificial sensory process that has higher higher bandwidth that audio/visual language based learning...
    - FeepingCreature 14y ago
      The singularity idea does not care how the self-improvement feedback occurs.
rvkennedy 14y ago
It's about computation improving the rate of computational progress But computation doesn't improve the rate of computational progress. That's achieved by improvements in manufacturing. Computation does help a little, giving us better tools for chip design. But mainly it's manufacturing. The only way for computational progress to exponentiate, past the limit of Moore's Law, is to use more physical resources - computers get literally bigger. And so expensive that only the five richest crowned heads of Europe can afford them!
- Anderkent 14y ago
  But computation doesn't improve the rate of computational progress. As long as it's people developing computing devices, that holds. But when you have a machine that is capable of improving its own software, you get a self reinforcing loop where computation improves the rate of computational progress.
  - loup-vaillant 14y ago
    Intelligence doesn't even have to feed on itself. Imagine a human-level hardware designing AI. As long as Moore's law applies, it can double its computational power every 18 subjective months. From our point of view, that's 18 months, then 9, then 4.5… If Moore's law where unlimited, such an AI would have infinite computational power in 36 months, with absolutely no increase in actual intelligence. But of course Moore's law is limited. Still, we could expect quite a bit of progress from a mere AI hardware designer.
    - wcarss 14y ago
      I do not believe that that is what Moore's Law means or how it works. Perhaps I just mistook your meaning - what do 'subjective months' mean? Why do they mean that the doubling time starts halving?
      - loup-vaillant 14y ago
        I mean this: http://dragonball.wikia.com/wiki/Hyperbolic_Time_Chamber http://dragonball.wikia.com/wiki/Hyperbolic_Time_Chamber When hardware is twice as fast, the AI works twice as fast. In 9 months, it will do 18 months worth of work. Now keep the doubling up, and it will have done infinite work after 36 months.
coldarchon 14y ago
Absolutely right, as I pointed out below, the very young author obviously compares scaling to scientific progress. As we currently have no model in AI which could create a cognitive skill, his writing becomes obsolete and will be laughed at when we do.
smoyer 14y ago
Actually he misses a second point too ... the reason you don't see exponential growth in human-led research is that we each start out with "empty" brains. To become truly knowledgeable in a field may require a life-time. I'm not arguing that the singularity is near either, but machines would have the advantage of creating offspring that are already as knowledgeable/intelligent as themselves. Something I couldn't do for my children.

fchollet 14y ago

How is throwing more (truly intelligent) computers at the problem any different than throwing more human brains at it? It's not just computing power that had been exploding, it is also the number of people trying to crack the nut of AI (for instance).

gd1 14y ago
Because the human brains can only communicate and acquire knowledge at a limited rate. This causes the "exponential decrease of discovery impact of each succeeding researcher" that the original post refers to, but he misses the point completely. We can't 'stand on the shoulders of giants' until we've climbed up there, slowly and painfully. An AI can just download the lot. And share any new discoveries with the hive mind.
- jpadkins 14y ago
  it's sci fi, but we haven't ruled out that human brains can't acquire knowledge at rates orders of magnitudes faster than current learning methods. Big advances in neuroscience and linguistics could make this possible. Right now, the limits on our bandwidth is language and our sensory inputs. It's not completely ridiculous to think we could encode certain kinds of knowledge in more compact forms, and transmit them via an artificial sensory process that has higher higher bandwidth that audio/visual language based learning...
  - FeepingCreature 14y ago
    The singularity idea does not care how the self-improvement feedback occurs.

rvkennedy 14y ago

It's about computation improving the rate of computational progress But computation doesn't improve the rate of computational progress. That's achieved by improvements in manufacturing. Computation does help a little, giving us better tools for chip design. But mainly it's manufacturing. The only way for computational progress to exponentiate, past the limit of Moore's Law, is to use more physical resources - computers get literally bigger. And so expensive that only the five richest crowned heads of Europe can afford them!

Anderkent 14y ago
But computation doesn't improve the rate of computational progress. As long as it's people developing computing devices, that holds. But when you have a machine that is capable of improving its own software, you get a self reinforcing loop where computation improves the rate of computational progress.
- loup-vaillant 14y ago
  Intelligence doesn't even have to feed on itself. Imagine a human-level hardware designing AI. As long as Moore's law applies, it can double its computational power every 18 subjective months. From our point of view, that's 18 months, then 9, then 4.5… If Moore's law where unlimited, such an AI would have infinite computational power in 36 months, with absolutely no increase in actual intelligence. But of course Moore's law is limited. Still, we could expect quite a bit of progress from a mere AI hardware designer.
  - wcarss 14y ago
    I do not believe that that is what Moore's Law means or how it works. Perhaps I just mistook your meaning - what do 'subjective months' mean? Why do they mean that the doubling time starts halving?
    - loup-vaillant 14y ago
      I mean this: http://dragonball.wikia.com/wiki/Hyperbolic_Time_Chamber http://dragonball.wikia.com/wiki/Hyperbolic_Time_Chamber When hardware is twice as fast, the AI works twice as fast. In 9 months, it will do 18 months worth of work. Now keep the doubling up, and it will have done infinite work after 36 months.

coldarchon 14y ago

Absolutely right, as I pointed out below, the very young author obviously compares scaling to scientific progress. As we currently have no model in AI which could create a cognitive skill, his writing becomes obsolete and will be laughed at when we do.

smoyer 14y ago

Actually he misses a second point too ... the reason you don't see exponential growth in human-led research is that we each start out with "empty" brains. To become truly knowledgeable in a field may require a life-time. I'm not arguing that the singularity is near either, but machines would have the advantage of creating offspring that are already as knowledgeable/intelligent as themselves. Something I couldn't do for my children.