4 ms·

This is a reasonably plausible argument for progress within fields and paradigm shifts; but consider between fields, and the rate of new field creation. To use

by 6ren 14y ago

This is a reasonably plausible argument for progress within fields and paradigm shifts; but consider between fields, and the rate of new field creation. To use the given examples, Newtonian physics, quantum physics and Information Theory began in 1687, 1930 and 1948. That seems quite accelerating (of course you'd need more complete stats to demonstrate this properly). BTW: I disagree that intelligence and imagination are separate. I think intelligence necessarily includes imagination, so AI would include artificial "imagination" too. I'd like progress to continue rather than to stagnate; and since progress has historically continued, that seems the way to bet. Those pundits predicting stagnation have historically been wrong. However, I find it curious that Kurzweil does not address the availability of innovations/discoveries (and the difficulty in finding them), in contrast to the compelling case he makes for historical progress. Are potential discoveries infinite? Is their density uniform, or do they become less likely as you go along because they are limited in supply (the assumption of the article)? How would one sensibly model the supply of potential discoveries, since it is intrinsically unknown? Perhaps a mathematical model of computation might help, of computable functions and Kolomogorov/Solomonoff program length to compute them. Using this model, the search space of functions increases exponentially with program length - but at what rate would the number of "interesting" computable functions increase? And with what density (with respect to the search space, i.e. how hard they are to find)?

fchollet 14y ago
While I agree with you that the rate of paradigm shift increases over time, I argue that their average impact/scope decreases. Newtonian physics opened more room for technological progress than quantum mechanics, etc. Earliest inventions have the most impact (the invention of the scientific method having the most impact of all?). So my point is that if you look at paradigm shift rate not in volume but in impact, you'd get a linear rate. For this reason, an AI given imagination and exploding resources would still only make linear progress.
- 6ren 14y ago
  I didn't notice your point about impact. Impact is hard to measure purely objectively, but I agree Newtonian physics, being more fundamental, has had more impact. But it's also had 325 years to do so, whereas Information Theory has had only 64 (to stay with these examples). Given an exponentially expanding search space, there's always more to see (but this is an assumption, not a conclusion; I think the nature of the search space is the essential question at issue here). How much impact did Newton have on the world by 1751? How much impact will Shannon have had by 2337?
- simonh 14y ago
  Transistors couldn't have been designed without understanding Quantum Mechanics. Bye, bye modern computers.
  - fchollet 14y ago
    And quantum mechanics could not have been thought up without newtonian mechanics...
- deleted 14y ago
  [deleted]
BlaineLight 14y ago
Intelligence and Imagination could not be more separate. I'm defining intelligence as sheer IQ, analogous to a computers' ability to process information. Imagination is so essentially human in nature that there is no way to describe current AI as "imaginative". For example, the person with the highest IQ in the world is not more creative than the best musician. Some of the smartest minds in the world cannot "think outside of the box" effectively, because their minds are not wired that way. Intelligence is not imagination. Can anyone please explain how an intelligence-self-improving computer will shift from being incredibly smart to wildly imaginative? That's the one hole in singularity that I can't figure out.
- mquander 14y ago
  You started off defining intelligence as something limited and not independently useful, and then you concluded that its presence was limited and not independently useful. The answer is that it's not actually obvious that "intelligence and imagination could not be more separate" or that "imagination is...essentially human in nature," so when most people talk about AI, they are not excluding those abilities.
- ajuc 14y ago
  Imagination isn't any more human specific, than inteligence, or walking upright. That we can't do machines that are imaginative now, doesn't meant it's impossible to do. Besides - what if imagination is just heuristic to search solution space quicker? Maybe machine with orders of magnitude more processing power can give better solutions without using this heuristic? That would mean it's us who thinks inside the box (because we use this heuristic), and computers can search solution space exhautively, so they will come up with solutions that escape our heuristic - we wouldn't even consider such solutions.
  - 6ren 14y ago
    Yes, it doesn't really matter how intelligence is defined, just that in aiming at creating an artificial mind, we'd try to include whatever was useful, including imagination. My reasoning is, if one machine can do it (us), it's possible for another machine to do it. (Though there's a question of whether we are smart enough to create that machine.) Exponential search space is a killer. Some form of heuristic seems necessary. Perhaps we will one day be able to simulate the heuristics we ourselves use - and then improve on them (e.g. apply them to more data than we can; use more complex heuristics; etc along the lines you say). Given variation between intellectual abilities of human beings, it is does not have an obvious fixed limit, and we may be able to extend the range of that variability. Though I expect increasing the associations an "imagination" can find suffers from exponential explosion.
benkant 14y ago
> Perhaps a mathematical model of computation might help, of computable functions and Kolomogorov/Solomonoff program length to compute them. Using this model, the search space of functions increases exponentially with program length - but at what rate would the number of "interesting" computable functions increase? And with what density (with respect to the search space, i.e. how hard they are to find)? Like AIXI? http://www.hutter1.net/ai/aixigentle.htm http://www.hutter1.net/ai/aixigentle.htm
kevinpet 14y ago
"To use the given examples, Newtonian physics, quantum physics and Information Theory began in 1687, 1930 and 1948." Some things that we consider significant today won't seem significant from the perspective of centuries. Similarly, someone in 1750 would be able to list significant new fields developed recently that we would omit. For example, Newton also did work in optics, which we pretty much forget about today.
- heja2009 14y ago
  While I would consider Newtionian physics to be much more fundamental, I would challenge the notion that it had the most impact. Compare the ~1900 feeling that almost everything had already been invended. Newtonian physics impact to technology was mainly mechanics. Then came thermodynamics which gave us (steam &) combustion engines and more. Nuclear & quantum physics lead to bombs, reactors and finally seminconductors and so on. I would say todays thnological relevance of thermodynamics and quantum physics is at least as high as that of Newtonian physics. Also: most of todays physicists do not look for Bosons, they do things like advancing semiconductor masking technology, liquid crystal physics and the like - i.e. they are not "fundamental" but "applied" physicists. Also - with respect to the original article - isn't the much faster exponential growth of the resources invested in science since around 1900 mostly an indicator of expected technological returns by society at large? Science is a huge economy success story since about a century ;-) The question then would be how long can we afford to sustain the current exponent?

by 6ren 14y ago

fchollet 14y ago

While I agree with you that the rate of paradigm shift increases over time, I argue that their average impact/scope decreases. Newtonian physics opened more room for technological progress than quantum mechanics, etc. Earliest inventions have the most impact (the invention of the scientific method having the most impact of all?). So my point is that if you look at paradigm shift rate not in volume but in impact, you'd get a linear rate. For this reason, an AI given imagination and exploding resources would still only make linear progress.

6ren 14y ago
I didn't notice your point about impact. Impact is hard to measure purely objectively, but I agree Newtonian physics, being more fundamental, has had more impact. But it's also had 325 years to do so, whereas Information Theory has had only 64 (to stay with these examples). Given an exponentially expanding search space, there's always more to see (but this is an assumption, not a conclusion; I think the nature of the search space is the essential question at issue here). How much impact did Newton have on the world by 1751? How much impact will Shannon have had by 2337?
simonh 14y ago
Transistors couldn't have been designed without understanding Quantum Mechanics. Bye, bye modern computers.
- fchollet 14y ago
  And quantum mechanics could not have been thought up without newtonian mechanics...
deleted 14y ago
[deleted]

BlaineLight 14y ago

Intelligence and Imagination could not be more separate. I'm defining intelligence as sheer IQ, analogous to a computers' ability to process information. Imagination is so essentially human in nature that there is no way to describe current AI as "imaginative". For example, the person with the highest IQ in the world is not more creative than the best musician. Some of the smartest minds in the world cannot "think outside of the box" effectively, because their minds are not wired that way. Intelligence is not imagination. Can anyone please explain how an intelligence-self-improving computer will shift from being incredibly smart to wildly imaginative? That's the one hole in singularity that I can't figure out.

mquander 14y ago
You started off defining intelligence as something limited and not independently useful, and then you concluded that its presence was limited and not independently useful. The answer is that it's not actually obvious that "intelligence and imagination could not be more separate" or that "imagination is...essentially human in nature," so when most people talk about AI, they are not excluding those abilities.
ajuc 14y ago
Imagination isn't any more human specific, than inteligence, or walking upright. That we can't do machines that are imaginative now, doesn't meant it's impossible to do. Besides - what if imagination is just heuristic to search solution space quicker? Maybe machine with orders of magnitude more processing power can give better solutions without using this heuristic? That would mean it's us who thinks inside the box (because we use this heuristic), and computers can search solution space exhautively, so they will come up with solutions that escape our heuristic - we wouldn't even consider such solutions.
- 6ren 14y ago
  Yes, it doesn't really matter how intelligence is defined, just that in aiming at creating an artificial mind, we'd try to include whatever was useful, including imagination. My reasoning is, if one machine can do it (us), it's possible for another machine to do it. (Though there's a question of whether we are smart enough to create that machine.) Exponential search space is a killer. Some form of heuristic seems necessary. Perhaps we will one day be able to simulate the heuristics we ourselves use - and then improve on them (e.g. apply them to more data than we can; use more complex heuristics; etc along the lines you say). Given variation between intellectual abilities of human beings, it is does not have an obvious fixed limit, and we may be able to extend the range of that variability. Though I expect increasing the associations an "imagination" can find suffers from exponential explosion.

benkant 14y ago

> Perhaps a mathematical model of computation might help, of computable functions and Kolomogorov/Solomonoff program length to compute them. Using this model, the search space of functions increases exponentially with program length - but at what rate would the number of "interesting" computable functions increase? And with what density (with respect to the search space, i.e. how hard they are to find)? Like AIXI? http://www.hutter1.net/ai/aixigentle.htm http://www.hutter1.net/ai/aixigentle.htm

kevinpet 14y ago

"To use the given examples, Newtonian physics, quantum physics and Information Theory began in 1687, 1930 and 1948." Some things that we consider significant today won't seem significant from the perspective of centuries. Similarly, someone in 1750 would be able to list significant new fields developed recently that we would omit. For example, Newton also did work in optics, which we pretty much forget about today.

heja2009 14y ago
While I would consider Newtionian physics to be much more fundamental, I would challenge the notion that it had the most impact. Compare the ~1900 feeling that almost everything had already been invended. Newtonian physics impact to technology was mainly mechanics. Then came thermodynamics which gave us (steam &) combustion engines and more. Nuclear & quantum physics lead to bombs, reactors and finally seminconductors and so on. I would say todays thnological relevance of thermodynamics and quantum physics is at least as high as that of Newtonian physics. Also: most of todays physicists do not look for Bosons, they do things like advancing semiconductor masking technology, liquid crystal physics and the like - i.e. they are not "fundamental" but "applied" physicists. Also - with respect to the original article - isn't the much faster exponential growth of the resources invested in science since around 1900 mostly an indicator of expected technological returns by society at large? Science is a huge economy success story since about a century ;-) The question then would be how long can we afford to sustain the current exponent?