3 ms·

Take a smart human and speed up their thinking a billion times, you get the time for free. What takes humanity a decade of effort could happen in a day (or fast

Take a smart human and speed up their thinking a billion times, you get the time for free. What takes humanity a decade of effort could happen in a day (or faster) that's the nature of thinking faster. People often mistakenly model a smart human compared to a dumb human, but the actual spectrum of capability is way wider than that. Lack of knowledge could be another way of saying "not aligned" which is the core issue.

sterlind 4y ago
that's the thing though, I don't believe superintelligence can possibly work like that. I think it's a logical fallacy to treat superintelligences like hypercomputers. they're constrained by existing on classical computers. like, as a trivial counterexample, I can speak at 500 wpm. let's say that ChatGPT can generate 500 words per second for a single thread of conversation - I think that's a generous overestimate. now that's a 60x speedup over me, not a 1,000,000,000x like you're talking about. do you honestly believe they they can make ChatGPT run 16.6 million times faster, without changing hardware? do you think ChatGPT will just like, hit an inflection point where it realizes how to refactor its inference code to run 16,666,666x faster? no, I think that's absurd. you're treating these things like black boxes but they are constrained by computational complexity, die area and the speed of light for Christ's sake.
- righttoolforjob 4y ago
  What is the limit of intelligence? What limits the human intelligence? How many more times intelligent would an AGI have to be to have the capacity to take over the world?
- sn9 4y ago
  The comparison shouldn't be to individual humans, but to societies of all scales. AIs can scale horizontally as well as vertically.
- aeternum 4y ago
  By that logic, we should have never broken the sound barrier because we're under the same physics constraints as other biological organisms. The key difference is that this new intelligence is not constrained by having to slowly pump ions across cell membranes in order to internally generate power for computation. Silicon transistors are much more efficient at computation than the human brain just as a jet engine is superior to to a peregrine falcon. ChatGPT could absolutely generate thousands of words per second on existing hardware.

sterlind 4y ago

that's the thing though, I don't believe superintelligence can possibly work like that. I think it's a logical fallacy to treat superintelligences like hypercomputers. they're constrained by existing on classical computers. like, as a trivial counterexample, I can speak at 500 wpm. let's say that ChatGPT can generate 500 words per second for a single thread of conversation - I think that's a generous overestimate. now that's a 60x speedup over me, not a 1,000,000,000x like you're talking about. do you honestly believe they they can make ChatGPT run 16.6 million times faster, without changing hardware? do you think ChatGPT will just like, hit an inflection point where it realizes how to refactor its inference code to run 16,666,666x faster? no, I think that's absurd. you're treating these things like black boxes but they are constrained by computational complexity, die area and the speed of light for Christ's sake.

righttoolforjob 4y ago
What is the limit of intelligence? What limits the human intelligence? How many more times intelligent would an AGI have to be to have the capacity to take over the world?
sn9 4y ago
The comparison shouldn't be to individual humans, but to societies of all scales. AIs can scale horizontally as well as vertically.
aeternum 4y ago
By that logic, we should have never broken the sound barrier because we're under the same physics constraints as other biological organisms. The key difference is that this new intelligence is not constrained by having to slowly pump ions across cell membranes in order to internally generate power for computation. Silicon transistors are much more efficient at computation than the human brain just as a jet engine is superior to to a peregrine falcon. ChatGPT could absolutely generate thousands of words per second on existing hardware.