3 ms·

> However, your broader point seems to imply that you can't "bootstrap" intelligence, which I don't find convincing. How about this weaker statement: "It is n

> However, your broader point seems to imply that you can't "bootstrap" intelligence, which I don't find convincing. How about this weaker statement: "It is not obviously true that humans (or a human-level AI) can bootstrap to a superhuman AI in a small number of years."

gary_0 2y ago
That one's a definite possibility, I think. I do think GenAI will prove to be very useful, possibly even world-changing (for better or worse), and the current frenzy of investment and research will probably turn up other useful ANN techniques (eventually). But the success of LLMs is not the Final Portent before the Singularity Arrives. I can think of quite a few major lines of research[0] that might be required before we can achieve superintelligence, and it's far from a given they'll succeed (or even get off the ground) any time soon. Especially if the economy loses the appetite for throwing billions of dollars into the AI furnace. [0] My pet theory is that embodiment might be required rather than solely relying on mostly language-based training data. A general intelligence might need to learn via interaction (initially; then you can just copy-paste the weights), because language is merely the hearsay of actual reality. Also, using attention-based "hacks" for easy parallelism to avoid needing exaflops that our hardware doesn't have might also be an issue (that's just a guess though, as I'm no AI expert).
ConspiracyFact 2y ago
This. It’s taken the universe (at least locally) several billion years to achieve human-level intelligence through trial and error. Somehow switching over to silicon is going to get us to superhuman intelligence and beyond within a decade? Why should the intelligence of a system surpass its complexity (if you’ll excuse my playing fast and loose with units)? For all we know we’ll achieve barely-superhuman AI and when we ask it why it doesn’t just make itself exponentially smarter, it will say, “Do you have any idea how complex I am?”

gary_0 2y ago

That one's a definite possibility, I think. I do think GenAI will prove to be very useful, possibly even world-changing (for better or worse), and the current frenzy of investment and research will probably turn up other useful ANN techniques (eventually). But the success of LLMs is not the Final Portent before the Singularity Arrives. I can think of quite a few major lines of research[0] that might be required before we can achieve superintelligence, and it's far from a given they'll succeed (or even get off the ground) any time soon. Especially if the economy loses the appetite for throwing billions of dollars into the AI furnace. [0] My pet theory is that embodiment might be required rather than solely relying on mostly language-based training data. A general intelligence might need to learn via interaction (initially; then you can just copy-paste the weights), because language is merely the hearsay of actual reality. Also, using attention-based "hacks" for easy parallelism to avoid needing exaflops that our hardware doesn't have might also be an issue (that's just a guess though, as I'm no AI expert).

ConspiracyFact 2y ago

This. It’s taken the universe (at least locally) several billion years to achieve human-level intelligence through trial and error. Somehow switching over to silicon is going to get us to superhuman intelligence and beyond within a decade? Why should the intelligence of a system surpass its complexity (if you’ll excuse my playing fast and loose with units)? For all we know we’ll achieve barely-superhuman AI and when we ask it why it doesn’t just make itself exponentially smarter, it will say, “Do you have any idea how complex I am?”