5 ms·

You don’t have to believe AGI is impossible to believe we have no route to get there. The burden of proof on “we have a way to do this” falls heavily on the per

You don’t have to believe AGI is impossible to believe we have no route to get there. The burden of proof on “we have a way to do this” falls heavily on the person proposing it. And being able to do something that wasn’t expected doesn’t imply the ability to do a specific other thing. It’s that belief that’s I’d say is perhaps more cultish than religious.

iknowstuff 3y ago
Who’s saying we definitely have a way to do this? Very few. Nobody’s trying to prove that we definitely do right now. But the “presumption” that we can’t is silly given our advancements
- dullcrisp 3y ago
  This logic doesn’t work with anything else. If I say we’re going to fly to Jupiter you’re going to ask me for specifics not just presume that I know what I’m talking about. Or any of the other infinite things I can claim to be able to do. Why is this one different?
  - eklavya 3y ago
    In that scenario, you are the one claiming we will "never" fly to jupiter.
    - timr 3y ago
      Just to close the metaphorical loop here, the current AI debate is something like: "There's a danger we'll be able to fly to Jupiter and bring home a Jupiterian life-form that will destroy humanity. Therefore we should stop all space exploration." The doomers take the equation with some wildly improbable step that we can't currently explain or justify, multiply the outcome of that step by "infinitely bad", and conclude that the whole thing must be dangerous. But to amp up the rhetoric, they always skip over the "wildly improbable" part.
      - adroniser 3y ago
        I guess the disagreement you have with doomers is the probablility you assign to us being able to create human level AI. For the record most "doomers" don't pull a pascal's wager here. Standard EA doctrine is that if we stagnate technologically at our current level for too long (read centuries) then we will inevitably be wiped out. So they want technological advance, they just want caution.
        timr 3y ago
        I don't know what "most" do, but I see that form of argument all the time -- particularly from the highest profile doomers. Personally, I think this line of argument is driven by the hype cycle more than reality. Use chatGPT or midjourney or whatever for a while, and it's pretty easy to see that we're dramatically overweighting theories of AGI risk, and dramatically underweighting stuff like "disemploying the bottom 80% of the intelligence bell curve" with technologies that automate away lots of formerly white-collar labor. If I had to put my money on an "existential risk" attributable to AI, it would be economic strife.
        adroniser 3y ago
        It's worth saying we still haven't seen a single model trained post-hype cycle from openAI. GPT-4 was summer 2022 stuff.
    - dullcrisp 3y ago
      I’m saying that there are very many things that I can’t prove are impossible that I don’t worry about happening. I’ve learned to live with that uncertainty.
  - anon84873628 3y ago
    Hmm. When the first basic black powder rockets were invented, would it have been unreasonable for someone to think that they could one day reach the moon? Of course there would be many uncertainties -- including what the moon even was. No one could say what technology would be required. But clearly there was a major paradigm shift that suddenly made it within the realm of consideration.
    - dullcrisp 3y ago
      I imagine it might have been unreasonable to suppose that, yes. The fact that the future is unpredictable doesn’t mean that all claims about the future are a priori equally valid. In the future where AGI takes over the world next year I might look silly for arguing with you about it, but that’s a risk I accept based on the fact that I don’t think that’s going to happen.
logicchains 3y ago
We absolutely have a way to get there. A sufficiently large neural network can approximate any function (the universal approximation theorem), which has been mathematically proven. LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it. Eventually we'll have LLMs trained on visual and audio input too (and be trained to predict what happens next), so they can receive an arbitrarily large amount of training samples. If the models keep growing and being trained on more and more data, we'd expect them to become at least as intelligent (good at modelling the world) as humans.
- tedivm 3y ago
  > LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it. I don't see any evidence that this is true or proven in any way. Also, the ability to brute force something doesn't mean that brute forcing it is easy or even feasible. You can apply the same logic to "calculating a private key from a public key" that you are to human intelligence. Sure, a large enough neural network can do either, but that doesn't mean that building them is actually realistic.
  - MostlyStable 3y ago
    It has been proven in much simpler toy models that LLMs just trained on token prediction can and do build "world" models[0]. That's not the same as evidence that they must build a world model, but it's proof that at least sometimes they do so even when just being trained on prediction. [0]https://thegradient.pub/othello/ https://thegradient.pub/othello/
- tsimionescu 3y ago
  This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function. So, there is no absolute certainty that they must be able to approximate human cognition, since there is no reason to believe cognition is continuous. Also, there is no proof and no reason to believe that any current architecture and, more importantly, that the currently known training algorithms can achieve anything close to cognition. After all, animals and humans seem to learn much much much faster (much smaller training sets) than any algorithm we have so far.

iknowstuff 3y ago

Who’s saying we definitely have a way to do this? Very few. Nobody’s trying to prove that we definitely do right now. But the “presumption” that we can’t is silly given our advancements

dullcrisp 3y ago
This logic doesn’t work with anything else. If I say we’re going to fly to Jupiter you’re going to ask me for specifics not just presume that I know what I’m talking about. Or any of the other infinite things I can claim to be able to do. Why is this one different?
- eklavya 3y ago
  In that scenario, you are the one claiming we will "never" fly to jupiter.
  - timr 3y ago
    Just to close the metaphorical loop here, the current AI debate is something like: "There's a danger we'll be able to fly to Jupiter and bring home a Jupiterian life-form that will destroy humanity. Therefore we should stop all space exploration." The doomers take the equation with some wildly improbable step that we can't currently explain or justify, multiply the outcome of that step by "infinitely bad", and conclude that the whole thing must be dangerous. But to amp up the rhetoric, they always skip over the "wildly improbable" part.
    - adroniser 3y ago
      I guess the disagreement you have with doomers is the probablility you assign to us being able to create human level AI. For the record most "doomers" don't pull a pascal's wager here. Standard EA doctrine is that if we stagnate technologically at our current level for too long (read centuries) then we will inevitably be wiped out. So they want technological advance, they just want caution.
      - timr 3y ago
        I don't know what "most" do, but I see that form of argument all the time -- particularly from the highest profile doomers. Personally, I think this line of argument is driven by the hype cycle more than reality. Use chatGPT or midjourney or whatever for a while, and it's pretty easy to see that we're dramatically overweighting theories of AGI risk, and dramatically underweighting stuff like "disemploying the bottom 80% of the intelligence bell curve" with technologies that automate away lots of formerly white-collar labor. If I had to put my money on an "existential risk" attributable to AI, it would be economic strife.
        adroniser 3y ago
        It's worth saying we still haven't seen a single model trained post-hype cycle from openAI. GPT-4 was summer 2022 stuff.
  - dullcrisp 3y ago
    I’m saying that there are very many things that I can’t prove are impossible that I don’t worry about happening. I’ve learned to live with that uncertainty.
- anon84873628 3y ago
  Hmm. When the first basic black powder rockets were invented, would it have been unreasonable for someone to think that they could one day reach the moon? Of course there would be many uncertainties -- including what the moon even was. No one could say what technology would be required. But clearly there was a major paradigm shift that suddenly made it within the realm of consideration.
  - dullcrisp 3y ago
    I imagine it might have been unreasonable to suppose that, yes. The fact that the future is unpredictable doesn’t mean that all claims about the future are a priori equally valid. In the future where AGI takes over the world next year I might look silly for arguing with you about it, but that’s a risk I accept based on the fact that I don’t think that’s going to happen.

logicchains 3y ago

We absolutely have a way to get there. A sufficiently large neural network can approximate any function (the universal approximation theorem), which has been mathematically proven. LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it. Eventually we'll have LLMs trained on visual and audio input too (and be trained to predict what happens next), so they can receive an arbitrarily large amount of training samples. If the models keep growing and being trained on more and more data, we'd expect them to become at least as intelligent (good at modelling the world) as humans.

tedivm 3y ago
> LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it. I don't see any evidence that this is true or proven in any way. Also, the ability to brute force something doesn't mean that brute forcing it is easy or even feasible. You can apply the same logic to "calculating a private key from a public key" that you are to human intelligence. Sure, a large enough neural network can do either, but that doesn't mean that building them is actually realistic.
- MostlyStable 3y ago
  It has been proven in much simpler toy models that LLMs just trained on token prediction can and do build "world" models[0]. That's not the same as evidence that they must build a world model, but it's proof that at least sometimes they do so even when just being trained on prediction. [0]https://thegradient.pub/othello/ https://thegradient.pub/othello/
tsimionescu 3y ago
This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function. So, there is no absolute certainty that they must be able to approximate human cognition, since there is no reason to believe cognition is continuous. Also, there is no proof and no reason to believe that any current architecture and, more importantly, that the currently known training algorithms can achieve anything close to cognition. After all, animals and humans seem to learn much much much faster (much smaller training sets) than any algorithm we have so far.