4 ms·

The problem is that we're going through what happened in computer vision all over again. Convnets were getting bigger and bigger. New improvements came about in

The problem is that we're going through what happened in computer vision all over again. Convnets were getting bigger and bigger. New improvements came about in training efficiency, making medium models better, etc. Until... marginal improvements became more and more marginal. LLMs are going through the same thing now. Better and better every iteration but increasingly we're starting to see a fast approaching wall on what they can really do with the current paradigm.

bottlepalm 2y ago
We are literally standing on the precipice of agency. After that, it's over. You see us approaching a wall, I see a cliff.
- WJW 2y ago
  To stick with the metaphor, it's not super unclear if agency is within grasp or on the other side of an abyss. LLMs are definitely an improvement, but it's not at all clear if they can scale to human-level agency. If they reach that, it's even more unclear if they could ever reach superhuman levels given that all their training data is human-level. And finally, we can see from normal human society that it is hardly ever the smartest humans who achieve the most or rise to the highest levels of power. There is no reason to believe that an AI with agency would an inherent "it's over" scenario
  - philistine 2y ago
    We kill millions of humans with agency every day on this planet. And none of them immediately die if we stop providing them with electrical power ... well I guess a small amount of them do. Anyway, we'll be fine. If the AI come, can they immediately stop us from growing food? How can an AI prevent my orchard from giving apples next year? Sure, it can mess up Facebook, but at this point that'd be a benefit.
    - WJW 2y ago
      I don't want to be a hater but according to the UN about 61 million humans die per year, so that only comes out to ~167k per day rather than millions. Most of those will die from old age too, rather than "being killed". Your main point is true though, even superhuman AI would have a rough time in actual combat. It's just too dependent on electricity and datacenters with known locations to actually have a chance.
      - labster 2y ago
        I’m sure the superintelligent AI will convince humans to transport its core while plugged into a potato battery. But honestly did the supply chain attack on Lebanese pagers last week teach you nothing? AIs should be great at that.
        philistine 2y ago
        What? Planting C4 in pagers? Those pagers did not blow up on their own. Hundreds of people needed to put all that explosive in all those devices. Our world is still analogue.
    - tivert 2y ago
      > We kill millions of humans with agency every day on this planet. And none of them immediately die if we stop providing them with electrical power ... well I guess a small amount of them do. Anyway, we'll be fine. If the AI come, can they immediately stop us from growing food? How can an AI prevent my orchard from giving apples next year? Sure, it can mess up Facebook, but at this point that'd be a benefit. 1. IMHO, genocidal apocalypse scenarios like you describe are the wrong way to think about the societal danger of AGI. I think a far more likely outcome, and nearly as disastrous, is AGI eliminating the dependency of capital on labor, leading to a global China Shock on steroids (e.g. unimaginable levels of inequality and wealth concentration, which no level of personal adaptability could overcome). 2. Even in your apocalypse scenario, I think you underestimate the damage that could be done. I don't have an orchard, so I know I'm fucked if urban life-support systems get messed up over large enough area that no aid is coming to my local area afterwards. And a genocidal AI that wasn't blindingly stupid would wait until it had control of enough robots to act in the real world, after which it could agent-orange your orchard.
  - bottlepalm 2y ago
    > There is no reason to believe... What is happening right now is so obvious that people have been predicting it for over 60 years. It is ingrained in our culture. Everyone knows what happens when the AI becomes smarter than us. If you 'see no reason' then you are looking through a microscope. Lift your head up and look around. Agency isn't black and white, it is a gradient. We already have agency to some degree, and it is improving fast.

bottlepalm 2y ago

We are literally standing on the precipice of agency. After that, it's over. You see us approaching a wall, I see a cliff.

WJW 2y ago
To stick with the metaphor, it's not super unclear if agency is within grasp or on the other side of an abyss. LLMs are definitely an improvement, but it's not at all clear if they can scale to human-level agency. If they reach that, it's even more unclear if they could ever reach superhuman levels given that all their training data is human-level. And finally, we can see from normal human society that it is hardly ever the smartest humans who achieve the most or rise to the highest levels of power. There is no reason to believe that an AI with agency would an inherent "it's over" scenario
- philistine 2y ago
  We kill millions of humans with agency every day on this planet. And none of them immediately die if we stop providing them with electrical power ... well I guess a small amount of them do. Anyway, we'll be fine. If the AI come, can they immediately stop us from growing food? How can an AI prevent my orchard from giving apples next year? Sure, it can mess up Facebook, but at this point that'd be a benefit.
  - WJW 2y ago
    I don't want to be a hater but according to the UN about 61 million humans die per year, so that only comes out to ~167k per day rather than millions. Most of those will die from old age too, rather than "being killed". Your main point is true though, even superhuman AI would have a rough time in actual combat. It's just too dependent on electricity and datacenters with known locations to actually have a chance.
    - labster 2y ago
      I’m sure the superintelligent AI will convince humans to transport its core while plugged into a potato battery. But honestly did the supply chain attack on Lebanese pagers last week teach you nothing? AIs should be great at that.
      - philistine 2y ago
        What? Planting C4 in pagers? Those pagers did not blow up on their own. Hundreds of people needed to put all that explosive in all those devices. Our world is still analogue.
  - tivert 2y ago
    > We kill millions of humans with agency every day on this planet. And none of them immediately die if we stop providing them with electrical power ... well I guess a small amount of them do. Anyway, we'll be fine. If the AI come, can they immediately stop us from growing food? How can an AI prevent my orchard from giving apples next year? Sure, it can mess up Facebook, but at this point that'd be a benefit. 1. IMHO, genocidal apocalypse scenarios like you describe are the wrong way to think about the societal danger of AGI. I think a far more likely outcome, and nearly as disastrous, is AGI eliminating the dependency of capital on labor, leading to a global China Shock on steroids (e.g. unimaginable levels of inequality and wealth concentration, which no level of personal adaptability could overcome). 2. Even in your apocalypse scenario, I think you underestimate the damage that could be done. I don't have an orchard, so I know I'm fucked if urban life-support systems get messed up over large enough area that no aid is coming to my local area afterwards. And a genocidal AI that wasn't blindingly stupid would wait until it had control of enough robots to act in the real world, after which it could agent-orange your orchard.
- bottlepalm 2y ago
  > There is no reason to believe... What is happening right now is so obvious that people have been predicting it for over 60 years. It is ingrained in our culture. Everyone knows what happens when the AI becomes smarter than us. If you 'see no reason' then you are looking through a microscope. Lift your head up and look around. Agency isn't black and white, it is a gradient. We already have agency to some degree, and it is improving fast.