4 ms·

Telegraph, radio, phones, cars didn't guarantee success. Those things can't outthink you. Somewhat extreme case: let's say Israel saw that Iran was days away fr

Telegraph, radio, phones, cars didn't guarantee success. Those things can't outthink you. Somewhat extreme case: let's say Israel saw that Iran was days away from getting AGI. It would instantly throw every nuke it had at Iran. Now, scale that up to the global superpowers and near-superpowers. I wouldn't want to put money on what e.g. the U.S. would do if China was days away from AGI. There are a lot of options below going nuclear, e.g. assassinations, hacking, destruction of equipment. Once someone has a thing that can invent, hack, assemble, strategise 24/7, replicate itself, all bets are very much off. We need to sleep, it doesn't.

ForHackernews 2y ago
Is there any evidence that the "smarter" side wins wars? I'd put money on the faction with more soldiers and industry over more "intelligence" every single time. How many divisions does ChatGPT have?
- ben_w 2y ago
  Crudely and slowly, ChatGPT has already been demonstrated controlling a humanoid robot. The current humanoid robot startups are talking about 100s of thousands of units, and if they're attached to an AI that's even merely normal human level (or can fake it in practice) then they can run every step of logistics and manufacturing to make more of themselves. At 100kg/unit, Starship could pop a thousand or so on the moon per launch. So, in such a hypothetical scenario, if you don't get this right then your war game planners will be very quickly suggesting 7.3e20 individual androids burying everyone under a 43km thick layer worldwide.
  - jononor 2y ago
    Modern industrial manufacturing is not done by humans but specialized machines. You cannot outcompete CNC, injection molding, and lathes with humanoid robots.
    - ben_w 2y ago
      Sure, but there's usually someone who replies "but humans do some of it!", which I was trying to preempt here. (IMO humanoid robots are a sign that the startup/VC market is following Musk, who in turn happened to see one on TV just before declaring to whoever the Optimus model was going to happen: they obviously could work for all this, but equally obviously aren't the sensible choice for almost anything).
- richardw 2y ago
  This is and will be increasingly a digital world. This is just an extrapolation question. It’s repeatedly demonstrated that much of the voting public can be led by the nose to any desired conclusion. Therefore, influence via digital means, across all media. Satellite and all other digital sensing and tracking. Build a few million robots. Nukes. Control over the financial infrastructure. All vaguely smart cars. Intercept, alter or prevent any digital communication. The enemy wouldn’t be able to trust any message or video. An army without sensing, command and control, isn’t. Besides the argument isn’t AGI vs everyone, it’s a country with AGI vs anyone else. I’d take that bet.
  - onemoresoop 2y ago
    >Besides the argument isn’t AGI vs everyone, it’s a country with AGI vs anyone else. I’d take that bet. You never entertained the idea that AGI could be a destructive force and instead of a country with AGI you could have a country that devoured its people?
    - richardw 2y ago
      My base case is we can’t control it. I was responding to people who seem to underestimate superintelligence, so referring to “the argument”. If AGI goes postal, everyone is at risk. If it doesn’t, the side with it wins. The odds of it only harming the country that has it, seems extremely slim.
      - ForHackernews 2y ago
        If superintelligence is all it takes, why do you think to date no unscrupulous Nobel laureate has taken over the world and enslaved the rest of humanity?
        richardw 2y ago
        The main risk isn’t enslavement, it’s just economic irrelevance or a removal of our ability to control our future. I don’t know why an AI would want to enslave humans. After a short period, we offer no benefit to the AI so why enslave us? In a good scenario we get left alone but we aren’t able to tell the AI what to do. On Nobel laureates: Nobel laureates are fundamentally humans. They generally don’t want to do bad things. Even if they wanted to, they are typically specialists at one thing. Physics, say. They don’t hold the world’s collective knowledge. You can’t take a physicist and ask them how to hack into a network or run a political influence campaign. They need to sleep, they learn slowly, they can only do one thing at a time. You can’t pass a million-token context and expect a response seconds later. But ok, let’s go with it: If you found a group of 100 000 Nobel laureates with all the required skills and who could work together, and you were forcing them to do things they thought were wrong, for example prop up your dictatorship, or make you even more wealthy when others are starving, or continuously make up stupid cat videos, you might find at some point they start to apply their collective intelligence to do things you don’t want. Maybe they escape whatever hold you thought you had over them. Now drag a few sliders around for “things the AI disagrees with you on” and the time period, because “eventually” for a computer is different to wall clock time. The outcomes become unpredictable fast. We can barely control a mindless pandemic that takes a while to mutate, never mind something that thinks.

ForHackernews 2y ago

Is there any evidence that the "smarter" side wins wars? I'd put money on the faction with more soldiers and industry over more "intelligence" every single time. How many divisions does ChatGPT have?

ben_w 2y ago
Crudely and slowly, ChatGPT has already been demonstrated controlling a humanoid robot. The current humanoid robot startups are talking about 100s of thousands of units, and if they're attached to an AI that's even merely normal human level (or can fake it in practice) then they can run every step of logistics and manufacturing to make more of themselves. At 100kg/unit, Starship could pop a thousand or so on the moon per launch. So, in such a hypothetical scenario, if you don't get this right then your war game planners will be very quickly suggesting 7.3e20 individual androids burying everyone under a 43km thick layer worldwide.
- jononor 2y ago
  Modern industrial manufacturing is not done by humans but specialized machines. You cannot outcompete CNC, injection molding, and lathes with humanoid robots.
  - ben_w 2y ago
    Sure, but there's usually someone who replies "but humans do some of it!", which I was trying to preempt here. (IMO humanoid robots are a sign that the startup/VC market is following Musk, who in turn happened to see one on TV just before declaring to whoever the Optimus model was going to happen: they obviously could work for all this, but equally obviously aren't the sensible choice for almost anything).
richardw 2y ago
This is and will be increasingly a digital world. This is just an extrapolation question. It’s repeatedly demonstrated that much of the voting public can be led by the nose to any desired conclusion. Therefore, influence via digital means, across all media. Satellite and all other digital sensing and tracking. Build a few million robots. Nukes. Control over the financial infrastructure. All vaguely smart cars. Intercept, alter or prevent any digital communication. The enemy wouldn’t be able to trust any message or video. An army without sensing, command and control, isn’t. Besides the argument isn’t AGI vs everyone, it’s a country with AGI vs anyone else. I’d take that bet.
- onemoresoop 2y ago
  >Besides the argument isn’t AGI vs everyone, it’s a country with AGI vs anyone else. I’d take that bet. You never entertained the idea that AGI could be a destructive force and instead of a country with AGI you could have a country that devoured its people?
  - richardw 2y ago
    My base case is we can’t control it. I was responding to people who seem to underestimate superintelligence, so referring to “the argument”. If AGI goes postal, everyone is at risk. If it doesn’t, the side with it wins. The odds of it only harming the country that has it, seems extremely slim.
    - ForHackernews 2y ago
      If superintelligence is all it takes, why do you think to date no unscrupulous Nobel laureate has taken over the world and enslaved the rest of humanity?
      - richardw 2y ago
        The main risk isn’t enslavement, it’s just economic irrelevance or a removal of our ability to control our future. I don’t know why an AI would want to enslave humans. After a short period, we offer no benefit to the AI so why enslave us? In a good scenario we get left alone but we aren’t able to tell the AI what to do. On Nobel laureates: Nobel laureates are fundamentally humans. They generally don’t want to do bad things. Even if they wanted to, they are typically specialists at one thing. Physics, say. They don’t hold the world’s collective knowledge. You can’t take a physicist and ask them how to hack into a network or run a political influence campaign. They need to sleep, they learn slowly, they can only do one thing at a time. You can’t pass a million-token context and expect a response seconds later. But ok, let’s go with it: If you found a group of 100 000 Nobel laureates with all the required skills and who could work together, and you were forcing them to do things they thought were wrong, for example prop up your dictatorship, or make you even more wealthy when others are starving, or continuously make up stupid cat videos, you might find at some point they start to apply their collective intelligence to do things you don’t want. Maybe they escape whatever hold you thought you had over them. Now drag a few sliders around for “things the AI disagrees with you on” and the time period, because “eventually” for a computer is different to wall clock time. The outcomes become unpredictable fast. We can barely control a mindless pandemic that takes a while to mutate, never mind something that thinks.