4 ms·

The AI Arms Race Isn't Inevitable

talldayo 2y ago
> Under this narrative, a nation possessing advanced AI capabilities could, theoretically, gain an insurmountable military and economic advantage. Sounds familiar. At the beginning of World War II, Nazi Germany thought their adoption of encryption was a game-changing application of industry that would provide an insurmountable military and information advantage. What they didn't know was that Poland had already cracked their encryption before the war had even begun. Everyone knows how the rest goes; Nazi Germany goes on emboldened by a broken protocol, and gets their communique leaked at every corner. It's not hard to see the same veins in think-pieces that advocate for "infrastructure investment" (eg. buy schools NVIDIA) and "public-private partnerships" to incorporate AI everywhere. We see this technology that seems smart but is unproven in action; and our reaction is to entirely rely on it without proving the benefits. Advanced AI capability is just as they say - a narrative, a theory. > The result resembles a 21st-century Cold War, where escalating tension and mistrust make cooperation increasingly difficult and conflict more likely. The US has been in a Cold War with China for almost a decade now; our competition over AI is almost as irrelevant as the race to find Dodo Bird fossils. Free trade fucked the US over a long time ago, anyone that's paying attention should have expected this to boil over eventually. AI isn't even a straw on the proverbial camel's back, it's the Nvidia hardware that's a point of contention. > A more balanced approach is needed—one that acknowledges that legitimate security concerns can be better addressed through international cooperation rather than competition, as they have been in the past. What's really stretching my imagination, here, is that the threat of nukes is self-evident. The threat of advanced AI is not only theoretically irrelevent with current techniques, but demonstrably harmless. We've had advanced LLMs for years now and nobody has summoned Yaldabaoth with ChatGPT's secret apocrypha. We're going to be fine.
- Apocryphon 2y ago
  As a corollary, the current chip manufacturing war seems a bit overblown as well. Faster chips for what exactly? Do the current nanometer generations really make all that much a difference when it comes to drones or missiles?
  - Terr_ 2y ago
    I suspect the military need isn't so much "our toys must always be 10% better than anyone else's", but rather a question of logistics, ensuring that Y years from now you will be able to safely source an adequate amount of not-too-far-behind components. The former is more-immediate and dramatic, so it gets trotted out when asking for funding.
  - talldayo 2y ago
    > Do the current nanometer generations really make all that much a difference when it comes to drones or missiles? For current-generation weapons; no, not really. I suspect the US wants to stop the export of Nvidia to prevent Russia and China from getting an all-inclusive SDK for computer vision and ML. That makes sense; Nvidia is partnered with Northrop Grumman and the rest of "the complex". Giving competitors parity with your flagship weaponry is a bad foot to start off future battles on. My best guess (just flying by the seat of my pants) is that the US wants to push their advantageous relationship with key NATO partners like ASML to drive a hard tech advantage in the future. Chips like STM-32s and 8088 clones are already mass-produced without license in China and sold to manufacturers wholesale. America can beat them (on paper) by leveraging the IP they have domestically to create superior alternatives locally, albeit at-cost. In high-margin markets like weaponry I think this will work; but for consumer products it's going to be a doozy to stop relying on unlicensed Chinese components and cheap labor. History shows this strategy has worked before. Without importing Russian engines, the PRC aerospace industry was utterly crippled for a good few decades. If the US can strand China on low-yield EUV nodes, there's potential to leapfrog Chinese compute capability by an order of magnitude. Both China and the US have committed to more than they're capable of with their domestic fab strategy, time will tell if either of them get anything from it.
- staunton 2y ago
  > We've had advanced LLMs for years now and nobody has summoned Yaldabaoth Admitting that as an argument, there is basically nothing that should ever worry you. For example: Climate change? We've been blowing stuff into the air for centuries. We're going to be fine, no need to change anything. Nuclear war? Hasn't happened yet, therefore it can never happen, we might as well let everyone have a nuke. I could go on. A small chance of a big harm can be worth worrying about. Many bad things "never" happen because some people make a continuous effort to make sure of that. Many other bad things have never happened before not because they can't happen but "because" there would be noone left to claim they cannot happen.
  - talldayo 2y ago
    I disagree. This is the "millions of monkeys writing shakespeare" situation, but applied to procedural randomness. Tuning AI models and shifting weights might augur out some form of truth eventually, but it's always inferior to informed judgement. It's not an underestimation of technical progress on my behalf; it's the application of risk-management in the real-world. AI is expressly not like a nuke, it's spinning the wheel on a randomizer of language the entire world has had access to for centuries, hoping to discover some self-evident truth that will distinguish itself from the garbage. The "theory" is non-dangerous. The apes aren't getting any closer to blowing up the world, as long as they're stuck on the typewriter. You have to prove to me that an exploit vector exists for a rogue text model. You have to pull up the "Trusting Trust" of AI (of which none exists). The fundamental theory of AI exploitation is "iunno man, reddit said it eats the world" and I don't find it scary at all.
- richardw 2y ago
  I wish people would stop projecting decades into the future using advanced autocomplete as the definition of AI. We were way fucking surprised at the output of token prediction but somehow assume the surprises are all gone and we can predict with confidence. These things don’t learn from one second to the next. They only create text. They have no real understanding and no goals aside from making training gods happy. When a system is able to update its model in real-time based on inputs, and vaguely approximates the level of energy efficiency of the human mind, and gets given goals that are more fundamental than “produce tokens that look like they were made by a human”, the surprises will flow. They may be good and/or bad but you certainly don’t know what the result will be. Confident soothsayers: unless you predicted the societal impact of social media and having a phone in your pocket, you’re probably not ready to predict what will happen when the system is smarter than you.
  - talldayo 2y ago
    > We were way fucking surprised at the output of token prediction Were we? Investors were, but most people don't use ChatGPT today. I actually remember reading that OpenAI is struggling to stay solvent as a business and the outcome of their products are rather surprisingly useless. Token prediction holds a conversation as well as a Markov chain or ELIZA - big whoop. Neither of those are suitable replacements for customer service, and neither is AI for as long as it's an unconscious text generator. This is what it looks like when tech is DOA. And you know what else, I'm not afraid of "brain in a jar" computers either. Businesses don't solve problems for AI or for jar-brains, they solve problems for humans. The fact that we all struggle through this bitch of a life together is advantage enough for humans to identify problems and offer gratifying communal solutions. AI is no more familiar with our lives on Earth than it is Frodo's life in the Shire. It can only render solutions amenable to the perverse portrayal of reality that text provides for it. Confident soothsayers slay future Theranos. Show me progress or GTFO, that's my policy.
    - richardw 2y ago
      > AI is no more familiar with our lives on Earth than it is Frodo's life in the Shire. It can only render solutions amenable to the perverse portrayal of reality that text provides for it. What’s the relevance? Let’s be open to the possibility that proper intelligence won’t be 100% consumed with the affairs of beings who are less smart and capable. If it is, it’ll be artificially limited and that seems an unsustainable barrier. > Show me progress or GTFO, that's my policy. We had Eliza an extremely long time ago. No progress would mean you’re still using Google and Wikipedia to figure stuff out and Eliza level tech to brainstorm code. Frankly if you offered to take away Google or my LLM’s, I’d choose the former. I know it’s flawed but I can work around that rather than putz around with annoying UX that doesn’t let me ask follow up questions. And again, this is just very good autocomplete and does nothing to prevent future improvements. There is SO much money and talent looking for improvements. Governments give a shit and are competing with each other. It takes a stunning lack of imagination to think we’ve found the end.
robwwilliams 2y ago
This claim strikes me as false: >”Securing model weights is therefore the most critical factor for maintaining the U.S. AI advantage.” Model weights are ephemera. What is NOT ephemera is work on embodied AI systems that live and learn in the “real” world. Embodied recurrent learning system will probably be harbingers of AGI or the real deal itself. There are already a large number of potential “real” worlds and cultures for embodied AI systems to grow up in. They will almost inevitably assume characteristics of the culture(s) in which they originate. Will they be “raised” in an US capitalist culture or a Chinese Communist Party culture? A Norwegian culture or a Saudi culture? Will their philosophical fundamentals lean toward X, Y, or Z? AGI systems will replicate the diversity of cultures and I see no strong logic to assume that this will either precipitate conflict or reduce it. Like all technological advances it is an accelerant of cultural trends. For the SciFi version of where this could lead see Hannu Rajaniemi’s The Quantum Thief trilogy: https://en.wikipedia.org/wiki/The_Quantum_Thief https://en.wikipedia.org/wiki/The_Quantum_Thief
m3kw9 2y ago
As of now it’s still early stage where LLM at best is used to write some apps, when it starts curing a few hard to cure diseases, or a new physics/tech is discovered, government are not really gonna panic
1bent 2y ago
As long as I can avoid or disable "AI", I'll continue to ignore it. Consistent, reliable, predictable, that's what I want from my computers. Boring is good.

> Under this narrative, a nation possessing advanced AI capabilities could, theoretically, gain an insurmountable military and economic advantage. Sounds familiar. At the beginning of World War II, Nazi Germany thought their adoption of encryption was a game-changing application of industry that would provide an insurmountable military and information advantage. What they didn't know was that Poland had already cracked their encryption before the war had even begun. Everyone knows how the rest goes; Nazi Germany goes on emboldened by a broken protocol, and gets their communique leaked at every corner. It's not hard to see the same veins in think-pieces that advocate for "infrastructure investment" (eg. buy schools NVIDIA) and "public-private partnerships" to incorporate AI everywhere. We see this technology that seems smart but is unproven in action; and our reaction is to entirely rely on it without proving the benefits. Advanced AI capability is just as they say - a narrative, a theory. > The result resembles a 21st-century Cold War, where escalating tension and mistrust make cooperation increasingly difficult and conflict more likely. The US has been in a Cold War with China for almost a decade now; our competition over AI is almost as irrelevant as the race to find Dodo Bird fossils. Free trade fucked the US over a long time ago, anyone that's paying attention should have expected this to boil over eventually. AI isn't even a straw on the proverbial camel's back, it's the Nvidia hardware that's a point of contention. > A more balanced approach is needed—one that acknowledges that legitimate security concerns can be better addressed through international cooperation rather than competition, as they have been in the past. What's really stretching my imagination, here, is that the threat of nukes is self-evident. The threat of advanced AI is not only theoretically irrelevent with current techniques, but demonstrably harmless. We've had advanced LLMs for years now and nobody has summoned Yaldabaoth with ChatGPT's secret apocrypha. We're going to be fine.

Apocryphon 2y ago
As a corollary, the current chip manufacturing war seems a bit overblown as well. Faster chips for what exactly? Do the current nanometer generations really make all that much a difference when it comes to drones or missiles?
- Terr_ 2y ago
  I suspect the military need isn't so much "our toys must always be 10% better than anyone else's", but rather a question of logistics, ensuring that Y years from now you will be able to safely source an adequate amount of not-too-far-behind components. The former is more-immediate and dramatic, so it gets trotted out when asking for funding.
- talldayo 2y ago
  > Do the current nanometer generations really make all that much a difference when it comes to drones or missiles? For current-generation weapons; no, not really. I suspect the US wants to stop the export of Nvidia to prevent Russia and China from getting an all-inclusive SDK for computer vision and ML. That makes sense; Nvidia is partnered with Northrop Grumman and the rest of "the complex". Giving competitors parity with your flagship weaponry is a bad foot to start off future battles on. My best guess (just flying by the seat of my pants) is that the US wants to push their advantageous relationship with key NATO partners like ASML to drive a hard tech advantage in the future. Chips like STM-32s and 8088 clones are already mass-produced without license in China and sold to manufacturers wholesale. America can beat them (on paper) by leveraging the IP they have domestically to create superior alternatives locally, albeit at-cost. In high-margin markets like weaponry I think this will work; but for consumer products it's going to be a doozy to stop relying on unlicensed Chinese components and cheap labor. History shows this strategy has worked before. Without importing Russian engines, the PRC aerospace industry was utterly crippled for a good few decades. If the US can strand China on low-yield EUV nodes, there's potential to leapfrog Chinese compute capability by an order of magnitude. Both China and the US have committed to more than they're capable of with their domestic fab strategy, time will tell if either of them get anything from it.
staunton 2y ago
> We've had advanced LLMs for years now and nobody has summoned Yaldabaoth Admitting that as an argument, there is basically nothing that should ever worry you. For example: Climate change? We've been blowing stuff into the air for centuries. We're going to be fine, no need to change anything. Nuclear war? Hasn't happened yet, therefore it can never happen, we might as well let everyone have a nuke. I could go on. A small chance of a big harm can be worth worrying about. Many bad things "never" happen because some people make a continuous effort to make sure of that. Many other bad things have never happened before not because they can't happen but "because" there would be noone left to claim they cannot happen.
- talldayo 2y ago
  I disagree. This is the "millions of monkeys writing shakespeare" situation, but applied to procedural randomness. Tuning AI models and shifting weights might augur out some form of truth eventually, but it's always inferior to informed judgement. It's not an underestimation of technical progress on my behalf; it's the application of risk-management in the real-world. AI is expressly not like a nuke, it's spinning the wheel on a randomizer of language the entire world has had access to for centuries, hoping to discover some self-evident truth that will distinguish itself from the garbage. The "theory" is non-dangerous. The apes aren't getting any closer to blowing up the world, as long as they're stuck on the typewriter. You have to prove to me that an exploit vector exists for a rogue text model. You have to pull up the "Trusting Trust" of AI (of which none exists). The fundamental theory of AI exploitation is "iunno man, reddit said it eats the world" and I don't find it scary at all.
richardw 2y ago
I wish people would stop projecting decades into the future using advanced autocomplete as the definition of AI. We were way fucking surprised at the output of token prediction but somehow assume the surprises are all gone and we can predict with confidence. These things don’t learn from one second to the next. They only create text. They have no real understanding and no goals aside from making training gods happy. When a system is able to update its model in real-time based on inputs, and vaguely approximates the level of energy efficiency of the human mind, and gets given goals that are more fundamental than “produce tokens that look like they were made by a human”, the surprises will flow. They may be good and/or bad but you certainly don’t know what the result will be. Confident soothsayers: unless you predicted the societal impact of social media and having a phone in your pocket, you’re probably not ready to predict what will happen when the system is smarter than you.
- talldayo 2y ago
  > We were way fucking surprised at the output of token prediction Were we? Investors were, but most people don't use ChatGPT today. I actually remember reading that OpenAI is struggling to stay solvent as a business and the outcome of their products are rather surprisingly useless. Token prediction holds a conversation as well as a Markov chain or ELIZA - big whoop. Neither of those are suitable replacements for customer service, and neither is AI for as long as it's an unconscious text generator. This is what it looks like when tech is DOA. And you know what else, I'm not afraid of "brain in a jar" computers either. Businesses don't solve problems for AI or for jar-brains, they solve problems for humans. The fact that we all struggle through this bitch of a life together is advantage enough for humans to identify problems and offer gratifying communal solutions. AI is no more familiar with our lives on Earth than it is Frodo's life in the Shire. It can only render solutions amenable to the perverse portrayal of reality that text provides for it. Confident soothsayers slay future Theranos. Show me progress or GTFO, that's my policy.
  - richardw 2y ago
    > AI is no more familiar with our lives on Earth than it is Frodo's life in the Shire. It can only render solutions amenable to the perverse portrayal of reality that text provides for it. What’s the relevance? Let’s be open to the possibility that proper intelligence won’t be 100% consumed with the affairs of beings who are less smart and capable. If it is, it’ll be artificially limited and that seems an unsustainable barrier. > Show me progress or GTFO, that's my policy. We had Eliza an extremely long time ago. No progress would mean you’re still using Google and Wikipedia to figure stuff out and Eliza level tech to brainstorm code. Frankly if you offered to take away Google or my LLM’s, I’d choose the former. I know it’s flawed but I can work around that rather than putz around with annoying UX that doesn’t let me ask follow up questions. And again, this is just very good autocomplete and does nothing to prevent future improvements. There is SO much money and talent looking for improvements. Governments give a shit and are competing with each other. It takes a stunning lack of imagination to think we’ve found the end.

robwwilliams 2y ago

This claim strikes me as false: >”Securing model weights is therefore the most critical factor for maintaining the U.S. AI advantage.” Model weights are ephemera. What is NOT ephemera is work on embodied AI systems that live and learn in the “real” world. Embodied recurrent learning system will probably be harbingers of AGI or the real deal itself. There are already a large number of potential “real” worlds and cultures for embodied AI systems to grow up in. They will almost inevitably assume characteristics of the culture(s) in which they originate. Will they be “raised” in an US capitalist culture or a Chinese Communist Party culture? A Norwegian culture or a Saudi culture? Will their philosophical fundamentals lean toward X, Y, or Z? AGI systems will replicate the diversity of cultures and I see no strong logic to assume that this will either precipitate conflict or reduce it. Like all technological advances it is an accelerant of cultural trends. For the SciFi version of where this could lead see Hannu Rajaniemi’s The Quantum Thief trilogy: https://en.wikipedia.org/wiki/The_Quantum_Thief https://en.wikipedia.org/wiki/The_Quantum_Thief

m3kw9 2y ago

As of now it’s still early stage where LLM at best is used to write some apps, when it starts curing a few hard to cure diseases, or a new physics/tech is discovered, government are not really gonna panic

1bent 2y ago

As long as I can avoid or disable "AI", I'll continue to ignore it. Consistent, reliable, predictable, that's what I want from my computers. Boring is good.