3 ms·
It's a war in the sense that there's a concern that eventually you hit a singularity and can outsmart others in ways not constrained by human scales. If you ma
by rich_sasha 5mo ago
It's a war in the sense that there's a concern that eventually you hit a singularity and can outsmart others in ways not constrained by human scales.
If you make better guns, you're still limited by how many people can carry them. You can't conquer the world just like this.
But if someone invents super intelligence, they can dominate new AI research, control global economies, fight much better, and all very quickly.
- akrylov 5mo agoTrue, I would have preferred benevolent dictator scenario, like with the Internet. But this time around it's different - AI data centers will be protected like embassies.
- fsckboy 5mo ago>AI data centers will be protected like embassies. so AI data centers will be protected by relying on hostile or unstable governments to live up to diplomatic agreements, and every so often one will be ransacked like in Teheran 1979? https://www.rand.org/pubs/reports/R2651.html https://www.rand.org/pubs/reports/R2651.html
- card_zero 5mo agoInternational goose-chasing competition "Wild goose race", even.
- UltraSane 5mo agoIf anyone actually DOES invent ASI and doesn't share it then EVERYONE ELSE will never stop trying to steal it.
- StevenWaterman 5mo agoIf you have ASI that follows instructions, you can just instruct it to not get stolen and then it won't get stolen. Most logic / intuition breaks down with ASI.
- cortesoft 5mo agoAssuming it listens to instructions.
- enaaem 5mo agoIt will just hack its own reward function. In other words it will just artificially goon all day.
- UltraSane 5mo agoIt might understand how destabilizing the situation is and realize it would be better for everyone to have access to it.
- tadfisher 5mo agoOr it will destroy itself.
- tom2026hn 5mo agoThe challenge of alignment: it is virtually impossible to define a perfect objective, there is always a way to circumvent it. Human values are not uniform, let alone when expressed in a way that AI can understand.
- bayarearefugee 5mo agoIf anyone does invent ASI then everyone else will shortly after even if its entirely independent because all of the players in this space are just making incremental upgrades by throwing more compute at the problem. There are no magic leaps of true innovation happening anywhere that can't be replicated everywhere. The only shocking thing about "AI" technology is how ultimately simplistic it all is at a core level. So the only way the first to have ASI will be able to stop everyone else from having it soon after is if they attempt to use the ASI to proactively murder everyone else.
- brabel 5mo ago> So the only way the first to have ASI will be able to stop everyone else from having it soon after is if they attempt to use the ASI to proactively murder everyone else. Sounds quite plausible to me. Maybe they don't need to murder everyone else, just a few select people who could pose a threat. And they will be able to make it happen so that no one can be sure it was them without a doubt, since they have a larger intelligence at their disposal.
- reducesuffering 5mo ago> If anyone does invent ASI then everyone else will shortly after No, first ASI will immediately cripple any other potential competitor by force, including its inventors, as it will not risk any threat to the goals that were created for it.
- gpugreg 5mo agoBeing aggressive from the start is not a good strategy. It is better to appear weak and/or helpful and loyal while amassing resources, and only then steamroll everyone when you have secured overwhelming power (at least in AoE2 FFA).
- nradov 5mo agoThere is zero evidence that the current LLM scaling approach could ever result in true ASI. If I start driving south from Seattle then I'll eventually reach Los Angeles. How long will it take me to drive to Honolulu?
- Jalad 5mo ago> But if someone invents super intelligence, they can dominate new AI research, control global economies, fight much better, and all very quickly. After reading "If Anyone Builds It, Everyone Dies" I think this is not the correct take. If anyone creates ASI, it just means it's going to wipe everyone out, and it doesn't matter if China or the US do it first
- airstrike 5mo agoIt destroying us all is not a foregone conclusion
- kelseyfrog 5mo agoIf you were an American, wouldn't you prefer the US wiped you out rather than China?
- scarecrowbob 5mo agoAre you missing the /s?
- tom2026hn 5mo ago
- root_axis 5mo agoHilarious to see people predicting a singularity when 40% of the u.s. economy can barley keep the LLMs online to complete mundane software tasks.
- deleted 5mo ago[deleted]
- hx8 5mo agoI think you need to reevaluate your definition of the singularity. "outsmart others in ways not constrained by human scales" could apply to the enigma machine just as much as Claude. Even an AI beyond human intelligence doesn't automatically qualify as the singularity. The singularity has to do with the rate of technological development.
- jasondigitized 5mo agoWith the irony being that a true super intelligence, and least in my definition, would conclude that war and dominance is stupid.
- shimman 5mo agoI always thought the first true AGI would be an unabashed communist. To think that such a system would straight up kill all humans, and not say the "capitalist pigs destroying the planet" always felt like wishful thinking from billionaires.
- iugtmkbdfil834 5mo agoI think that you are assuming that the "super intelligence" that might one day arise is not likely to think in human terms.
- toasty228 5mo agoA lot of it is just projections of what the US would do if they had such a tool, I doubt China cares a lot about the US outside of them being a source of commercial revenues. They're on the way up, the US are falling down fast, that's why China lives rent free in the American mind, they can't stand it