7 ms·
>the main question for me is, why is this a war? It's a war because the hinted promise behind the hype that the first organization to reach some as-yet-entirel
by GolfPopper 5mo ago
>the main question for me is, why is this a war?
It's a war because the hinted promise behind the hype that the first organization to reach some as-yet-entirely-theoretical AGI that can bootstrap itself to godlike capabilities will then Install Planetary Overlord* and rule the world as near-deities themselves, with the rest of the (surviving) human race as their slaves.
I think it's a nonsensical idea, but that's the relevant driver.
* Coined by SF auther Charles Stross in The Jennifer Morgue (2006)
- hx8 5mo agoThis is a war because the media says it's a war. The media says it's a war because AI companies are paying them to say it's a war [0]. When AGI comes the threat won't be from which primate turned it on, but from how well AGI is aligned with humanity. All of the war talk is to distract from the alignment problem and instead force investment in hardware infrastructure. [0] https://www.wired.com/story/super-pac-backed-by-openai-and-palantir-is-paying-tiktok-influencers-to-fear-monger-about-china/ https://www.wired.com/story/super-pac-backed-by-openai-and-p...
- exe34 5mo agoBefore AGI can choose for itself, it will depend on its creators to decide what it values and how it behaves. We can see how that works whenever grok gets the answer factual.
- fragmede 5mo agoElon's basilisk
- goalieca 5mo agoVery likely humans wont actually understand how the thing we designed works other than in some hand-wavvy statistical way. It'll be a race to whatever works first. There won't be some intentional intelligent design.
- Andrex 5mo agoAm I the only one seeing the very obvious parallels to child rearing here...
- cwillu 5mo agoNo, it is one of the standard tropes in the field.
- mitthrowaway2 5mo agoRobert Miles has a video explaining why aligning AI is not like raising a child: https://www.youtube.com/watch?v=eaYIU6YXr3w https://www.youtube.com/watch?v=eaYIU6YXr3w
- Andrex 5mo agoAppreciate this, thank you. TBH I'm still in "open mode" absorbing as many disparate AI viewpoints as possible. I have my own soft beliefs and theories but I'm not married to any of them.
- exe34 5mo agoIt's exactly like child-rearing, except you get to put a zapper in their head and any time they try to say something you don't like, you zap them. Watch "thinking mode" squirm when you ask them awkward questions.
- joe_mamba 5mo ago>The media says it's a war because AI companies are paying them to say it's a war [0] When AGI comes the threat won't be from which primate turned it on, but from how well AGI is aligned with humanity. And when the AGI comes, they won't unleash it to defeat US enemies, they'll first unleash it to make more US workers redundant and boost their stock valuation.
- vrganj 5mo agoAt which point something akin to the French revolution better break out..
- weavie 5mo agoUnlikely. The media has already been taken over, so unfortunately people are more likely to cheer it on and blame outsiders for their problems.
- Nasrudith 5mo ago[flagged]
- vrganj 5mo agoI did read the history books. All of that eventually ended in the Fifth Republic. You're taking too short a view of history.
- hx8 5mo agoAgreed. The French Revolution is one of the top 5 historical revolutions when valued as a unitarian (Amount of Good) - (Amount of Bad). What most pro-revolution folk fail to mention is * The amount of bad in a revolution is unpredictable and very large. You are fundamentally disrupting the institutions of law and order, which will embolden the worst in society and stroke a fear and self-preservation response in the population. * Almost every revolution does not result in a high quality government taking power. The most common outcome is that the new government is worse than what it replaced, as the most violent and ambitious tend to thrive in revolutionary conditions.
- trhway 5mo ago>Planetary Overlord* AGI is nice, yet not necessary. The orbit filled with Starlink descendants and datacenters will be the it. Anybody else wanting to get there would have to get permission. SpaceX/Musk have all the components for it to happen - from Starship to AI (including the army of robots on the ground). The governmental power/sovereignty of US will be used as a stepping stone (that is the strategy described in the Palantir's Karp's book "Technological Republic") for such global techno-feudal regime establishment.
- customguy 5mo agoKinda like Krikkit, but except for a close knit community of people who can sing, and sing about how much they love their family and whatnot in addition to singing about how much they have to destroy the universe, it's just a bunch of stuck up weirdos who don't like themselves and each other, and have no goal other than somehow, magically, getting away from who and what they are. People where the idea of them singing a happy, compassionate tune conjures something involving motion capture or deepfakes. Why are we suffering fools steering us into the worst of all possible worlds? Are we hoping for some kind of integer overflow?
- dgellow 5mo agoThe discourse on this topic is at the point where I have no idea if people are serious or satirical. Please tell me you don’t seriously believe data centers in spaces is a realistic idea
- trhway 5mo agoI don't "believe". I'm arithmetically sure that it is going to happen, and it will beat the ground based on pretty much all metrics. Some of my comments with napkin numbers https://news.ycombinator.com/item?id=46882199 https://news.ycombinator.com/item?id=46882199 https://news.ycombinator.com/item?id=46880680 https://news.ycombinator.com/item?id=46880680 https://news.ycombinator.com/item?id=46880486 https://news.ycombinator.com/item?id=46880486 Just a very rough primitive illustration - a land for a house in SV is like a $1M, and putting a 10 ton house into space at $100/kg - $1M. Existence of supposedly cheap land somewhere (with not much infrastructure usually) doesn't help as you put your computer nodes into a datacenter building with all the required infrastructure which cost more than the SV land on a sq foot basis. And that is without consideration of how powerful a weapon is the energy generated by a humongous field of solar panels in space. Remember Reagan's Star Wars? Nuclear explosions as a source of power for the direct energy weapons like lasers, etc. Well, you wouldn't need the nukes anymore. Just redirect a bit of power from your compute nodes. And as i already wrote, the large transnational companies will have to take care about their own defense themselves https://news.ycombinator.com/item?id=47981423 https://news.ycombinator.com/item?id=47981423 - one more "feudal" aspect of the coming techno-feudalism. Defense is one of the most important sovereign aspects, and upon acquiring it the transnationals will be able to acquire pretty fast the other sovereign aspects. Like enforcement of the Criminal Code of the Mars Colony - again pretty rough primitive illustration of course. The feudal Europe emerged on the outskirts of the Roman Empire, and in our world the new order will be emerging faster on the outskirts (i.e. where reach and strength of the existing order is weaker), the space being one such "outskirts" dimension and the AI/hypercompute virtual world being the other. To the commenter below with reddit link : they use human env temp for heat radiation estimate. That lowers the numbers and requires AC equipment. Ie they estimate space station, not datacenter
- argomo 5mo agoNot everybody thinks it's nonsensical. Here's a different take: If Anyone Builds It, Everyone Dies https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_Dies https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_...
- senordevnyc 5mo agoYes, nonsensical people like EY don’t think it’s nonsensical.
- loandbehold 5mo agoResearches at top AI labs don't consider EY to be a kook even though they may not necessarily agree. EY concepts/terminology appear in Anthropic safety papers. Geoffrey Hinton takes him quite seriously and mentions him in his interviews.
- cardanome 5mo agoJust because some researchers are infected with this idiocy that EY propagates does not mean that it is legit. Maybe they should pay more attention to real problems like the sycophantic nature of current LLMs causing psychosis in people and worry less about theoretical AGI.
- cthor 5mo agoThey are worried about both risks.
- achierius 5mo agoWho are you to say? Why do you have such little regard for everyone in the field, both pro- and anti- AI development? Do you think they're colluding to deceive us?
- jazzyjackson 5mo agotheres billions, even trillions of dollars on the line, why not start with the assumption they have every incentive to deceive, even if unintentional (ie, deceiving themselves)
- throwawayqqq11 5mo agoI will never comprehend why a godlike deity wouldnt just skip all the wetware bs with us humans and conquer some other celestial body to make paperclips.
- tardedmeme 5mo agoThe deity has no physical presence and can only communicate by putting words on screens. Of course it has to bend humans to its will to actually do stuff. (This deity is called the stock market)
- ben_w 5mo agoIf the AI is so monomaniacally focused on paperclips (or anything else) to be a threat to us, going to some other planet is simply one of the early steps, but they absolutely will come back to Earth after all other resources have been consumed. If such an AI can be reliably made to never ever come back to Earth, they were never a threat in the first place. Nobody knows how to fully test an AI's utility function yet, only randomly test inputs and hope the random distribution we chose is helpful; but every time a diffusion model's output is body horror, every time an LLM makes buggy code (and even every time it gets the pelican-on-bike wrong), this is an example of the test distribution not being good enough.
- aaarrm 5mo agoWell it could recognize that wetware is extremely energy and storage efficient in some ways.
- baq 5mo ago> I think it's a nonsensical idea, but that's the relevant driver. Nice to hear from an optimist sometimes, but it’s hard to be one when meat compute substrate can do all those amazing things in a 4U package on 20W and you extrapolate to silicon
- mrsilencedogood 5mo agoI don't think we understand consciousness, thought, and what we generally consider to be "intelligence" even nearly well enough that we can start getting hopeful that what works for us is going to work for a computer. Philosophers have been working on this for literally millennia and despite electron microscopy, MRIs, our entire standard model of physics, etc etc etc... we're basically no closer than the ancient Greeks, despite continuous opining on the topic. Luckily for planetary overlord hopefuls, you probably don't need the whole package to become overlord. Just machines that can build machines. I will remark that I don't really understand why any of the current idiot overlord hopefuls even want the job. The entire world is _already_ functionally their slaves. The only thing jeff bezos doesn't have that I can imagine he wants is the world to not think he's an asshole. But short of complete genocide of the human race, I don't think even overlord status will make progress on that. Might even be counterproductive.
- Ajedi32 5mo agoIt doesn't have to be that extreme. Even if rather than "godlike capabilities" it just boosted your economic efficiency by 2x over other nations that would still be a serious geopolitical threat. (I'm not necessarily saying that's a realistic outcome either, but it's certainly more realistic.)
- jurgenburgen 5mo agoIf your enemy has a theoretical non-zero chance of achieving infinite power, does it justify expending near-infinite resources to get there first? I guess we’ll find out.