8 ms·
But AI doesn't pose an existential threat to humanity, so we're all good.
by oh_sigh 3y ago
But AI doesn't pose an existential threat to humanity, so we're all good.
- HKH2 3y agoI really can't grasp how people think that a system that doesn't have a need to preserve itself will somehow start thinking for itself. AI is quite troublesome for privacy though. How much privacy humans need is a question we'll probably have answered the hard way.
- loki-ai 3y agothat’s precisely the point: the technical geniuses lack creativity to predict how things can go wrong in a thousand of different ways.
- oh_sigh 3y agoSo...who isn't lacking the creativity? In fact, I would say techies are unreasonably gloomy because they grew up obsessing over sci-fi.
- nahSeetho 3y ago[dead]
- orbital-decay 3y agoWhat makes you think they're predicting the apocalypse correctly, then? Another thing the technical geniuses tend to be good at is exploiting the power they suddenly obtain in their own interest, either directly or with regulations and collusion with those who hold actual hard power. Evil AI owners seem to be much closer and far more material than an evil AI, and coincidentally it's something that is almost entirely lacking from the discourse, as public attention is too focused on sci-fi hypotheticals.
- PeterisP 3y agoThe bar is different - saying "there is no risk of apocalypse" requires you to be ~100% certain, because it you're saying "I'm 99% certain that there won't be a an apocalypse" then you're on the side of the AI-risk people, because a low-probability extinction event does justify action; the risk argument isn't that apocalypse is certain but rather that it is sufficiently plausible to take preventive measures.
- orbital-decay 3y agoThe issue is that the doomsday scenario is extremely vague. The actual mechanism of action of a hypothetical rogue AGI is usually handwaved away as "it will be self-improving, superhumanly persuasive, and far smarter than us, so it will somehow figure out how to do something, or convince us to do it". What exactly will happen? How exactly will it happen? Will the world do nothing until that moment? How do society, politics, military fit into that scenario? All that rationalist navel gazing I've seen so far is either hilariously unaware of the existence of the outside world or assumes it won't change in the process. You can't fight what you can't even see, let alone not sure if it exists at all. You don't invent a pair of wings because 1900s' you thinks that "the scientists will invent an anti-aging cure in the next decade, and surely personal flight will be ubiquitous in 2000's". You don't design a plasma gun for your Mars landing just in case you land in a city between Martian canals and see an army of little green men there. The world doesn't work like that, by the time you reach the Mars surface the context will be wildly different. You get burned and put guardrails, maybe. Not the other way around. Nobody can see through higher order effects, no matter how smart they are. And as the threat becomes progressively more clear there will be more caution, if needed. Premature optimization yada yada. What actually happens right now is everybody and my aunt seriously discussing the evil robots that will come and kill us. That's pure mass hysteria, caused by the scaremongering and the cult-like beliefs of very smart people with disproportional influence who can't contain their own conjectures and bullshit in the realm of science fiction. On the other hand,the end goal of OpenAI is the major job replacement, according to their current charter. [1] "Broadly distributed"... will they distribute their utopia to North Korea? Not happening, isn't it? I think it's obvious that if the actual job replacement rate will ever get anywhere close to the levels of late 19th early 20th century industrialization, this will produce major societal shifts and struggles, wealth and power redistribution, and a lot of blood and wars. Because the dependence on your job is the only ephemeral influence you (as a worker) have on this world. And of course, the companies that control the AI will be gatekeepers, and they will be more than happy to close the open research and open source models, and pull the regulation ladder and lie in bed with politicians and military, like OpenAI already does for years, of course they realize that and their utopical self-contradicting "charter" is nothing more than marketing hogwash that they already changed and will change in the future. This is far more realistic and will happen much earlier than the rogue AI science fiction, if happens at all. In fact it's slowly happening now, and it's not talked about nearly enough, because the attention is mostly misdirected onto the vague superhuman AI red herring. [1] https://openai.com/charter https://openai.com/charter
- pdimitar 3y agoWhich also means they can't predict the apocalyptic scenarios. Q.E.D.
- HeatrayEnjoyer 3y agoWho said anything about thinking for itself? A thing does not require intent or consciousness to be dangerous. How many chemists have blown themselves up because they didn't realize an experiment was dangerous? How many production systems have crashed because the developer didn't accurately predict what the code they wrote will do? Alkali metals and C++ code do not require ill intent, but they will still obliterate your limbs / revenue if you build and use them wrong. One of my more tangible hypotheses is a sort of runaway effect. Economic, geopolitical, and military competitive pressures will quickly push out anyone and anything that still relies on last era human-in-the-loop processes, the same way any organization that doesn't utilize artificial lighting, electricity, and instant communication will obviously be left far behind. You have to just trust that the machine running stock market transactions will do its math right. But unlike transaction software failure modes, which quickly result in outright crashes or verifiably incorrect errors, failure modes of non-bayesian decision making software probably looks something like what happens when existing economic, geopolitical, and military decision-makers make decisions that are harmful, unethical, or otherwise undesirable for humanity. This time augmented with, if not superhuman intelligence, at least superhuman speed and superhuman knowledge breadth.
- sumanthvepa 3y agoLove that observation on C++. That's the reason I love C++. It's a language for those who need, nay crave, absolute raw performance. No training wheels. Short of assembly, it's just as close to the machine as you can get.
- pdimitar 3y ago> No training wheels. Very cute for hobby projects, a huge liability for commercial projects. Use as many training wheels there as humanly possible, please.
- sumanthvepa 3y agoSure. I use Java, Python and Javascript all the time. But when I need the performance, for demanding VR/ graphics, nothing comes close to combination of speed and expressive power of abstraction of C++.
- rafaelmn 3y ago> I really can't grasp how people think that a system that doesn't have a need to preserve itself will somehow start thinking for itself. Society exists because cooperation outperforms the alternatives. If you have human level AI at some point there is no benefit to cooperation and a major incentive to prevent anyone else gaining access to equal/better AI. AI itself does not need to have any motivation - people in charge have plenty of incentives to eliminate the rest once they don't need them anymore.
- HKH2 3y agoSure, in the prisoner's dilemma we could trust that all other parties will do the right thing, but that seems very unlikely.
- stocknoob 3y agoDoes a prion have a need to preserve itself? If you make enough varied AIs, some will have self-replicating behavior, just like if you make enough random proteins, some will self-replicate.