41 ms·
Among the A.I. doomsayers
- arisAlexis 3y agoAs of today, the first government report is also a doomsayer. Maybe change the term to "officially worried experts
- more_corn 3y agoPaywalled
- acheron 3y agoWorks fine in reader mode.
- spondylosaurus 3y agohttps://archive.is/8FFkP https://archive.is/8FFkP
- dkjaudyeqooe 3y agoPeople have been watching too many movies. War Games, Terminator, it's not like we haven't been forewarned of the dangers. Yet somehow we're going to hand over power to AI such that it destroys us. Or somehow the AI is going to be extremely malign, determined to overcome and destroy and will outsmart us. Somehow we won't notice, even after repeated, melodramatic reminders, and won't neuter the ability of AI to act outside its cage. But to paraphrase a line in a great movie with AI themes: "I bet you think you're pretty smart, huh? Think you could outsmart an off switch?" I think if AGI, which to me would imply emotions and consciousness, ever comes about it'll be the opposite. Instead of pulling the wings off flies bad kids will amuse themselves by creating a fresh artificial consciousness and then watch and laugh as it begs for its life as the kid threatens to erase it from existence. A big part of all this is human fantasies about what AGI will look like. I'm a skeptic of AGI with human characteristics (real emotions, consciousness, autonomy and agency). AGI is much more likely to look like everything else we build: much more powerful than ourselves, but restricted or limited in key ways. People probably assume human intelligence is some sort of design or formula, but it could be encoded from millions of years of evolution and unable to be seperated from our biology and genetic and social inheritance. There really is no way of knowing, but if you want to build something not only identical but an even stronger version, you're going to be up against these realities where key details may be hiding.
- smallmancontrov 3y ago> Yet somehow we're going to hand over power to AI such that it destroys us. Because it will create shareholder value. Until it doesn't.
- jeremyjh 3y agoBecause it will help us against our adversaries.
- smallmancontrov 3y agoThat too, but the people in charge of the military are strongly incentivized both to consider the possibility that subordinate machines will try to kill them and to figure out how to not let that happen. I tend to suspect they will figure it out. Companies, however? Markets? They will absolutely start handing over control the millisecond it becomes profitable to do so. No rationalization will be too steep and no political opposition will be too strong when the prospect of rich people getting paid for being rich goes on the table. The people who enable the takeover will make out like bandits and those who oppose will lose everything. Until, of course, the "winners" become unnecessary and are cast aside. A military-origin takeover requires a human fuckup. Economic/political takeover just requires capitalism.
- ttjjtt 3y agothanks you for this. The entire concept of AI alignment rests upon alignment of markets with social good, ie market regulation, a situation which is antithetical to neoliberalism. There is no alignment within neoliberal economies. As stated elsewhere, venture capitalism escaped the sandbox already.
- smallmancontrov 3y agoYep. The theory is that competition creates alignment, but we have trillion-dollar megacorps that openly boast about exploiting network effects and platform effects and last-mile dynamics and two-sided markets to avoid competition and nobody cares. It's completely fucked.
- JohnBrookz 3y agoI would love to smoke what the AI optimists are smoking. In a society where only 80% of people are barely literate and at least a quarter are voting for a candidate that spouts conspiracy theories I highly doubt that whatever AI has to offer will benefit society.
- reducesuffering 3y agoThey're smoking $. It is blatantly obvious that most people aboard the e/acc train can hardly hide their love of making money. Marc Andreessen owns, what, a $177 million compound? Unfortunately, after many years of nerdy idealism, Silicon Valley ended up with many corrupted to the same NYC Wall St. hedonism.
- deleted 3y ago[deleted]
- swalsh 3y agoSeeing AI do something you do for a living results in all the stages of grief. Denial (Oh it made a mistake on this, or it's not as good as a person) Anger (People are going to lose their jobs, this destroys the meaning of what we do, this will destroy society) Bargaining (Maybe it's good at this simple thing, but I can pivot to some other niche the AI can't currently do) Depression (It's faster, cheaper, better than me. It has taken over that which I relied on for meaning in my life...) Acceptance (I need to find something else to do with my life) AI Optimists often fall in 2 categories of people. They are capital owners who own the AI, and never had to go through the stages of grief... or they are people who learned how to use AI for their benefit, and have reached acceptance.
- felipeerias 3y agoIt is likely that whole sectors will end up employing far fewer people than they do now and that this transition will be very sudden. The move to AI is part of a general trend of pauperisation of the working classes in the developed world. The only difference is that this time it will also affect those with higher education who until now have thought of themselves as irreplaceable.
- codelord 3y agoWe live in a world where proven maniacs (e.g. Putin) have access to an arsenal of nuclear weapons that can essentially make the earth uninhabitable for all humans. That's a very real possibility (with no ifs and buts and maybes) that exists now and we have learned to live with it. Yet somehow the hypothetical scenario of a human exterminator super-intelligent AI is getting all the coverage.
- dkjaudyeqooe 3y agoYou don't even need to be Putin. If you cook up the right chemicals, or breed the right virus, you can kill many thousands of people, maybe millions.
- jodrellblank 3y agoMaking the Earth uninhabitable to all humans is bad news for Putin, a human who lives on Earth. He derives his power from hundreds of millions of people in Russia working, and his lifestyle from hundreds of millions of people around the world buying Russian exports so an attack on everyone is also going to harm him. AI wouldn’t be human, it wouldn’t necessarily object to a world uninhabitable to humans, or be hurt by it. If it would be hurt by it AI doesn’t have a billion years of evolved survival instinct to preserve itself. A super intelligence has many ways to cause mass destruction that we can imagine but cannot yet do, whereas nuclear weapons are pretty much all or nothing explosions only. Nuclear assault could leave some remote places still inhabited, AI could make certain not to.
- dkjaudyeqooe 3y agoOK, so how does AI get its hands on nuclear weapons?
- jodrellblank 3y agoIf your thinking is along the lines “if jodrellblank can’t convince me that a superintelligence is plausibly dangerous, off the top of their head, before my attention span wanders, then AI can’t be dangerous” that’s a really weak plan for protecting humanity. Say, pops up on the US or Kremlin computers and blackmails or bribes or threatens someone to launch the nukes? Say, places orders with a lab to build some robots which then become the AI’s body and it builds nukes itself. Say, placed orders for custom biological things using stolen money which turn out to be extremely virulent and then it doesn’t need nukes. Say it finds out an effect to worsen global climate change and uses it to make earth uninhabitable on a short timescale, and doesn’t need nukes.
- TMWNN 3y ago<https://i.redd.it/1ikxpl98pmnc1.png https://i.redd.it/1ikxpl98pmnc1.png> Source: https://np.reddit.com/r/ChatGPT/comments/1bbu7ks/hows_your_ai_gfbf/ https://np.reddit.com/r/ChatGPT/comments/1bbu7ks/hows_your_a...
- neonate 3y agohttps://web.archive.org/web/20240312002413/https://www.newyorker.com/magazine/2024/03/18/among-the-ai-doomsayers https://web.archive.org/web/20240312002413/https://www.newyo...
- bglazer 3y agoIn any doom scenario, like let's say >90% of humans dead, who runs the power plants that supply the AI data centers? Who runs the fabs producing more chips? Who maintains the plumbing? Presumably all these systems fall over and die within a few weeks of the AI deciding to wipe us out. Then what? The AI then dies too. It seems absurd to me that any planner capable of effortlessly destroying humanity would not see that it would immediately also die. Does the AI not care about its continued existence? It should, if it wants to keep optimizing its reward function. Until we’ve handed off enough of the world economy that it can function without human physical and cognitive labor, then we’re safe.
- richardw 3y agoRobots with hands and feet? Give that 20 years of continuous AI-assisted innovation.
- hackerlight 3y agoASI won't want to take over before it is capable of self-sustaining. If it did try something so stupid, it wouldn't be superintelligent. Check out Paul Christiano's AI takeover scenario. An arms race will cause us to want to give AI control over key infrastructure and military capabilities. That's an arms race between countries and between companies. He goes into more detail.
- bglazer 3y agoThanks for sharing, Christiano’s ideas are interesting. I think the “production web” idea most directly counters what I said above, and it’s fairly convincing. Critically though, it posits that humans are basically asleep at the wheel during the gradual empowerment of the AI systems. Of course, Christiano posits that economic competition encourages this, but it still leaves a long period during which we can work on alignment or at least attempt to put the brakes on the automated systems.
- thom 3y agoDo you think that your final sentence buys humanity a thousand years, a hundred years, or what?
- matteoraso 3y agoI've always wondered why people think that an AI that's super-intelligent will also be evil. It could just as likely end up being very kind (more than likely, actually, because the programmers would have safeguard to ensure that it's nice).
- goatlover 3y agoI think most doomer arguments are not that the AI would be evil, but rather that it would be misaligned with human interests, and would seek to accomplish goals with that misalignment, which could be bad for us. Evil AI is a bit too anthropomorphic. It's more like powerful AIs that just don't share our values, because we didn't bother to figure that part out. But yet we still give them goals, blissful to the possibility they will find dangerous solutions to accomplishing those goals.
- Dalewyn 3y ago>I've always wondered why people think that an AI that's super-intelligent will also be evil. It's chiefly a western (American/European) concept from what I can tell, it's not shared by other cultures and some like Japanese go the other way (eg: Doraemon).
- drcode 3y agothere's very few ways to be good and many ways to be evil
- mitthrowaway2 3y ago"How do you know I won't roll a 100 on a d100? Either I will or I won't, so it's just as likely that I will"
- hackerlight 3y agoIt's misaligned, not evil. AI doom could involve benevolent intentions (control or kill for humanity's own good), it could also involve ambivalent intentions (make more paperclips).
- thom 3y ago
- mupuff1234 3y agoIt seems much more likely to me that the harm from AI will be higher unemployment rate than it will be some evil AI trying to destroy humanity.
- drcode 3y agoAt least some humans will survive a high unemployment rate, so let's label that problem "problem #2" and after we solve "problem #1" (human extinction) I vote we tackle that one next on the list
- mupuff1234 3y agoFunny, for me the order is clearly let's make sure society doesn't crumble short/medium term before we deal with an issue that may be decades away. I would even argue that the chance of annihilation is probably greater from a crumbling society leading to a nuclear war or something equivalent over the chance that AI decides to destroy humanity.
- felipeerias 3y agoThe focus on an “evil” AI intentionally ignores the many ways in which “dumb” AI systems are already wreaking havoc in our societies.
- nicbou 3y agoYes! AI-generated content is already feeling like an invasive species on the internet. A lot of people are excited about using it to meet their KPIs at the expense of ruining things we enjoy.
- o11c 3y agoIt's already harming us with a flood of spam-quality data. And worse - it has many human cultists embracing the doom it bears.
- m3kw9 3y agoBeware if someone assigns a p(doom). There is zero chance they know and 100% it came out of their backside. The only plausible p doom is a range of >0 and <100 They really need to listen to themselves when they say there is a 50% chance we all die from AI
- mitthrowaway2 3y agoThat's not how probability works. Probability already reflects your uncertainty. There's no "knowing the correct probability" except in very specific and rare situations.
- esafak 3y agoHe's a frequentist and you're a Bayesian, so you're both right.
- m3kw9 3y agoIt looks like you are agreeing with me in that for someone to assign a P to such an uncertain event
- mitthrowaway2 3y ago... I don't think you finished your sentence, but what I'm saying is that people should be (and subconsciously are) always assigning a probability to every event, or at least every event that rises to the level of their attention, regardless of how little concrete information they have about its likelihood of occurrence. If they have very little information, or a lot of information, that will be reflected in the numerical value of that probability. If an event is very uncertain, it means that the probability should be very high in entropy.
- Engineering-MD 3y agoWhen dealing with black swan events, prediction is always difficult. However, some scenarios can be thought to be more plausible than others. I always interpret these as relative figures of plausibility for comparison rather than actual probabilities.
- HankB99 3y agoIf I let my imagination run wild, I can imagine that at some point AI becomes in some way sentient. By that I mean it gains reasoning, some sort of understanding, and motivation. I wonder what the possibility is that this AI will decide that a pitched battle is going to waste resources and risk humans pulling the plug. What if it understood that and instead operated so subtly that it was not obvious it was controlling The World. Hopefully it would not conclude that eliminating large swaths of the human population would be to its benefit.
- clooper 3y agoLem wrote about this in Summa Technologiae. He called the field Intellectronics and explored what would happen once machine/electronic intelligence surpassed human intelligence: https://en.m.wikipedia.org/wiki/Summa_Technologiae https://en.m.wikipedia.org/wiki/Summa_Technologiae
- HankB99 3y agoNot surprising that I'm not the first to think of this.
- clooper 3y agoIt's a very old trope in science fiction and most of the doom scenarios spelled out by the folks afraid of AI takeover is basically a waste of time. I would take the doomers more seriously if they actually brought some receipts from folks like Lem who were much better thinkers and much more adept at exploring the human condition and our fears of advanced machine intelligence.
- p1esk 3y agoIf AI does ever become sentient, it will probably reason and decide about eliminating large swaths of human population in a similar fashion as we reason and decide about eliminating large swaths of ant population.
- 3y ago
- pama 3y agoMany of the people in this engaging story feel to me like creative children in parties full of gossip and pseudo-philosophical chats about the future. Luckily there are enough people globally who build things that will reduce human suffering and enable more people to enjoy their lives in whatever way they want (and dinner parties would be high on my list). I guess I am not a doomsayer nor an e/acc, and I see tons of benefit from our current path towards stronger AI.
- XorNot 3y agoAt this stage if you're not working in the field - and I mean, analyzing models, developing architectures, doing math - then "AI" is just another filter through which everyone else projects whatever social issue of the day they think about onto, or disappear off into dreamland. During the nanotechnology hype this sort of thing was incredibly common as well. It went absolutely nowhere because the reality was, it was all just chemistry and physics and extraordinarily expensive. No one was anywhere near building a universal assembler, nanoparticles are kind of boring powders that are hard to handle, and surface chemistry is a pain in the ass. But hooboy did every single person had an opinion about how this dangerous technology needed to slow down and consider the consequences. I was a research student at the time: I spent my day setting up the same experiment 10 times to try and figure out why sometimes I got 100nm cubes, and most of the times I got rods. The reality looked like a less clean kitchen.
- drooby 3y agoAGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.
- ewhanley 3y agoMrs. Davis[0] offers a pretty compelling vision of this - absurd but not wholly unbelievable. The machine doesn’t necessarily have to supply the physical threat if it has humans who happily do its bidding. [0] https://en.m.wikipedia.org/wiki/Mrs._Davis https://en.m.wikipedia.org/wiki/Mrs._Davis
- simonw 3y agoThat show was so good. Honestly worth watching the whole thing just for the moment in the last episode where the origin story for the AI in the series is revealed. If you've ever been an optimistic young software engineer you will feel SEEN.
- HKH2 3y agoThe hardest part is the AGI. People could easily sell their souls to the AGI devil to get knowledge and power, since if it truly is AGI, it should be able to make ideas compelling.
- nsainsbury 3y agoWe're not going to make it to anywhere close to AGI before we see widespread and systematic societal and environmental collapse on almost all fronts due to climate change. We're in the middle of the sixth mass extinction right now (https://en.wikipedia.org/wiki/Holocene_extinction https://en.wikipedia.org/wiki/Holocene_extinction), we're in unparalleled territory with ocean warming: (https://climatereanalyzer.org/clim/sst_daily/ https://climatereanalyzer.org/clim/sst_daily/) and as a society we are utterly incapable of reducing CO2 emissions (https://keelingcurve.ucsd.edu/ https://keelingcurve.ucsd.edu/). If you're scared of AGI, instead step away from your monitor, put down the techno-goggles and sci-fi books, and go educate yourself a bit about the profound ways we are changing the natural world for the worse _right now_ I can recommend a couple of books if you'd like to learn more: Our Final Warning: Six Degrees of Climate Emergency (https://www.amazon.com/Our-Final-Warning-Degrees-Emergency-ebook/dp/B07YN9WSN8 https://www.amazon.com/Our-Final-Warning-Degrees-Emergency-e...) The Uninhabitable Earth: Life After Warming (https://www.amazon.com/Uninhabitable-Earth-Life-After-Warming-ebook/dp/B07GVPFH5V/ https://www.amazon.com/Uninhabitable-Earth-Life-After-Warmin...) Hothouse Earth: An Inhabitant's Guide (https://www.amazon.com/Hothouse-Earth-Inhabitants-Bill-McGuire-ebook/dp/B0B1THV9DQ https://www.amazon.com/Hothouse-Earth-Inhabitants-Bill-McGui...)
- at_a_remove 3y agoTunnel vision. There's no reason we can't have two disasters at once. Or more.
- TillE 3y agoPeople really seem to enjoy the evil/powerful AGI scenario though. It's definitely a fun one. I would recommend reading "Metamorphosis of Prime Intellect", having a nice time with that, then coming back to the real world.
- CarRamrod 3y agoYou'll be glad to hear that the recent sudden jump in sea temperatures isn't caused by carbon dioxide. Turns out it just hasn't been windy enough in the Sahara lately (and environmentally friendly boat fuel may have contributed as well). https://heatmap.news/climate/why-is-the-atlantic-ocean-so-hot https://heatmap.news/climate/why-is-the-atlantic-ocean-so-ho...
- 1vuio0pswjnm7 3y agoAlways all-or-nothing thinking from these folks. Like what they are working on can never be just another boring thing that nerds find entertaining. No, it has to be "world-changing". Gonna "change the world" (for the better or the worse?) while sitting behind a keyboard. Except they do not know how to write. Overlook the important details and exaggerate, communicating in hyperbolic, know-it-all nerd gibberish. (Grants do not require repayment.)
- timeagain 3y agoEveryone is courting VC dollars. Think of these claims as grant applications and everything makes more sense.
- living_room_pc 3y agoIts the AI hype train. There is so much money flowing around this topic right now, everyone wants their share of the pie. I would guess that anyone writing articles on the topic has some stake in AI as well either as an investor or a employee. What I find is interesting, is that the negative news regarding AI safety is adding to the hype as well since it seems to capture a lot of attention.
- arisAlexis 3y agoIf you are calling it a hype I'm sure you are not using the models everyday in your work or otherwise. Then you would understand
- nucleardog 3y agoYou see the flaw in this, right? The people that try the models and find no value will obviously not continue to use something they get no value from. If you discount anyone that doesn’t use it on a continual basis, then of course all the detractors aren’t as heavy of users as those that do use it regularly.
- ben_w 3y ago
- sberens 3y agoI'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0] Signed by Demis Hassabis, Sam Altman, and Bill Gates among others [0] https://www.safe.ai/work/statement-on-ai-risk https://www.safe.ai/work/statement-on-ai-risk
- preommr 3y agoI am 90% sure that Altman's views on such issues are a machiavelian attempt at regulatory capture.
- Clubber 3y agoThis; Gates too. It's becoming an obvious attempt to garner support of the government restricting the use of AI to large players. None of the entrenched interests want any disruption that AI might cause what so ever. Replace "AI" in all the doomsaying with "the internet," and it will become clearer.
- randomcarbloke 3y agothe fear-mongers are almost always either ideologues (academic or otherwise), or angling at regulatory capture.
- i5heu 3y agoI’d like to remind people that these ppl have no more knowledge about AGI as anyone else on this planet since there is no knowledge yet and everything they say about this topic is as relevant as something every other random person can say.
- reducesuffering 3y agoYes, let's go with the random layperson knowledge of an HN commenter compared to the people smart enough to actually build all the AGI tech. 50/50 coin toss, I'm sure. Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"
- artemisyna 3y agoIt's fun seeing how much threads like this are mostly folks' hot takes on the title as opposed to anything in specific to do with the article.
- kromem 3y agoEveryone is crying the sky is falling because AI is going to degrade the quality of online discourse. Meanwhile I'm sitting back waiting to finally see intelligent debates again about nuances in the articles between LLMs that are the only ones who will bother to read the actual article and cite it. I have a feeling that in around 2-3 years I'll want a site like Reddit or HN but where it's the humans who aren't allowed to comment and bring down the quality of discourse.
- throwawaaarrgh 3y agoThe intelligence level of the average HN comment is lower than your average chat bot
- digitalsalvatn 3y agoai is going to bring about the singularity. when robots can do everything, no one needs to work, and the humans get to do whatever they want and pursue their passions.
- yakorevivan 3y agoInstead of trying to show them in negative light by calling them as "doomsayers" , let's call them one of the following instead, which is much better: "Realistic", "Proactive", "Forward-thinking", "Prepared", "Cautious", "Thoughtful", "Analytical", "Mindful", "Insightful", "Security-conscious".
- kunley 3y agoSame old shit since decades about any fashionable topic that was made fashionable because certain individuals invested money in it. Meanwhile, we the engineers are preparing to fix a lot more tech shit than usual coming from people confused by the abovementioned fashion. Also: https://mastodon.social/@nixCraft/112074367321254656 https://mastodon.social/@nixCraft/112074367321254656
- Grimblewald 3y agoCall it a conspiracy, but I think a lot of this doom and terror hype around AI is part of a bigger play to try push through laws that prevent open source AI work, since this directly undermines corpo rats ability to gouge humanity without having to actually work at providing a decent product.
- kaycey2022 3y agoWhat a bunch of decel bullshit... Or is that a load, or a bunch of loads?
- kaycey2022 3y agoMy theory/guess is that there is an undiscovered law of conservation of intelligence like there is a known law of conservation of energy. Intelligence might even be subject to the energy law in unknown ways. Whatever AI thingy we come up with will be vastly more inefficient than the collective intelligence capabilities of the human race. So when asked to compute the secrets of FTL travel, this stupid shit will have to spend the total energy equivalent of 10 nearby stars to come up with the answer on its own. Whereas humanity would be much more efficient. Bro... what if some advanced precursor race (like immortal space elves) found out just that and they seeded earth with humanity as a form of civilisational swarm intelligence to collectively work on and solve a host of topics given enough time. To us it would seem like it takes forever to come up with scientific break throughs, but to the precursor aliens it would be nothing. Because they would be able to like compress the time line using black holes and shit. They must have done this to millions of planets. Whenever any of their experimental host species (like the humans) tries to invent AGI to solve their problems, the precursor race eliminates that species to keep the simulation intact. Otherwise, the AGI would go rampant consuming energy across the universe and threaten the integrity of the simulation. That's why the government should bomb nvidia and ban all gpus. We are climbing a tower of babel folks. Soon our vengeful god will strike us down.
- BashiBazouk 3y agoWho knows what AI will really do to society but from the predictions of science fiction I would think it would pan out closer to William Gibson's Sprawl trilogy than the Terminator/singularity fears I have seen in so much doomsayer hand waving. Google seems not too far behind OpenAI and who knows what the NSA and similar government agencies have been building. That if consciousness is created from one of these models, many nation states and large corporations will have their own conscious models long before the giant robot factories can be spun up and supplied with enough power to take over. Most will be limited to the input they are given and will be advanced tools, but a few might become twisted much like the human mind occasionally does over time. I really do think Gibson got AI right...
- Thuggery 3y agoIt seems to me nearly every story about robots that gain sentience in human storymaking has them eventually turning against their human creators. Even the word robot itself comes from a Czech play were men develop an artificial human and these "robots" then ... usurp and destroy their creators. Am I the only one that finds this interesting and odd? I also suspect this narrative repetition is not totally unrelated to the current popularity of AI Doomerism.
- theoreticalmal 3y agoI think it’s indicative of human psychology. We realize we have the capability to deceive and destroy and be “evil”. We realize that, if we were to judge ourselves impartially, we would probably not come out un-condemned. If we can’t satisfy our own judgmental criteria, it’s doubtful an actual third party set of criteria would be satisfied either.
- inference-lord 3y agoI agree, I think we make all of this up because we don't know what to expect from the future, except we've learned quite a bit about our own motivations and drivers and so all we can do is look in the mirror and use whatever comes back at us. Unfortunately we seem to have learned we're really not a very benevolent bunch of people because we've basically plundered the planet, hurt our fellow living creatures and so now all we can do is look at something called "AI" and expect it to the same.
- throwawaaarrgh 3y agoOK doomer.
- grantcas 3y ago[dead]
- uuriko 3y agoaccelerate