4 ms·
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and sen
by stratos123 1mo ago
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger.
A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk.
Watch people read this, ignore it completely, and continue commenting about marketing stunts on every piece of news about an LLM-done advance or felony.
- acivitillo 1mo agoHe is resigning from a job, what else should we think? If something really dangerous was happening he would be doing a whistleblower or at minimum talk to a lawyer. The thing is, the complete lack of transparency makes it hard to assess OpenAI and Anthropic. If they were quoted on the stock market, we could at least rely on some basic audits and reporting requirements.
- hegelstoleit 1mo agoThere's no whistleblower program for this, they're not breaking any laws. What are you proposing he should do if not this?
- akman 1mo agoAt least from an interview with Amanda, key philosopher at Anthropic: https://www.youtube.com/watch?v=I9aGC6Ui3eE&t=1912s https://www.youtube.com/watch?v=I9aGC6Ui3eE&t=1912s
- zdragnar 1mo agoHaving witnessed so many people treat LLMs as a something divine, I can only assume the reasonable people at openai and anthropic were all pushed out long ago, and the majority that remain believe the crazy hype despite Tesla-self-driving-level predictions from these companies that don't come true. I'm not worried about what they think. I'm worried that too much infrastructure- water, power, defense systems, etc- remain running on tech from an outdated era of understanding security. > they believe no one else will act responsibly, so they must do it themselves, despite the risk. This genuinely makes no sense. Them getting there first in no way precludes bad actors from also getting there. It might as well be another marketing stunt.
- deleted 1mo ago[deleted]
- reducesuffering 1mo ago> I can only assume the reasonable people at openai and anthropic were all pushed out long ago Typical uninformed take on the side of "doomers are crazy". Both CEO's of OpenAI, Sam Altman and Dario Amodei, and many in their leadership, believe AGI has a very real probability of causing humanity's extinction. Both companies were founded upon this belief, it is at the core of the company. Only later were mercenaries hired chasing $1m compensation packages.
- moogly 1mo agoIf they truly, truly believed that, would they be speeding towards building it? If yes, that would make them truly insane, right? Not as in a quaint "off their rocker" but more "non compos mentis".
- reducesuffering 1mo agoBoth companies have deluded themselves into thinking the arms race is going to happen anyway and they need to rush to it first, as if somehow that helps. They have publicly stated as such repeatedly. They think the ~10-50% chance of extinction sucks, but that it's going to happen anyway and they believe they can steer it towards something good the best and unlock all the potential positives like infinite life.
- nozzlegear 1mo agoWhy is the take uninformed? You didn't address anything about the part you quoted, wherein reasonable people were allegedly pushed out long ago.
- reducesuffering 1mo agoHow is it not addressed? The company never contained only "reasonable people" that believe AGI is not an existential risk to humanity. At both inceptions were people who believed in AGI x-risk, even the founders. Only after time, did there become more "reasonable people" who were mercenaries and only believed it to be a typical tech job. Today, there are more "reasonable people" than ever there. They haven't been pushed out. We're just witnessing some prescient mercenaries smart enough to Eureka the grave implications of what is actually happening.
- cyanydeez 1mo agoCults are like this. Are you saying you believethem ?
- impulser_ 1mo agoSo you think in 3 years AI is going to kill 8.5 billion people because they were used to hack into HuggingFace?
- jeremyjh 1mo agoWho used them?
- teeray 1mo agoJust wait until it gets its hands on a shady biolab just outside of oversight. “Claude, make me Captain Tripps”
- sakesun 1mo agoExactly. These people really need to get a life.
- underyx 1mo ago"So you think in 3 years AI is going to solve longstanding math problems because it was used to write some coherent sentences?" — people with the same amount of foresight in 2023
- brookst 1mo agoAre you saying at anything that can solve longstanding math problems necessarily has the means, motive, and capability to kill 8 billion people in just 3 years?
- famouswaffles 1mo agoHumans weren't built to handle long term risks. We just weren't. For basically all of our evolutionary history, we were almost overwhelmingly concerned with the short term. What will you eat today, How will you sleep tonight. Problems on the order of days or weeks. At best, the next season. Our intelligence evolved to disregard super long term risks because it simply didn't matter (what use is worrying about 5 years from now if you're starving and a tiger is stalking you?). So when long term risks manifest in our modern world, our brains get scrambled - Climate Change, Fertility Rates etc. "Safety regulations are written in blood" isn't a saying for nothing. Humans have a strong tendency to let long term risks become imminent risks before doing anything about it, and i don't expect this will be any different.
- brookst 1mo agoReligion does pretty well with the long term risk of hell if you die, the antichrist, etc. a substantial portion of human output has gone into those things over the millennia.
- jrflowers 1mo agoTrying to imagine seeing years of transparently obvious marketing stunts and retconning my own memory because I read a tweet Or seeing a tweet saying that a thing doesn’t count as a publicity stunt if some unknown number of employees mumble about it being spooky behind closed doors and thinking “that makes sense and sounds true”
- platinumrad 1mo ago> At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk. OpenAI are mercenaries, Anthropic is a cult. I know which I prefer.
- duolanda 1mo agosounds just like GDI and Nod
- rahulyc 1mo agoIt was pretty disheartening to hear that only a single scientist quit the Manhattan Project after the Nazi's were defeated. I'm pleasantly surprised that the people working on this seem wiser. He is not the first, and hopefully will not be the last to do this.
- skew-aberration 1mo agoMany also claimed altruistic motivations for continuing their work, sharing technology with the Soviets
- xlbuttplug2 1mo agoWell, are you planning to do something with this information or are you just claiming to be self aware? :)
- EA-3167 1mo agoThis is the "Pilot testimony of UFO sighting" levels of naive. What's more likely? Anthropic is doing some deeply unethical marketing in the lead up to their multi-trillion dollar IPO? Or they're inventing a machine god? There's ample evidence of the former because that's their entire business model, but no evidence whatsoever to support the latter claims. If you want an extreme claim to be taken seriously, provide commensurate evidence.
- x312 1mo agoThe proof is that LLMs could barely solve arithmetic 3 years ago, but now surpass the best human mathematicians, and that this has all occurred from simple principles (RL + compute) that will continue to scale up by factors of millions in the coming years. Also, advocating for slowing LLM progress does not benefit Anthropic or OpenAI.
- platinumrad 1mo agoThey surpass the best human mathematicians in one specific way: they don't get tired or bored.
- xyzsparetimexyz 1mo agoIt won't scale up by factors of millions, that's just obscene hyperbole. Since chatgpt we've probably made things 10x more intelligent on the same hardware. We've also made way more expensive models. Maybe we get a maximum of another 10x efficiency and 5x model size/expense from this point but millions is a joke.
- EA-3167 1mo agoHaven't people learned about the peril of assuming, "Line go up" yet?! TRENDS HAVE FEEDBACK
- skybrian 1mo agoTrends don't go on forever, but the market can stay irrational longer than you can stay solvent. There's no good rule of thumb for this, other than maybe the Lindy effect.
- surgical_fire 1mo agoI read this. I still think it's complete bullshit. The person posting this may very well believe in all this crap, I don't dispute that. People believe in all sorts of shit.
- Mithriil 1mo agoWhat's almost certainly true is the amount of insanity he encountered at Anthropic.
- __d 1mo agoSo, let’s quantify things: what’s the chance they’re right? And what’s the cost of that chance happens?
- par1970 1mo agoThis is the way. And, this is just for us girls, notice that Anthropic just believing that they are making a machine god is sufficient for their public announcements to not jUsT bE mArKeTiNg.
- surgical_fire 1mo agoDo you quantify the chance of any doomsday cult being right too?
- block_dagger 1mo agoI came in expecting the highest voted comment to be that this was some kind of marketing (which I disagree with). I'm glad your comment was what I saw first.
- brookst 1mo agoOr, read it, and remember the openai researcher who deeply, truly believed GPT3 or whatever was sentient. The fact that people working in the space think it’s going to (eradicate poverty / usher in utopia / kill us all) is not a signal that that’s true. Think of it this way: if an exec at Anthropic told you “wow, our stuff is going to lead to universal happiness”, would you believe them? If not, why are you more willing to believe them if they say it will kill us all?
- nvdc 1mo agoi don't think everything that comes out like this is marketing. however, i do think that these companies are largely staffed by "true believers" (anthropic especially) -- people who are so lost in the sauce and embedded in very specific, very peculiar, sf-based rationalist circles where the ai apocalypse is a foregone conclusion. i understand that these models are powerful and pose certain risks. i use them daily for work and the pace of improvement has been pretty remarkable. that said, i don't buy for a second the borderline-religious proclamations coming from some of these researchers, even if i believe that they are making these claims in earnest