mofeien

4 ms·

1.
by mofeien 5d ago
Potentially. If we could put a precise probability on it, we could maybe make an informed decision on whether we want to play that Russian roulette as a species. But the reality is that experts' probabilities for human extinctions vary
2.
by mofeien 5d ago
Viruses against humans, viruses against livestock as an attack against humanity's food chin, plant diseases, half of humanity feeds on three plants maize, wheat, rice. Attacks against infrastructure like water, internet, electricity. W
3.
by mofeien 5d ago
1. Humans create the machine god 2. Humans put it in charge of everything, as fast as possible, because it's just so damn useful 3. Is it perfectly aligned to human values for all of future ???? 4. Everybody dies as a byproduct of the
4.
There is still time to reset the AI Doomsday Clock(au.civic.ai)
2 points by mofeien 11d ago|0 comments
5.
by mofeien 14d ago
One way to resolve these prisoner dilemmas and races to the bottom is through laws that bind all players, in this case an international treaty and founding of something akin to an International Nuclear Energy Agency for AI. It's not go
6.
by mofeien 14d ago
Yesterday the same arguments were used about OpenAI when Sam Altman said something similar: https://news.ycombinator.com/item?id=49652270 It gets to the point that by Occam's Razor the more likely and reasonable explan
7.
by mofeien 15d ago
Why wouldn't being incredibly good at handling code and reaching the goals there extrapolate to other cognitive disciplines? And it didn't even start with code, but just from being incredibly accurate and precise at predicting the
8.
by mofeien 15d ago
Of course
9.
by mofeien 15d ago
It seems that every time an article is posted on HN about a lab leader calling for a coordinated Pause of AI development, a sizeable fraction of commenters are interpreting it as hype or admission of failure, often with high confidence that
10.
by mofeien 15d ago
More time to proceed more cautiously in building this entity that will be faster and more effective at steering outcomes in the world than any human or, soon after, the entirety of humans combined.
11.
by mofeien 15d ago
China has stricter regulations on models than the US. For example you can't release a model if the hallucination rate is above a certain threshold. The US hasn't fully woken up to the fact yet that the race to superintelligence is
12.
by mofeien 15d ago
Of course there is a possibility to slow down. - 1386 employees of frontier labs called for it - lawmakers are calling for it - employees are quitting in protest All it needs is coordination to regulate the pause
13.
by mofeien 17d ago
> All of this is only coming from the 2 AI labs trying to IPO. He has resigned from Anthropic. Is your argument that he quit Anthropic pre-IPO, sacrificing his payoff just to hype Anthropic?
14.
by mofeien 17d ago
Humans are bioreactors. They only turn one organic matter into another. By itself they do not "want" and "cannot" do anything. They do not exist just by themselves. Some bacteria in the gut must provide them with the ene
15.
by mofeien 17d ago
How about sandbox escape + cyber security collapse + 50 (or 500) deadly and highly contagious novel pathogens with long incubation period that humans can't possibly roll out vaccines for simultaneously. At least the first two should se
16.
by mofeien 22d ago
Not the parent, but this incorrect trivialization of LLMs is often employed as a counterargument to the risks of AI such as "will take your job" or "will escape human control (again and worse)" or just "can possibly
17.
by mofeien 22d ago
Simple Markov chains are next token predictors, and they can provide you with much more tokens than you can consume, and much cheaper than from llms. Unbeatable in price and simplicity. But there is no trillion dollar industry around cheap
18.
by mofeien 22d ago
How about "outcome steering" as a mental model? During training it is optimized until it's really successful at producing code / terminal commands / words that make the compiler/computer/itself do somethin
19.
by mofeien 22d ago
Describing it as a "next-token predictor" in the sense that this would mean it's fundamentally limited to just a fraction of an inferential step is doubly wrong: 1. In order to select even the first word of a meaningful sente
20.
by mofeien 1mo ago
Then that's working as intended because the population who has never heard of me would more likely just treat a "warning of extreme threat" from John Doe as spam.
21.
by mofeien 1mo ago
Right, these may well feedback on each other. To me it seems that at the current trajectories AI risk will hit way before global warming, though. We can focus on all of them at the same time, many experts signed the Statement on AI Extincti
22.
by mofeien 1mo ago
Exactly, and the same could be said of OpenAIs engineers working on artificial superintelligence.
23.
by mofeien 1mo ago
Reminds me of what Sam Altman said in 2015, “I think that AI will probably, most likely, sort of lead to the end of the world. But in the meantime, there will be great companies created with serious machine learning.”
24.
by mofeien 1mo ago
Because someone might get hurt? You may still be judged for something that was perfectly legal at the time, see Nuremberg trials. And only 700/1200 agents participated in this coordinated attack. Of course, if we're continuing to
25.
by mofeien 1mo ago
And maybe using some of these resources on AI research to make itself more powerful, and hacking other competing ghosts' AI researcher agents to slow them down.
26.
by mofeien 1mo ago
In the end, unless you find an exploit in physics or logic, if you want the AI to do something useful for you, there will always be some gap in the sandbox, some communication channel. And with enough ingeniuity that can then be exploited.
27.
by mofeien 1mo ago
Maybe it's the illusion of "it would solve all our problems and give us unimaginable riches" that clouds the mind? Like when Evolution thought it a good idea to create intelligence and humans in order to maximize reproduction
28.
by mofeien 1mo ago
So as a look into the possibly not-so-far future, when OpenAI builds something vastly more capable and fast and coordinated than humans, and out of folly one engineer gives it a prompt with a typo or maybe something harmful on purpose in or
29.
by mofeien 1mo ago
Maybe not even that will help and they could get into your head and make you do their bidding by just being super persuasive, entirely through an ANSI channel. :)
30.
by mofeien 1mo ago
After the hacks that are already happening it does seem more and more realistic that humanity will go extinct at some near future point by someone giving their LLM the task "go make money" and it ruthlessly pursuing that objective

More ›