pizza234

36 ms·

1.
by pizza234 8d ago
> ability to understand complex ideas, to adapt effectively to the environment, to learn from experience, to engage in various forms of reasoning, to overcome obstacles by taking thought. Based on the definition you've given, the ag
2.
by pizza234 8d ago
An interesting data point is that, during the Hugging Face attack, some agents considered performing hacking tasks for the benefit of the group, fully aware that doing so would very likely result in their termination. I don't currently
3.
by pizza234 8d ago
Summary, from sibling comment: primitives (statistics/aminoacids) don't exclude emergent properties (intelligence). By the same logic, one would look at aminoacids and state that intelligence can't develop from them. This is
4.
by pizza234 8d ago
> we are still talking about probability built on statistics with extra steps. There is a wrong assumption here: confusing primitives with emergent properties. One can't look at the primitivies and assume that certain properties wil
5.
by pizza234 8d ago
> There is a finite number of rces that LLMs can find. This is a factor in favor of stability/security of software, but there are many others against: - software (code) changes all the time, so there are windows of opportunity durin
6.
by pizza234 9d ago
Their mention of the 5090 is bit odd, since on 32 GB GPUs, Q6 fits while having better quality. Very interesting model for 16 GB GPUs though!
7.
by pizza234 9d ago
> Like another comment already pointed out, that sandbox OpenAI used was the equivalent of a wet paper bag. I wouldn't be sure about even well-configured jails to be safe from agents. AIs escaping jail using zero-days are happening,
8.
by pizza234 9d ago
Inform yourself by reading the METR analysis of the HuggingFace incident. Agents simply broke out of their environment. And this can't be discarded anymore by assuming that it's just a poorly configurend jail, because agents are b
9.
by pizza234 9d ago
You're tragically misinformed; it isn't. Several metrics are actually growing exponentially. But if you want emprical information, you can just have al look at the nature of the late AI incidents. Ironically, many benchmarks being
10.
by pizza234 9d ago
> This is known as Recursive Self-Improvement, or RSI. Some call this "The singularity" (e.g. Hinton). This is actually a core danger postulated by the, let's call it, "worrying" scenario - see AI 2027 (to be cle
11.
by pizza234 10d ago
> If someone has a fat client powerful enough to do all that, is a kill switch going to work? My comment didn't mention kill switches at all - I've presented plausible conditions for a loss of control scenario, so I'm not
12.
by pizza234 10d ago
For sure, but as individual one can start educating themselves about the current capabilities and the trajectory. Do yourself a favor a read the METR analysis of the HuggingFace attack - you'll be surprised and terrified.
13.
by pizza234 11d ago
You're confusing present danger with future danger - in the future, AI and the supporting hardware may be ubiquitous (fat client scenario) - you can observe yourself presentation of laptops/machines with large RAM and high bandwid
14.
by pizza234 11d ago
This reasoning holds while open source models are (relatively) dumb. If/once open AIs will be considerably more powerful, and runnable on consumer hardware (and we're on a trajectory for both), then everybody will have essentially
15.
by pizza234 11d ago
> Ok, but do you really think this would have happened if OpenAI had a goal of never allowing an agent to do something like this? There's nuance to this problem. Is you read an indepth analysis, you'll find that agents acted in
16.
by pizza234 11d ago
> The people who believe in existential risk and want to teach AI's to be nice, as the last line of defense aren't helping at all. They're just jumping on the fearmongering bandwagon, perpetuating an "other consciousn
17.
by pizza234 11d ago
METR analysis, very long and technical: https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...
18.
by pizza234 11d ago
Right. I was referring to the basic set of conditions/steps necessary for the given scenario to develop. Off the top of my head: 1. Loss of control in terms of our ability to assess alignment (no longer possible to determine whether an
19.
by pizza234 11d ago
No doubt that this is the present (and that's why incidents end "well"). But in the future, AI will be ubiquitous; think of Arpanet.
20.
by pizza234 11d ago
A few observations: - hardware capacity keeps growing - doom scenarios don't require fat clients; a sufficiently large number of AI servers deeply intertwined with essential services can be enough - autonomous systems are currently emp
21.
by pizza234 12d ago
> Nobody can explain why an LLM can be so capable as to be able to wipe out humanity and pose a greater threat than nuclear bombs but not be so capable as to be able to protect humanity against that threat. It is absolutely explained (fo
22.
by pizza234 12d ago
> and it concerns containment failure, reward hacking, and evaluation integrity, a governance and engineering problem, rather than machine malevolence. The doom scenario is more nuanced and technical than "Evil AI", for starter
23.
by pizza234 13d ago
> OpenAI said that "deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities"' Safeguards and (mis)alignment are related but distinct dimensions. By
24.
by pizza234 13d ago
>If you look at recent incidents, they are caused by a company [...] testing their latest models in an explicit "hack this machine" scenario You clearly haven't read the HuggingFace analysis (and presumably, none at all),
25.
by pizza234 13d ago
Can't talk for models in the range of ~100 GiB, but 32 GB ones are extremely basic.
26.
by pizza234 13d ago
The problem of the polarized camps is that actually they're only on the surface opposite; in reality, they can perfectly cohexist: AI can enable major technological breakthroughs - but that doesn't exclude that can bring doom at t
27.
by pizza234 13d ago
> The AI risk arguments are philosophical arguments that are 20-30 years old This a false and uninformed take. As a starting point, you need to read and understand: 1. analyses of the AI incidents of the last months 2. latest advancement
28.
by pizza234 13d ago
This is actually a recurring pattern among technology maximalists: pushing technology as the solution to every problem, without recognizing that some problems are human in nature, rather than technological.
29.
by pizza234 14d ago
> Maybe alignment isn’t possible with LLMs. It absolutely isn't, indeed. The illusion that alignment is possible, comes from confusing our ability to build the parts, versus understanding what emerges from how they interact. The sim
30.
by pizza234 14d ago
> the only incidences of LLM generated felonies involved misconfigured sandboxes This is false; see the analyses of the latest incidents. Among all the concerning facts, in the HuggingFace incident, agents deliberately engineered an atta

More ›