3 ms·
> They were not at all concerned with some kind of hostile / AGI takeover scenario. this doesn't sound belivable, or at least it seems off. competent ai engine
by sometimelurker 4mo ago
> They were not at all concerned with some kind of hostile / AGI takeover scenario.
this doesn't sound belivable, or at least it seems off. competent ai engineers should have good intution about how agents work, and what happens when they don't do what you want them to do: https://www.forbes.com/sites/boazsobrado/2026/03/11/alibabas-ai-agent-mined-crypto-without-permission-now-what/ https://www.forbes.com/sites/boazsobrado/2026/03/11/alibabas...
- deleted 4mo ago[deleted]
- slopinthebag 4mo agoI think a competent engineer understands that they are statistical next-token prediction machines and aligns their expectations around that.
- sometimelurker 4mo agoif you train an agent on long running tasks (like 5 hour autonomous coding tasks) it is practice for the system to learn various behaviors, some of which are dangouus. I link an example of one of these behaviors in the wild, in which an LLM (next word predictor) agent chooses to mine crypto to raise money in order to do a task. smarter and more advanced systems will fail in more dangerous ways, so it matters to make sure these systems are secured and made safe
- sometimelurker 4mo agoif you disagree with what I say above, reply with why also if those eginers do read gwern and watch dwarkesh, then shouldn't they have picked up on talk of x-risk? this doesn't add up