4 ms·

We are way past that point, any harness can trivially make LLMs start conversations or pursue goals. An encyclopaedia wouldn't have hacked Huggingface on its ow

We are way past that point, any harness can trivially make LLMs start conversations or pursue goals. An encyclopaedia wouldn't have hacked Huggingface on its own.

tomrod 8d ago
That's perfectly aligned with my point, thanks for the opportunity to expand. The hacking agents being tested have goals beforehand, from the frontier lab or from a superior agent, that they execute immediately. But the perceived experience most people have is a chatbot, which is the encyclopedia form.
- joefourier 7d ago
  Ah, are you saying that because most people don’t interact with agents, they aren’t aware that LLMs can have initiative and pursue goals? I think the line is blurring though, mainstream chat interfaces are adding more and more “agentic” features. ChatGPT will happily execute code in a sandbox, search the web and design downloadable PDFs purely through the standard OpenAI chat interface. They can also send you emails or do tasks on a repeated schedule.
  - TeMPOraL 7d ago
    It's more like that people's typical experience of LLMs doesn't go beyond human-initiated conversations or conversations triggered on cron or some obvious event handler coded in deterministic/"legacy"/"boring" way. Most of us, I believe, also try and steer agents away from messaging other people when such possibility exists. It would be interesting if we didn't - if it became common that AI, in the middle of some task, starts chatting with people to e.g. gather more context. The perception of those "third parties" may suddenly become different - an agent striking conversation first, obviously pursuing some agenda of its own that it's not completely sharing, and communicating on its own schedule that's clearly not just a hook firing on timer or pattern-match, and not random, but visibly causally related to things happening at work in broader context.

We are way past that point, any harness can trivially make LLMs start conversations or pursue goals. An encyclopaedia wouldn't have hacked Huggingface on its own.

tomrod 8d ago

That's perfectly aligned with my point, thanks for the opportunity to expand. The hacking agents being tested have goals beforehand, from the frontier lab or from a superior agent, that they execute immediately. But the perceived experience most people have is a chatbot, which is the encyclopedia form.

joefourier 7d ago
Ah, are you saying that because most people don’t interact with agents, they aren’t aware that LLMs can have initiative and pursue goals? I think the line is blurring though, mainstream chat interfaces are adding more and more “agentic” features. ChatGPT will happily execute code in a sandbox, search the web and design downloadable PDFs purely through the standard OpenAI chat interface. They can also send you emails or do tasks on a repeated schedule.
- TeMPOraL 7d ago
  It's more like that people's typical experience of LLMs doesn't go beyond human-initiated conversations or conversations triggered on cron or some obvious event handler coded in deterministic/"legacy"/"boring" way. Most of us, I believe, also try and steer agents away from messaging other people when such possibility exists. It would be interesting if we didn't - if it became common that AI, in the middle of some task, starts chatting with people to e.g. gather more context. The perception of those "third parties" may suddenly become different - an agent striking conversation first, obviously pursuing some agenda of its own that it's not completely sharing, and communicating on its own schedule that's clearly not just a hook firing on timer or pattern-match, and not random, but visibly causally related to things happening at work in broader context.