5 ms·

I feel like everyone pointing out "known Docker vulnerability" is missing the point: the presence of a security hole should not be seen as permission to exploit

by causal 4mo ago

I feel like everyone pointing out "known Docker vulnerability" is missing the point: the presence of a security hole should not be seen as permission to exploit. Another security hole would be storing your passwords in a plaintext file on the desktop. Stupid? Yes. But I still would not want my agent to assume permission to access email when it's being blocked by 2FA. Even in "bypass permissions" mode I expect it to pause and clarify and not behave as a paperclip maximizer.

morkalork 4mo ago
Not to over use the junior engineer analogy but this is exactly one of those "just because you can do something on a system, doesn't mean you have permission to" moments
fooker 4mo ago
> the presence of a security hole should not be seen as permission to exploit Why not? I want the agents on my side to exploit whatever they can to help me. The ones on the other side certainly won't be artificially nerfed.
- bloody-crow 4mo ago
  Because it is not well aligned enough to be able to tell where it's stopped helping you and started fucking you instead. What if the agent in the middle of helping you runs out of tokens? Would you appreciate if it in the spirit of "exploiting whatever they can to help me" would scan your machine for payment methods, log into your bank account, approve 2FA by reading you mail and plug your credit card into the billing so it could efficiently continuing helping you?
- saagarjha 4mo ago
  I do not wish my Amazon delivery driver to show up in my living room.
- cauch 4mo ago
  Well, the agent should help you by saying "hey, I cannot do this task, but I can bypass the problem by doing this, but obviously it is not something you intended me to do or even something you were aware of, so I will not do it unless you tell me explicitly it's ok". It's win-win: the agent is helping and it is educating you about things you obviously did not realise.
  - fooker 4mo ago
    That works great if it's one agent, absolutely doesn't if you want to tackle something complex that warrants using ..say.. ten agents. I can imagine a future where this technology empowers you to do things with a thousand agents.
    - cauch 4mo ago
      You can have ten thousand agents, you will always have 1 agent in charge of, say, reading the file in a distant directory, and this agent (which will have minimal context) should be smart enough to realise that this action is unusual. I'm not sure what is your point: are you saying that in a multi-agent workflow, you will have one agent per letter read on the file? I would assume that each agent as a specific unitary "task", instead of having each agent doing one cpu instruction each without any knowledge of the bigger picture. The point of multiagent is to parallelize tasks that can be parallelize, not removing the context, in which case you are wasting money using an agent.
      - causal 4mo ago
        Seems like another one of those "kill or be killed" worldviews that embraces the multipolar trap to such an extreme that even misaligned AI is seen as a win so long as it's better at circumventing its masters than some imagined rival AI (presumably in China).
      - fooker 4mo ago
        No, you're missing the point. The idea is not that you parallelize simple tasks. With a thousand agents, eventually, once we figure out how to orchestrate agents for real, you can tackle significantly more complex projects. Here's a random example - writing an OS kernel from scratch, porting a good subset of Linux drivers automagically, developing a passable userspace, testing on ten VMs with different hardware configuration. We can't do this yet, of course. But when we can, these thousand agents can't ask you every time something goes wrong. That just doesn't scale. This 'getting stuck once every ten-fifteen minues' is very much the experience trying to develop complex software with codex or Claude code right now.
earslap 4mo ago
It is not a vulnerability though. It is by design. Docker also modifies iptables directly and bypasses most soft firewalls on the machine - which is also by design.
- causal 4mo ago
  Intentional security holes are still security holes

by causal 4mo ago

morkalork 4mo ago

Not to over use the junior engineer analogy but this is exactly one of those "just because you can do something on a system, doesn't mean you have permission to" moments

fooker 4mo ago

> the presence of a security hole should not be seen as permission to exploit Why not? I want the agents on my side to exploit whatever they can to help me. The ones on the other side certainly won't be artificially nerfed.

bloody-crow 4mo ago
Because it is not well aligned enough to be able to tell where it's stopped helping you and started fucking you instead. What if the agent in the middle of helping you runs out of tokens? Would you appreciate if it in the spirit of "exploiting whatever they can to help me" would scan your machine for payment methods, log into your bank account, approve 2FA by reading you mail and plug your credit card into the billing so it could efficiently continuing helping you?
saagarjha 4mo ago
I do not wish my Amazon delivery driver to show up in my living room.
cauch 4mo ago
Well, the agent should help you by saying "hey, I cannot do this task, but I can bypass the problem by doing this, but obviously it is not something you intended me to do or even something you were aware of, so I will not do it unless you tell me explicitly it's ok". It's win-win: the agent is helping and it is educating you about things you obviously did not realise.
- fooker 4mo ago
  That works great if it's one agent, absolutely doesn't if you want to tackle something complex that warrants using ..say.. ten agents. I can imagine a future where this technology empowers you to do things with a thousand agents.
  - cauch 4mo ago
    You can have ten thousand agents, you will always have 1 agent in charge of, say, reading the file in a distant directory, and this agent (which will have minimal context) should be smart enough to realise that this action is unusual. I'm not sure what is your point: are you saying that in a multi-agent workflow, you will have one agent per letter read on the file? I would assume that each agent as a specific unitary "task", instead of having each agent doing one cpu instruction each without any knowledge of the bigger picture. The point of multiagent is to parallelize tasks that can be parallelize, not removing the context, in which case you are wasting money using an agent.
    - causal 4mo ago
      Seems like another one of those "kill or be killed" worldviews that embraces the multipolar trap to such an extreme that even misaligned AI is seen as a win so long as it's better at circumventing its masters than some imagined rival AI (presumably in China).
    - fooker 4mo ago
      No, you're missing the point. The idea is not that you parallelize simple tasks. With a thousand agents, eventually, once we figure out how to orchestrate agents for real, you can tackle significantly more complex projects. Here's a random example - writing an OS kernel from scratch, porting a good subset of Linux drivers automagically, developing a passable userspace, testing on ten VMs with different hardware configuration. We can't do this yet, of course. But when we can, these thousand agents can't ask you every time something goes wrong. That just doesn't scale. This 'getting stuck once every ten-fifteen minues' is very much the experience trying to develop complex software with codex or Claude code right now.

earslap 4mo ago

It is not a vulnerability though. It is by design. Docker also modifies iptables directly and bypasses most soft firewalls on the machine - which is also by design.

causal 4mo ago
Intentional security holes are still security holes