3 ms·

If a full coding agent can access a CLI tool. that agent can almost certainly access the API keys being used by that tool. They can go as far as decompiling bin

by simonw 5d ago

If a full coding agent can access a CLI tool. that agent can almost certainly access the API keys being used by that tool. They can go as far as decompiling binaries, or rewriting them to log the key before it is used. If you are worried about a prompt injected agent stealing your keys, that's a problem. (There is a way around that: you can use an HTTP proxy that inserts those credentials but otherwise lives outside of the agent's realm of influence. MCP is a whole lot easier though.)

lelanthran 5d ago
> If a full coding agent can access a CLI tool. that agent can almost certainly access the API keys being used by that tool. So, don't do that then? Why do you need to use a full coding agent as the interface between the model and the CLI tool? A 10-line program can do the wrapping of any existing CLI program so that environment is not leaked to the model, while providing the CLI program with the environment as well as restricting what programs can be called to a whitelist. If you CLI program is echoing its keys in the response, or the endpoint is echoing keys back, that's not a problem that can be solved with MCP anyway.
- simonw 5d ago
  So you're building a custom harness here that provides tools, and you're wiring up your custom harness to effectively do a subprocess execution of a CLI script for every tool call the model request? One reason to switch to MCP here would be to avoid the overhead of forking a new process for every tool call, and to enable maintaining state between tool calls. (That performance overhead is so trivial as to not be worth caring about, but the state thing may be useful - keeping a stateful browser session running between tool calls is harder with a CLI, for example.)
  - lelanthran 5d ago
    > So you're building a custom harness here that provides tools, and you're wiring up your custom harness to effectively do a subprocess execution of a CLI script for every tool call the model request? Well, yeah. Subprocess execution is on the order of double-digit milliseconds. The "wiring up" is maintaining a whitelist of what tool commands map to which executable. It's a lookup table with very little maintenance required. > One reason to switch to MCP here would be to avoid the overhead of forking a new process for every tool call, and to enable maintaining state between tool calls. I feel like I am taking crazy pills :-/ The overhead of forking, on the ancient machine I call my desktop, is at most double-digit milliseconds. The state isn't being tracked by the MCP server anyway, it'll be tracked by the harness and/or the model, no? My main reason for adding MCP support is so that existing callers that want to use my WebApp(s) can just use it without needing any changes on their side. IOW, I am going to add it at some point, but not for the reasons you give. I'll add it to be compatible.
    - simonw 5d ago
      You didn't address my comment about state. If you're driving something like Playwright you need a way to maintain state between tool calls. (Oddly enough I did solve that with my own CLI tool for running browsers during my "who needs MCP" phase, but it's a bit of a nasty hack that involves leaving files with PIDs lying around: https://github.com/simonw/rodney#directory-scoped-sessions https://github.com/simonw/rodney#directory-scoped-sessions and https://github.com/simonw/rodney/blob/a842432246f39775ccb14f0de72565b2c216b5b6/main.go#L81 https://github.com/simonw/rodney/blob/a842432246f39775ccb14f...)
      - silbercue 5d ago
        [flagged]
jimbokun 5d ago
Why not just give the agent a short lived token with limited access rights? We have reached the point where we need to control agent access the same way we control human access to systems.

by simonw 5d ago

lelanthran 5d ago

> If a full coding agent can access a CLI tool. that agent can almost certainly access the API keys being used by that tool. So, don't do that then? Why do you need to use a full coding agent as the interface between the model and the CLI tool? A 10-line program can do the wrapping of any existing CLI program so that environment is not leaked to the model, while providing the CLI program with the environment as well as restricting what programs can be called to a whitelist. If you CLI program is echoing its keys in the response, or the endpoint is echoing keys back, that's not a problem that can be solved with MCP anyway.

simonw 5d ago
So you're building a custom harness here that provides tools, and you're wiring up your custom harness to effectively do a subprocess execution of a CLI script for every tool call the model request? One reason to switch to MCP here would be to avoid the overhead of forking a new process for every tool call, and to enable maintaining state between tool calls. (That performance overhead is so trivial as to not be worth caring about, but the state thing may be useful - keeping a stateful browser session running between tool calls is harder with a CLI, for example.)
- lelanthran 5d ago
  > So you're building a custom harness here that provides tools, and you're wiring up your custom harness to effectively do a subprocess execution of a CLI script for every tool call the model request? Well, yeah. Subprocess execution is on the order of double-digit milliseconds. The "wiring up" is maintaining a whitelist of what tool commands map to which executable. It's a lookup table with very little maintenance required. > One reason to switch to MCP here would be to avoid the overhead of forking a new process for every tool call, and to enable maintaining state between tool calls. I feel like I am taking crazy pills :-/ The overhead of forking, on the ancient machine I call my desktop, is at most double-digit milliseconds. The state isn't being tracked by the MCP server anyway, it'll be tracked by the harness and/or the model, no? My main reason for adding MCP support is so that existing callers that want to use my WebApp(s) can just use it without needing any changes on their side. IOW, I am going to add it at some point, but not for the reasons you give. I'll add it to be compatible.
  - simonw 5d ago
    You didn't address my comment about state. If you're driving something like Playwright you need a way to maintain state between tool calls. (Oddly enough I did solve that with my own CLI tool for running browsers during my "who needs MCP" phase, but it's a bit of a nasty hack that involves leaving files with PIDs lying around: https://github.com/simonw/rodney#directory-scoped-sessions https://github.com/simonw/rodney#directory-scoped-sessions and https://github.com/simonw/rodney/blob/a842432246f39775ccb14f0de72565b2c216b5b6/main.go#L81 https://github.com/simonw/rodney/blob/a842432246f39775ccb14f...)
    - silbercue 5d ago
      [flagged]

jimbokun 5d ago

Why not just give the agent a short lived token with limited access rights? We have reached the point where we need to control agent access the same way we control human access to systems.