4 ms·

This is a good summary of the dangers of using agentic clis, but the title & general focus on opencode is odd for two reasons: 1. Most obviously & importantly

This is a good summary of the dangers of using agentic clis, but the title & general focus on opencode is odd for two reasons: 1. Most obviously & importantly this is a complaint without a straightforward suggested alternative. A sibling commenter mentions suggesting fixes to Opencode would be more productive: I don't necessarily agree since many of these issues are fundamental & would likely require an almost ground up rethink & rewrite, but the issue is that the article contains no constructive proposal at all: it may as well be titled "Stop Using LLMs" 2. None of the major issues listed are unique to OpenCode. At least the full list within the "Alarming Things" seems in my mind to apply to Claude CLI, & I would guess most other agents from frontier model providers. Granted it's worthwhile documenting these issues as a plea for someone to build better tooling from the ground up, so the article is far from worthless - on the contrary I've bookmarked it & will be sharing & referencing it widely & often. But the title & focus is just very odd & seems misguided, especially when the contents of the piece is otherwise so good.

qarl2 3mo ago
I wonder how he imagines you can give shell access but securely prevent arbitrary command execution. His complaint that "echo git | bash" still works is particularly nutty.
- wren6991 3mo ago
  Sure, the two main approaches are revoking execute permission from the executable itself, and write-protecting the things that it would modify (like $PWD/.git). Both of these are achievable with sandbox-exec or landlock_restrict_self(2). This works for the `bash` example too. Spawning a new shell and running arbitrary commands is not an escalation because the child shell inherits the parent's restrictions.
  - qarl2 3mo ago
    And what would prevent the LLM from downloading and compiling its own version of git? But it names it "not-git".
    - wren6991 3mo ago
      I mean, ideally enforce W ^ X, and enforce stricter permissions on executables where the agent has flipped the W to X. My broader point is we should be using operating system primitives instead of regex and wishful thinking.
      - qarl2 3mo ago
        I don't know what you mean by "enforce W ^ X". You can't give shell access and then revoke special parts of it. This is not controversial. All your criticisms apply to all LLM harnesses - and everyone who genuinely cares about these things is using security on top of the harness, because OBVIOUSLY. Your complaints are misguided.
        wren6991 3mo ago
        Here: https://en.wikipedia.org/wiki/W%5EX https://en.wikipedia.org/wiki/W%5EX This doesn't require cooperation from the shell. If writable paths are non-executable, and executable paths are non-writable, then executing `cp $(which git) my-new-favourite-binary` doesn't grant you permission to run it. The part which does require cooperation (from outside of the shell, i.e. from the harness) is the ability to upgrade writable paths to executable, at the cost of writability and whatever other restrictions you place on the binaries generated by your agent. This kind of W^X invariant is widely deployed in every desktop operating system, and the type of W-X upgrade hook I described is used by every JIT engine.
        qarl2 3mo ago
        You are confusing CPU-level security with filesystem-level security. I'm curious - how do you expect an LLM harness to build and test executables without being able to build and execute executables?
        wren6991 3mo ago
        Replying inline because this hit max depth: > You are confusing CPU-level security with filesystem-level security. > I'm curious - how do you expect an LLM harness to build and test executables without being able to build and execute executables? W^X applies to file systems as well as page tables, and this is called out in paragraph 4 of the Wikipedia article you didn't read. Respectfully, I'm leaving this conversation at this point because it doesn't seem productive.
- lucideer 3mo ago
  This is very doable - it's a shell you're creating within the cli so you have a lot of control over the environment. If you look at tools like sandbox-exec & apparmor, they specifically target binaries, not string input. E.g. if I write a bash script & put it in my path, OpenCode's allowlists aren't looking at the commands my bash script is exec-ing, nor monitoring subshells it creates. A proper sandbox has full context of the execution chain. Not only that, it's potentially much more powerful & expressive in that you can allowlist indirect filesystem access from commands run. For example, I recently ran opencode via sandbox-exec & naively granted the sandbox access to the aws cli. First attempt to exec it failed because I hadn't granted the sandbox access to the credential file in ~/.aws. This took 2 seconds to resolve but effectively highlights how utterly useless the tree-sitter string-parsing they're doing is. It's so concerning because implementing tree-sitter for that is non-trivial to the point that any competent dev should surely realise quickly while implementing it that this avenue is utterly useless. Yet every high-profile agent cli has taken this approach.
  - qarl2 3mo ago
    Yes, I am aware that sandboxing technology exists. I use them. I don't expect them to be built into my LLM harness because I value modular software composition - and sandboxing should be provided by a product that specializes in that area. OpenCode's allowlists are not there for security. They exist for LLM behavior guidance.
    - lucideer 3mo ago
      > allowlists are not there for security I don't honestly believe that the allowlist isn't (naively) intended to be there for security. It's certainly not a very useful ux for providing LLM guidance: prompts, memories & AGENTS.md already provide an avenue for this & having it in a formalised, structured format isn't conducive to LLM guidance. The implementation brings in a lot of dependencies (tree-sitter) & contains enough complex targeted string parsing that's definitely superfluous for something intended to be fed to an LLM as guidance. Perhaps most importantly - the LLM behaviour on encountering limits of the allowlist is neither intuitive nor consistent enough to be considered reliably guiding (& system prompt doesn't do anything to mitigate this). > I don't expect them to be built into my LLM harness because I value modular software composition I value software modularity, in particular where software architecture is concerned, but I wouldn't go as far as being prescriptive about the unix philosophy. Direct integration does have its pros, & most of the cons associated with it in practice are specific to bad architecture (or black box software). For example Claude cli does integrate sandbox-exec internally, but it does do in a closed-source inaccessible way that negates its value. I am confident it can be done well though, even if I'm somewhat less confident that it ever will be.

qarl2 3mo ago

I wonder how he imagines you can give shell access but securely prevent arbitrary command execution. His complaint that "echo git | bash" still works is particularly nutty.

wren6991 3mo ago
Sure, the two main approaches are revoking execute permission from the executable itself, and write-protecting the things that it would modify (like $PWD/.git). Both of these are achievable with sandbox-exec or landlock_restrict_self(2). This works for the `bash` example too. Spawning a new shell and running arbitrary commands is not an escalation because the child shell inherits the parent's restrictions.
- qarl2 3mo ago
  And what would prevent the LLM from downloading and compiling its own version of git? But it names it "not-git".
  - wren6991 3mo ago
    I mean, ideally enforce W ^ X, and enforce stricter permissions on executables where the agent has flipped the W to X. My broader point is we should be using operating system primitives instead of regex and wishful thinking.
    - qarl2 3mo ago
      I don't know what you mean by "enforce W ^ X". You can't give shell access and then revoke special parts of it. This is not controversial. All your criticisms apply to all LLM harnesses - and everyone who genuinely cares about these things is using security on top of the harness, because OBVIOUSLY. Your complaints are misguided.
      - wren6991 3mo ago
        Here: https://en.wikipedia.org/wiki/W%5EX https://en.wikipedia.org/wiki/W%5EX This doesn't require cooperation from the shell. If writable paths are non-executable, and executable paths are non-writable, then executing `cp $(which git) my-new-favourite-binary` doesn't grant you permission to run it. The part which does require cooperation (from outside of the shell, i.e. from the harness) is the ability to upgrade writable paths to executable, at the cost of writability and whatever other restrictions you place on the binaries generated by your agent. This kind of W^X invariant is widely deployed in every desktop operating system, and the type of W-X upgrade hook I described is used by every JIT engine.
        qarl2 3mo ago
        You are confusing CPU-level security with filesystem-level security. I'm curious - how do you expect an LLM harness to build and test executables without being able to build and execute executables?
        wren6991 3mo ago
        Replying inline because this hit max depth: > You are confusing CPU-level security with filesystem-level security. > I'm curious - how do you expect an LLM harness to build and test executables without being able to build and execute executables? W^X applies to file systems as well as page tables, and this is called out in paragraph 4 of the Wikipedia article you didn't read. Respectfully, I'm leaving this conversation at this point because it doesn't seem productive.
lucideer 3mo ago
This is very doable - it's a shell you're creating within the cli so you have a lot of control over the environment. If you look at tools like sandbox-exec & apparmor, they specifically target binaries, not string input. E.g. if I write a bash script & put it in my path, OpenCode's allowlists aren't looking at the commands my bash script is exec-ing, nor monitoring subshells it creates. A proper sandbox has full context of the execution chain. Not only that, it's potentially much more powerful & expressive in that you can allowlist indirect filesystem access from commands run. For example, I recently ran opencode via sandbox-exec & naively granted the sandbox access to the aws cli. First attempt to exec it failed because I hadn't granted the sandbox access to the credential file in ~/.aws. This took 2 seconds to resolve but effectively highlights how utterly useless the tree-sitter string-parsing they're doing is. It's so concerning because implementing tree-sitter for that is non-trivial to the point that any competent dev should surely realise quickly while implementing it that this avenue is utterly useless. Yet every high-profile agent cli has taken this approach.
- qarl2 3mo ago
  Yes, I am aware that sandboxing technology exists. I use them. I don't expect them to be built into my LLM harness because I value modular software composition - and sandboxing should be provided by a product that specializes in that area. OpenCode's allowlists are not there for security. They exist for LLM behavior guidance.
  - lucideer 3mo ago
    > allowlists are not there for security I don't honestly believe that the allowlist isn't (naively) intended to be there for security. It's certainly not a very useful ux for providing LLM guidance: prompts, memories & AGENTS.md already provide an avenue for this & having it in a formalised, structured format isn't conducive to LLM guidance. The implementation brings in a lot of dependencies (tree-sitter) & contains enough complex targeted string parsing that's definitely superfluous for something intended to be fed to an LLM as guidance. Perhaps most importantly - the LLM behaviour on encountering limits of the allowlist is neither intuitive nor consistent enough to be considered reliably guiding (& system prompt doesn't do anything to mitigate this). > I don't expect them to be built into my LLM harness because I value modular software composition I value software modularity, in particular where software architecture is concerned, but I wouldn't go as far as being prescriptive about the unix philosophy. Direct integration does have its pros, & most of the cons associated with it in practice are specific to bad architecture (or black box software). For example Claude cli does integrate sandbox-exec internally, but it does do in a closed-source inaccessible way that negates its value. I am confident it can be done well though, even if I'm somewhat less confident that it ever will be.