3 ms·

I used dagger for the sandbox in my personal harness. Docker also has a sandbox (microvm based) product now. Both obscure the credentials, but I do not think it

I used dagger for the sandbox in my personal harness. Docker also has a sandbox (microvm based) product now. Both obscure the credentials, but I do not think it eliminates them base64 encoding it (or some other transformation). It does step it from being visible if it is the same string, regardless where/how it is printed. Service accounts / IAM / WIF / proxies, which remove credentials from the environment all together, are more advanced options. I'm working on a custom "tool calling policy" guardrail agent. There are a number of models which have been trained to accept a policy document and user content, returning a truthy value for the harness to use during approval.

alanfuNZ 2mo ago
Agree on the sandbox side. dagger or a microVM is the right blast-radius layer, and moving creds out of the environment entirely (service accounts, WIF, an egress proxy that injects them) is the real fix for the secret-then-outbound-call case. Obscuring the string is weaker than it looks: the agent can base64 it, split it, or just describe it, and the sandbox has no way to know that the request body was derived from what it read ten minutes ago. That's why I ended up gating at the tool-call level with session state instead of at the network level. Same curl gets a different verdict depending on whether the session already read something secret-shaped. Deterministic rules only, and the approval prompt times out to deny, logged as a timeout rather than a denial so I can tell the two apart later. On the policy-model approach: I'd be careful about what a model's truthy value is allowed to do. A classifier crossing a threshold is a guess with a confidence attached, and a guess that hard-blocks real work gets the whole guardrail disabled by the end of the day. The split I settled on is that a probability can ask (escalate to a human) but only a predicate can refuse. Curious whether you're letting the model produce the deny directly, or routing its output through a human when it's uncertain. For what it's worth this is what I've been building: https://github.com/DobermanCore/Doberman-Core https://github.com/DobermanCore/Doberman-Core. Apache-2.0, 100% open source

alanfuNZ 2mo ago

Agree on the sandbox side. dagger or a microVM is the right blast-radius layer, and moving creds out of the environment entirely (service accounts, WIF, an egress proxy that injects them) is the real fix for the secret-then-outbound-call case. Obscuring the string is weaker than it looks: the agent can base64 it, split it, or just describe it, and the sandbox has no way to know that the request body was derived from what it read ten minutes ago. That's why I ended up gating at the tool-call level with session state instead of at the network level. Same curl gets a different verdict depending on whether the session already read something secret-shaped. Deterministic rules only, and the approval prompt times out to deny, logged as a timeout rather than a denial so I can tell the two apart later. On the policy-model approach: I'd be careful about what a model's truthy value is allowed to do. A classifier crossing a threshold is a guess with a confidence attached, and a guess that hard-blocks real work gets the whole guardrail disabled by the end of the day. The split I settled on is that a probability can ask (escalate to a human) but only a predicate can refuse. Curious whether you're letting the model produce the deny directly, or routing its output through a human when it's uncertain. For what it's worth this is what I've been building: https://github.com/DobermanCore/Doberman-Core https://github.com/DobermanCore/Doberman-Core. Apache-2.0, 100% open source