3 ms·

Looking ahead a bit, how do you see the key ownership / trust model evolving as systems scale? Right now it seems very reasonable for the human-in-the-loop to

Looking ahead a bit, how do you see the key ownership / trust model evolving as systems scale? Right now it seems very reasonable for the human-in-the-loop to be the signing authority, which makes the cryptographic certificates more about binding human authorization to agent actions than proving agent correctness. As agents become more autonomous or higher-throughput, do you imagine humans delegating scoped signing authority to sub-agents? time or capability-limited keys? multi-sig / quorum models where humans only intervene on boundary cases? Curious how you’re thinking about preserving accountability and auditability as the human loop inevitably gets thinner.

alexgarden 8mo ago
Yah... how does this evolve... this is the big question. Honest answer? We'll see. My opinion? Human-in-the-loop will get thinner over time. As that happens, the accountability chain has to thicken. If we want any notion of reliable trust, these scales have to balance. Note: I don't think this scales without it. Broadly speaking (I've talked a lot about life in the post-rules universe), we (humans) stop signing actions and start signing policies - policies in this case are declarative envelopes of defined agent automation boundaries. Couple this with a proof system that can (cryptographically) prove that the agent stayed between the lines. Build on that... trust between agents becomes computable. If A trusts B, you have a derivable trust score (with ~ decay) and naturally Quorum models fall out of that. Then you get to proof composition - essentially "instead of verifying /checkpoint you verify a proof for an entire session - the math guarantees nothing was skipped. Human only needs to see the summary. All of this presumes the policy was correct to begin with. This approach isn't a substitute for "don't write sloppy policy or be an asshole."

alexgarden 8mo ago

Yah... how does this evolve... this is the big question. Honest answer? We'll see. My opinion? Human-in-the-loop will get thinner over time. As that happens, the accountability chain has to thicken. If we want any notion of reliable trust, these scales have to balance. Note: I don't think this scales without it. Broadly speaking (I've talked a lot about life in the post-rules universe), we (humans) stop signing actions and start signing policies - policies in this case are declarative envelopes of defined agent automation boundaries. Couple this with a proof system that can (cryptographically) prove that the agent stayed between the lines. Build on that... trust between agents becomes computable. If A trusts B, you have a derivable trust score (with ~ decay) and naturally Quorum models fall out of that. Then you get to proof composition - essentially "instead of verifying /checkpoint you verify a proof for an entire session - the math guarantees nothing was skipped. Human only needs to see the summary. All of this presumes the policy was correct to begin with. This approach isn't a substitute for "don't write sloppy policy or be an asshole."