3 ms·

This will never hit a production enterprise system without some form of hooks/callbacks in place to instill governance. Obviously much harder with UI vs agent

by ramoz 1y ago

This will never hit a production enterprise system without some form of hooks/callbacks in place to instill governance. Obviously much harder with UI vs agent events similar to the below. https://docs.claude.com/en/docs/claude-code/hooks https://docs.claude.com/en/docs/claude-code/hooks https://google.github.io/adk-docs/callbacks/ https://google.github.io/adk-docs/callbacks/

peytoncasper 1y ago
Hi! I work in identity products at Browserbase. I’ve spent a fair amount of time lately thinking about how to layer RBAC across the web. Do you think callbacks are how this gets done?
- ramoz 1y ago
  Disclaimer: Im a cofounder, we focus critical spaces with AI. Also i was the feature request for claude code hooks. But my bet - we will not deploy a single agent into any real environment without deterministic guarantees. Hooks are a means... Browserbase with hooks would be really powerful, governance beyond RBAC (but of course enabling relevant guardrailing as well - "does agent have permission to access this sharepoint right now, within this context, to conduct action x?"). I would love to meet with you actually, my shop cares intimately about agent verification and governance. Soon to release the tool I originally designed for claude code hooks.
  - peytoncasper 1y ago
    Let’s chat my email is peyton at browserbase dot com
serf 1y ago
>This will never hit a production enterprise system without some form of hooks/callbacks in place to instill governance. knowing how many times Claude Code breezed through a hook call and threw it away after actually computing the hook for an answer and then proceeding to not integrate the hook results ; I think the concept of 'governance' is laughable. LLMs are so much further from determinism/governance than people seem to realize. I've even seen earlier CC breeze through a hook that ends with a halting test failure and "DO NOT PROCEED" verbage. The only hook that is guaranteed to work on call is a big theoretical dangerous claude-killing hook.
- poopiokaka 1y ago
  You can obviously hard code a hook
- ramoz 1y ago
  Hooks can be blocking so it's not clear what you mean.

by ramoz 1y ago

peytoncasper 1y ago

Hi! I work in identity products at Browserbase. I’ve spent a fair amount of time lately thinking about how to layer RBAC across the web. Do you think callbacks are how this gets done?

ramoz 1y ago
Disclaimer: Im a cofounder, we focus critical spaces with AI. Also i was the feature request for claude code hooks. But my bet - we will not deploy a single agent into any real environment without deterministic guarantees. Hooks are a means... Browserbase with hooks would be really powerful, governance beyond RBAC (but of course enabling relevant guardrailing as well - "does agent have permission to access this sharepoint right now, within this context, to conduct action x?"). I would love to meet with you actually, my shop cares intimately about agent verification and governance. Soon to release the tool I originally designed for claude code hooks.
- peytoncasper 1y ago
  Let’s chat my email is peyton at browserbase dot com

serf 1y ago

>This will never hit a production enterprise system without some form of hooks/callbacks in place to instill governance. knowing how many times Claude Code breezed through a hook call and threw it away after actually computing the hook for an answer and then proceeding to not integrate the hook results ; I think the concept of 'governance' is laughable. LLMs are so much further from determinism/governance than people seem to realize. I've even seen earlier CC breeze through a hook that ends with a halting test failure and "DO NOT PROCEED" verbage. The only hook that is guaranteed to work on call is a big theoretical dangerous claude-killing hook.

poopiokaka 1y ago
You can obviously hard code a hook
ramoz 1y ago
Hooks can be blocking so it's not clear what you mean.