3 ms·

I would argue that lot of the tools will be hosted on GitHub - infact, most of the existing repos are potentially a tool (in future). And the discovery is just

I would argue that lot of the tools will be hosted on GitHub - infact, most of the existing repos are potentially a tool (in future). And the discovery is just a GitHub search btw gh repos are already part of training the llm So you don't even need internet to search for tools, let alone TEO

michaelanckaert 11mo ago
Security nightmare inbound... The example given by Anthropic of tools filling valuable context space is a result of bad design. If you pass the tools below to your agent, you don't need "search tool" tool, you need good old fashion architecture: limit your tools based on the state of your agent, custom tool wrappers to limit MCP tools, routing to sub-agents, etc. Ref: GitHub: 35 tools (~26K tokens) Slack: 11 tools (~21K tokens) Sentry: 5 tools (~3K tokens) Grafana: 5 tools (~3K tokens) Splunk: 2 tools (~2K tokens)
- mkagenius 11mo ago
  Don't see whats wrong in letting llm decide which tool to call based on a search on long list of tools (or a binary tree of lists in case the list becomes too long, which is essentially what you eluded to with sub-agents)
  - michaelanckaert 11mo ago
    I was referring to letting LLM's search github and run tools from there. That's like randomly searching the internet for code snippets and blindly running them on your production machine.
    - mkagenius 11mo ago
      For that, we need sandboxes to run the code in an isolated environment.
      - michaelanckaert 11mo ago
        Sure to protect your machine, but what about data security? Do I want to allow unknown code to be run on my private/corporate data? Sandbox all you want but sooner or later your data can be exfiltrated. My point is giving an LLM unrestricted access to random code that can be run is a bad idea. Curate carefully is my approach.
        mkagenius 11mo ago
        For data security, you can run sandbox locally too. See https://github.com/instavm/coderunner https://github.com/instavm/coderunner

michaelanckaert 11mo ago

Security nightmare inbound... The example given by Anthropic of tools filling valuable context space is a result of bad design. If you pass the tools below to your agent, you don't need "search tool" tool, you need good old fashion architecture: limit your tools based on the state of your agent, custom tool wrappers to limit MCP tools, routing to sub-agents, etc. Ref: GitHub: 35 tools (~26K tokens) Slack: 11 tools (~21K tokens) Sentry: 5 tools (~3K tokens) Grafana: 5 tools (~3K tokens) Splunk: 2 tools (~2K tokens)

mkagenius 11mo ago
Don't see whats wrong in letting llm decide which tool to call based on a search on long list of tools (or a binary tree of lists in case the list becomes too long, which is essentially what you eluded to with sub-agents)
- michaelanckaert 11mo ago
  I was referring to letting LLM's search github and run tools from there. That's like randomly searching the internet for code snippets and blindly running them on your production machine.
  - mkagenius 11mo ago
    For that, we need sandboxes to run the code in an isolated environment.
    - michaelanckaert 11mo ago
      Sure to protect your machine, but what about data security? Do I want to allow unknown code to be run on my private/corporate data? Sandbox all you want but sooner or later your data can be exfiltrated. My point is giving an LLM unrestricted access to random code that can be run is a bad idea. Curate carefully is my approach.
      - mkagenius 11mo ago
        For data security, you can run sandbox locally too. See https://github.com/instavm/coderunner https://github.com/instavm/coderunner