3 ms·

I’m just getting into MCP (building my own server and trying some canned ones), and one thing I’ve noticed — some servers seem to devour your context window bef

by cowmix 1y ago

I’m just getting into MCP (building my own server and trying some canned ones), and one thing I’ve noticed — some servers seem to devour your context window before you’ve even typed a single token / question. My favorite example is the public Atlassian one — https://www.atlassian.com/blog/announcements/remote-mcp-server https://www.atlassian.com/blog/announcements/remote-mcp-serv... Even with Claude or Gemini CLI (both with generous limits), I run out of context and resources fast. With local LLMs via LM Studio? Forget it — almost any model will tap out before I can get even a simple question in.

jjani 1y ago
Love how the MCPs mimic the company they represent in that sense.
- cowmix 1y ago
  Heh.. this comment made my day.
- catlifeonmars 1y ago
  Conway’s law in action?
markerz 1y ago
MCPs are the most basic solution possible. Shoving the tool definitions into a vector store and having a subagent search for relevant tools, then another subagent to run the tools would greatly reduce impact on context. I think it’d work in theory, but it’s so annoying to have to do something like this. We’re still in a world where we have some building blocks rather than full fledged toolboxes.
- dragonwriter 1y ago
  > MCPs are the most basic solution possible. Shoving the tool definitions into a vector store and having a subagent search for relevant tools, then another subagent to run the tools would greatly reduce impact on context. That's a toolchain design approach that is independent of MCPs. A toolchain using MCP could do that and there would be no need for any special support in the protocol.
dragonwriter 1y ago
Most UIs currently are unsophisticated and let you turn on or off the tools on server-by-server basis. For some large servers (especially if they act as aggrgeators) this approach isn't going to be desirable and you are going to want to select individual tools to activate, not servers. But that's a UI issue more than a protocol issue.
- jngiam1 1y ago
  We've been thinking that an intermediate (virtual) server layer might be helpful here. Actively working on something to solve that now and looking for feedback, please reach out if interested.
- typpilol 1y ago
  Vscode let's you go tool by tool. It's great
jameshart 1y ago
There’s very little actual engineering going in to designing MCP interfaces to actually efficiently work with the way LLM workflows actually operate. Many MCPs offer tools that allow an LLM to retrieve a list of ‘things that exist’ with the expectation the LLM will then pick something out of that list for further action with a different tool. There’s very little evidence that LLMs are actually good at using tools that work like that, and massive lists of ‘things that exist’ eat tokens and context. Many businesses are rushing to put out something that fits the MCP standard but not taking the time to produce something that lets an LLM achieve things with their tool.
- whattheheckheck 1y ago
  All new models are probably trained to understand the tool use paradigm
  - jameshart 1y ago
    They’re trained in using tools that take actions (like sending an email) and tools that retrieve information (like pulling down a Wikipedia article), yes. The specific pattern I see in a lot of SaaS MCPs though is tools for traversing a database which is not really either of those things.
    - typpilol 1y ago
      The only one I've had any use for like that was the sqlite mcp they let's it pull the database index and structure Seems to help it during database coding. But this could also be done easily with a file.
- ncallaway 1y ago
  > Many businesses are rushing to put out something that fits the MCP standard but not taking the time to produce something that lets an LLM achieve things with their tool I think they'll have a while where they can get away with this approach too. For a good while, most people will probably blame the AI or the model if it doesn't use Atlassian tools well. It'll probably be quite some time before people start to notice that Atlassian specifically doesn't work well, but the almost all their other tools do. (More technical users might notice sooner—obviously, given the context of this thread—but I mean enough of a broader user base noticing to have reputational impact)
britannio 1y ago
I discovered the "Layered Tool Pattern" a few days ago which exposes three tools to discover and execute an arbitrary number of service endpoints. https://engineering.block.xyz/blog/build-mcp-tools-like-ogres-with-layers https://engineering.block.xyz/blog/build-mcp-tools-like-ogre...
cadamsdotcom 1y ago
There other ways to get the data & do the interactions you need. Eg. give the model your login token/cookies so it can curl the pages and interact with them - or have it log in as you with Playwright MCP.

by cowmix 1y ago

jjani 1y ago

Love how the MCPs mimic the company they represent in that sense.

cowmix 1y ago
Heh.. this comment made my day.
catlifeonmars 1y ago
Conway’s law in action?

markerz 1y ago

MCPs are the most basic solution possible. Shoving the tool definitions into a vector store and having a subagent search for relevant tools, then another subagent to run the tools would greatly reduce impact on context. I think it’d work in theory, but it’s so annoying to have to do something like this. We’re still in a world where we have some building blocks rather than full fledged toolboxes.

dragonwriter 1y ago
> MCPs are the most basic solution possible. Shoving the tool definitions into a vector store and having a subagent search for relevant tools, then another subagent to run the tools would greatly reduce impact on context. That's a toolchain design approach that is independent of MCPs. A toolchain using MCP could do that and there would be no need for any special support in the protocol.

dragonwriter 1y ago

Most UIs currently are unsophisticated and let you turn on or off the tools on server-by-server basis. For some large servers (especially if they act as aggrgeators) this approach isn't going to be desirable and you are going to want to select individual tools to activate, not servers. But that's a UI issue more than a protocol issue.

jngiam1 1y ago
We've been thinking that an intermediate (virtual) server layer might be helpful here. Actively working on something to solve that now and looking for feedback, please reach out if interested.
typpilol 1y ago
Vscode let's you go tool by tool. It's great

jameshart 1y ago

There’s very little actual engineering going in to designing MCP interfaces to actually efficiently work with the way LLM workflows actually operate. Many MCPs offer tools that allow an LLM to retrieve a list of ‘things that exist’ with the expectation the LLM will then pick something out of that list for further action with a different tool. There’s very little evidence that LLMs are actually good at using tools that work like that, and massive lists of ‘things that exist’ eat tokens and context. Many businesses are rushing to put out something that fits the MCP standard but not taking the time to produce something that lets an LLM achieve things with their tool.

whattheheckheck 1y ago
All new models are probably trained to understand the tool use paradigm
- jameshart 1y ago
  They’re trained in using tools that take actions (like sending an email) and tools that retrieve information (like pulling down a Wikipedia article), yes. The specific pattern I see in a lot of SaaS MCPs though is tools for traversing a database which is not really either of those things.
  - typpilol 1y ago
    The only one I've had any use for like that was the sqlite mcp they let's it pull the database index and structure Seems to help it during database coding. But this could also be done easily with a file.
ncallaway 1y ago
> Many businesses are rushing to put out something that fits the MCP standard but not taking the time to produce something that lets an LLM achieve things with their tool I think they'll have a while where they can get away with this approach too. For a good while, most people will probably blame the AI or the model if it doesn't use Atlassian tools well. It'll probably be quite some time before people start to notice that Atlassian specifically doesn't work well, but the almost all their other tools do. (More technical users might notice sooner—obviously, given the context of this thread—but I mean enough of a broader user base noticing to have reputational impact)

britannio 1y ago

I discovered the "Layered Tool Pattern" a few days ago which exposes three tools to discover and execute an arbitrary number of service endpoints. https://engineering.block.xyz/blog/build-mcp-tools-like-ogres-with-layers https://engineering.block.xyz/blog/build-mcp-tools-like-ogre...

cadamsdotcom 1y ago

There other ways to get the data & do the interactions you need. Eg. give the model your login token/cookies so it can curl the pages and interact with them - or have it log in as you with Playwright MCP.