4 ms·

i really dont get why we cant just feed the openapi spec to the LLM instead of having this intermediate MCP representation. Don't really buy the whole 'the api

i really dont get why we cant just feed the openapi spec to the LLM instead of having this intermediate MCP representation. Don't really buy the whole 'the api docs will overwhelm an LLM" - that hasn't been my experience.

wild_egg 1y ago
I haven't looked at MCP payloads properly to compare but often the raw OpenAPI spec is overly verbose and eats context space pretty quick. Really trivial to have the LLM first filter it down to the sections it cares about and then condense those sections though. Wrap that process in a small tool and give that to the LLM along with a `fetch` tool that handles credentials based on URLs and agent capabilities explode pretty rapidly.
crystal_revenge 1y ago
I see this question frequently related to MCP, but I'm guessing these questions come from people who haven't built a lot of products using LLMs? Even if you're LLM could learn the openai spec, you still have to figure out how to concretely receive a response back. This is necessary for virtually any application build using an LLM and requires support for far, far more use cases than just calling an API. Consider the following use case: - You need to include some relevant contextual data from a local RAG system. - There are local functions that you want the model to be able to call - The API example you describe - You need to access data from a database In all of these cases, if you have experience working with LLMs, you've implemented some ad hoc template solution to pass the context into the model. You might have writing something like "Here is the info relevant to this task {{info}}" or "These are the tools you can use {{tools}}", but in each case you've had to craft a prompting solution specific to one problem. MCP solves this by making a generic interface to sending a wide range of information to the model to make use of. While the hype can be a bit much, it's a pretty good (minus the lack of foresight around security) and obvious solution to this current problem in AI Engineering.
- otabdeveloper4 1y ago
  Just ask the model to respond with JSON. Give it a template example response. You don't need a spec. For sending prompts to the LLM you will absolutely need to hand-craft custom prompts anyways, as each model responds slightly different.
- wild_egg 1y ago
  > you still have to figure out how to concretely receive a response back Isn't that handled by whatever Tool API you're using? There's usually a `function_call_output` or `tool_result` message type. I haven't had a need for a separate protocol just to send responses.
truemotive 1y ago
If you're working from OpenAPI, ideally you want to be able to process any, potentially full of shit formatting spec file. I find that half the integrations I run into have some old weird version of Swagger, and the rest work like hell to stay up to date with the 3.x spec track. I agree, I wish, it will be a solved problem eventually. Just feeding a complex data model like that to the paper shredder that is the LLM, for making decisions about whether DELETE or POST is used is just asking for trouble.

wild_egg 1y ago

I haven't looked at MCP payloads properly to compare but often the raw OpenAPI spec is overly verbose and eats context space pretty quick. Really trivial to have the LLM first filter it down to the sections it cares about and then condense those sections though. Wrap that process in a small tool and give that to the LLM along with a `fetch` tool that handles credentials based on URLs and agent capabilities explode pretty rapidly.

crystal_revenge 1y ago

I see this question frequently related to MCP, but I'm guessing these questions come from people who haven't built a lot of products using LLMs? Even if you're LLM could learn the openai spec, you still have to figure out how to concretely receive a response back. This is necessary for virtually any application build using an LLM and requires support for far, far more use cases than just calling an API. Consider the following use case: - You need to include some relevant contextual data from a local RAG system. - There are local functions that you want the model to be able to call - The API example you describe - You need to access data from a database In all of these cases, if you have experience working with LLMs, you've implemented some ad hoc template solution to pass the context into the model. You might have writing something like "Here is the info relevant to this task {{info}}" or "These are the tools you can use {{tools}}", but in each case you've had to craft a prompting solution specific to one problem. MCP solves this by making a generic interface to sending a wide range of information to the model to make use of. While the hype can be a bit much, it's a pretty good (minus the lack of foresight around security) and obvious solution to this current problem in AI Engineering.

otabdeveloper4 1y ago
Just ask the model to respond with JSON. Give it a template example response. You don't need a spec. For sending prompts to the LLM you will absolutely need to hand-craft custom prompts anyways, as each model responds slightly different.
wild_egg 1y ago
> you still have to figure out how to concretely receive a response back Isn't that handled by whatever Tool API you're using? There's usually a `function_call_output` or `tool_result` message type. I haven't had a need for a separate protocol just to send responses.

truemotive 1y ago

If you're working from OpenAPI, ideally you want to be able to process any, potentially full of shit formatting spec file. I find that half the integrations I run into have some old weird version of Swagger, and the rest work like hell to stay up to date with the 3.x spec track. I agree, I wish, it will be a solved problem eventually. Just feeding a complex data model like that to the paper shredder that is the LLM, for making decisions about whether DELETE or POST is used is just asking for trouble.