5 ms·

>The toolchain is parsing, validating, and mapping the data into the format preferred by the chosen models promot template, the LLM has nothing to do with doing

>The toolchain is parsing, validating, and mapping the data into the format preferred by the chosen models promot template, the LLM has nothing to do with doing that The LLM has everything to do with that. The LLM is literally choosing to do that. I don't know why this point keeps getting missed or side-stepped. It WILL, at some point in the future and given enough executions, as a matter of statistical certainty, simply not do that above, or pretend to do the above, or do something totally different at some point in the future.

dragonwriter 1y ago
> The LLM has everything to do with that. The LLM is literally choosing to do that. No, the LLM doesn't control on a case-by-caae basis what the toolchain does between the LLM putting a tool call request in an output message and the toolchain calling the LLM afterwards. If the toolchain is programmed to always validate tool responses against the JSON schema provided by MCP server before mapping into the LLM prompt template and calling the LLM again to handle the response, that is going to happen 100% of the time. The LLM doesn't choose it. It CAN'T because the only way it even knows that the data has come back from the tool call is that the toolchain has already done whatever it is programmed to do, ending with mapping the response into a prompt and calling the LLM again. Even before MCPs or even models specifically trained and with vendor-provided templates for tool calling (but after the ReAct architecture was described), it was like a weekend project to implement a basic framework supporting tooling calling around a local or remote LLM. I don't think you need to do that to understand how silly the claim that the LLM controls what the toolchain does with each response and might make it not validate it is, but certainly doing it will give you a visceral understanding of how silly it is.
- whoknowsidont 1y ago
  I think you are, for whatever reason, missing a fact of causality here and I'm not sure I can fix that over text. I mean that in the most respectful way possible.
  - gnat 1y ago
    Are you two talking at cross-purposes because you don't have a shared understanding of control and data flow? The pieces here are: * Claude Code, a Node (Javascript) application that talks to MCP server(s) and the Claude API * The MCP server, which exposes some tools through stdin or HTTP * The Claude API, which is more structured than "text in, text out". * The Claude LLM behind the API, which generates a response to a given prompt Claude Code is a Node application. CC is configured in JSON with a list of MCP servers. When CC starts up, CC"s Javascript initialises each server and as part of that gets a list of callable functions. When CC calls the LLM API with a user's request, it's not just "here is the user's words, do it". There are multiple slots in the request object, one of which is a "tools" block, a list of the tools that can be called. Inside the API, I imagine this is packaged into a prefix context string like "you have access to the following tools: tool(args) ...". The LLM API probably has a bunch of prompts it runs through (figure out what type of request the user has made, maybe using different prompts to make different types of plan, etc.) and somewhere along the way the LLM might respond with a request to call a tool. The LLM API call then returns the tool call request to CC, in a structured "tool_use" block separate from the freetext "hey good news, you asked a question and got this response". The structured block means "the LLM wants to call this tool." CC's JS then calls the server with the tool request and gets the response. It validates the response (e.g., JSON schemas) and then calls the LLM API again bundling up the success/failure of the tool call into a structured "tool_result" block. If it validated and was successful, the LLM gets to see the MCP server's response. If it failed to validate, the LLM gets to see that it failed and what the error message was (so the LLM can try again in a different way). The idea is that if a tool call is supposed to return a CarMakeModel string ("Toyota Tercel") and instead returns an int (42), JSON Schemas can catch this. The client validates the server's response against the schema, and calls the LLM API with { "type": "tool_result", "tool_use_id": "abc123", "is_error": true, "content": [ { "type": "text", "text": "Expected string, got integer." } ] } So the LLM isn't choosing to call the validator, it's the deterministic Javascript that is Claude Code that chooses to call the validator. There are plenty of ways for this to go wrong: the client (Claude Code) has to validate; int vs string isn't the same as "is a valid timestamp/CarMakeModel/etc"; if you helpfully put the thing that failed into the error message ("Expect string, got integer (42)") then the LLM gets 42 and might choose to interpret that as a CarMakeModel if it's having a particularly bad day; the LLM might say "well, that didn't work, but let's assume the answer was Toyota Tercel, a common car make and model", ... We're reaching here, yet these are possible. But the basic flow has validation done in deterministic code and hiding the MCP server's invalid responses from the LLM. The LLM can't choose not to validate. You seemed to be saying that the LLM could choose not to validate, and your interlocutor was saying that was not the case. I hope this helps!
whartung 1y ago
I don’t know how this works, just to start off. How does the AI bypass the MCP layer to make the request? The assumption is (as I understand it) the AI says “I want to make MCP request XYZ with data ABC” and it sends that off to the MCP interface which does the heavy lifting. If the MCP interface is doing the schema checks, and tossing errors as appropriate, how is the AI routing around this interface to bypass the schema enforcement?
- cle 1y ago
  What you described is essentially how it works. The LLM has no control over how the inputs & outputs are validated, nor in how the result is fed back into it. The MCP interface (Claude Code in this case) is doing the schema checks. Claude Code will refuse to provide the result to the LLM if it does not pass the schema check, and the LLM has no control over that.
  - whoknowsidont 1y ago
    >The LLM has no control over how the inputs & outputs are validated Which is completely fucking irrelevant to what everyone else is discussing.
    - dragonwriter 1y ago
      > > The LLM has no control over how the inputs & outputs are validated > Which is completely fucking irrelevant to what everyone else is discussing. Not sure what you think is going on, but that is literally the question this subthread is debating, starting with an exchange in which the salient claims were: From: https://news.ycombinator.com/item?id=44849695 https://news.ycombinator.com/item?id=44849695 > Claude Code validated the response against the schema and did not pass the response to the LLM. From: https://news.ycombinator.com/item?id=44850894 https://news.ycombinator.com/item?id=44850894 > This time. > Can you guarantee it will validate it every time ?
- whoknowsidont 1y ago
  >How does the AI bypass the MCP layer to make the request It doesn't. I don't know why the other commenters are pretending this step does not happen. There is a prompt that basically tells the LLM to use the generated manifest/configuration files. The LLM still has to not hallucinate in order to properly call the tools with JRPC and properly follow MCP protocol. It then also has to make sense of the structured prompts that define the tools in the MCP manifest/configuration file. It's system prompts all the way down. Here's a good read of some the underlying/supporting concepts: https://huggingface.co/docs/hugs/en/guides/function-calling https://huggingface.co/docs/hugs/en/guides/function-calling Why this fact is seemingly being lost in this thread, I have no idea, but I don't have anything nice to say about it so I won't :). Other than we're all clearly quite screwed, of course. MCP is to make things standard for humans, with expected formats. The LLM's really couldn't give a shit and don't have anything super special about how the interact with MCP configuration files or the protocol (other than some additional fine-tuning, again, to make it less likely to get the wrong output).

dragonwriter 1y ago

> The LLM has everything to do with that. The LLM is literally choosing to do that. No, the LLM doesn't control on a case-by-caae basis what the toolchain does between the LLM putting a tool call request in an output message and the toolchain calling the LLM afterwards. If the toolchain is programmed to always validate tool responses against the JSON schema provided by MCP server before mapping into the LLM prompt template and calling the LLM again to handle the response, that is going to happen 100% of the time. The LLM doesn't choose it. It CAN'T because the only way it even knows that the data has come back from the tool call is that the toolchain has already done whatever it is programmed to do, ending with mapping the response into a prompt and calling the LLM again. Even before MCPs or even models specifically trained and with vendor-provided templates for tool calling (but after the ReAct architecture was described), it was like a weekend project to implement a basic framework supporting tooling calling around a local or remote LLM. I don't think you need to do that to understand how silly the claim that the LLM controls what the toolchain does with each response and might make it not validate it is, but certainly doing it will give you a visceral understanding of how silly it is.

whoknowsidont 1y ago
I think you are, for whatever reason, missing a fact of causality here and I'm not sure I can fix that over text. I mean that in the most respectful way possible.
- gnat 1y ago
  Are you two talking at cross-purposes because you don't have a shared understanding of control and data flow? The pieces here are: * Claude Code, a Node (Javascript) application that talks to MCP server(s) and the Claude API * The MCP server, which exposes some tools through stdin or HTTP * The Claude API, which is more structured than "text in, text out". * The Claude LLM behind the API, which generates a response to a given prompt Claude Code is a Node application. CC is configured in JSON with a list of MCP servers. When CC starts up, CC"s Javascript initialises each server and as part of that gets a list of callable functions. When CC calls the LLM API with a user's request, it's not just "here is the user's words, do it". There are multiple slots in the request object, one of which is a "tools" block, a list of the tools that can be called. Inside the API, I imagine this is packaged into a prefix context string like "you have access to the following tools: tool(args) ...". The LLM API probably has a bunch of prompts it runs through (figure out what type of request the user has made, maybe using different prompts to make different types of plan, etc.) and somewhere along the way the LLM might respond with a request to call a tool. The LLM API call then returns the tool call request to CC, in a structured "tool_use" block separate from the freetext "hey good news, you asked a question and got this response". The structured block means "the LLM wants to call this tool." CC's JS then calls the server with the tool request and gets the response. It validates the response (e.g., JSON schemas) and then calls the LLM API again bundling up the success/failure of the tool call into a structured "tool_result" block. If it validated and was successful, the LLM gets to see the MCP server's response. If it failed to validate, the LLM gets to see that it failed and what the error message was (so the LLM can try again in a different way). The idea is that if a tool call is supposed to return a CarMakeModel string ("Toyota Tercel") and instead returns an int (42), JSON Schemas can catch this. The client validates the server's response against the schema, and calls the LLM API with { "type": "tool_result", "tool_use_id": "abc123", "is_error": true, "content": [ { "type": "text", "text": "Expected string, got integer." } ] } So the LLM isn't choosing to call the validator, it's the deterministic Javascript that is Claude Code that chooses to call the validator. There are plenty of ways for this to go wrong: the client (Claude Code) has to validate; int vs string isn't the same as "is a valid timestamp/CarMakeModel/etc"; if you helpfully put the thing that failed into the error message ("Expect string, got integer (42)") then the LLM gets 42 and might choose to interpret that as a CarMakeModel if it's having a particularly bad day; the LLM might say "well, that didn't work, but let's assume the answer was Toyota Tercel, a common car make and model", ... We're reaching here, yet these are possible. But the basic flow has validation done in deterministic code and hiding the MCP server's invalid responses from the LLM. The LLM can't choose not to validate. You seemed to be saying that the LLM could choose not to validate, and your interlocutor was saying that was not the case. I hope this helps!

whartung 1y ago

I don’t know how this works, just to start off. How does the AI bypass the MCP layer to make the request? The assumption is (as I understand it) the AI says “I want to make MCP request XYZ with data ABC” and it sends that off to the MCP interface which does the heavy lifting. If the MCP interface is doing the schema checks, and tossing errors as appropriate, how is the AI routing around this interface to bypass the schema enforcement?

cle 1y ago
What you described is essentially how it works. The LLM has no control over how the inputs & outputs are validated, nor in how the result is fed back into it. The MCP interface (Claude Code in this case) is doing the schema checks. Claude Code will refuse to provide the result to the LLM if it does not pass the schema check, and the LLM has no control over that.
- whoknowsidont 1y ago
  >The LLM has no control over how the inputs & outputs are validated Which is completely fucking irrelevant to what everyone else is discussing.
  - dragonwriter 1y ago
    > > The LLM has no control over how the inputs & outputs are validated > Which is completely fucking irrelevant to what everyone else is discussing. Not sure what you think is going on, but that is literally the question this subthread is debating, starting with an exchange in which the salient claims were: From: https://news.ycombinator.com/item?id=44849695 https://news.ycombinator.com/item?id=44849695 > Claude Code validated the response against the schema and did not pass the response to the LLM. From: https://news.ycombinator.com/item?id=44850894 https://news.ycombinator.com/item?id=44850894 > This time. > Can you guarantee it will validate it every time ?
whoknowsidont 1y ago
>How does the AI bypass the MCP layer to make the request It doesn't. I don't know why the other commenters are pretending this step does not happen. There is a prompt that basically tells the LLM to use the generated manifest/configuration files. The LLM still has to not hallucinate in order to properly call the tools with JRPC and properly follow MCP protocol. It then also has to make sense of the structured prompts that define the tools in the MCP manifest/configuration file. It's system prompts all the way down. Here's a good read of some the underlying/supporting concepts: https://huggingface.co/docs/hugs/en/guides/function-calling https://huggingface.co/docs/hugs/en/guides/function-calling Why this fact is seemingly being lost in this thread, I have no idea, but I don't have anything nice to say about it so I won't :). Other than we're all clearly quite screwed, of course. MCP is to make things standard for humans, with expected formats. The LLM's really couldn't give a shit and don't have anything super special about how the interact with MCP configuration files or the protocol (other than some additional fine-tuning, again, to make it less likely to get the wrong output).