6 ms·

Except that any properly written software will respond to protocol and schema violations by throwing an error. Claude will happily cast your int into a 2023 To

Except that any properly written software will respond to protocol and schema violations by throwing an error. Claude will happily cast your int into a 2023 Toyota Yaris and keep on hallucinating things.

starkparker 1y ago
For the curious: > Cast an integer into the type of a 2023 Toyota Yaris using Javascript (GPT-4o mini) > To cast an integer into the type of a 2023 Toyota Yaris in JavaScript, you would typically create a class or a constructor function that represents the Toyota Yaris. Then, you can create an instance of that class using the integer value. Here's an example of how you might do this: // Define a class for the 2023 Toyota Yaris class ToyotaYaris { constructor(modelYear, value) { this.modelYear = modelYear; this.value = value; } displayInfo() { console.log(`Model Year: ${this.modelYear}, Value: ${this.value}`); } } // Function to cast an integer into a Toyota Yaris object function castToYaris(integerValue) { const modelYear = 2023; // Set the model year return new ToyotaYaris(modelYear, integerValue); } // Example usage const integerValue = 20000; // Example integer value const yaris = castToYaris(integerValue); yaris.displayInfo(); // Output: Model Year: 2023, Value: $20000
- mrits 1y ago
  Billy, it's becoming increasingly harder to believe you are writing this code yourself
- tempaccount420 1y ago
  You really tried to inflict the most amount of damage to the reader by choosing GPT-4o mini
cle 1y ago
I just tried this in Claude Code. I made an MCP server whose tool output is declared as an integer but it returns a string at runtime. Claude Code validated the response against the schema and did not pass the response to the LLM. test - test_tool (MCP)(input: "foo") ⎿ Error: Output validation error: 'bar' is not of type 'integer'
- ohdeargodno 1y ago
  This time. Can you guarantee it will validate it every time ? Can you guarantee the way MCPs/tool calling are implemented (which is already an incredible joke that only python brained developers would inflict upon the world) will always go through the validation layer, are you even sure of what part of Claude handles this validation ? Sure, it didn't cast an int into a Toyota Yaris. Will it cast "70Y074" into one ? Maybe a 2022 one. What if there are embedded parsing rules into a string, will it respect it every time ? What if you use it outside of Claude Code, but just ask nicely through the API, can you guarantee this validation still works ? Or that they won't break it next week ? The whole point of it is, whichever LLM you're using is already too dumb to not trip when lacing its own shoes. Why you'd trust it to reliably and properly parse input badly described by a terrible format is beyond me.
  - cle 1y ago
    This is deterministic, it is validating the response using a JSON Schema validator and refusing to pass it to an LLM inference. I can't gaurantee that behavior will remain the same more than any other software. But all this happens before the LLM is even involved. > The whole point of it is, whichever LLM you're using is already too dumb to not trip when lacing its own shoes. Why you'd trust it to reliably and properly parse input badly described by a terrible format is beyond me. You are describing why MCP supports JSON Schema. It requires parsing & validating the input using deterministic software, not LLMs.
    - whoknowsidont 1y ago
      >This is deterministic, it is validating the response using a JSON Schema validator and refusing to pass it to an LLM inference. No. It is not. You are still misunderstanding how this works. It is "choosing" to pass this to a validator or some other tool, _for now_. As a matter of pure statistics, it will simply not do this at some point in the future on some run. It is inevitable.
      - cle 1y ago
Groxx 1y ago
This can even be a feature: https://github.com/Zorokee/ArtificialCast https://github.com/Zorokee/ArtificialCast

starkparker 1y ago

For the curious: > Cast an integer into the type of a 2023 Toyota Yaris using Javascript (GPT-4o mini) > To cast an integer into the type of a 2023 Toyota Yaris in JavaScript, you would typically create a class or a constructor function that represents the Toyota Yaris. Then, you can create an instance of that class using the integer value. Here's an example of how you might do this: // Define a class for the 2023 Toyota Yaris class ToyotaYaris { constructor(modelYear, value) { this.modelYear = modelYear; this.value = value; } displayInfo() { console.log(`Model Year: ${this.modelYear}, Value: ${this.value}`); } } // Function to cast an integer into a Toyota Yaris object function castToYaris(integerValue) { const modelYear = 2023; // Set the model year return new ToyotaYaris(modelYear, integerValue); } // Example usage const integerValue = 20000; // Example integer value const yaris = castToYaris(integerValue); yaris.displayInfo(); // Output: Model Year: 2023, Value: $20000

mrits 1y ago
Billy, it's becoming increasingly harder to believe you are writing this code yourself
tempaccount420 1y ago
You really tried to inflict the most amount of damage to the reader by choosing GPT-4o mini

cle 1y ago

I just tried this in Claude Code. I made an MCP server whose tool output is declared as an integer but it returns a string at runtime. Claude Code validated the response against the schema and did not pass the response to the LLM. test - test_tool (MCP)(input: "foo") ⎿ Error: Output validation error: 'bar' is not of type 'integer'

ohdeargodno 1y ago
This time. Can you guarantee it will validate it every time ? Can you guarantee the way MCPs/tool calling are implemented (which is already an incredible joke that only python brained developers would inflict upon the world) will always go through the validation layer, are you even sure of what part of Claude handles this validation ? Sure, it didn't cast an int into a Toyota Yaris. Will it cast "70Y074" into one ? Maybe a 2022 one. What if there are embedded parsing rules into a string, will it respect it every time ? What if you use it outside of Claude Code, but just ask nicely through the API, can you guarantee this validation still works ? Or that they won't break it next week ? The whole point of it is, whichever LLM you're using is already too dumb to not trip when lacing its own shoes. Why you'd trust it to reliably and properly parse input badly described by a terrible format is beyond me.
- cle 1y ago
  This is deterministic, it is validating the response using a JSON Schema validator and refusing to pass it to an LLM inference. I can't gaurantee that behavior will remain the same more than any other software. But all this happens before the LLM is even involved. > The whole point of it is, whichever LLM you're using is already too dumb to not trip when lacing its own shoes. Why you'd trust it to reliably and properly parse input badly described by a terrible format is beyond me. You are describing why MCP supports JSON Schema. It requires parsing & validating the input using deterministic software, not LLMs.
  - whoknowsidont 1y ago
    >This is deterministic, it is validating the response using a JSON Schema validator and refusing to pass it to an LLM inference. No. It is not. You are still misunderstanding how this works. It is "choosing" to pass this to a validator or some other tool, _for now_. As a matter of pure statistics, it will simply not do this at some point in the future on some run. It is inevitable.
    - cle 1y ago

Groxx 1y ago

This can even be a feature: https://github.com/Zorokee/ArtificialCast https://github.com/Zorokee/ArtificialCast