4 ms·

> A reasoning error has an infinite, unpredictable blast radius. Says who? It’s quite easy to limit the blast radius of a reasoning error.

• amazingamazing 5mo ago
  How so? Suppose you had: Math() Add() Subtract() Program() Math(“calculate rate”) This is intentionally written vaguely. How do you limit that these implementations ensure Program() runs and does the right thing when there is no guarantee Math() or its components are correct? Normally you could use a typed programming language, unit tests, etc, but if LLM is the ultimate abstraction programs will be written line above. At some point traditional software engineering principles will need to apply.
• distalx 5mo ago
  In 2024, a Chevy dealership deployed an AI chatbot that confidently agreed to sell a customer a 2024 Chevy Tahoe for $1. It executed a catastrophic business failure simply because it didn't know the logic was wrong. Sure, you can patch that specific case with guardrails, but how many unpredictable edge cases are you going to cover? It only takes a user with a bit of ingenuity to circumvent them. There are already several examples of AI agents getting stuck in infinite loops, burning through massive API bills while achieving absolutely nothing. You can contain a system failure, but you cannot contain a logic failure if the system doesn't know the logic is wrong.
  • pear01 5mo ago
    This would be more convincing if a single car had been exchanged for $1. It didn't happen. Seems the bug was "contained". Sort of undermines your point re "catastrophic business failure" don't you think?
  • yunwal 5mo ago
    > but how many unpredictable edge cases are you going to cover? This is the wrong question. The correct question is what specific subsets of cases do you allow, similar to any security question
  • yunwal 5mo ago
    You literally can contain a logic failure. If I execute logic on my computer that’s not connected to the internet it can’t get out of the box. Done. Contained