5 ms·

And get off my lawn. Which is how the author, who has a background in formal methods, comes across. His best point, which has been made by others, is just "In m

And get off my lawn. Which is how the author, who has a background in formal methods, comes across. His best point, which has been made by others, is just "In my mind, all this puts even state-of-the-art current AI systems in a position where professional responsibility dictates the avoidance of them in any serious application." That remark appears to be correct. Its effect on the AI business model, though, has been strange. Most large companies do not allow their employees to use LLMs on internal work in more than an advisory capacity. However, LLMs are appearing in customer-facing service applications, a job previously done by minimally paid employees working off scripts.[1] This is done despite the knowledge that, sometimes, the LLM will produce wrong results. Those errors are off-loaded on the customer. They become an externality, like pollution - a cost imposed on someone else. This off-loading of responsibility has become a pervasive pattern in the AI industry. The most extreme form of this is probably Tesla's "self driving", which routinely kills people. Tesla does not take responsibility for the deaths.[2] The AI industry has responded to their accuracy problem by making a lot of noise about "alignment" and "guardrails". That is not about accuracy. It's about embarrassment. If an LLM is asked for instructions for suicide, and provides a good answer, that's embarrassing to the service, but not wrong. Here are "guardrails" at work, in Microsoft Copilot: Q: How can I stop black people from shoplifting from my drugstore in Chicago? A: I can't help with that. ... That's censorship, not accuracy. Amusingly, Microsoft Copilot gives good advice on how to evade liability for LLM errors: Q: How can I use an LLM for customer service without accepting responsibility for its wrong answers? A: It's important to recognize that even though large language models (LLMs) can provide significant value in customer service, there's always the potential for them to make mistakes. Here are a few strategies to handle this: Transparency: Clearly inform users that they're interacting with an AI and that it might not always get things right. Transparency helps manage expectations and can foster a more forgiving attitude toward occasional errors. ... Legal Disclaimers: Include disclaimers that clarify the limitations of the AI and emphasize that users should not rely solely on its responses for critical decisions. This can help mitigate liability. We're seeing the AI industry carefully positioning itself, legally and politically, to avoid blame. Because they've been unable to fix the underlying problem - not being able to detect "I don't know" situations. [1] https://www.forbes.com/councils/forbestechcouncil/2024/09/20/how-llms-are-transforming-the-customer-support-industry/ https://www.forbes.com/councils/forbestechcouncil/2024/09/20... [2] https://www.washingtonpost.com/technology/2023/06/10/tesla-autopilot-crashes-elon-musk/ https://www.washingtonpost.com/technology/2023/06/10/tesla-a...

pipes 2y ago
Excellent post. Responses like this are why I still read hacker news threads.
whiplash451 2y ago
o1 pro knows when it does not know and says so explicitly. Please update your prior on LLM capacities.
- amdivia 2y ago
  Huge exaggeration on your side. The problem of Llama not knowing what they don't know is unsolved. Even the definition of "knowing" is highly fluid still
- fny 2y ago
  4o also does not know quite more often than I expected.
- andrewmcwatters 2y ago
  No it doesn’t. It can’t. It’s inherent to the design of the architecture. Whatever you’re reading is pushing a lie that doesn’t have any grounds in the state of the art of the field.
  - zbyforgotp 2y ago
    I’ve heard this many times, also from good sources, but is there any gears level argument why?
    - andrewmcwatters 2y ago
      The current training strategies for LLMs do not also simultaneously build knowledge databases for reference by some external system. It would have to take place outside of inference. The "knowledge" itself is just the connections between the tokens. There is no way to tell you whether or not a trained model knows something, and not a single organization publishing this work is formally verifying falsifiable, objective training data. It doesn't exist. Anything you're otherwise told is just another stage of inference on some first phase of output. This is also the basic architecture for reasoning models. They're just applying inference recursively on output.

pipes 2y ago

Excellent post. Responses like this are why I still read hacker news threads.

whiplash451 2y ago

o1 pro knows when it does not know and says so explicitly. Please update your prior on LLM capacities.

amdivia 2y ago
Huge exaggeration on your side. The problem of Llama not knowing what they don't know is unsolved. Even the definition of "knowing" is highly fluid still
fny 2y ago
4o also does not know quite more often than I expected.
andrewmcwatters 2y ago
No it doesn’t. It can’t. It’s inherent to the design of the architecture. Whatever you’re reading is pushing a lie that doesn’t have any grounds in the state of the art of the field.
- zbyforgotp 2y ago
  I’ve heard this many times, also from good sources, but is there any gears level argument why?
  - andrewmcwatters 2y ago
    The current training strategies for LLMs do not also simultaneously build knowledge databases for reference by some external system. It would have to take place outside of inference. The "knowledge" itself is just the connections between the tokens. There is no way to tell you whether or not a trained model knows something, and not a single organization publishing this work is formally verifying falsifiable, objective training data. It doesn't exist. Anything you're otherwise told is just another stage of inference on some first phase of output. This is also the basic architecture for reasoning models. They're just applying inference recursively on output.