3 ms·

But if an LLM says "I don't know" should you pay for the tokens?

by amelius 4mo ago

guerrilla 4mo ago
Why not? It did the work. Why should you expect it to be omniscient? We can rank them based on how much they know and people will gravitate towards those that do know more. It's a market after all.
- mmmattt 4mo ago
  If it’s a market, wouldn’t the incentive be to lie about knowing and thus to keep the hallucinations?
  - BDPW 4mo ago
    If you had an llm that could accurately predict when a claim is uncertain it would be very popular, I think. I would pay for that kind of reliability tbh
    - intended 4mo ago
      This would break reality. There’s some underlying physical law that prevents the existence of any algorithm of truth.
      - embedding-shape 4mo ago
        > There’s some underlying physical law that prevents the existence of any algorithm of truth Haven't heard about that law, but seems unlikely we can come up with ("discover") any sort of law that uses a concept ("truth") humans can't even agree what it means, and that's not for a lack of trying, we've been trying to figure it out for millenniums already with no end in sight.
      - derektank 4mo ago
        If you accept certain axioms a priori, it’s fine. If you simply let the machine intelligence take it for granted that induction works because nature is uniform and give it some way to test its predictions, it would have all the building blocks it needs to reason out a lot of very useful information. Which as the parent comment points out, people would absolutely pay a lot of money for.
        intended 4mo ago
        If I am reading you correctly, that ends in testing, or essentially doing science. That wouldn't satisfy the requirement to look at a claim and be able to tell if a claim was uncertainm roughly speaking.
  - Greenpants 4mo ago
    Up to the point where consumers notice and decide to stop using these models because of it. Might be why we're already rarely seeing models output an "I don't know".
  - nutjob2 4mo ago
    According to your logic the market will produce an LLM that consists only of 'PRINT "I don't know."'.
embedding-shape 4mo ago
Depends on what your understanding of the product is. If someone sold you a "Solved all your problems" machine, and it suddenly doesn't solve all your problems, then probably no, you shouldn't pay. But the way I'm being sold LLMs, is basically "A text generator that gives your plausible-sounding human text that sometimes hallucinates and gets things wrong, based on your input", then regardless of what the outcome is, I still made use of the "Input > Output" part, which is what I bought into, so I should still pay for that. Now of course bunch of people will say they been sold the former, but the companies themselves seem to be selling the latter. That's my perspective from a person who doesn't follow "influencers" and what not though, which seem to be selling the public on the former rather than the latter.
- al_borland 4mo ago
  Let's pretend I am someone who has heard people talk about ChatGPT, but have on idea what it actually is. I go to the website and am not presented with any information, just a prompt. So I ask it what it is and what it can do for me. My ask: > In a couple sentences, explain to me the product I'm being sold with ChatGPT. What does it do for me? The Reply from ChatGPT: > ChatGPT is a conversational AI that helps you think, create, learn, analyze, and get things done faster. You can use it to answer questions, draft and edit writing, summarize information, brainstorm ideas, learn new topics, write code, plan projects, and increasingly act as an assistant that can search for information, work with documents, generate images, and help complete tasks. > In simple terms: you're buying access to an AI that turns natural language into useful work—saving time, expanding your capabilities, and giving you an always-available collaborator for both everyday tasks and specialized knowledge work. This sounds much more like the former, a "solve all your problems" machine.... not a plausible-sounding text generation machine. Only two weeks ago Sam Altman said their new data center "could" be where cancer gets cured[0]. It is only the people who deeply understand AI who see it as a text generator of plausible-sounding text. That isn't what the marketing department, the CEO, or the product itself seem to be saying. I'm using OpenAI as the example here, but the others don't seem much different. [0] https://www.youtube.com/watch?v=9-tOtbDDrJA https://www.youtube.com/watch?v=9-tOtbDDrJA
  - 4mo ago
nutjob2 4mo ago
'I don't know' is the correct answer for infinitley more questions than those that can be answered.
skillina 4mo ago
"I don't know" has positive value, presumably you could prompt further to learn more about where it got stuck. It also increases the value of correct answers, by improving confidence that answers are actually correct. "Confidently incorrect" has negative value. At best, a human realizes the answer is wrong and At worst, the incorrect information makes is not identified and can cause untold damage. By having the potential to be so severely wrong, it lessens the value of correct answers because there is a lower confidence value on their output.
ludwik 4mo ago
I would be very willing to pay more! The choice between “you may get a correct answer, or you may get lied to, without a clear way to distinguish between the two” and “you may get a correct answer, or a clear indication that the answer was not found” is pretty clear. One is a much more useful tool than the other. I don’t see any real incentives for companies making LLMs to keep their AI factually unreliable. (Full disclosure: I work for one, but I’m definitely not in the rooms where such decisions would be made.)
deleted 4mo ago
[deleted]
maxbond 4mo ago
Would you rather pay for a nonsensical explanation?

But if an LLM says "I don't know" should you pay for the tokens?

by amelius 4mo ago

But if an LLM says "I don't know" should you pay for the tokens?

Why not? It did the work. Why should you expect it to be omniscient? We can rank them based on how much they know and people will gravitate towards those that do know more. It's a market after all.

mmmattt 4mo ago
If it’s a market, wouldn’t the incentive be to lie about knowing and thus to keep the hallucinations?
- BDPW 4mo ago
  If you had an llm that could accurately predict when a claim is uncertain it would be very popular, I think. I would pay for that kind of reliability tbh
  - intended 4mo ago
    This would break reality. There’s some underlying physical law that prevents the existence of any algorithm of truth.
    - embedding-shape 4mo ago
      > There’s some underlying physical law that prevents the existence of any algorithm of truth Haven't heard about that law, but seems unlikely we can come up with ("discover") any sort of law that uses a concept ("truth") humans can't even agree what it means, and that's not for a lack of trying, we've been trying to figure it out for millenniums already with no end in sight.
    - derektank 4mo ago
      If you accept certain axioms a priori, it’s fine. If you simply let the machine intelligence take it for granted that induction works because nature is uniform and give it some way to test its predictions, it would have all the building blocks it needs to reason out a lot of very useful information. Which as the parent comment points out, people would absolutely pay a lot of money for.
      - intended 4mo ago
        If I am reading you correctly, that ends in testing, or essentially doing science. That wouldn't satisfy the requirement to look at a claim and be able to tell if a claim was uncertainm roughly speaking.
- Greenpants 4mo ago
  Up to the point where consumers notice and decide to stop using these models because of it. Might be why we're already rarely seeing models output an "I don't know".
- nutjob2 4mo ago
  According to your logic the market will produce an LLM that consists only of 'PRINT "I don't know."'.

embedding-shape 4mo ago

Depends on what your understanding of the product is. If someone sold you a "Solved all your problems" machine, and it suddenly doesn't solve all your problems, then probably no, you shouldn't pay. But the way I'm being sold LLMs, is basically "A text generator that gives your plausible-sounding human text that sometimes hallucinates and gets things wrong, based on your input", then regardless of what the outcome is, I still made use of the "Input > Output" part, which is what I bought into, so I should still pay for that. Now of course bunch of people will say they been sold the former, but the companies themselves seem to be selling the latter. That's my perspective from a person who doesn't follow "influencers" and what not though, which seem to be selling the public on the former rather than the latter.

al_borland 4mo ago
Let's pretend I am someone who has heard people talk about ChatGPT, but have on idea what it actually is. I go to the website and am not presented with any information, just a prompt. So I ask it what it is and what it can do for me. My ask: > In a couple sentences, explain to me the product I'm being sold with ChatGPT. What does it do for me? The Reply from ChatGPT: > ChatGPT is a conversational AI that helps you think, create, learn, analyze, and get things done faster. You can use it to answer questions, draft and edit writing, summarize information, brainstorm ideas, learn new topics, write code, plan projects, and increasingly act as an assistant that can search for information, work with documents, generate images, and help complete tasks. > In simple terms: you're buying access to an AI that turns natural language into useful work—saving time, expanding your capabilities, and giving you an always-available collaborator for both everyday tasks and specialized knowledge work. This sounds much more like the former, a "solve all your problems" machine.... not a plausible-sounding text generation machine. Only two weeks ago Sam Altman said their new data center "could" be where cancer gets cured[0]. It is only the people who deeply understand AI who see it as a text generator of plausible-sounding text. That isn't what the marketing department, the CEO, or the product itself seem to be saying. I'm using OpenAI as the example here, but the others don't seem much different. [0] https://www.youtube.com/watch?v=9-tOtbDDrJA https://www.youtube.com/watch?v=9-tOtbDDrJA
- 4mo ago

nutjob2 4mo ago

'I don't know' is the correct answer for infinitley more questions than those that can be answered.

skillina 4mo ago

"I don't know" has positive value, presumably you could prompt further to learn more about where it got stuck. It also increases the value of correct answers, by improving confidence that answers are actually correct. "Confidently incorrect" has negative value. At best, a human realizes the answer is wrong and At worst, the incorrect information makes is not identified and can cause untold damage. By having the potential to be so severely wrong, it lessens the value of correct answers because there is a lower confidence value on their output.

ludwik 4mo ago

I would be very willing to pay more! The choice between “you may get a correct answer, or you may get lied to, without a clear way to distinguish between the two” and “you may get a correct answer, or a clear indication that the answer was not found” is pretty clear. One is a much more useful tool than the other. I don’t see any real incentives for companies making LLMs to keep their AI factually unreliable. (Full disclosure: I work for one, but I’m definitely not in the rooms where such decisions would be made.)

deleted 4mo ago

[deleted]

maxbond 4mo ago

Would you rather pay for a nonsensical explanation?