6 ms·

I feel like there's some semantic slippage around the meaning of the word "accuracy" here. I grant you, my print Encyclopedia Britannica is not 100% accurate.

• jononor 2y ago
  Encyclopedia Britannica is also wrong in a reproducible and fixable way. And the input queries a finite set. It's output does not change due to random or arbitrary things. It is actually possible to verify. LLMs so far seem to be entirely unverifiable.
  • lukev 2y ago
    They don’t just seem it. They are by design. We talk about models “hallucinating” but that’s us bringing an external value judgement after the fact. The actual process of token generation works precisely the same. It’d be more accurate to say that models always hallucinate.
    • madeofpalk 2y ago
      Yes - this is what i've been saying all the time. The term 'hallucinations' is misleading because the whole point of LLMs is that they recombine all their inputs into something 'new'. They only ever hallucinate outputs - that's their whole point!
      • wizzwizz4 2y ago
        Into something probable. The models that underlie these chatbots are usually overfitted, so while they usually don't repeat their training data verbatim, they can.
    • DougBTX 2y ago
      > The actual process of token generation works precisely the same I’d be wary of generalising it like that, it is like saying that all programs run on the same set of CPU instructions. NNs are function approximators, where the code is expressed in model weights rather than text, but that doesn’t make all functions the same.
      • lukev 2y ago
        You misunderstand. I mean that the model itself is doing exactly the same thing whether the output is a “hallucination “ or happens to be fact. There isn’t even a theoretical way to distinguish between the two cases based only on the information encoded in the model.
      • skydhash 2y ago
        > it is like saying that all programs run on the same set of CPU instructions Turing machine is the embodiment of all computer programs. And then you come across the halting problem. LLMs can probably generate all books in existence, but it can't apply judgement to it. Just like you need programmers to actually write the program and verify that it correctly solves the problem. Natural languages are more flexible. There are no functions libraries, or paradigms to ease writing. And the problem space can't be specified and usually relies on shared context. Even if we could have snippets of prompts to guide text generations, the result is not that valuable.
    • intended 2y ago
      YES. Humans can hallucinate, its a deviation from what is observable reality. All the stress people are feeling with GenAI comes from the over anthropomorphisation of ... stats. Impressive syntatic ability is not equivalent to semantic capability.
      • tremon 2y ago
        The human definition of hallucination has to do with sensory experience, i.e. inputs. Saying that LLMs hallucinate means that we're ascribing them control over their inputs that they simply do not have -- by design. Or, in other words, if a chatbot really were hallucinating, it would probably start giving unprompted responses. > Humans can hallucinate, its a deviation from what is observable reality. What is "observable reality" then, for an LLM? Its training set?
  • ta8645 2y ago
    > LLMs so far seem to be entirely unverifiable. I don't understand this complaint. Are they any less verifiable than a human?
    • threeseed 2y ago
      I can ask a human to explain the steps they took to answer a question. I can ask a human a question 100 times and I don't get back 100 different answers. None of those applies to an LLM.
      • ta8645 2y ago
        You can ask an LLM to explain itself.. it will give you a logical stepwise progression from your question to its answer. It will often contain a mistake, but the same is true for a human. And if your LLM is giving you 100 different answers, then it has been configured to do so. Because instead, it could be configured to never vary at all. It could be 100% reproducible if so desired.
        • soloist11 2y ago
          [dead]
        • bluefirebrand 2y ago
          > it will give you a logical stepwise progression from your question to its answer. No, it will generate a new hallucination that might be a logical stepwise progression from the question you asked to the answer you gave, but it is not due to any actual internal reasoning being done by the LLM
          • ta8645 2y ago
            So what? You have no way to know for sure if the human you ask the same question, does either. The question that started this thread was related to verifiability. And i still think it is a spurious complaint, given that we have exactly the same limitations when dealing with any human agent.
  • somenameforme 2y ago
    LLMs are completely deterministic even if that's kind of weird to state because they output things in terms of probabilities. But if you simply took the highest probability next word, you'd always yield the exact same output given the exact same input. Randomness is intentionally injected to make them seem less robotic through the 'temperature' parameter. Why it's not just called the rng factor is beyond me.
    • dleeftink 2y ago
      Maybe some models can be deterministic at a point in time, but train it for another epoch with slight parameter changes and a revised corpus and determinism goes out the proverbial (sliding) window real quick. This is not unwanted per se, and the exact feedback loop that needs improving to better integrate new knowledge or revise knowledge artefacts incrementally/post-hoc.
      • Vetch 2y ago
        If you train it then it's no longer the same model. If I have f(x) = x + 1 and change it to f(x) = x + 1 + 1/1e9, it would not mean that `f` is not deterministic. The issue would be in whatever interface I was exposing the f's at.
        • jononor 2y ago
          But current models must be retrained to incorporate new information. Or to attempt to fix undesirable behavior. So just freezing it forever does not seem feasible. And because there is no way to predict what has changed - one has to verify everything all over again.
          • avar 2y ago
            Would you by extension argue that e.g. modern relational database aren't deterministic in their query execution? Their query plans tend to be chosen based on statistics about the tables they're executed against, and not just the query itself. I don't see how that's different than the LLM case, a lot of algorithms change as a function of the data they're processing.
  • ein0p 2y ago
    LLMs are deterministic if you want them to be. If you eagerly argmax the output you will get the same sequence for the same prompt every time
• eynsham 2y ago
  > it’s not /justified/ belief Beliefs derived from the output of LLMs that are ‘right most of the time’ pass one facially plausible precisification of ‘justification’ in that they are generated by a reliable belief-generation mechanism (see e.g. Goldman). To block this point one must engage with the post-Gettier literature at least to some extent. There is a clear difference between beliefs induced by reading the outputs of LLMs and those induced by the contents of a reference work, but it is inessential to the point and arguably muddies the water to present the distinction as difference in status as knowledge or non-knowledge.
  • nicklecompte 2y ago
    To be clear he is saying that the LLM is not capable of justified true belief, not commenting on people who believe LLM output. I don’t think your comment is relevant here.
    • lukev 2y ago
      I do think trusting an LLM is less firm ground for knowledge than other ways of learning. Say I have a model that I know is 98% accurate. And it tells me a fact. I am now justified in adjusting my priors and weighting the fact quite heavily at .98. But that’s as far as I can get. If I learned a fact from an online anonymously edited encyclopedia, I might also weight that a 0.98 to start with. But that’s a strictly better case because I can dig more. I can look up the cited sources, look at the edit history, or message the author. I can use that as an entry point to end up with significantly more than 98% conviction. That’s a pretty important difference with respect to knowledge. It isn’t just about accuracy percentage.
    • eynsham 2y ago
      That reading of the comment did occur to me, but I think neither dictionaries nor LLMs are capable of belief, and the comment was about the status of beliefs derived from them.
      • nicklecompte 2y ago
        Okay we are speaking past each other, and you are still misunderstanding the subtlety of the comment: A dictionary or a reputable Wikipedia entry or whatever is ultimately full of human-edited text where, presuming good faith, the text is written according to that human's rational understanding, and humans are capable of justified true belief. This is not the case at all with an LLM; the text is entirely generated by an entity which is not capable of having justified true beliefs in the same way that humans and rats have justified true beliefs. That is why text from an LLM is more suspect than text from a dictionary.
• hnfong 2y ago
  The Gettier problem is an indication that the definition has (at least) a bug. There are other formulations of "knowledge" which does not involve justification, see eg. Gnosticism. Of course, for a publicly available frequently used service, the "JTB" formulation of knowledge is probably the only one we can practically use, but this kind of indicates that the whole idea of search engines, knowledge systems, or expert systems is flawed due to the Gettier problem.
• seanp2k2 2y ago
  +1, AIs don’t really “understand” anything, like human anatomy and deformity rates and societal norms when tasked with image generation, so you get hands with weird numbers of digits and other topological errors which even a very unintelligent human wouldn’t make. AI doesn’t understand and build knowledge in interconnected layers, it can’t “think” and link things back to first principles, it can’t really reason about things, and it’s not going to get significantly better until we start approaching it differently. This generation of AI might be useful for some things, but it’s being applied wayyyy too broadly too quickly and I see a big pullback coming. “Expert systems” are not a new thing, and they’re still not all that useful except in some very small niches. Phone trees that can keyword match FAQs are useful for a lot of low-effort callers who put zero effort into solving their problem on their own first, but frustrating for callers who are only calling because there’s literally no other way for them to resolve their issue. Unfortunately for consumers, the cost is very low for businesses to make everyone wade through junky phone tree systems and penalize anyone who tries to mash zero to talk to a real person, even if that’s the only thing which will actually help them.