4 ms·

"The LLMs have no ground truth" claim (around chapter 2) that's core to the "bullshit machines" argument is itself wrong. Of course LLMs have ground truth. What

• ImPleadThe5th 2y ago
  So if I ask ChatGpt about bears and in the middle of explaining their diet it tells me something about how much they like porridge and in the middle of habitat it tells me they live in a quaint cabin in the woods, that's ... True? Statistically we certainly have a lot of words about 3 bears and their love for porridge. That doesn't mean it's true, it just means it's statistically significant. If I asked someone a scientific question about bears and they told me about Goldilocks, id think it was bullshit.
  • tmnvdb 2y ago
    Having a ground truth doesn't mean it does not make huge and glaring mistakes.
  • vaidhy 2y ago
    If that were the case, then you are right.. However, the current crop of LLMs seem to be good at understanding the context. A scientific data point about bears is unlikely to have Goldilocks in there (unless talking about evolution of life and Goldilocks zone). You can argue that there is meaning hidden in words that is not captured by words themselves in a given context - psychic knowledge as opposed to reasoned out knowledge. That is a philosophical debate.
    • TeMPOraL 2y ago
      Words don't carry meaning. Meaning exists in how words are or are not used together with other words. That is, in their.... statistical relationships to each other!
  • TeMPOraL 2y ago
    ChatGPT has enough dimensions in its latent space to represent and distinguish between the various meanings of porridge and is able to be informed by the Goldilocks story without switching to it mid-sentence. It's actually a good example of what I have in mind by saying human text isn't random. The Goldilocks story may not be scientific, but it's still highly correlated with scientific truth about matters like food, bears, or the daily lives of people. Put yourself in the shoes of an alien trying to make heads or tails of that story, you'll see just how many things in it are not arbitrary.
  • empath75 2y ago
    LLM's demonstrably don't do this, nor do they say that they live in the hundred acre woods and love honey. Unless you ask about a specific bear.
• sesm 2y ago
  > Statistical ground truth isn't any worse than explicitly given one There are multiple kinds of truths. 'Statistical truth' is at best 'consensus truth', and that's only when LLM doesn't hallucinate.
  • TeMPOraL 2y ago
    That's the only one that's available, though. When a kid at school is being taught, say, Newton's laws of motion, or what happened in 476 CE, they're not experiencing the empirical truth about either. They're only learning the consensus truth, i.e. the correct answer to give to the teacher, so they get good grade instead of bad grade, and so their parents praise them instead of punishing them, etc. This covers pretty much everything any human ever learns. Few are in position to learn any particular things experimentally. Few are in position to verify most of what they've learned experimentally afterwards. We live in a "consensus reality", but that works out fine, because establishing consensus is hard, and one of the strongest consensus-forcing forces that exist is "close enough to actual reality".
    • sesm 2y ago
      I've heard about at least 4 theories of truth: Correspondence, Coherence, Consensus and Pragmatic (as described, for example, here https://commoncog.com/four-theories-of-truth/ https://commoncog.com/four-theories-of-truth/). If we look at Newtonian mechanics, then various independently verifiable experiments are examples of Correspondence truth, and the minimal mathematical framework that describes them is an example of Coherence truth.
      • TeMPOraL 2y ago
        Fine, but it's not how any of us learned of it either - whether the Newtonian mechanics or the "4 theories of truth". I mean, coherence is sure a an important aspect of truth, and just by paying attention whether it all "adds up" you can easily filter 90% of the bullshit you hear people (or LLMs for that matter) saying - but even there, I'm not a physicist, I don't do much experiments in a lab, so when I evaluate if some information is coherent with Newton's laws of motion, I'm actually evaluating some description of a situation against a description of Newton's laws. It's all done in "consensus space" and, if an answer is expected, the answer is also a "consensus space" one. We're all so used to evaluating inputs and outputs through the lens of "is this something I expect others believe, and others expect me to believe", that we're almost always just mentally folding the indirection through "consensus reality" and feel like we're just checking "is this true". It works out okay, and it can't really be any other way - but we need to remember this is what we're doing.
• fancyfredbot 2y ago
  Statistical truths based on observation of reality is the basis for science. Statistical truth based on the text on the internet is the basis for something else, and I would personally not like to call whatever that is science, or any truths established this way "ground truths".
  • TeMPOraL 2y ago
    Statistical truths based on observation of reality is the basis for incremental additions science. Statistical truths based on what you read in the textbook is what actually is almost all science, to everyone, at almost all times. There's few things in our lives any of us actually learned first-hand, empirically. Everything else we learned the same way LLMs did - as what is expected to be the right completion to a question or inquiry. The objective truth as we experience is things that, were we to make a prediction conditioned on them, our prediction would turn out correct. That doesn't mean we actually make such predictions often, or that we ever made such predictions learning it.
  • empath75 2y ago
    What year did the Normans invade England? What's Newton's second law? Who was the last czar of Russia? How many moons does Jupiter have? I bet you "know" a lot of those facts not because you have observed them empirically, but because you read about them in books. And in fact, almost all scientists rely on reading for nearly everything they know about science, including nearly everything they know about their own specialties. Nobody has time to derive all of their knowledge of the world from personal observation, and most people who can read have probably learned almost all "facts" they know about the world from books.
    • TeMPOraL 2y ago
      Yup, and importantly, the correct answer to those is, in anyone's personal learning experience, almost always just what the authority figures in their lives (parents, teachers, peers they respect, or textbooks themselves) would accept as the correct answer! "Consensus reality" works so well for us most of the time, that we habitually "substitite the Moon for the finger pointing at the Moon" without realizing it.
• krupan 2y ago
  It's true that LLMs aren't trained on strings of random words, so in a sense you correct are that they have some "ground truth." They wouldn't generate anything logical at all if not. Does that even need to be stated though? You don't need AI to generate random words. The more important point is, they aren't trained on only factual (or statistically certain) statements. That's the ground truth that's missing. It's easy to feed an LLM a bunch of text scraped from the internet. It's much harder to teach it how to separate fact from fiction. Even the best human minds that live or ever have lived have not been able to do that flawlessly. We've created machines that have a larger amount of memory than any human, much quicker recall, the ability to converse with vast numbers of people at once, but it performs at about par with humans in discerning fact from fiction. That's my biggest concern about creating super powered artificial intelligence. It's super powers are only super in a couple dimensions and people mistake that for general intelligence. I came across someone online that really believed chatGPT was creating a custom diet plan tailored to their specific health needs, base on a few prompts. That is scary!
• HarHarVeryFunny 2y ago
  Well, duh ... of course there are statistical regularities in the data, which is what LLMs learn. However, the LLM has no first hand experience of the world described by those words, so the words (to an LLM) are not grounded. So, the LLM is doomed to only be able to predict patterns in the dataset, with no knowledge or care as to whether what it is outputting is objectively true or not. Calling it bullshitting would be an anthromorphism since it implies intention, but it's effectively what an LLM does - just spits out words without any care (or knowledge) as to the truth of them. Of course if there is more truth than misinformation and lies in the dataset, then statistically that is more likely to be output by an LLM, but to the LLM it's all the same - just word stats, just the same as when it "hallucinates" and "makes something up", since there is always a non-zero possibility that the next word could be ... anything.
• 1970-01-01 2y ago
  >Of course LLMs have ground truth. It is my understanding that LLMs have no such thing, as empiric truth is weighted. For example, if Newton's laws are in conflict with another fact, the LLM will defer to the fact that it finds more probable in context. It will then require human resources to undo and unfold it's core error, else you receive bewildering and untrue remarks or outputs.
  • jononor 2y ago
    Where "probable" means: occurs the most often in the training data (approx the entire Internet). So what is common online is most likely to win out, not some other notion of correctness.
  • TeMPOraL 2y ago
    > For example, if Newton's laws are in conflict with another fact, the LLM will defer to the fact that it finds more probable in context. Which is the correct thing to do. If such a context would be, for example, an explanation of an FTL drive in a science fiction story, both LLMs and humans would be correct to put Newton aside. LLMs aren't Markov chains, they don't output naive word frequency based predictions. They build a high-dimensional statistical representation of entirety of their training data, from which then completions are sampled. We already know this representation is able to identify and encode ideas as diverse as "odd/even" and "fun" and "formal language" and "golden gate bridge"-like. "Fictional context" vs. "Real-life physics" is a concept they can distinguish too, just like people can.
• nickpsecurity 2y ago
  Ground truth is possibly used in the sense that humans’ brains tie what they create to the properties of observed reality. Whatever new information comes in is compared to, or checked by, that. Whereas, LLM’s will believe anything you feed them in training no matter how unrealistic it is. I do think that, after much training data, they do have specific beliefs that are ingrained in them. That changing those is difficult. We’ve seen that on some political and scientific claims that must have been prominent in their pre-training data or RLHF tuning. They will argue with us over those points, like it’s a fight. Otherwise, I’ve seen continued pre-training or fine-tuning can change everything up to their vocabularies.
  • TeMPOraL 2y ago
    > Ground truth is possibly used in the sense that humans’ brains tie what they create to the properties of observed reality. Whatever new information comes in is compared to, or checked by, that. What do you compare your knowledge of history to, other than what you expect other people will say? Most knowledge we learn in life is tied only to our expectations of other people's reactions. Which works out fine, most of the time, because even with questions of scientific fact, it's enough some people are in position to ground information in empirical experiments, and then set everyone else's expectations based on that.
    • nickpsecurity 2y ago
      To start with, we know what a person is, rudimentary things about how they behave, our senses, how they commonly work, and can do mental comparisons (reality checks). We know LLM’s don’t start with that because we initialize them with zero’d or random weights. Then, their training data can be far more made up, even works of fiction, that the reality most humans observe with is almost always real. We could raise a human in VR or something where technically there would be comparisons. Most humans’ observations connect to expectations in their brain which was designed for the reality we operate in. Finally, the brain has multiple components that each handle different jobs. They have different architectures to do those jobs well. Sections include language, numbers, reasoning, tiers of memory, hallucination prevention, mirroring, and even meta-level stuff like reward adjustment. We don’t just speculate that they do different things: damage to those regions shuts down those abilities. Tied to the physical realm are vision, motor, and spatial areas. We can feel objects, even temperature or pressure changes. That we can do a lot of that without supervised learning shows we’re tailor-made by God for success in this world. LLM’s have one architecture that does one job which we try to get to do other things, like reasoning or ground truth. We pretend it’s something it’s not. The multimodal LLM’s are getting closer with specialized components. Even they aren’t all trained in a coherent way using real-world, observations in the senses. There’s usually a gap between systems like these and what the brain does in the real world just in how it gets reliable information about its operating environment.