> The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. I think even that's too-op

> The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. I think even that's too-optimistic: The LLM is a document-extender, so its "belief" is whether a token seems like it would statistically fit-next in a partial document, based on prior documents. This is usually not the kind of analytic truth we're interested in, and we've already figured out how to constantly extract it. If we peek at vectors and weights, we'll we'll probably end up measuring the moods and styles for whatever tokens are about to get emitted next, whether that's dialogue for a fictional character (of various kinds), a narrator, or an impersonal memo conclusion paragraph. We'll be measuring "earnestness and conviction", on the same level as "loquaciousness" or "pleading" or "talking like a pirate." So is Truthiness [0] what we really want? Probably not. If our document described the character as Yoda, then The Force connecting all existence ends up truthy. Using "a really gullible person" can repeat anything you supply as truthy. Even if we set things up as "a respected encyclopedia article" or "a relentlessly logical super-genius", we're really changing the influence mix of styles and biases, rather than creating a logical mind independent of text inside the LLM. [0] https://en.wikipedia.org/wiki/Truthiness https://en.wikipedia.org/wiki/Truthiness