2 ms·
How do you know it’s a novel question?
by MichaelZuo 1y ago
How do you know it’s a novel question?
- IshKebab 1y agoIt's not exactly difficult to come up with a question that's so unusual the chance of it being in the training set is effectively zero.
- troupo 1y agoAnd as any programmer will tell you: they immediately devolve into "hallucinating" answers, not trying to actually reason about the world. Because that's what they do: they create statistically plausible answers even if those answers are complete nonsense.
- MichaelZuo 1y agoCan you provide some examples of these genuinely unique questions?
- pdabbadabba 1y agoI'm not sure what you mean by "genuinely." But in the coding context LLMs answer novel questions all the time. My codebase uses components and follows patterns that an LLM will have seen before, but the actual codebase is unique. Yet, the LLM can provide detailed explanations about how it works, what bugs or vulnerabilities it might have, modify it, or add features to it.
- MichaelZuo 1y agoIt must not have existed prior in any text database whatsoever.
- pdabbadabba 1y agoIt certainly wasn't. The codebase is thousands of lines of bespoke code that I just wrote.
- drw85 1y agoWhich pretty much every line in it was written similarly somewhere else before, including an explanation and is somehow included in the massive data set it was trained on. So far i have asked the AI some novel questions and it came up with novel answers full of hallucinated nonsense, since it copied some similarly named setting or library function and replaced a part of it's name with something i was looking for.
- pdabbadabba 1y agoAnd this training data somehow includes an explanation of how these individual lines (with variable names unique to my application) work together in my unique combination to produce a very specific result? I don't buy it. And... > pretty much Is it "pretty much" or "all"? The claim that the LLM simply has simply memorized all of its responses seems to require "all."
- hackinthebochs 1y agoYou have probably seen examples of LLMs doing the "mirror test", i.e. identifying themselves in screenshots and referring to the screenshot from the first person. That is a genuinely novel question as an "LLM mirror test" wasn't a concept that existed before about a year ago.
- MichaelZuo 1y agoElephant mirror tests existed, so it doesn’t seem all that novel when the word “elephant” could just be substituted for the word “LLM”?
- hackinthebochs 1y agoThe question isn't about universal novelty, but whether the prompt/context is novel enough such that the LLM answering competently demonstrates understanding. The claim of parroting is that the dataset contains a near exact duplicate of any prompt and so the LLM demonstrating what appears to be competence is really just memorization. But if an LLM can generalize from an elephant mirror test to an LLM mirror test in an entirely new context (showing pictures and being asked to describe it), that demonstrates sufficient generalization to "understand" the concept of a mirror test.
- MichaelZuo 1y agoHow do you know it’s the one generalizing? Likely there has been at least one text that already does that for say dolphin mirror tests or chimpanzee mirror teats.